A comparison of knowledge retention between online and in-class problem-based learning

Objective: To compare knowledge retention of the two learning methods: online (OPBL) and in-class problem-based learning (IPBL). Methods: A pre and post-test study design of the three-week research project was performed in volunteered students from two-academic year. After completing the pretest, the IPBL group performed an activity test. Then the instructor held an open discussion for further explanation and clarification. Afterwards, students performed a diagnostic test to earn their achievement score. The 3-hour activities in each learning specification part took place exactly a week apart. The contrary, the OPBL group performed all tests by log on the website. The online program not only established their weaknesses and urged them to explore core knowledge, but also recorded students’ profiles. After three weeks, the post-test was arranged for both groups. Four week later, the final test was managed without prior notice. The pretest, post-test and final test forms were parallel under the same table of specifications. Results: The pretest, post-test and final test scores of the IPBL group and the OPBL group were 4.57± 2.92, 23.74± 7.58, and 12.70± 5.19; 4.94± 6.31, 31.67± 7.07, and 25.77± 5.9 respectively. The growths of knowledge after the post-test and after the final test as well as the retention of knowledge of the IPBL group and the OPBL group were 54.32%, 22.64%, and 68.32%; 76.01%, 59.63%, and 83.64% respectively. Conclusion: The OPBL yielded the better knowledge retention. The key success factors might depend on students’ achievement motive and a sense of self-actualization.


Introduction
Problem-based learning (PBL) proposed to help students to exhibit sufficient retention of. information, develop an integrated knowledge. base, exposure to clinical experience at an earlier stage, set up student-staff liaison, and express overall motivation towards lifelong learning (Pereira, Telang, & Butler, 1993;Robbs & Meredith, 1994;Tipping, Freeman, & Rachlis, 1995). It was under the belief that, given different background knowledge, students could equally learn successfully and express their understanding of core knowledge of the subjects. The teaching program breaks down the course material into manageable units, each with their own clear objectives and assessment on patient problems. Students perform self-directed learning on the selected subject in small groups. As student-centered learning, they must demonstrate mastery on criterion-referenced tests or correct deficiencies before they move on in their work. Students who do not satisfactorily complete a topic need to attend an additional instruction until they succeed. Instructional staffs work like a coach, devote to help students deal with their deficiencies and motivate them until the entire This learning method supports conceptual framework on assorted educational matters. First, it is a child-centered learning approach laid down by John Dewey in 1940; a selected subject represents the body of knowledge for students' interest, since they are about to perform their career advancement. Second, it applies empowerment evaluation defined by Fetterman (1996), which has taken into account stakeholder engagement. By means of student empowerment, PBL has a strong reputation for enhancing student motivation towards learning tasks and providing an unconstrained environment (Honebein, Duffy, & Fishman, 1991;Greening, 1998). The method promotes the learners' role as the decision-maker and planner, self-assessment designer, and the implementer of the discovered information. The instructor plays the role of a 'good friend', facilitating learning and stimulating students with enthusiasm, discipline, and self-respect.
At present, the diversity and complexity of current medical information require students to be on the alert for progress in their knowledge. In addition, the online learning method is increasing in popularity. It is challenging and stimulating to the curiosity of students, as well as promptly interacting with other databases (Towle, Godolphin, Grams & Lamarre, 2006;Rossett & McDonald, 2006;Heath, Higgs & Ambruso, 2008;Cadwell, 2008;Modica & et al., 2009). Students are highly independent in studying to the selected subjects of their current interest with a complete scope of content and quality. As a result, we would like to know which learning procedures between the in-class problem-base learning (IPBL) and the online problem-based learning (OPBL) is better in helping students to fulfill their goals in terms of learning achievement and retention of knowledge. The analysis of arterial blood gases (ABGs), a procedure to assess the pathophysiological changes in respiratory, renal, cardiovascular, or metabolic systems of critically ill patients, was chosen as prototype subject in this study.

Methods
Twenty-three and 36 nurse anesthetist students on the corresponding academic year 2008 and 2009, Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, assigned as IPBL and OPBL group respectively. They volunteered to join the three-week study project without any honorarium; their only benefit was knowledge to be applied clinically. To ensure that all gave it their best efforts, students were informed of the significance of this research project under the support of the faculty. Whether or not they participated did not affect their educational records in any way. They also indeed posed a research challenge, as their background knowledge was little regarding the clinical application. The selected subject was likely to be applied to patient care in the operating theatre and intensive care unit in the next few weeks. In class, each group first completed a paper-pencil pretest of 40 short answers written exam totaling one hour.
The scope of ABGs concepts are classified into three learning parts. This was to find out students' misconceptions in the selected subject. Each part had at least 20% of the subjects related to each other and consisted of two tests, an activity and a diagnostic test. All tests are developed by using a concept and knowledge map of the selected subject to determine the table of specifications (Treagust, 1988;Novak, 1996). An activity test had fifteen open-ended questions, while a diagnostic test consisted of ten questions with each question containing four sub-items. Each question in all test required short answers regarding case-based. Each case had a brief patient's history, a report of laboratory tests and assessment questions. Students had to pass all three parts in an orderly way. To qualify for the next level, a student might complete each test with a score of no less than 80%.
After the pre-test, the IPBL group performed activity/diagnostic tests using a paper pencil method. During a one-hour activity test, students were able to gain subject contents through any available textbooks, which provided them with current and relevant information. In addition, for further explanation and clarification, an instructor held a one-hour open discussion. Afterwards, students performed another one-hour diagnostic test to earn the achievement score. An activity and a diagnostic test in each learning specification part took place exactly a week apart. The contrary, the OPBL group performed activity/diagnostic tests by log on to the website where students could spend free time at their own pace during the next three weeks. The program identified learning weaknesses while avoiding giving those direct answers immediately after each activity or diagnostic test. In other words, to get the proper answer

South-East Asian Journal of Medical Education
Vol. 5 no. 2, 2011 43 for each item of the test, students needed to study the content appearing on the website or lessons in detail. After finishing each test, they were able to see their score in order to modify their study strategies. During the study, students could discuss the topic among friends or read supplementary articles. No efforts to evaluate whether students completed their assignments, as the computer program not only established their weaknesses and urged them to explore for core knowledge, but also recorded students' profiles in a real-time fashion (name, ID and password, frequency, date and time of access, the number of exercises, and score earned).
After three weeks, a paper-pencil post-test of 40 short answers written exam arranged in class for both groups, to those that finished their studies and were still keen on participation. After the post-test, students were encouraged to continuously revise the content materials to gain experience and confidence in the operating theatre and intensive care unit environment. Four week later, the final test of 40 short answers held in class for both groups without prior notice. The pretest, post-test and final test forms were parallel under the same table of specifications.
Both pretest and post-test score as well as pretest and final test score were computed as growth of knowledge after the post-test (G-1) and growth of knowledge after the final test (G-2) respectively (Kanjanawasee, 1989). The difference between G-1 and G-2 was the level of knowledge decrease after a period of time.
The less the level of knowledge decrease was, the more the retention of knowledge might be.

Validation and reliability of the test
The correctness and appropriateness of all test (content validity and index of item objective congruence, IOC) determined by four anesthesiologists who were not involved in the project and had at least 10 years of experience in educating medical science. For additional review, 16 second-year residents in anesthesiology performed all tests to verify the assessment of criterion-referenced test item difficulty, discrimination and internal consistency (Cronbach's alpha). Score graded by four non-physician evaluators measured the outcome.

Statistics
The pretest, post-test and final test score within group the and between the group were compared by ANOVA. The growth of knowledge after the post-test (G-1) and after the final-test (G-2) within the group and between the group were compared by t dependent and t independent test respectively using the Statistical Package for Social Sciences for Windows, release 17. The statistically significant differences were considered when there were a p value < 0.05 with a 95% confidence interval. G = growth of knowledge F = full test score X = pretest score Y = post-test score Z = final test score There was no statistically significant difference of the pretest score between the two groups. In both groups, the post-test score was significantly higher than the pretest and the final test score. The post-test and final test score as well as the G-1 of the OPBL group appeared to increase significantly as compared to the IPBL group. Both groups particularly the IPBL group showed to decrease significantly in the G-2. In addition, the retention of knowledge showed statistically significant difference between the two groups. (Table 2)

Discussion
We administered the (pre-test) to nurse anesthetist students to determine their prior knowledge of ABGs because parts of the subject were about basic science and previous background had an effect on the value of the material. This reduced the possibility that study results might be confounded by baseline differences in pre-existing knowledge among the groups. However, both learning groups close yielded the pretest score. This narrow range implied that though registered nurses who attended one-year specific training in anesthesiology, had work experience in some nursing care units for years; they did not comprehend ABGs and other laboratory information. As a result, it was appropriate to screen these students for basic information on the selected subject and to monitor their progress.
Significantly, students in the OPBL group have shown an increase in the post-test score, the final test score and the growth of knowledge after taking the post-test and better retention of knowledge as compared to the IPBL group. This has implied that the online learning yielded highly practical ways to help students achieve their study in medical practices. The key success factors might include many learning aspects.
In the OPBL group, an instructor played the role of an 'organizer', facilitating learning and stimulating referral to content databases while students played the role of an 'independent', studying the subjects of their current interest.

** p < 0.01
Students were stimulated to apply problemsolving strategies, individual help-seeking strategies, and self-regulation, as well as developing critical thinking with discretion for self-assessment and keeping on learning with a sense of self-actualization and self-esteem in education. They should be convinced that only they are accountable for what they have learned while being committed to the learning method and yearning to learn new knowledge. Thus, it has depended largely on the enthusiasm of the students, notably their learning skill. This is in agreement with a study performed by Schublova (2008) who has stated that a PBL method via computer simulation promoted active learning via critical thinking, self-directed learning and problemsolving ability.
The 'anytime anywhere' internet implied a friendly atmosphere, total communication freedom, courtesy, fairness, uniform practice, borderless education and privacy of tuition. Once students had passed certain courseware, they were able to review all learning materials as often as they wanted to. This facilitated learners' accessibility problems while motivating them to achieve their missions. In addition, each student was free to save, copy, or print any information appearing on the website, learn the subject as preferred, and do additional search. This opportunity of downloading materials freely also supported them to learn individually and comfortably with or without a computer. This has been shared by Euliano (2003) and Valdez (2007) regarding the application of technology that has appeared to be an effective stepping-stone for students to begin developing their higher-level learning and problem-solving skills.
In addition, the online program provided a unique learning format that required application of knowledge rather than memorization.
It yielded personalized feedback, which did not simply show the actual answers, but instead revealed the answers by linking it to a specific webpage or lesson page, where one could find detailed clues. This might foster the development of self-directed learning by allowing them to select a modality that best matched their preferred learning style. Through this means, students gained in content understanding, problem-solving strategies and retention of knowledge. It has seemed to yield highly practical ways to help professional students achieve their studying. This idea has agreed with a study performed by many authors (Robbs & Meredith, 1994;Branch & Paranjape, 2002;Paukert, Richards & Olney, 2002;Naidr & et al., 2004;Kripalani & et al., 2006), who have shown that the feedback mechanism was significantly better in bringing about students' content understanding and achievement.
In contrast, the IPBL group had some aspects affecting the success in this study (Albanese & Mitchell, 1993;DesMarchais & Vu, 1996). An instructor played the role of a 'manager', controlling the whole learning process and fostering a context conducive to learning in order to help students deal with their deficiencies until they reach a mastery level of learning materials. However, the students who played the role of a 'dependent,' studied the subjects specified by an instructor. They were bounded in the classroom with the rigid control over formal pedagogy. In addition, interactivity of learning materials and confidence in overcoming challenging assignments seemed to be limited by the functionality of the tools, exhausted instructor or learner's control of the trial-and-error strategy.
Though selected subject was a challenging and inspiring procedure for the active participants their ability to work effectively in teams, their reactions of some students to this assignment included dread of performance on exams, and reluctance to participate in open discussions. Apparently, slow students dared not disclose their misconceptions. Interactivity of learning materials and confidence in overcoming challenging assignments seemed to be a boring process. These findings agreed with Pereira (1993) and Zimitat (1994) who documented that PBL might involve perceptions of increased cognitive load, resulting in a hidden source of anxiety and resistance for many students. Besides, a few studies (Honebein, Duffy & Fishman, 1991;Morrison and Murray, 1994) confirmed that at least some basis for the low score in perceived relevance could be directly related to exam anxiety. As a result, a formally informative discussion for further explanation and clarification among students with different background knowledge consumed much time to comprehend the core subject. The unfavourable perception of time-consuming could possibly be the result of repetition in the educational format.
The IPBL method has yielded collective feedback, which provided accurate and specific answers with some details while neglecting some crucial points. On this matter, many studies have suggested that in the presence of cognitive complexity, adequate time was necessary for gradual development of these essential managerial skills in order to empower the student and facilitate deep approaches to learning (Honebein, Duffy & Fishman, 1991;Morrison & Murray, 1994;Heath, Higgs & Ambruso, 2008;Amesse & et al., 2008). It seemed effective only for learning matters of schoolchildren with an undersized curriculum. It was insignificant in bringing about content understanding and achievement of professional learners who craved to succeed the overwhelming current medical knowledge, particularly certain subjects, not formally included in regular class hours and commonly referred to as a hidden curriculum in an academically competitive atmosphere (Kripalani & et al., 2006). This opinion agreed with Drinan (1991) who suggested that PBL demanded a level of maturity that may not be applicable to all learners.

Conclusion
The OPBL significantly yielded the better of knowledge retention as compared to the IPBL in the selected subject. It seemed practical to help professional students gained content understanding. The key success factors might depend on individual time management, students' achievement motive and enthusiasm, a sense of self-actualization and self-esteem for lifelong learning.

Suggestion for future study
The OPBL represents a technological innovation that can change the learning process, accuracy in data processing, and streamlining of operational research steps, as seen in the presence and accessibility of a growing number of online courses and programs in higher education today. The pretest and post-test of learning materials in class were still required. If future innovations could verify the identity of exercise/test examiners and information submitters, then an educational research via online systems will be filled with maximum quality and efficiency.