A Comparative Study of Online and In-class Problem-Based Learning for Nurse Anesthetists

Objective: To compare the online (OPBL) and in-class problem-based learning (IPBL) in terms of educational achievement. Methods: A pre-test and post-test study design of a three-week research project was performed in volunteering students of two academic years. After completing the pre-test, the IPBL group underwent a formative assessment. The instructor then held an open discussion for further explanation and clarification. Afterwards, students faced another formative test to earn their achievement scores. The three-hour activities in each learning specification part took place exactly a week apart. On the other hand, the OPBL group performed all tests by logging on to a website. The program not only established their weaknesses and urged them to explore for core knowledge, but also recorded students’ profiles. After three weeks, a post-test was arranged for both groups. Results: The pre/post-test score and growth of knowledge of the IPBL and OPBL group were 4.57 2.92, 23.74 7.58, 54.32% and 4.94 6.31, 31.67 7.07, 76.01%. There was no statistically significant difference between pre-test scores of the two groups. In both groups, post-test scores were higher than the pre-test ones. Conclusions: The on-line learning yielded flexible arrangement for self-directed learning (SDL) with better learning achievement.


Introduction
Problem-based learning (PBL) is proposed to help students exhibit sufficient retention of information, develop an integrated knowledge base, expose them to clinical experience at an earlier stage, set up student-staff liaison, and express overall motivation towards lifelong learning (Tipping, 1995;Robbs, 1994;Ritchie, 1994;Pereira, 1993). It is believed that given different background knowledge, students could learn equally successfully and express their understanding of core knowledge of the subjects. The PBL teaching program breaks down course materials into manageable units, each with their own clear objectives and assessment on patient problems. Students perform selfdirected learning on the selected subject in small groups. As student-centred learning, they must demonstrate mastery on criterionreferenced tests or correct deficiencies before moving on in their work. Students who do not satisfactorily complete a topic need to attend an additional instruction until they succeed. Instructional staff work like coaches, devoted to help students deal with their deficiencies and motivate them until the entire class can progress together (Schmidt, 1983;Morrison & Murray, 1994;Savery & Duffy, 1995;Barrows, 1996;Peterson, 1997;Greening, 1998). 19 Vol. 11, no. 2, 2017 This learning method supports a conceptual framework on varied educational matters. First, it is a student-centered learning approach as laid down by John Dewey in 1940; a selected subject represents the body of knowledge for students' interest since they are about to perform their career advancement.
Second, it applies empowerment evaluation defined by David M. Fetterman in 1996, which take into account stakeholder engagement.
Through student empowerment, PBL has a strong reputation for enhancing student motivation towards learning tasks and providing an unconstrained environment (Honebein et al., 1991;Greening, 1998). The method promotes the learner's role as the decision-maker and planner, self-assessment designer, and the implementer of the discovered information. The researchers play the role of a 'good friend', facilitating learning and stimulating referral to databases, while fostering a context conducive to learning.
At present, the diversity and complexity of current medical information require students to be on the alert for progress in their knowledge. In addition, the online learning method is increasing in popularity. It is challenging and stimulating to the curiosity of students, as well as promptly interacting with other databases (Levine, 2002;Towle et al., 2006;Rossett & McDonald 2006;Heath et al., 2008;Amesse et al., 2008;Cadwell, 2008;Bridge et al., 2009;Modica et al., 2009). Students are highly independent in studying the selected subjects of their current interest with a complete scope of content and quality. Therefore, we would like to know which learning procedures between the in-class problem-based learning (IPBL) and the online problem-based learning (OPBL) is better help to fulfil students' goals. Analysis of arterial blood gases (ABGs), a procedure to assess pathophysiological changes in respiratory, renal, cardiovascular, or metabolic systems of critically ill patients, was chosen as the prototype subject under this study.

Methods
Twenty-three and thirtysix (n=59)nurse anesthetist students in two academic years, Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, were included in theIPBL and OPBL groups respectively. They all volunteered to join the three-week study project without any honorarium; their only benefit was knowledge to be applied clinically. To ensure that all gave their best efforts, students were informed of the significance of this research project under the support of the faculty. Their participation did not affect their final summative examination scoresin anyway. They indeed posed a research challenge, as their background knowledge was minimal for clinical application. The selected subject was likely to be applied to patient care in the operating theatre and intensive care unit in the next few weeks. In class, each group first completed a written pre-test of 40 short answer questions within one hour.
The concepts and scope of ABGs were classified into three learning parts. This was designed to identify students' misconceptions in the selected subject. Each part had at least 20% of the subjects related to each other and consisted of two formative tests. All tests were developed by using a concept and knowledge map of the selected subject to determine the table of specifications (Treagust, 1988;Novak, 1996).
The first formative test had 15 open-ended questions, while the second one consisted of 10 questions, with each containing four subitems. Each question in all tests required short case-based answers, with each case including brief patient's history, a report of laboratory tests and assessment questions. Students had to pass all three parts in an orderly way. To qualify for the next level, a student must complete each test with a score of no less than 80%.
After the pretest, the IPBL group performed two formative tests using the paper-pencil method. During a one-hour formative test, students were able to refer subject contents through any available textbooks, which provided them with current and relevant information. In addition, for further explanation and clarification, the instructor held a one-hour open discussion. Afterwards, students performed another one-hour formative test to earn their achievement scores. Both formative tests in each learning specification part took place exactly one week apart.
In comparison, the OPBL group performed two formative tests by logging on to a website where students could spend free time at their own pace over the next three weeks. The program identified learning weaknesses while avoiding giving direct answers immediately after each test. In other words, to get the proper answer for each item of the test,

South-East Asian Journal of Medical Education
Vol. 11, no. 2, 2017 students needed to study the content appearing on the website or lessons. After finishing the test, they were able to see their scores to modify their study strategies. During the study, students could discuss the topic among friends or read supplementary articles. This aimed to foster the development of selfdirected learning by allowing them to select a modality that best matched their preferred learning style. No further steps were taken to evaluate whether students completed their assignments, as the computer program not only established their weaknesses and urged them to explore for core knowledge, but also recorded students' profiles in a real-time manner(name, ID and password, frequency, date and time of access, number of exercises, and score earned).
After three weeks, a paper-pencil post-test of 40 short answers written examination was arranged in class for both groups, and for nonparticipants who had finished their studies and were still keen on assessing themselves. Both pre-test and post-test forms were developed in a parallel manner under the same table of specifications.

Validation and reliability of the test
The correctness and suitability of all tests (content validity and index of item objective congruence, IOC) were determined by four anesthesiologists not involved in the project and with at least 10 years of experience in medical science education. For additional review, 16 second-year residents in anesthesiology performed all tests to verify the assessment of criterion-referenced test item difficulty, discrimination and internal consistency (Cronbach's alpha). Scores graded by four non-physician evaluators measured the outcome. In addition, the pretest and post-test scores were used to compute the growth of knowledge (G) in accordance with the following formula (Kanjanawasee, 1989). G = growth of knowledge F = full test score X = pre-test score Y = post-test score

Statistics
Scores within group (the pre-test and post-test scores) and scores between groups (the pretest and post-test scores, as well as growth of knowledge), were compared by t-dependent and t-independent test using the Statistical Package for Social Sciences for Windows, release 17. The statistically significant differences were noted when p value < 0.05 with a 95% confidence interval.
For pre/post-tests, the IOC was 0.88. The assessment of criterion-referenced test item difficulty, discrimination and internal reliability was 0.59, 0.38 and 0.91.The pre-test and post-test scores, as well as growth of knowledge of the IPBL and OPBL group, were 4.57  2.92, 23.74  7.58, 54.32% and 4.94  6.31, 31.67  7.07, 76.01% (Table 1).There was no statistically significant difference between pre-test scores of the two groups. In both groups, post-test scores were much higher than the pre-test ones. In the OPBL group, post-test scores and growth of knowledge showed sharp rises (Table 2).

Discussion
We administered a pre-test for nurse anesthetist students to determine their previous knowledge of ABGs ascomponents of the subject were about basic science and previous knowledge would affect the value of the material. This reduced the possibility that study results might be confounded by baseline differences in pre-existing knowledge among the groups. However, both learning groups yielded close pre-test scores. This narrow range implied that although registered nurses, who attended one-year specific training in anesthesiology had some experience in some nursing care units for years, they did not comprehend ABGs and other laboratory information. As a result, it was appropriate to screen these students for basic information on the selected subject and to monitor their progress.
Although both learning methods yielded higher post-test scores, the OPBL group showed better learning outcomes.This implied that online learning yielded highly practical help to students' achievement of their study in medical practices.
Key success factors may include several variables observed during the study. First, we learned from students' recorded profiles that most participants preferred to study in the evening hours through midnight, spending less than five hours a night, possibly because they felt free from their daily rounds and less worried with the privacy of self-directed learning tuition and online communication (Metcalf, 2010). Second, thanks to the increased growth of knowledge, students seemed convinced that only they were accountable for what they have learned while being committed to the learning method and motivated to learn new knowledge. Therefore, it depended largely on students' enthusiasm and notably their learning skills. This agrees with studies performed by Levine in 2002 andSchublova in 2009, who said that PBL via computer simulation promoted active learning via critical thinking, and that students with high scores tended to show their self-directed learning and problem-solving ability. Third, the program provided a unique learning format that required application of knowledge rather than memorization. The students were stimulated to apply problem-solving strategies, individual help-seeking strategies, and selfregulation with discretion for self-assessment and keeping on learning with a sense of selfactualization and self-esteem in education. This finding echoed the opinion suggested by Euliano in 2003 andValdez in 2007, regarding the application of technology that appeared to be an effective stepping stone for students to begin developing their higher-level learning and problem-solving skills. Finally, online learning yielded personal feedback, which did

South-East Asian Journal of Medical Education
Vol. 11, no. 2, 2017 not simply show the actual answer once a test was finished, but instead revealed the answer by linking it to a specific webpage or lesson page, where one could find detailed clues. Through this means, students gained in content understanding, problem-solving strategies, retention of knowledge, and learning success.
This idea agreed with studies performed by many authors (Robbs & Meredith, 1994;Branch & Paranjape, 2002;Paukert et al., 2002;Naidr et al., 2004;Kripalani, 2006), which showed that the feedback mechanism was significantly better in bringing about students' content understanding and achievement.
In contrast, the IPBL group had aspects affecting success in this study (Albanese & Mitchell, 1993;DesMarchais & Vu, 1996). First, a formally informative discussion among students with different background knowledge consumed much time in order for them to comprehend a core subject. Slower students dared not disclose their misconceptions during interactive learning among friends and very few problems were solved. Additionally, collective feedback provided accurate and specific answers with some details while neglecting some crucial points. On this matter, many studies suggest that in the presence of cognitive complexity, adequate time was necessary for gradual development of these essential managerial skills in order to empower the student and facilitate deep approaches to learning (Honebein et al., 1991;Morrison & Murray, 1994;Health et al., 2008;Amesse et al., 2008).
Second, although a selected subject was a challenging and inspiring procedure for active participants who can work effectively in teams, the reactions of some students to this assignment included performance anxiety for an examination, and reluctance to participate in open discussions. Interactivity of learning materials and confidence in overcoming challenging assignments seemed a boring process. A policy for instilling in them enthusiasm, discipline, self-respect, and earnest dedication was limited by the functionality of the tools, exhausting both the instructor and the learners or students' control of the trial-and-error strategy. These findings agreed with Pereira in 1993 andZimitat in 1994, who documented that PBL might involve perceptions of increased cognitive load, resulting in a hidden source of anxiety and resistance of many students. Besides, studies (Honebein et al., 1991;Morrison & Murray, 1994) confirmed that at least some lower scores in perceived relevance could be directly related to examination anxiety. Finally, IPBL proved insignificant in bringing about content understanding and achievement of professional learners who craved to overcome the current overwhelming medical knowledge.

Conclusion
OPBL significantly improved the learning achievement and growth of knowledge when compared with IPBL in the selected subject. It seemed practical to help professional students gain content understanding, problem-solving strategies and growth of knowledge.

Suggestions for future studies
Although OPBL represents a technological innovation that could change the learning process, accuracy in data processing, and streamlining of operational research steps, as seen in the presence and accessibility of a growing number of online courses and programs in higher education today, the pretesting and post-testing of learning materials in class were still required. If future innovations could verify the identity of exercise/test examiners and information submitters, then educational research via online systems will feature maximum quality and efficiency.

Limitations
This study was performed in a small segment of learning where the sample of volunteers was not randomized/balanced in term of numbers.