Evaluation of Problem-Based Learning Tutors from Students ’ Perspective

Since the evolvement of the problem-based curriculum as a complementary or a substitute to the comprehensive learning methods that had been followed for decades, few factors were acknowledged to be crucial in accomplishing a successful problem-based environment; among which are the tutors or the facilitators. The role of the tutor/facilitator has been subjected to debate for a long time, and though it is difficult to define, medical educators often declined the referral to the facilitator as a “tutor” as they argue that the conducting person must be a group facilitating expert rather than an educator. To monitor and assess the newly-born and developing process, series of feedbacks evaluating the roles and performances of facilitators and that particular learning method are conducted every now and then. This study is conducted in Cyberjaya University College of Medical Sciences (CUCMS) throughout Semesters 3 and 4 of the second-year medical program. PBL tutors‟ levels of knowledge, skills, and attitude are observed and coded. The total number of assessed facilitators being observed comprises 24 lecturers. The number of students involved in this study totals 86 students. Collectively, each facilitator has been evaluated on average for 32 times. Students led by knowledgeable, skilled facilitators have expressed higher levels of satisfaction. However, we are trying to review this evaluation process from a medical educator‟s prospect in an attempt to conclude whether it is significant for the conductor role to be that of a content expert rather than as a mere group learning facilitator.


Introduction
Problem-Based Learning (PBL), is also referred to as a Partnership and Bonding in Learning between tutors and students and between individuals and society.
Though different descriptions have been mentioned in educational literature and there are also different approaches in which various institutions have adapted PBL as an educational method, most of the literature has similar descriptions and definitions for PBL.
According to Barrows and Tamblyn (1980), PBL is the learning process that results from understanding and solving of a problem.The encountered problem will provoke the reasoning skills and induce knowledge attainment, required to approach and solve the problem in hand.On one hand PBL is referred to as "an instructional method," which uses medical cases and problems to help students obtain knowledge and critical-thinking skills (Albanese & Mitchell 1993).On the other hand, Schmidt (1993) described it as a learning process in which a small group of students will tackle a problem under a supervision of a tutor, whereby the problem is presented to them as a set of clinical data, Vol. 12, no. 1, 2018 phenomena or events, which will be analyzed during tutorial sessions, which will then explain the relations and the mechanisms of the signs and symptoms given earlier.Regardless of the approach and the way used in adopting PBLs as a teaching method, a few factors were recognized to be critical in order for the process to be successful in leading towards attaining students' motivation, understanding and enthusiasm for the procedure, especially as studies have actually shown that learning in small groups do indeed reflect positively on students" performances.In addition, both factors may affect and influence each other to together accomplish better-quality PBL sessions that produce higher-level outcomes (Michaelsen et al., 1996).Some of the texts have stressed the importance upon the conductor"s role as that of being both a content and a process expert, while in other scenarios, the tutor is merely a facilitator and is expected to be neither a content nor a resource expert (Schimidt, 1983), while at the same time the facilitator is not expected to be authoritarian either.Barrows and Tamblyn (1980) thought that the tutor should have expertise, both in supervising a small group and in group facilitation rather than in a particular subject area.Ross disliked the tutorial label; he viewed PBL sessions more as professional and strategic meetings rather than as teaching sessions (Ross, 1997).Tutors should help students via listening, observing, assessing, suggesting in constructive ways through questioning, probing, encouraging critical thinking and challenging the group he/she is facilitating in a functional way, with this being considered as a skill that will face an inexpert facilitator with the "how-tos" and the "wheres" to intervene in accordance to Shields" point of view (Shields et al., 2007).Tutors might intrude a student"s activity, which could result in the tutors concerned dominating the session.Nevertheless, a passive tutor is not as helpful as the dominant one (Dobbs, 2008).
Notwithstanding of the tutor"s dexterity in handling a small-group learning session, several factors were found to affect the tutor"s behavior.For instance, a group that has the best basic knowledge on the subject matter and has shown a higher performance would drastically motivate the tutor (Schmidt & Moust, 1995).Likewise, cooperative groups have shown a much better outcome in comparison to non-cooperating ones; which either influenced or discouraged the tutors respectively (Dolmans et al., 1994).
In order to perfect the PBL method, a few evaluation procedures on the tutors' performance, PBL cases and feedback forms had been developed, and such instruments were utilized to draw attention of each party towards the drawbacks that they may be able to avoid in the future.The observations reported here were perceived by the authors over the past few years guided by the students" feedback.
In this study, we assessed the elements that affect the tutor"s personal performance upon taking into consideration the students" point of view.These elements were categorized into three classifications that we use to portray the tutor"s performance, which are "Knowledge," "Skills" and "Attitude" in relation to the years of experience and the educational background of the tutors.

Statement of Problem
Inexperienced tutors may face initial difficulties in their playing their roles.Some facilitators tend to intervene over students" lack of indepth knowledge.They may go to the extreme, especially when the topic is in line with the teachers" knowledge expertise; as a result, the tutor transpires to be dominating the PBL session and thereby indirectly affects the learning process.Others might misunderstand their role by being very passive.Therefore, they won"t be able to provide the proper guidance when it is called for.

Objectives
The prime objective of this study is to evaluate the tutors" performance in PBL sessions in relation to their educational background, knowledge, skills and attitude by year 2 medical (MBBS) students at a private university in Selangor, Malaysia.
The secondary objective is to present the students" feedback concerning the tutors" involvement in PBL sessions.
Research Questions: This study addresses the following research questions: 1. Do medical background and duration of experience positively enhance the tutors" performances in PBL sessions?
2. What are the students" perceptions towards PBL facilitators?

Target population
The study was conducted in a private university in Selangor, Malaysia.Second-year MBBS students were involved in the assessment of PBL facilitators for one whole academic year.During that year, we had collectively administered 12 PBL packages.
The number of students who participated in this study was 86; and each PBL group comprising 8-10 students assessed a single tutor"s performance during two PBL sessions.

PBL setup
Each PBL package was chosen in accordance to the teaching block running at that time.In general, we had 1-2 PBL topics per block.The PBL packages were prepared and distributed to previously briefed facilitators.The main intention is to give enough time to do topicrelated own reading in advance.Each PBL was conducted in two meetings.The first meeting will require the small group of students to establish a set of learning outcomes or objectives that they are expected to prepare for before the second meeting.
Some of these learning outcomes are essential and are prerequisites for that package, and the facilitators are requested to probe the students to establish these learning outcomes.Additional learning outcomes can be included as well whenever the facilitator or the students believed it to be necessary and beneficial.The students are expected to prepare a report that explains the studied case and send it to the facilitator few days prior to the second PBL session.The facilitator may ask the students to moderate the report whenever it was found necessary, and in addition each student will prepare a case summary that will be handed to the facilitator at the beginning of the second session.During the second meeting, the facilitator will go through the case with the students as they explain the causes, pathophysiology, diagnostic methods and proper management of the medical condition in hand.Both the tutor and students are assessed by the end of the second session.

A
self-administered questionnaire is distributed to all students as a soft copy through a batch common Gmail (Table 1).

Yes
No Comment Knowledge  Behave as a source of information during discussions  Share with the students relevant stories from real life experiences  Guide students to discuss uncertain information given by the students Skills  Address the students by name  Listen carefully during students" discussions  Say "well done", "good question" whenever a student gives an important comment  Give equal opportunity for all students to express their viewpoints  Keep on asking questions to direct students" attention towards important facts  Encourage the silent/shy student to participate  Give the group some suggestions to help them to absorb the desired knowledge  Control dominant students or those who interrupt the discussion  Keep the discussion within the limits of predetermined objectives

Figure 1: Frequency of facilitators' evaluation
The questionnaire was set in a form of a checklist that assesses tutors" knowledge, skills, and attitude.The questionnaire is used as an anonymous feedback instrument, and the tutors were given free access to these feedbacks, and they could also be used during annually held PBL workshops as well.We concentrated more on some components under each session that we assumed to be more valid for the assessment.Tutors who have useful information share relevant stories with the students, and direct students to discuss uncertain data, which are considered to be of knowledgeable value.
The main criteria for skillful facilitators are; keep on asking questions to direct students" attention towards important facts, give the group some suggestions to help them to absorb the desired knowledge, and keep the discussions within the confines of the predetermined objectives.Though many factors can be included within the attitude session, we deemed punctuality, sociability, and communication skills to be of great interest to us.

Data collection
Following the second PBL session, the hard copy of the students" feedback was collected, the data were inserted into an excel sheet to be analyzed later at the end of the following semester and the academic year.Repeated comments (whether positive or negative) about the same facilitator were very useful for our study.

Data Analysis
A Chi-square test was used to assess the association between the knowledge, skills, attitude and socio-demographic variables.Stataversion13 was used in the data analysis.

Results
The total number of assessed tutors was 24.Collectively, each facilitator was evaluated on an average 32 times altogether.Fulfillment of each component in the checklist granted the assessed facilitator a score of 1 for this element while zero scores were be given for not complying with that element (Figure 1).The collected results were analyzed in relation to the educational background of the tutors and years of experience in that field.Although it was hard to eliminate some factors related to personal preference when it comes to assessing the assigned facilitator, we tried to eliminate such a possibility by excluding evaluations done by a group for their personal mentor who was assigned to be a facilitator for a certain PBL session.
Seven hundred and forty seven questionnaires were collected.Following tabulation, only facilitators, courses and topics that were evaluated more than five times have been included in the results.Twenty-four facilitators were assessed.80% of them hold a basic medical degree, and the rests are of nonmedical background.Quite a large number (46 %) has less than a five-year teaching experience (Figure 2).Data analysis showed that medical facilitators were acting more as a source of information during the PBL sessions in comparison to nonmedical ones.It was also found that the (5-10 and >10 years of experience) groups respectively were more informative in comparison to the less-experienced facilitators (Table 2).Medical facilitators shared more real-life stories related to the subjects that were discussed during the small-group sessions compared to the non-medical.
Students had again attributed this category to the more experienced groups (Table 3).On the other hand, analysis showed that nonmedical facilitators displayed better skills in handling the PBL sessions.According to the students; the 5-10 year experience group was better in all the above skills except when it comes to giving equal chances and encouraging shy participants in which case the more experienced facilitators scored the highest (Table 4).

Discussion
The aim of the study was to evaluate the facilitators" performance and their didactic skills during the PBL session; though as academicians, some facilitators were more than efficient in running a small-group session.Small-group learners may have a different point of view, especially when they are at the receiving end.
Ensuring the quality of PBL tutoring has been an on-going issue.Researchers reported that the greatest problem acknowledged by all schools, is the variation in quality of teachers (Hay & Katsikitis, 2008).It is the link represented by the teachers" ability as a tutor where the strength or weakness of the PBL chain lies.We also noticed in our study that there are some consistent excellent comments and positive feedback for some PBL tutors.
We used the educational background and the years spent in the academic line as a base for our assessment, and we had chosen these two categories for the reason that they were raised by a number of students in few occasions during the private interviews.Quoting one of the students" comments "I just like to express my disappointment that we were not given a doctor for this session, should the college lack doctors, it would be better for them to invite and pay external doctors to facilitate our session rather than a statistician".
Another student raised an important issue related to facilitators" professionalism during the session "PBL should be handled by experienced lecturers, our first PBL was a real waste of time, I would say because our lecturer was too sarcastic and he wasn't concentrating, and he was busy using his hand phone".
The study was based on a self-administrated feedback form that has been produced locally for the PBL sessions; some of the questions are similar to other survey forms used by other institutions.
In addition of having full access to feedbacks submitted by the small groups; there were a number of workshops throughout the academic year that aimed to improve facilitators" performances, during these workshops where students" feedbacks were shared with all the tutors.Those workshops were aimed at improving the pedagogical and managing skills by discussing the strong points identified by the students for outstanding facilitators and in a few occasions by discussing a few of contrary aspects that require immediate actions.
Facilitators" assessments were done before and after the workshops, but the changes (according to the students" feedback) were not significant.Regardless of their motivations, however, it seems that teachers will practice PBL according to their own "inner lights,", subtly and sometimes unsubtly altering it, and adding to the confusion about what PBL is all about (Taylor & Miflin, 2008).One of our tutor's roles was as a discussion tutor rather than as a facilitator resembling Shield's description (Shields et al., 2007), and others did not want to direct or intervene the PBL sessions and believed that students should be self-directed.
There is an unfortunate tendency in current available literature to perceive the views of faculty, teachers and students as faulty, and to "blame" them for less-than-satisfactory PBL curricular change.For example, in medical education, (Dolmans et al., 2005) it was argued that the problems encountered in educational practice usually stem from poor implementation of PBL, which was obviously observed in a few cases in our study.In, other occasions, some of the old faculty showed irresponsiveness to PBL.Fisher (1991) claims that the reason for resistance to PBL in medical education is that Some (faculty) cannot survive the loss of influence and prestige that can occur while others have difficulties with having to lose the traditional conventional disciplinary badge as a source of professional identity.
PBL facilitators are required to have group facilitating skills rather than just be role models and the process of PBL may interrupt the conveying of a tutor"s enthusiasm about the topic in the group (Paice, 2002).In some of the cases where the tutor is a subject expert, he/she unconsciously will tend to dominate the discussion rather than probe the group to seek for information (Azer, 2005).Young lecturers were more adherent to the facilitator role rather than to the lecturer role played by most of the senior facilitators.Subject specialists are more likely to be inefficient facilitators as they may interrupt the learning process and revert to lecturing (Wood, 2008), however, that"s not always the case, as some topic experts can be great facilitators, not to mention too that students always value expertise related to the PBL topic.
Some of the medical educators will insist on having a subject expert or a medical personnel as a PBL tutor, though it is possible for certain curriculums that have adopted problems and case-based learning as an essential medium for the course, while some other settings have found this to be a hard requirement to provide, hence, they hold on to the assumption that enthusiasm and expertise in small group dynamics is more vital than deep subject knowledge and as a result it is acceptable to have a non-medical personnel as a PBL facilitator and this has been shown to be the case by a few non-medical facilitators in our study (Sobral, 1994).Some studies have shown that a senior medical student can facilitate a PBL session just as good as some of the experienced family members.
For Asian students who have been taught using traditional approaches; many of them have assumed a passive learning process rather than project active and dynamic progression; and we find some of the students to be shut to interactive methods like-problem based learning.The authors of this paper as PBL tutors agree that a large portion of the students were passive learners, where they looked up to the facilitator for information, and to lead them through the session, and that explains the reason behind favoring the facilitators who deliver "mini lectures" to the group, share stories and the more knowledgeable (in the students" opinion) medical facilitators.There have been so many studies over the past years that tried to reflect on the reasons behind passive and quiet behavior in the academic setting and most of them have attributed this to a few factors, including attitude, language proficiency, perception, learning styles and cultural characteristics (Xia, 2009), in addition the fear of making mistakes (Jackson, 2002).Nevertheless, some may reflect and link the passivity and uncommunicativeness to unsuitable teachers" methodology and a lack of understanding between the facilitator and the group members (Cheng, 2000) or even the less embraced active learning styles among East-Asian students.This may explain, to a certain extent, the reason behind the lack of participation by a large percentage of the students that led to domination of one or two active individuals in the PBL session, but this doesn"t exclude the lack of controlling skills, unsuitable pedagogical style of the facilitator or a lack of connection to some of the students Vol. 12, no. 1, 2018 who made them unenthusiastic to contribute at all.On the other hand, Liu explicated the negative effect of highly active peers on Asian students, which led them to develop "anxiety from high-performance expectations" (Liu & Littlewood, 1997) which others thought that having such peers should be stimulating and encouraging for the rest of the group.Another problem is with the mentality of the Asian students, for those who have been used to traditional methods of teaching and self-study, it has been found that they will have problems sharing what they have learned during the collaborative learning periods due to their prioritization for high self-achievement and the existence of a competition-oriented culture (Holmes, 2004).

Limitations of the Study
Self-administrated questionnaires were designed as feedback forms, therefore the answers for each individual question was with a yes or a no (1 or 0), the study would have had better results if the questions followed a rating scale and that what we have noticed in some of the forms filled by the students as they weren"t able to conclude their answers.Some of the students were more expressive during the direct interview feedbacks; the questionnaire didn"t include all the responses that they mentioned during the oral interview.Consequently, we are considering oral interviews for similar studies in the future, though it will be time and manpower consuming, results are most likely to be more conducive.

Conclusion
Continuous training for tutors is a must for all young institutions that are adopting a problembased curriculum, including tutors who consider themselves experts within the field, not to mention using MOC PBL and peerfeedback sessions or may be even videotaped PBLs to bring to the attention of the tutors some of the aspects that they may have missed somewhere along the way.It is also important to remind the tutors of the purpose behind the PBL method and not to try to dominate the sessions and to guide the group into obtaining the desired outcome as they actively seek for the desired information.Tutors" participation in group-based learning should enthusiastic about the process.Accordingly, it is not essential for the tutors to be subject experts, field experts or of medical background, as long as they have basic evidence upfront of what is the outcome desired of the session.As it was shown in our study, the different between medical and nonmedical tutors" performance wasn"t significant.Although students preferred medical tutors, non-medical ones followed the essence of PBL more strictly than medical ones most of the time Dobbs, V. ( 2008


Help the students to finish on time  Give a mini lecture related to the topic Attitude  Punctual  Smile and show cheerfulness at the beginning of BPL session  Keep eye contact and use nonverbal communication during the discussion  Give the group feed back at the end of each session

0 Figure 2 :
Figure 2: Number of Facilitators with Qualification versus Number of Years of Experience