Objective Structured Practical Examination (OSPE) versus Viva Voce: The Indian Students’ and Faculty Perception

Introduction: In most of the medical colleges in India, practical examination in Anatomy is conducted in the form of conventional viva-voce. Viva-voce is however limited by its lack of validity, reliability and objectivity. Objective Structured Practical Examination (OSPE) has been proposed as objective, reliable and valid method of assessment. The present study was conducted to assess the feasibility of conducting OSPE in our set-up and to determine perception of first year undergraduate medical students and anatomy faculties towards OSPE vis-a-vis viva-voce. Method: A total 200 1st MBBS students and 13 anatomy faculties participated in this study. The students were exposed to conventional viva followed by OSPE. The perception of students and faculties about conventional viva and OSPE was assessed through separate pre-tested, pre-validated structured feedback forms. Result: OSPE was conducted smoothly. Positive feedback was received from both students and faculties about quality of OSPE. Students perceived that OSPE was significantly better than viva-voce on parameters like effect of examiner’s mood on scores awarded, personal bias, level of transparency, carry over effect, equal opportunity to all students and level of stress (p<0.0001). Overall 81% students opined that if option is given they will prefer OSPE. Less stress, no fear of examiner, equal opportunity for all students, less bias and sufficient time to think were reasons cited by students for favoring OSPE. All faculty favored appropriate combination of OSPE and viva voce. Conclusion: OSPE is preferred over conventional viva by students, while the faculty preferred appropriate combination of both these methods.


Introduction
In most of the medical colleges in India, practical examination in Anatomy for undergraduate students is conducted predominantly in the form of conventional viva voce. However it is often blamed for its lack of validity, reliability and objectivity (Davis et al., 2005;Sierles et al., 2001;Varuni et al., 2012;Weingarten et al., 2000).
During informal discussions with peer examiners, faculty and students, many drawbacks of viva voce are talked about; like the mood of examiner affecting the scores awarded to the students, favorable/unfavorable bias towards particular students by the examiner and so on. Different students are likely to face different questions, with the bias that some students face easy questions while others face difficult ones. Likewise, sometimes the questions asked maybe of subjective nature making assessment erroneous. Thus injustice to the students is a possibility in conventional viva. Previous study reported that the subjectivity involved in conventional viva may reduce the correlation coefficient, between marks awarded by different examiners for the same candidates'

South-East Asian Journal of Medical Education
Vol. 13, no. 2, 2019 performance, to as low as 0.25 (Ananthakrishnan et al., 1993). From another study it was observed that the marks awarded reflect the overall performance of the candidate and are not based on demonstration of individual competencies (Malik et al., 1988). Thus, examiner variability, though not justifiable, forms a significant factor in marks scored by the students.
Attempts are being made to overcome the drawbacks of conventional viva (Stiliman et al., 1977;Edelstein and Ruder, 1990;Newbie, 1991). The earliest attempt in this regard was the introduction of Objective Structured Clinical Examination (OSCE) which was applicable to clinical subjects. Later on, in 1975, Objective Structured Practical Examination (OSPE) was designed for pre-and para clinical subjects. Since then, OSPE has been widely tried abroad for practical assessments in different pre-and para clinical subjects. Notable among these are OSPE in Physiology, Pathology, Forensic Medicine and Anatomy (Dissanayake et al., 1990;Sandila, 2001;Rahaman, 2007;Abraham, 2009;Feroze and Jacob, 2002;Menezes, 2011;Yaqinuddin et al., 2013). In India, few studies have evaluated OSPE, though it remains largely in experimental phase still (Nayer et al., 1986;Malik et al., 1988;Rao et al., 2018).
Like in most of the medical colleges in India, barring histology spots, the practical examination in Anatomy for undergraduate medical students in our medical college is conducted in the form of conventional unstructured viva voce. Though we focus on 'studentcentered teaching', very few studies are conducted on how our students perceive conventional viva as well as OSPE. With this background the present study was undertaken to determine the feasibility of conducting OSPE in the Department of Anatomy, B.J. Government Medical College, Pune. We further evaluated the perception of first year undergraduate medical students and Anatomy faculty about the conventional viva and OSPE.

Method
The study was approved by the Institutional Ethical Committee of B.J. Government Medical College, Pune. The study was conducted in the Department of Anatomy B.J. Government Medical College, Pune. A total of 200 first year undergraduate medical students and 13 Anatomy faculties participated in the study. Informed written consent was taken from all the participants.
The faculty and supporting staff in the department were sensitized and trained in OSPE. The practical examination was conducted by both methods, namely, conventional viva and OSPE, on 'Living Anatomy of Head, Neck and Face'. Initially the students were exposed to conventional viva and then followed by OSPE evaluation. OSPE was conducted on eight stations of which four were procedure stations and four response stations. Each procedure station was followed by a response station. The procedure stations included questions to test practical skills. The response stations included relevant questions to test analytical thinking, higher level of knowledge and application of knowledge in clinical practice. Each station was allotted 5 minutes. The stations were designed by the same faculty members who conducted the conventional viva, anonymously and the difficulty level of questions was maintained same for both conventional viva and OSPE. While designing the stations, care was taken that the questions were in line with the objectives of the curriculum. Each student went through all eight stations. The stations were same for all students. Enough care was taken to avoid interaction between students who faced OSPE and those who had to face it.
Separate structured questionnaires were designed to assess the perception of students and faculties about traditional viva and OSPE. These were then validated and tested. Both the questionnaires included three sets of questions. Set 1 had four questions to test whether the OSPE was conducted properly. Set 2 had eight statements to assess the perception of the students and faculty about conventional viva and OSPE. This set dealt with a wide range of issues like transparency, chances of bias, level of comfort and anxiety, equal opportunity to all examinee, etc. Each item was selfdescriptive, unambiguous and conveyed single complete thought. The items were arranged randomly. Likert's 5 point rating scale was used to assess the perception. Set 3 included two questions seeking the opinion of the participants about which assessment method they prefer and the rationale behind it. To ensure frank response by the participants, anonymity was maintained.
The responses were analyzed statistically. Fisher's exact test was applied.

Results
The OSPE was conducted smoothly in the Department of Anatomy. We got good

19
Vol. 13, no. 2, 2019 cooperation from the faculty, the technical staff and full support from higher administrative authorities. All the 200 students returned the questionnaire resulting in 100 % response rate. All the 13 faculty members answered the questionnaire (100% response rate). Table 1 shows the students' and faculties' feedback about important aspects of OSPE conducted. It was observed that both faculties and students were satisfied with the quality of OSPE conducted.  Table 2 gives the comparative data of the students' perception about conventional viva and OSPE. It was observed that in all parameters studied, namely, effect of examiner's mood on scores awarded, personal bias, level of transparency, carry over effect, equal opportunity to all students and level of stress, the students opined OSPE to be better than conventional viva, the difference being statistically significant (p<0.0001). More than three-forth (81%) students opined that if option was given they would prefer OSPE as mode of assessment. Of the remaining, 10% students favored conventional viva, while 3% opined that they will prefer a combination of OSPE and conventional viva.

Figures in the parenthesis indicate percentages
The gender-wise data of perception of students about OSPE is provided in Table 3. Barring personal bias, on rest all parameters, the views of male students were comparable to those of their female counterparts (p>0.05). The female students opined more in favor of the statement 'there are cognizable chances of personal bias in OSPE' than their male counterparts, the difference being statistically significant (p=0.029). Figure 1 depicts the comparative perception of students and faculties about OSPE. From it, it is clear that the ratings given by both the groups for various aspects of OSPE are comparable (p>0.05). It was observed that faculties were more comfortable during conducting OSPE (comfort score 78.85%) than conventional examination (comfort score 59.5%), p=0.036. When asked 'If option is given, which method of practical assessment you will prefer, all 100% of the faculty members opined that 'an appropriate combination of both these methods would be preferable'.
Figure 2 depicts the reasons cited by students for preferring OSPE over conventional viva. The common reasons cited were, there is less anxiety and more comfort while facing OSPE (29.4%), no fear of 'facing an examiner' (14%), no chance of personal bias (8.4%), equal opportunity for all students (8.4%) and OSPE provides sufficient time to think for answer (7%). Remaining one-fourth of the students (25.9%) preferred OSPE over conventional viva citing reason that it is transparent, mood of examiner do not affect the students' scores, they do not have to face stares/looks of examiner, advantageous for those who have better writing skills, complete thought is   Ten percent students opined that they would prefer conventional viva method. The reasons cited for this preference were that other questions are asked by the examiner in conventional viva (3%) if student fails to answer the particular question, examiners give hints in conventional viva that may lead to correct answer (3%), and it helps to improve confidence and communication skills (3%), and quick decision making skills are developed (1%). One student opined that it is more stressful and hence advantageous as one gets an opportunity to learn to work under stress.

South-East Asian Journal of Medical Education
Vol. 13, no. 2, 2019

Discussion
Students' feedback forms the most useful and important basis for modifying and improving medical education. This type of evaluation helps to assess the areas of strength and/or lacunae of teaching methodologies and helps to rectify the difficulties and revise the curriculum suitably (Verhoeven et al., 2000;Rafique and Rafique, 2013 mentally and physically taxing (Malik et al., 1988).
A good assessment tool should be acceptable to all those using it. Our study shows that most of the students preferred OSPE over conventional viva. This finding is concordant with, Al-Mously, Rehman and Rao (Al-Mously et al., 2012;Rehman et al., 2012;Rao et al., 2018). Our students opined OSPE as a better assessment tool than the conventional viva as OPSE was not affected by the mood of examiner on scores, personal bias and carry over effect and was more transparent. In an ideal examination, it is expected that all the candidates are assessed uniformly. Majority of the students in our study strongly agreed that OSPE assessment was uniform and provided equal opportunity to all students.
Examination is well known source of stress and anxiety. Further, increased stress decreases performance of the candidate during examinations. Hence, as far as possible, during assessment, students should be comfortable and under minimal stress. In our study, students felt that OSPE was less stressful and more comfortable than conventional viva. Similar findings were reported by other researchers. Rehman stated that OSPE is better and unbiased system of examination as it is neither stressful nor dependent on mood of the examiner and there is no fear of facing the examiner (Rehman et al., 2012). In corroboration with us there are studies that have reported that OSPE is fairer, more transparent and objective in comparison to Traditional Practical Examination and thus effective and useful assessment tool (Al-Mously, 2012;Rao et al., 2018). Our findings are in agreement with Harden and Gleeson who recommended that OSCE should be combined with traditional long case (Harden and Gleeson, 1979).
Our study had limitation. It was conducted in a single medical college in India. Our findings need to be confirmed by conducting similar studies at multiple centers across India so that appropriate policy recommendations can be made.

Conclusion
To conclude, the present study shows that OSPE is feasible to implement and is acceptable for both students and faculty. The results indicate students favor OSPE over conventional viva, while faculty opined that an appropriate combination of both these methods would be preferable.