Utility of OSLER for Assessing Enhancement of Learning in Postgraduate Students

Background: Assessing the clinical acumen of the medical students both at undergraduate and postgraduate levels is a challenge faced by the medical teachers worldwide. For years knowledge and skills were being assessed using traditional long cases and short cases. Structuring the long case, like Objective Structured Long Examination Record (OSLER) can help improve the reliability of the long case. Method: A six month interventional study was conducted in Department of Obstetrics & Gynaecology, JNMC, Wardha. It included, 12 first year and 12 second year postgraduate students, who were subjected to two OSLER interventions at interval of one month giving a total sample size of 48 OSLER assessments. Result: It was observed that the scores obtained by the students, had improved significantly in the 2nd OSLER intervention as compared to the 1st OSLER intervention. For the first year postgraduate students the mean scores were 48.33 in the 1st encounter as compared to 60.83 in the 2nd encounter. For the second year postgraduate students, the means scores were 53.75 in the 1st encounter and 65.41 in the 2nd encounter. The ‘P’ value was found to be 0.0001 which was statically significant. Conclusion: OSLER can be used effectively for testing and for assessing the enhancement of learning and clinical acumen of postgraduate students. Interventions done regularly can go a long way in improving the clinical skills of the students.


Introduction
Assessing the clinical acumen of the medical students both at undergraduate and postgraduate levels is a challenge faced by the medical teachers worldwide. For years knowledge and skills were being assessed using long cases and short cases since the time, long case examination was introduced as a method of assessing the clinical acumen by Cambridge University in nineteenth century (Dare et al., 2008).
Clinical reasoning has been defined as the ability to sort through a maze of clinical features presented by the patient, to accurately use diagnostic methods, and to develop an appropriate treatment strategy at the end.
Direct observation is a unique and useful tool in the assessment of medical students and residents. Assessing learners in natural settings offers the opportunity to see beyond what they know and into what they actually do, which is fundamentally essential to training qualified physicians.
Traditional long cases are highly valued by many clinicians as they closely resemble the performance in a real life situation with real patients. The long case consists of testing the nuances of medical practice, in a given format of history-taking, examination, decision regarding laboratory tests, formulation of

South-East Asian Journal of Medical Education
Vol. 13, no. 2, 2019 differential diagnosis and planning the management. The case presentation and the following discussion in which the students are required to defend their findings and conclusions, helps the examiners to get a true sense of the candidates' clinical skills and competence. However different cases are allotted to different students as per availability and performance in one case cannot be correlated with performance in another. Also there is a lot of difference in the scores given by different examiners.
The weaknesses of the traditional long cases are the lack of objectivity and low validity and reliability (Norcini, 2002). In order to make the long examination more objective, valid and reliable, many modifications in the original format were suggested (Newble, 1991;Abouna, 1999;Gleeson, 1992).
Structuring of long case, like in Objective Structured Long Examination Record (OSLER) can help improve the reliability of the long case. Structuring may make it more time efficient and also it can provide an opportunity for imparting feedback. Gleeson introduced the OSLER 'objective structured long case examination record' as a more valid, reliable and objective tool to assess clinical competence. The objective structured long examination record (OSLER) is a 10-item analytical record of the traditional long case, with an examiner observed history-taking and physical examination process. It also incorporates a criterion-referenced marking scheme in order to improve the reliability of the long case (Gleeson, 1992;Gleeson, 1997). The ten items include (1) pace and clarity of presentation, (2) communication process, (3) systematic approach, (4) establishment of case facts, (5) systematic examination, (6) examination technique, (7) establishment of correct physical findings, (8) formulation of appropriate investigations, (9) formulation of appropriate treatment and (10) clinical acumen or the ability to identify and solve a problem.
Literature shows that comparisons of OSLER with traditional long case examination found OSLER to be student and examiner-friendly as also it improved the format of long examination. The examiner had to be consistent with all candidates and all candidates were assessed using the same time frame (Sood, 2001).

Rationale of the study
Drawbacks have been identified by many researchers in the traditional long case. Hence in this study OSLER 'objective structured long case examination record' was used as an tool for formative assessment. The aim was to find out whether OSLER helped to improve the clinical acumen of the postgraduate students in the Department of Obstetrics & Gynaecology.

Aim
Aim of the study was to analyse the utility of OSLER for assessing the enhancement of learning and clinical acumen of postgraduate students in the Department of Obstetrics & Gynaecology. Initially the 12 first year post graduate students were subjected to OSLER assessment.

Objectives
(Consent was taken from the postgraduate students. They not only agreed but participated enthusiastically) Then another set of 12 second year post graduate students were subjected to OSLER assessment.
The OSLER were taken by different faculty members of the rank of Professor.
The students were randomly allotted to the faculty members in order to reduce bias.
The standard format of OSLER was used [Annexure 1].
Thereafter at the interval of one month both the groups of first year and second year postgraduate students were again subjected to OSLER assessment.
This gave a total sample size of 48 OSLER assessments The scores obtained by first and second year postgraduate students in the two OSLER assessments was compared to see if there was any improvement.

Results
The first and second year postgraduate students in the Department of Obstetrics & Gynaecology were subjected to two OSLER interventions at the interval of one month each. It was observed that the scores obtained by the students, had improved significantly in the 2nd OSLER intervention as compared to the 1st OSLER intervention. This was evident from the line graphs ( Figure 1 & Figure 2).

Figure 1: Scores of 1 st year PG students
Comparison of scores obtained by postgraduate students JR I and JR II was done. For the first year postgraduate students the mean scores were 48.33 in the 1st encounter as compared to 60.83 in the 2nd encounter. The 'P' value was found to be 0.0001 which was statically significant. For the second year postgraduate students, the means scores were 53.75 in the 1st encounter and 65.41 in the 2nd encounter. The 'P' value was 0.0001 which was statically significant.

South-East Asian Journal of Medical Education
Vol. 13, no. 2, 2019

Figure 2: Scores of 2 nd year PG students
Comparison of scores obtained by postgraduate students JR I and JR II was done. For the first year postgraduate students the mean scores were 48.33 in the 1st encounter as compared to 60.83 in the 2nd encounter. The 'P' value was found to be 0.0001 which was statically significant. For the second year postgraduate students, the means scores were 53.75 in the 1st encounter and 65.41 in the 2nd encounter. The 'P' value was 0.0001 which was statically significant.   It was observed that the Cronbach's alpha was more than 0.75 and hence the OSLER intervention was found to be reliable.

Discussion
'Assessment drives learning' is the reality of education. The assessment strategies influence the way students learn. The traditional long case is the most commonly used method for testing the clinical acumen in many countries including India. Newer approaches to learning and assessment have been introduced in modern education, for improvement of teaching, learning and assessment.
The objective-structured long examination record (OSLER) was developed as a structured presentation of the long case by Gleeson in 1997. It included direct observation of the student-patient interaction (Gleeson, 1997). Also it was noted that OSLER had a profound effect on student learning in all areas of clinical competence (Gleeson, 1994).
In our study, it was observed that the scores of the students improved significantly in the second encounter of OSLER as compared to the first. For the first year postgraduate students the mean scores obtained were 48.33 in the 1st encounter as compared to 60.83 in the 2nd encounter. The 'P' value was found to be 0.0001 which was statically significant. For the second year postgraduate students, the means scores obtained were 53.75 in the 1st encounter and 65.41 in the 2nd encounter. The 'P' value was 0.0001 which was statically significant.
The drawback of the traditional long case was the lack of objectivity and low validity and reliability (Norcini, 2002). Attempts were made to improve the validity and reliability of the traditional long case. Wass and Jolly used two pairs of examiners and introduced examiner observation of the history taking in the traditional long case in the final MBBS examination at a London medical school. The authors found that observation of the long case history taking, constituted a distinct component of clinical competence, which the traditional long case fails to measure (Wass & Jolly, 2001).
In our study an attempt was made to find out the Cronbach's alpha. This was found to be more than 0.75 and hence the OSLER assessments were found to be reliable.
Van der Vleuten found that reliability can be improved by increasing the number of cases, rather than the focus on observation of the student during the long case (Vleuten, 1996;. Norcini (Norcini, 2002) proposed that the reliability of the long case can be improved by modification of three factors, as follows: 1. By increasing the number of cases (instead of one long case) 2. By increasing the number of examiners to minimise the difference in marking by two examiners.
3. By improving the standardisation across examiners. This was done by training the examiners, increasing the number of aspects of competence assessed, providing the examiners with lists of competencies and using examiner observed student-patient interactions.
Troncon proposed a standardized and structured modification of the traditional longcase examination (Troncon, et al., 2000). The students were presented with a sequence of two long cases and each case was observed by a different examiner. The student was assessed by the examiner by using four standardized questions. A 10-item checklist covering five groups of major clinical skills was made. The time frames were standardised for the two cases which was 25 minutes and time for student interaction was 10 minutes. At the end immediate feedback was given to the student.
Agreement between different examiners was as high as 89% during assessment of clinical skills of the same student. Hence the study concluded that  (Traynor et al., 2016).

Conclusion
Choosing the correct assessment method is necessary, whether the evaluation is summative or formative in nature. Many times new modalities are not used because of lack of exposure to them or because they are not in common use. Faculty development and sensitisation of the faculty and the students becomes necessary for using the innovative modalities. Now in the era of CBME -Competency Based Medical Education, adapting to newer teaching, learning and assessment modalities has become the need of the hour. As was evident from our study, OSLER 'objective-structured long examination record' can be effectively used for assessing the enhancement of learning and clinical acumen of postgraduate students. Interventions done regularly can go a long way in the improvement of clinical skills of the postgraduate students.

Limitations
1. Short duration of the study 2. Small sample size