Faculty and Resident Perceptions on Mini-Clinical Examination Exercise (Mini-CEX) as an Assessment Tool in Medical and Surgical Super-Specialty Departments of a Teaching Hospital

Introduction: Many teaching hospitals in India find it difficult to conduct formative assessment of residents due to limitation of resources like manpower and time. The aim of this study is to evaluate faculty and resident perceptions on the feasibility of using Mini-CEX as a formative assessment tool in medical and surgical speciality departments. Settings and Design: An observational study to evaluate the performance and elicit perceptions on the Mini-CEX tool by 30 faculty and 30 residents was conducted in a large public sector tertiary care medical college, which is also a regional centre for medical education technologies (RCMET) under Medical council of India. Methods: Mini-CEX conducted by standard methodology of Norcini et al. using downloadable ABIM forms. Faculty and resident feedback obtained, opinion regarding assessment recorded on a 5 point Likert score along with few open ended questions. Data obtained from 30 encounters in various clinical settings was assessed on ABIM forms, assessor (faculty) feedback form and resident feedback form. Data analysis was done using SPSS V18 statistical software package. Continuous variables were expressed as mean with standard deviation or median with interquartile range (IQR). Categorical were summarized as frequency with proportions. Paired t test used for comparison of scores. Results: The overall satisfaction score on assessment process was 69.6% for faculty and 68.3% for residents. Satisfaction was higher for medical super specialty departments [6.50(SD+/-1.2)]. Clinical judgement (p value 0.012), overall clinical competence (p value 0.037) and counselling skills (p value 0.044) were best assessed. Familiarity with Mini-CEX improved assessment. 60% residents felt that the 1:1 faculty interaction and immediate feedback motivates further learning [4.23(SD+/-0.679)] . Conclusions: Our study demonstrates that Mini-CEX is a feasible assessment option in resource constrained super-specialty departments.


Introduction
India has adopted a revised curriculum for medical colleges since 2019.
It defines the requisite skills to be possessed by every Indian medical graduate and mandates their certification at each stage of learning. With the curricular reforms, there is increasing emphasis being placed for assessing competency of medical students by observing them in actual workplace settings and providing timely feedback to facilitate their learning (Singh and Sharma, 2010). It is an aberration therefore that such a structured approach to learning is not used in post graduate teaching DOI: http://doi.org/10.4038/seajme.v14i2.219 and assessment in India (Sood and Singh, 2012;Jain et al., 2016). Summative assessment of residents is still by long and short case presentations which consider only the final outcome without observing how the trainees reach their clinical diagnosis (Behere, 2014). According to Kassebaum and Eaglen (1999), less than one fourth medical students get observed or given feedback by the faculty during a clinical encounter with a structured format. Fraser et al. (1987) have shown that timely feedback has the single most important influence on learning. In many public sector teaching institutions of India, the medical 'super-specialty' departments (Cardiology, Gastroenterology, Nephrology and Neurology) and the surgical 'super-speciality' departments (Urology, Neurosurgery, Paediatric surgery and Plastic surgery), have acute shortage of human resources to meet the demands of patient care (Rao et al., 2011;Bajpai, 2014). This shortage makes it very difficult to conduct regular formative assessments in resident training. It was perceived that Mini-CEX will prove an ideal tool for clinical skill training and assessment of residents in these busy departments. A preliminary evaluation of the faculty and resident perceptions on the use of Mini-CEX as formative assessment tool in super-specialty departments was assessed in our institution.
Mini-CEX is a structured assessment tool based on observed clinical encounter, used globally for formative assessment of medical students. This method has gained acceptance as a work place based assessment (WPBA) and is aligned to the relatively new concept of competency based medical education (CBME). Generic form for Mini-CEX can be downloaded fromhttps://www.abim.org/pdf/paper-tools/minicex.pdf and the results can be recorded in a standardised format in logbooks or portfolios. Since the exercise can be completed in 30 minutes with provision for immediate feedback on performance from faculty to the learner, Mini-CEX is most suited as an assessment tool for residents in departments with shortage of manpower and time resources. We decided to evaluate the feasibility of Mini-CEX for performance assessment in our institution. Faculty and resident satisfaction on the feedback component of the tool was elicited for each clinical encounter.

Subjects and Methods
Thirty faculty and residents from Medical superspecialities (Cardiology, Neurology, Nephrology, Gastro-enterology) and Surgical superspecialities (Neurosurgery, Urology, Plastic surgery, Paediatric surgery) of Government Medical College, Kozhikode, a 2000 bedded tertiary care centre participated in this exercise. This institution is also a regional centre for medical education technologies (RCMET) under Medical council of India. Faculty development programs are conducted regularly, sensitising faculty to the current methods of teaching and assessment including WPBA.
Faculty and residents were sensitisedabout the method to conduct MINI-CEX. The standardised ABIM assessment forms, faculty and resident feedback forms were distributed (Appendix 1 & 2). The exercise was conducted using standard methodology described by Norcini et al (2003). In the forms, 1-9 scores were assigned to skills in -Medical Interviewing, Physical examination, Humanistic qualities/Professionalism, Clinical judgement, Counselling, Organisation/Efficiency and Overall clinical competence. The type of cases and areas of assessment were decided by the faculty and residents. Each resident had a 1:1 interaction with the assessor.

Data collection
Data was obtained from 30 encounters in various clinical settings, assessed by the faculty using ABIM Mini CEX assessment form, assessor (faculty) feedback form and resident feedback form. At the end of the session, the time taken and the overall satisfaction scores were noted in the standard form. The faculty and residents were then asked to fill in the feedback forms provided. Opinion regarding assessment was recorded on a 5-point Likert score along with few open-ended questions for feedback (Kogan et al., 2002). Data was collected on a) performance of residents using Mini-CEX tool, b) faculty feedback and c) resident's feedback on the clinical encounter. Response to open ended questions were collected and transcript analyzed. For closed end questions, Continuous variables were summarized as mean and standard deviation (SD) and categorical variables were summarized as frequency with percentages. Independent sample t test was used for comparison of scores between the groups like prior experience with Mini-CEX. Data analysis was done with SPSS 18 statistical software packages.

Results
Among the total assessments,40% were from medical super-specialty departments and 60% were from surgical super specialty departments. While 83% of assessments were conducted in the inpatient setting, 17% were done in outpatient setting. Of assessments, 53%were focused on data gathering, 23% on diagnosis, 17% on therapy and 7% on counselling (Fig 1). Complexity of the task was set as moderate in 90%, high in 7%and low in 3%. Area of focus and complexity of assessment were decided based on the year of residency. Among the residents who participated in the study, 30% were in their first year, 33% were in the second year and 37% were in their third year of residency.
Comparison between assessor characteristics, domains assessed and Mini CEX scores were done ( Table 1). The average assessor satisfaction score on tool administration was 69.6% of the possible maximum for the faculty and 68.3% for the residents. The median+ range scores for competencies on the assessment form ranged from 5.69 (SD 1.87) to 6.04 (SD 1.45). Among the faculty, the designations were Professor (53%), Associate Professor (20%) and Assistant Professor (27%). It was found that the senior faculty gave higher competency scores and better overall satisfaction scores in comparison to their junior colleagues. Overall assessor satisfaction was higher for medical super specialty departments [6.50 (SD +/-1.2)] than surgical super specialty departments [6.42 (SD+/-1.497)]. Medical super-specialities perceive that MINI CEX is useful for selfassessment (p value 0.037) compared to

Figure 2: Comparison between the scores of Medical and surgical super specialty departments
Familiarity with MINI CEX improved faculty satisfaction compared with first time users. While 43% of our faculty were already using MINI-CEX assessments, 57% were unfamiliar and experiencing the technique for the first time.
Among the faculty, only 23 % had attended the basic course workshops for faculty training, which are mandated by the Medical Council of India. Those who attended the workshop were familiar with the MINI-CEX assessment method. The attendees gave higher mean scores for assessing medical interviewing and counselling skills. They also identified areas of development (p values 0.003), identified strengths (p value 0.012) and formed action plan (p value 0.028) for improved performance in future sessions during feedback.
The faculty perceived that the three parameters best assessed by MINI-CEX were -clinical judgement (p value 0.012), overall clinical competence (p value 0.037) and counselling skills (p value 0.044) (fig 3). Qualitative data from open questions on the difficulties faced during Mini CEX and suggestions to modify Mini CEX to suit resident assessment were analysed and added in Tables 2 and 3. The assessors pointed out difficulties in assessing clinical judgement and unfamiliarity with the technique. Residents perceived this to be a time-consuming exercise and a few were feeling stressed in the presence of the assessor. Suggested areas for modification by assessors included professionalism, procedural skills and patient feedback Residents suggested frequent assessments, global assessment combining multiple aspects in one session and gaining familiarity with Mini CEX.  Assessment of areas such as Investigation and recent advances should be included".

Frequent assessments
Global assessment combining multiple aspects Increase familiarity with the assessment process

Discussion
In Indian medical colleges, most of the clinical departments use long case presentation method as assessment tool for residents. Clinical skill assessment using routine case presentations is time consuming, requires optimum clinical environment and can affect patient management in a busy department. Students are often evaluated for presentation skills rather than clinical competences.
Feedback, if at all given is usually inadequate and non-actionable. 'Super-speciality' departments of most medical colleges in India, often find it difficult to do a proper formative assessment of residents due to shortage of manpower and time resources.
Mini-CEX is a positive tool for resident assessment in busy clinical departments 1,9 . Holmboe et al states that direct observation of clinical skills by faculty is the critical first step to help improve their trainees' clinical acumen (Holmboe et al., 2004). With Mini CEX, clinical competencies are directly observable, can be conducted in any setting (inpatient or outpatient), requires no advance planning and can be completed in 30 minutes. It is an objective, reliable and valid assessment tool. Bias can be avoided by multiple sessions with different faculty (Singh and Sharma, 2010). There is provision for giving immediate feedback to learner, which is prompt, specific and actionable. It helps learning process in residents through self-assessment and reflective practice. It is a powerful tool to provide good interactive and structured feedback and has proven to improve trainees' clinical skills (Liao et al., 2013). By addressing the cognitive and psychomotor domains as well as the attitude and communication skills, this assessment method is expected to encourage problem based learning and self-directed learning in medical students. Periodic assessments with multiple encounters of MINI-CEX will enable supervision of learner's progress in real time.
In our study, average satisfaction scores were 69.6% for assessors and 68.3% for the residents.23% of our faculty had attended BCW and 43% were familiar with MINI CEX format and were been able to give better assessment than those unfamiliar with the format. In their study on the development and implementation of Mini-CEX, Liao et al have stated that faculty development is the prerequisite for successful implementation of this assessment (Liao et al., 2013). We also found that the senior faculty noted better satisfaction rates though Goel and Singh (2015) consider the experience of teacher is irrelevant due to the objectivity of this assessment. But a study on rater training by Cook et al. (2009) states that by conducting a two-level workshop, rater confidence and interrater reliability can be improved.
Majority of the studies on Mini-CEX in literature search were done for Internal medicine residency training. In our study, the overall assessor satisfaction with Mini-CEX was higher for medical super specialty departments [6.50 (SD+/-1.2)] than surgical super specialty departments [6.42 (SD +/-1.497)], with surgical faculty commenting that the surgical skills assessments were inadequately addressed in the score sheet. Lowest ratings for physical examination and highest ratings in professionalism has been reported in the study by Norcini et al. (1995) suggesting that the tool may need modifications for satisfactory use in surgical departments. Medical departments expressed that there was no scope for interpretation of investigation reports using this format. Several studies have shown that mini-CEX, when used alone may be insufficient to assess individual competencies and hence will have to be coupled with other assessment tools like DOPS or OSPE for procedural skills (Norcini et al., 2003;Durning et al., 2002).
Residents from three years of training were equally represented in the study. Though majority of residents felt that being observed by the faculty in MINI CEX is helpful, it was stressful for some. Such apprehensions can be overcome by repeated assessments with the same or different faculty. Malhotra et al. (2008) have noted that the anxiety level reduces with familiarity to the exercise. Residents felt that this exercise helped them have 1:1 interaction with faculty. They felt that the assessment through faculty feedback will help to motivate further learning and improve their attitude and communication skills. Many residents expressed difficulty in communication skills due to language barrier since they were from different linguistic regions, such issues have been reported by other researchers (Goel and Singh, 2015). Schopper et al. (2016) have studied students' perspectives on the effect of observation and feedback on the development of their communication skills and elicited their suggestions to maximize the educational value like increasing the number of observations, disassociating observation from numerically scored evaluation, providing regular feedbacks starting early in residency. Mini CEX tool for post graduate training has been studied and found feasible in in various departments of medical and dental colleges in India and other developing countries (Alam et al., 2016;Gupta et al., 2017;Bhatnagar et al., 2014;Castanelli et al., 2016;Meresh et al., 2018;Joshi et al., 2017;Al-Jewair and Kumar, 2019).
To our knowledge, feasibility of such assessment tools has not been studied in DM or MCh course residents of super-specialty departments of India so far. Our preliminary study demonstrates that the Mini-CEX is a feasible option for formative assessment of residents in the super specialty departments of various medical colleges in India. Limitations of our study are that psychometric properties of this tool have not been tested by repeated assessments before it is used as a formative assessment tool in the concerned departments (Nair et al., 2008). There are conflicting opinions about the impact of work place based assessments on doctors' education and performance. Miller et al. (2010) in a systemic review have shown that there is no evidence to show that WPBA tools other than multisource feedback leads to improvement in performance, subjective reports on their educational impact are positive. Lorwald et al. (2018) in their systematic review and metaanalysis have identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS (Directly observed procedural skills) on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b). They found two implementation characteristics, "quality" and "participant responsiveness" to have potential influence on the educational impact. Hejri et al. (2017) are analysing the psychometric properties of Mini CEX, in order to identify gap of knowledge in this field. Assessors familiarity with Mini CEX is an important finding that influences the competency scores as noted from this study. Our future studies are planned to evaluate the impact of multiple formative assessments with mini CEX on clinical acumen of residents at the end of their training in super-specialty departments.
Limitations of our study: Being small sample results cannot be generalised and study needs to be replicated using larger samples.

Conclusions
MINI CEX is a feasible tool which was well accepted by faculty and residents for formative assessment in the super-specialty departments of our setting. Familiarity with Mini CEX tool and attendance of workshop for faculty training improves assessment. Residents feel that the 1:1 faculty interaction and immediate feedback motivates further learning.

Conflict of interest
Authors declare no conflicts of interest.