Medical student selection as the ‘first assessment’: international trends

Greater accountability and professional regulation and a more mobile medical workforce means that selecting students with the right attributes to practice medicine is increasingly important. Recruiting and retaining doctors who will stay and practice in the country that trained them, especially with doctor shortages such as rural and remote areas, is a huge social challenge. Selection for medical school is a crucial step in addressing such issues and medical schools have responsibility to ‘get it right’. Whilst cultural and regional differences exist, international trends in medical selection indicate two main shifts: the first towards seeing selection for medical school as ‘the first assessment’, the second is towards using a wider range of selection methods than simply selecting the brightest students as determined by school leaving or university academic qualifications. All selection methods have advantages and disadvantages and, depending on the course of study and (most importantly) the nature of current and future medical practice, schools can tailor selection methods to meet health service needs. Newer methods reflect changes in assessments which are more objective, seeking to formally assess professional attributes and behaviours (non technical skills) as well as cognitive ability. Methods discussed include the application form; personal statement; interview; multiple mini interview (MMI); personality tests, and newer methods such as situational judgement tests (SJTs). Schools need to ensure students are not only fit to study but will be ultimately fit to practice medicine and identify the expertise and resources to carry out what may be labour intensive or expensive activities.


Introduction
Medical educators and those in charge of the medical workforce worldwide have expressed concerns about the quality of medical graduates and the appropriateness of modern medical courses. This has international importance because of ensuring patient safety, professional practice and improving health outcomes (which lie at the core of good medical practice). As the health workforce becomes more mobile, it is increasingly vital to produce graduates who are fit for practicing global medicine.
Health service shifts have influenced medical education, not only that graduating doctors from any country need the knowledge, skills and attitudes to practice safely and effectively, but also that the majority of doctors produced through any medical education programme want to work (a) in the country in which they were trained and (b) in all the specialties and all the geographical communities that need doctors. In most countries, there are difficulties in recruiting and retaining doctors (particularly general practitioners) to work in remote and rural areas (Hsueh et al., 2004), and in areas of high deprivation such as poor urban communities (Hayden, 2010). In Australia for example, in recent years there has been a net loss of GPs, the reasons for which have been widely reported in the literature (Hays et al., 2011;Hays et al., 1997;Barnett, 1992). In many countries there is a heavy reliance on internationally-trained GPs and other specialists for provision of healthcare to rural countries for relatively short periods before leaving for positions in other countries. It is essential therefore that selection processes not only draw on international best practice but are also designed to recruit the most able and suitable applicants to work in each country's health services.

The role of selection in an integrated approach to medical workforce planning
Although the causes for many medical workforce issues lie largely outside undergraduate medical education, undergraduate programmes and the medical schools that deliver them are one part of the solution. Countries, medical schools and communities have responded to workforce issues through a range of means including wholesale curriculum reform, affirmative action programmes (linked to local communities or to encourage and support applicants from specific ethnic groups) and the establishment of new medical schools.
From selection onwards, medical training throughout the continuum should be more integrated, better co-ordinated and address the 'choke points' at various stages of training. This can be achieved partly through managing the medical workforce through education and training to match workforce needs, and also through better communication systems and aligned curricula to smooth out the educational continuum. However, it is vital to select and recruit the most appropriate students to meet the needs of existing and future health services and who can also cope with the demands of contemporary curricula.
Increasing attention is being paid to how and from where medical students are selected (McManus & Powis, 2007). In Australia (as in many countries) specific streams, supported by scholarships and tied into bonding arrangements, exist for students from rural and remote areas. In Australia, New Zealand and Canada, affirmative action programmes are well established for indigenous groups (e.g. Aborigine, Torres Strait Islanders, Māori and Pacific groups) and for students from rural regions. Evaluation of medical programmes in Australia indicates high long term retention of doctors (Joyce & McNeil, 2006). However, the long term success of programmes aimed at addressing workforce need through expanding and reshaping undergraduate medical education is still to be determined and issues still remain around attracting doctors to work in such areas.
However, as Prideaux (2006) and others point out, the other side of the solution is around policies and strategies for workforce planning, noted by many to be a hugely difficult problem particularly with a global and increasingly mobile health workforce. In the UK NHS next stage review (Department of Health, 2008) Lord Darzi identified workforce planning and education, and training commissioning as areas that needed strengthening. The review suggests a new better co-ordinated approach aimed at sustaining the NHS in the future. Other writers have suggested that taking narrow, health service based approaches to workforce planning stifles both innovation and the capacity for change, and a reliance on a primarily medical workforce to deliver care that can be delivered by other health or social care workers is short sighted (Wilson et al., 2009).
It has been suggested that more attention should be paid to formalising the roles of community health workers, mid-level health workers, advanced practitioners or 'physicians assistants', particularly in areas where primary care, public health and preventive services need to be strengthened (WHO, 2003;Pick et al., 2001). What implications do these shifts have on the recruitment and selection of medical students?
Over the last two decades, many countries have increased the number of medical graduates, primarily in response to increasing populations, changing demographics and shifting workforce trends, to address shortages in rural, regional and remote areas and to encourage recruitment and retention amongst certain sectors of the population, such as indigenous or under-represented groups. Increasing medical student numbers has resulted in four main responses by universities and governments.
• Establishing new medical schools in areas of previously underserved populations.
• Allocating additional numbers to existing schools for existing programmes.
• Allocating additional numbers to existing schools to develop and introduce new programmes, quotas or establish clinical or urban schools in rural, remote or regional areas of deprivation or health need • Supporting affirmative action schemes and programmes for groups under-represented in medicine In addition, many countries or regions (e.g. Malaysia, some provinces in Canada, Cyprus) have formed collaborative arrangements with universities or governments in other countries under which a selected group of students are admitted either to a jointly delivered and validated programme, a split programme (where part of the programme is taken in the home country and part overseas) or have franchised out education and training to another country. This has required ongoing debate and agreement over selection criteria and methods. Finally, in many countries, private medical schools exist (sometimes less well regulated than public or state schools) which again meet students' demand to study medicine.

Recruitment and selection for Medical School: methods and approaches
Assessing cognitive skills

Academic aptitude and attributes
Students entering a five or 6 year programme are usually school leavers, graduates with a degree that is not deemed relevant for graduate entry or graduates with a relevant degree (e.g. in Biosciences) but where a Graduate Entry Programme is not on offer.
Entry requirements are usually high as there is great competition for places, typically between 12 and 20 applicants per place. Graduate Entry Medicine (GEM) programmes are usually four year programmes typically for bioscience or other graduates, offered in many countries including the US, Canada, the UK, Ireland and Australia.
Special arrangements exist in many countries e.g. New Zealand and Australia for students from indigenous and rural backgrounds. At Auckland University for example, the MAPAS (Maori and Pacific Islanders Admissions Scheme) and ROMPE (Rural Origin Medical Programme Entrants) are positive or affirmative action programmes which provide quotas for students from these backgrounds, plus additional entry and support mechanisms. Many state funded medical programmes also include a quota (typically 7-10%) for international students, some on study scholarships, others full fee paying students. Selection for medical school is typically through a combination of measuring academic achievement at secondary school or university (e.g. through secondary school certificate, baccalaureate or A-levels or degree level); the personal statement or 'open space' in the application; referees' report and interview.
Most medical schools require students to have studied biology, and sometimes chemistry and physics at least to school leaver level. In New Zealand and Australia, applicants also take the UMAT (Undergraduate Medical and Health Sciences Admissions Test), administered by the Australian Council for Educational Research (ACER) to assist in the selection of students into certain health science courses including most medical (MBBS or MBChB) and Dentistry (BDSc. or BDS) courses, physiotherapy and pharmacy.
The US medicine equivalent of UMAT is MCAT (Medical College Admissions Test). In the UK, some traditional entry medical students take UKCAT (UK Aptitude Test), which is equivalent to UMAT. Students wishing to enter graduate-entry programmes in Australia, the UK and Ireland take the GAMSAT (Graduate Medical Schools Admission Test). Each year, these tests are held on a single day, typically during either late July or early August.
Selections processes need to get a balance between aptitude and achievement and whilst ideally methods should judge a candidate's aptitude to study medicine, most tests are more likely to be achievement focused. The number of studies of the use of GAMSAT is small, and reports demonstrate that there is poor or only modest correlation between GAMSAT score and later performance outcomes (Groves et al., 2007;Coates, 2008, Wilkinson et al., 2008. In addition there is negative correlation between GAMSAT score and clinical reasoning tests (Groves et al., 2007).

Interviews
Interviewing is often used as part of the selection process for medicine, although as McManus & Powis observed 'selection sometimes seems more to ensure the correct number of entrants on day one, than to identify those best suited to the course and profession ... the University of Adelaide recently reduced its emphasis on selection interviews, the University of Queensland may be ending interviews and a meta-analysis in Medical Teacher (Goho & Blackman, 2006) suggested that selection interviews have only a 'modest ' predictive validity and 'little' or 'limited' practical value'. (2007, p.1) However in some countries and schools (e.g. mainland Europe, the UK, China) selection to medical school is determined only by achievement or highest grade point average (GPA) at the relevant level. In countries where the volume of applicants far exceeds supply, this can be a pragmatic approach to selection, however it fails to address issues relating to suitability and motivation for a highly pressurised profession which demands sophisticated nontechnical and communication skills.
A number of studies looking at face-to-face interviews found wide variations in reported reliability, the general consensus being that little evidence exists on the reliability of interviews (Kreiter et al., 2004). One issue is that of interviewer bias, especially in situations using clinicians as single interviewers, where candidates whose personality traits matched those of the interviewer, had favourable outcomes for selection (Quintero et al., 2009). Despite its face validity as a measure of noncognitive abilities, much controversy remains regarding the reliability, predictive validity and cost-effectiveness of the selection interview. (Salvatori, 2001). Hays et al. (2011) identify a set of features and behaviours exhibited by medical students in need of extra support, suggesting that students may be admitted who are unsuitable for medical practice and who struggle, for personal or academic reasons. Student professional problems are associated with a higher risk of professional problems in later practice (Papadakis et al., 2004). Relatively few problems have serious consequences but those that do require substantial resource input and often result in students not completing the course. Identifying these applicants during the selection process may provide the key to minimising non-completion rates and preventing future poor performance at medical school and later professional practice. These issues have given rise to the increasing attention being paid to assessing 'softer' (non technical) skills required of doctors as well as academic achievement, such as:

Assessing non-cognitive skills
• The later in the medical programme that students encounter real patients and the complexities of medical practice, the later these struggling students are identified. This poses a risk to students, patients and a great cost to the public purse. It makes sense, therefore to attempt to select medical students on the basis of these generic attributes, whilst recognising that the role of the doctor in today's society is changing and a number of other attributes may also be key.
Selection has been termed the 'first assessment' in that many of the aspects that medical school assessments are trying to measure are similar to those that selectors are struggling with. The principles which underlie assessment are as important to the process of selection as they are to assessing the student once admitted. By considering selection as the first assessment, the same quality assurance principals utilised throughout the course are also valid, namely: blueprinting of assessable domains and attributes, utilising appropriate strategies, using an evidence-based approach, demonstrating transparent decision making and evaluating impact (Prideaux et al., 2011) Having blueprinted assessable attributes, the next step is to identify appropriate formats within which to assess them. A number of approaches have been taken to address this: MMIs are now commonly used in medical selection and interviewing in Canada, the United States, Australia and the UK (Eva et al., 2004;Jerant et al., 2012;Dowell et al., 2012).

Multiple mini interviews (MMIs)
Discuss issues relating to various topics, including ethical issues, career choices, motivation for studying medicine, previous work experience, motor skills tasks and activities designed to explore attitudes. MMIs replicate the OSCE (Objective Structured Clinical Examination), in that after all applicants have completed the stations, the scores and comments are fed back and collated. The interview scores are then aggregated with other application data so that decisions can be made on who should be offered a place. Whilst MMIs have been shown to have good predictive validity and reliability , some studies suggest they may favour more mature candidates with more life experience (Jerant et al., 2012;Dowell et al., 2012).

6
South East Asian Journal of Medical Education Vol. 6 no.1, 2012

Situational judgement tests (SJTs)
The limitations of many selection instruments are that they assess what an applicant says they would do in a certain situation. More robust selection methods should seek to discover what an applicant would do, given a certain situation. SJTs purport to do this; as well as assessing whether applicants can discuss situations, they also seek to show that they can demonstrate the actual competencies that we are trying to select. SJTs normally involve either written or filmed hypothetical scenarios in which the candidate has to report on how they would respond in that situation (Lievens & Sackett, 2006).
Whilst they are now used to test clinical reasoning in postgraduate medical admissions in the UK (Koczwara et al., 2012), they have also been applied to admissions processes in Canada (Dore et al., 2009), Belgium (Lievens & Sackett, 2006) and Israel (Ziv et al., 2008). SJTs show good validity and reliability (Lievens & Sackett, 2012). Whilst Whetzel et al. (2008) report that white candidates outperform the other ethnic groups tested, the difference is believed to be smaller than that seen when using academic achievements alone (Koenig et al., 2013).

Personality and emotional intelligence assessment
A number of studies have profiled unprofessional behaviours, through the use of self-report questionnaires, and suggested the use of personality indicators and profiles have predictive value in identifying students that go on to cause public concern (Hodgson et al., 2007;Lumsden et al., 2005). Whilst evidence suggests that it may be unwise to exclude candidates solely on this basis , they may help identify students preadmission who may need additional support or guidance during the course. For example, previous studies using personality testing have shown that conscientiousness (as measured by NEO-PI-R) can be used to predict academic success in medical school (Doherty & Nugent, 2011). However, students with high levels of neuroticism and conscientiousness are believed to be more vulnerable to stress and burnout (Tyssen et al., 2007).

Fitness to practice
It should be noted that health or disability issues or other personal circumstances that come to light during the selection process should be dealt with separately from the selection methods described above. This is important to ensure schools do not inadvertently discriminate against applicants. In some circumstances however, a health or disability issue may be so severe as to exclude an applicant from enrolling at medical school as they would not be able to practice as a doctor once qualified. Each country's regulatory or professional bodies should provide guidance to schools on such cases and limits on practice.

Conclusions
Selecting the right medical students from a large number of intelligent young people in a highly competitive environment is very challenging. Patients expect more from their doctors than pure knowledge and 'doing'. Fish & de Cossart (2007) assert that doctors who only possess these attributes demonstrate just the tip of 'the iceberg of professional practice'. They liken the base of the iceberg as the 'invisibles' of practice. These include the feelings, expectations, assumptions, beliefs, intuitions (sixth sense), attitudes, insight/ imagination and personal values of the clinician.
So, although it is vital to ensure students have the academic ability to sustain a medical programme, schools also need to determine how best to select the future "wise doctor". Schools need valid, reliable and reproducible methodologies to ensure selecting students with the necessary insight, integrity and resilience at the start of their professional journey. This may well involve a combination of semi structured interviews, questioning the applicants on several ethical dilemmas which they could face, written scenarios, situational judgement tests or simulation.
Whilst some of the newer methods (such as MMIs and situational judgement tests) are enabling schools to assess non-technical skills and proto-professional behaviours in a more objective and structured way, the perfect medical school selection process is yet to be determined. However, on current evidence it is clear that a combination of methods helps to assess a wide range of attributes and skills, that some methods are better than others in predicting future professional performance and that selectors need to take into account not only the academic journey the student will encounter but also the healthcare needs of the country or region. This is particularly vital for countries that struggle to recruit and retain doctors, especially those from indigenous populations.