Predictors of First Year Medical School Academic Performance at University of Santo Tomas Faculty of Medicine and Surgery: A Retrospective Observational Cohort Study

Background: This study investigated whether undergraduate general weighted average (uGWA), National Medical Admission Test (NMAT), gender and premedical course were able to predict academic performance based on the general weighted average of the first year of medical school (medGWA). Methods: Records of medical students accepted from 2012 to 2015 were reviewed and analyzed. Results: 1528 records were included. uGWA (r = 0.55 p<0.001) and NMAT scores (r = 0.45 p<0.001) had significant moderate linear correlations with medGWA. An increase of one point in uGWA and NMAT scores increased chance of passing by 40% [OR = 1.4; 95% CI: 1.3 to 1.5] and 5% [OR = 1.05; 95% CI: 1.04 – 1.07], respectively. For a passing grade (>75points), optimal cut-off score for uGWA was 86.96 with predictive accuracy of 81% [95% CI: 0.77 to 0.84], while for NMAT it was 79.5 with predictive accuracy of 75% [95% CI: 0.71 to 0.80]. Males were 40% less likely to pass [OR = 0.6; 95% CI: 0.4 to 0.9]. Nursing, Physical/Occupational Therapy have final scores higher, while Pharmacy and Psychology had final grades lower than other premedical courses. Conclusion: uGWA and NMAT had significant moderate linear correlations with medGWA with the optimal cut-off score to get a passing grade for uGWA at 86.96 and NMAT at 79.5. Males were less likely to pass. Graduates of Nursing, Physical/Occupational Therapy garnered significantly higher medGWA compared to other premedical courses. Findings may guide the admissions committee and add to knowledge about which criteria would help predict medical school academic performance.


Introduction
The University of Santo Tomas Faculty of Medicine and Surgery prides itself with more than 140 years of academic excellence. To date, it has been named a Center of Excellence by the Commission on Higher Education (CHED). It accepts only 450 incoming first year students out of the usual 1700-1900 applicants each year. Medicine and Surgery, Manila, Philippines. Email: eccunanan@ust.edu.ph Preferred preparatory programs for entry into medical school are Biological Sciences (BS) primarily structured as pre-medicine courses such as Biology, Medical Technology, Pharmacy, Nursing, Public Health, and Physical/Occupational Therapy, but other BS and BA degree holders may still apply. With the growing diversity of students applying and the limited freshmen slots, it is pertinent that the admissions committee identifies the best applicants and ensures that the selection process is fair. Scholastic standing based on undergraduate general weighted average (uGWA) and National Medical Admission Test (NMAT) performance are given the biggest weight when narrowing the choice for potential candidates (University of Santo Tomas Faculty of Medicine and Surgery UST-FMS, 2015).

South-East Asian Journal of Medical Education
Vol. 14, no. 1, 2020 NMAT is a nationwide norm-referenced multiple-choice examination designed to determine scholastic aptitude and basic science knowledge of those desiring to pursue a career in medicine. Globally, most medical schools use a combination of one or more of the following areas as criteria to select medical students: academic ability: undergraduate grades, entrance exams and interviews. (Boelen & Boyer, 2001).
The study conducted by Siu and Reiter (2009) indicates that academic scores, grade point average, aptitude tests and multiple miniinterviews show positive predictive value to the academic performance of students, as compared to personal interviews, letters of reference and emotional intelligence, which were not able to predict performance.
Undergraduate grade point averages and Medical College Admission Test (MCAT) scores were considered to be strong predictors of test performances during medical schooling in the US (Basco et al., 2002). The MCAT, moreover, demonstrates incremental validity (compared with undergraduate grades alone) for predicting performance as early as Year 1 of medical school. A meta-analysis of MCAT studies showed correlations of 0.58-0.67 between the total test score and Year 1 performance. Over the span of the medical degree course, MCAT explained about 19% of the variance in early or pre-clinical performance (Donnon et al., 2007).
The United Kingdom Clinical Aptitude Test (UKCAT), however, did not predict Year 1 performance at two UK medical schools studied (Lynch et al., 2009). It only had a modest predictive power for overall course performance at the University of Glasgow Medical School (Sartania et al., 2014).
Locally, a study at the Cebu Institute of Medicine (CIM) showed a positive relationship between NMAT score, premedical grade point average (GPA) and year 1 final grade of first year medical students admitted in school year 2008-2009 with a problem-based learning curriculum. 16% and 27% of the final grade were explained by the NMAT score and combined NMAT score and premedical GPA, respectively (Barbosa et al., 2009). The CIM problem-based curriculum, however, is different from UST's more traditional medical curriculum.
Aside from the abovementioned study, published data on the predictive validity of NMAT and undergraduate GWA on academic performance in medical schools in the Philippines are sparse. Aditionally, despite rigorous screening procedures, some chosen medical students still fail the course and eventually drop out as early as the first year of medical school. This leads to unnecessary waste of resources and lost opportunities for those who were not chosen.
Since uGWA and NMAT are critical criteria for admission, it is crucial to investigate whether uGWA and NMAT scores accurately predict the academic success of students accepted in UST medical school. It would be important to find out whether the premedical course impacts their academic performance as well.
This study, therefore, aimed to investigate the extent to which uGWA, NMAT and premedical course predict UST medical students' first-year academic performance.
The findings may serve as guide to the admissions committee in their decision-making and may help in the development of future admission plans and student retention programmes.
Results can also guide undergraduate school counselors by being able to better identify students at risk for failing or dropping out from medical school. Furthermore, this may add knowledge about which criteria are more significant in predicting students' medical school academic performance.

Research Design
This is a retrospective observational cohort study. The cohort included first year medical students admitted at UST-FMS from the following school years: 2012-2013, 2013-2014, and 2014-2015. Three different school years 48 Vol. 14, no. 1, 2020 were included to make the results more generalizable.

Undergraduate General Weighted Average (uGWA)
The term "uGWA" indicates the average of all undergraduate course grades.

National Medical Admission Test (NMAT)
The NMAT is a two-part test: The four subtests that compose Part I measure aptitude, while the four subtests that compose Part II measure subject proficiency in areas which are part of the academic background required of applicants to the medical course.
Part I has five scores, one for each of the four aptitude subtests and their summative score, the APT Composite. Part II produces five scores, one for each of the four special subject areas and their summative score, the SA Composite. The General Performance Score (GPS) is the summative score derived from the eight subtests.
The results on the test are converted to standard scores from normalized scales with points ranging from 200 to 800 and a midpoint of 500. These scales were derived from the NMAT performance of the norm group. The test results of examinees are automatically compared to those of the norm group who has mean score of 500 and standard deviation of 100. A percentile rank scale is generated from the GPS for purposes of ranking. This scale has points ranging from 1-to 99+, with a midpoint of 50 which corresponds to the midpoint of 500 in the normalized standard score scales. (CEM, 2015)

Medical school GWA (medGWA)
Year 1 medical school final grades were treated as general weighted averages (medGWA). For each medical student, medGWA was computed by multiplying each course grade by the number of hours for that course, summing the weighted grades across courses, then dividing this sum by the total number of hours for that year.
Each medical student's medGWA, together with the student's individual final scores in the major subjects Anatomy, Biochemistry, Physiology, Histology and Neurology were extracted from the UST-FMS dean's office database.

Failed
Grades < 75points were considered as failures.

Data gathering procedures
Information regarding gender and premedical course together with the students' uGWA and NMAT scores were obtained from the UST-FMS admissions office database. medGWA was obtained from the UST-FMS dean's office database.
Necessary information were collected and recorded using a data collection form that did not contain any personal identifiers. Only special code numbers appeared on the collection form.

Ethical Considerations
Any information that were obtained from the records was kept strictly confidential. The subjects were not identified by name in the data collection form and will not be identified in any future publication of the results. Moreover, information contained within the data collection form will only be used for this study.

Statistical Analyses
Means and ranges were utilized to summarize quantitative datasets uGWA, NMAT, and medGWA, while counts and percentages were used for qualitative datasets, gender and premedical course.
Linear regression was performed to determine factors that contributed to the students' final grades. Diagnostic checkings were performed to validate the results of the regression analysis. Univariate and multivariate logistic regression were performed to identify factors affecting whether medical students fail in their final grade. Area under the curve was used to determine optimal cut-off. The maximum Youden Index J identified the optimal cut-off in NMAT and uGWA scores in obtaining a passing mark in the medGWA, including the individual final grades in Anatomy, Biochemistry, Physiology, Histology and Neurology All tests used 5% level of significance in SPSS version 20.0.

South-East Asian Journal of Medical Education
Vol. 14, no. 1, 2020

Results
A total of 1567 students were in the record but five had missing NMAT scores. Additionally, 34 students dropped out of the program prior to start of the first year of medical schooling, leading to 1528 students to be included in this study. The demographic profile of the students are shown in Table 1.
The linear regression model had R2 = 0.681, indicating that 68.1% of medGWA was predicted by the factors presented in Table 2.
In identifying whether medical students will pass or fail in their final grade, the optimal cutoff for NMAT and uGWA grades were 79.5 and 86.96, respectively. These have sensitivities of 0.82 and 0.66, respectively, and specificities were 0.59 and 0.83, respectively. These models have predictive accuracy (AUC) of 75% [95% CI: 0.71 to 0.80] and 81% [95% CI: 0.77 to 0.84] for NMAT and uGWA, respectively.
Other cut-offs in predicting performance in Anatomy, Biochemistry, Physiology, Histology and Neurology are also shown in Table 4. All these models have acceptable predictive accuracy (AUC > 0.5, p<0.05).

Discussion
This study showed that both uGWA scores (r = 0.55 p<0.001) and NMAT scores (r = 0.45 p<0.001) positively correlated with final grades (medGWA) which is similar to the findings in a study at the Cebu Institute of Medicine which showed a positive relationship between NMAT score, premedical grade point average and year 1 final grade of first year medical students admitted in school year 2008-2009 with a problem-based learning curriculum.
The NMAT score has moderate correlation (r = 0.45 p<0.001) with the final grades (medGWA) similar to the results of a meta-analysis where MCAT (Medical College Admissions Test in the USA and Canada) also showed moderate correlations of 0.58-0.67 with Year 1 performance (Donnon et al., 2007). This is in contrast to the United Kingdom Clinical Aptitude Test (UKCAT) which did not predict Year 1 performance at two UK medical schools studied (Lynch et al., 2009).
Vol. 14, no. 1, 2020 What is unique about this study is that the premedical course was proven to significantly affect medGWA. Graduates of premedical courses in BS Nursing and BS Physical/Occupational Therapy significantly garnered higher scores compared to graduates of other premedical courses.
The premedical course also significantly affected medGWA, with graduates of Nursing and Physical/Occupational Therapy significantly garnering higher scores compared to graduates of other premedical courses. Males were 40% less likely to pass [OR = 0.6; 95% CI: 0.4 to 0.9] compared to females. Graduates of Pharmacy and Psychology were 50% [OR = 0.5; 95% CI: 0.3 to 0.9] and 70% [OR = 0.3; 95% CI: 0.2 to 0.5], respectively, less likely to pass compared to graduates of other premedical courses.
This study is mainly limited by its retrospective nature. The author recommends extending the study to determine whether the same factors also predict second and third year final grades and eventual performance in the Philippine Physicians' licensure examinations. The author also recommends coming up with a validated predictive scoring system utilizing the factors found to be significant predictors as the parameters.