Pre and post-lecture test scores for assessment of short term effectiveness of didactic lectures in anatomy and as a predictor for performance in summative evaluation

Introduction: Didactic lectures play an important role as a method of student instruction in anatomy in India. The aims of the present study were to quantify whether students' derived any short term benefit from didactic lectures and to assess whether their performance in tests based on the lecture was a good indicator of their performance at the end of the year university examinations. Method: A total of eight theory classes, four each on embryology and gross anatomy, for sixty first year undergraduate medical students were conducted. At the beginning of each class, the students were administered a pre-lecture test. The post-lecture test was administered after the lecture. Subsequently, the marks obtained by the students in the theory component of the end of the year university examination in anatomy were noted and used for further correlation analysis using Spearman’s correlation coefficient. Paired and unpaired t tests (2 tailed) were performed to check for significant differences in the pre and post-lecture test scores for each class. The mean scores in the pre and post-lecture test scores for each student over the eight classes were used in correlation analysis. Results: In all eight classes, the mean post-lecture test scores were significantly greater than the mean pre-lecture test scores. There was a significant positive correlation between mean post-lecture test scores and university exam scores. Conclusion: Tests incorporated into didactic lectures may be a useful method not only to improve students' benefit from lectures, but also to predict their performance in the theory component of the university examination.


Introduction
Undergraduate medical students at the college where the authors teach, study the pre-clinical subjects including anatomy in the first year of their course. This is then followed by one and a half years in which the paraclinical subjects are covered. The final two years of the course are utilized for the clinical subjects. There are periodic examinations at the end of each year. Once the students have passed all the examinations, they go through one year of practical training in the hospital. Very little vertical or horizontal integration of the course material occurs during the period of study.
There are numerous research papers which show that actively involving students in the learning process improves learning outcomes in students (Michael, 2006). Many methods to harness the potential of active learning, like problem based and team based learning (PBL and TBL), have been used in medical colleges all over the world for teaching anatomy (Nieder et al., 2005;Yiou and Goodenough, 2006). In addition, both vertical and horizontal integration within the medical curriculum helps students to gain a wider perspective, as well as an appreciation of the inter-relatedness of the preclinical, paraclinical and clinical subjects (Vyas et al., 2008). However, in most medical colleges in India teaching tends to be compartmentalized into specific subjects and the didactic lecture still continues to be an integral part of the teaching methodology.
There have been few studies to assess the effectiveness of didactic lectures in teaching anatomy. The effectiveness of didactic lectures in student learning can be improved by utilizing various innovative methods that involve the students in the learning process (Richardson, 2008). One such method has been attempted in the present study. The aims of the present study were to quantify whether students derived any short term benefit from didactic lectures, and to assess whether the students' performance in pre and post-lecture tests was a good indicator of their performance in the end of the year university examinations.

Course structure
The university to which the medical college is affiliated, stipulates that 650 hours be utilized to teach anatomy. Of this time, 160 hours are meant for theory classes, with the remaining time being used for dissections, small group discussions, tutorials and practical classes. The subjects covered by the theory classes in anatomy are general anatomy, gross anatomy including neuroanatomy, histology, embryology and genetics. With very few exceptions, the method used to cover these subjects is the didactic lecture. In didactic lectures, one faculty member speaks to the whole class of 60 students about a particular topic for an hour, usually with the help of the blackboard, transparencies or Power Point presentations. The faculty conducts these lectures on a rotation basis throughout the academic year. The subjects for the present study were sixty first year undergraduate students at a medical college attached to a tertiary hospital in South India, who participated after giving their informed consent. The time period of the study was from August, 2008 to May, 2009.

Pre and post-lecture test
A single faculty member conducted a total of eight theory classes, four each in embryology and gross anatomy. At the beginning of each class, after the attendance was taken, the students were administered a pre-lecture test for about 5 minutes. The pre-lecture test consisted of ten single responses multiple choice questions (MCQs'). The didactic lecture then commenced with a listing of the specific learning objectives for the session. The topic was then covered with the help of a Power Point presentation displayed through a LCD projector. The blackboard was also utilized as and when required. This lecture went on for forty five to fifty minutes after which the postlecture test was administered, with identical questions as the pre-lecture test.
The pre and post-lecture tests were designed in such a manner that they followed the learning objectives of the lecture. To prevent any copying, four versions of the pre and postlecture tests were distributed, such that students sitting adjacent to one another got different versions. The different versions had the same questions but in a different serial order. At the end of the eighth lecture a questionnaire was distributed to the class asking them whether they found the pre and post-lecture test useful. A five point Likert scale was used to elicit their responses, with higher scores indicating that they found the tests useful. They were also asked to state the reason for their response as well as to give their comments. The pre and post-lecture tests as well as their responses were evaluated by the same faculty member who took the classes.
Subsequently all of the sixty students answered their end of the year university examination which determined whether they would pass on to the next academic year. This examination has a theory, viva voce and a practical component. Once the results of the examination were announced, the marks obtained by the students in the theory component in anatomy were noted and used for further statistical analysis.

Statistical analysis
The mean and standard deviation of the pre and post-lecture test scores for each of the eight classes were calculated. Both the paired and unpaired t-tests (2 tailed) were performed to check for statistically significant differences in the pre and post-lecture test scores for each class. The paired and unpaired t-tests were utilized to look for within-subject differences and differences in the means of the pre and

South East Asian Journal of Medical Education
Vol. 6 no.1, 2012 35 post-lecture tests scores respectively. The mean and standard deviation of the subjective scores on the Likert scale were also calculated. The mean scores in the pre and post-lecture test for each student over the eight classes was calculated and used in the subsequent correlation analysis. The correlation between the following parameters was calculated using Spearman's correlation coefficient: 1) mean pre-lecture test and mean post-lecture test scores; 2) mean pre-lecture test and university exam scores; 3) mean postlecture test and university exam scores; 4) mean post-lecture test and subjective evaluation scores. A p value of less than or equal to 0.05 was considered statistically significant.

Results
The mean and standard deviation of the pre and post-lecture test scores are shown in Table 1. In all eight classes the mean post lecture test scores were significantly greater than the mean pre-lecture test scores (p<0.0001). The paired t-test also showed highly significant differences in the pre and post-lecture test scores (p<0.0001). When considered separately the mean pre-lecture test scores of the embryology lectures were lower than that of gross anatomy lectures (2.97 vs. 3.36). However, the mean post test scores were higher for the embryology classes (6.74 vs. 6.29).
In response to the question about whether they found the tests useful, the mean Likert score of the 60 students was 3.57 ± 0.89. This indicated that on the whole the students found the tests beneficial. When asked to comment on the tests, many of the students remarked that the tests helped them to focus better in class, as well as to assess how much knowledge they gained from the lecture. Some of the students felt that as the questions were based on the learning objectives, they gave them a good idea of the important aspects of the lecture. A few students felt that the prelecture test was not useful, but the post-lecture test was useful. They felt that if they had prepared for the class in advance they might have benefitted more from the tests. A small number of students felt that such tests would be taken more seriously if the results counted for the final evaluation of the student.
There was a significant positive correlation between the means of the pre and also postlecture test scores each and mean pre as well as post-lecture test scores with the university exam scores. The highest positive correlation was noted between the mean post-lecture test scores and the university exam scores. However, there was only a weak positive correlation between the mean subjective scores and the mean post-lecture test scores (Table 2).

Discussion
Didactic lectures still play an important role as a method of student instruction in anatomy in India. This method of teaching has its advantages and disadvantages (Moni, 2000). There has been a definite move in many countries to replace the traditional didactic lecture method with the more active learning methods such as problem-based learning (PBL) and team-based learning (TBL). Some studies have shown that there is a comparable level of knowledge in anatomy in students who have studied through a PBL method and the conventional lecture based method (Last et al., 2001;Prince et al., 2003). However other studies have shown that students in a conventional curriculum have a significantly higher level of anatomical knowledge as compared to peers who went through a PBL curriculum for anatomy (Hinduja et al., 2005). Nayak et al. (2006) are of the opinion that a hybrid method, incorporating features of both methods, would be the most suitable method for teaching anatomy. Even when lectures are used as a method of instruction, there has been an attempt to incorporate active learning elements into it to make it more effective (Richardson, 2008).
Keeping in mind the scenario in many medical universities in India where recall is tested more than analytical ability and problem solving, didactic lectures still remain the predominant method of teaching anatomy to medical students. However, there have been only a few studies to analyze whether lecture classes actually improve short-term knowledge gain. Another objective of this study was to see whether students' performance in the pre and post tests had a predictive value on their performance in the theory component of the University examinations.
In this study, the post-lecture test scores were significantly higher than the pre-lecture test scores. The increase in the mean post-lecture test scores as compared to mean pre-lecture test scores was higher for embryology classes than for gross anatomy classes. This could be because students are not exposed as much to embryology as they are to gross anatomy during the course. Many of the structures that are discussed in the lecture are actually visualized during the dissection practical, possibly explaining the higher pre-lecture test scores in the gross anatomy lectures. A survey of the literature did not reveal any study that utilized pre and post-lecture test scores to evaluate individual lecture classes.
A project conducted in India utilized multiple choice questions (MCQs) at the end of surgery lecture sessions for third year undergraduate medical students. The students were of the opinion that the MCQs were useful and made them more attentive in class. They also felt that the MCQs emphasized the important points of the lecture and enabled them to think about the material covered in the lecture (Prasad, 2004). The feedback received was similar to the comments made by the students in the present study. The poor correlation between the subjective evaluation scores and the post-lecture test scores suggests that subjective evaluation alone may be an inadequate method of determining the success or failure of an educational program. Formative assessment of students is important because it gives them feedback even while learning is taking place (Hudson and Bristow, 2006). The use of formative assessment of first year medical students has been attempted with success at many medical universities around the world (Geist & Soehren, 1997;Lukic et al., 2001;Hudson & Bristow, 2006;Krasne et al., 2006;Poljicanin et al., 2009). The role of formative assessment scores as a predictor of performance in summative examinations has also been documented. The present study shows that both pre-lecture test scores as well as post-lecture test scores had a significant positive correlation with the scores in the theory component of the university examinations. These results are similar to a study conducted in Croatia, in which students' performances in daily quizzes in anatomy had a significant positive correlation with their performance in both the theory and oral components of the final examination (Poljicanin et al., 2009). An interesting finding of the present study is that pre-lecture test scores also had a significant correlation with both post-lecture test scores as well as university examination scores. Thus, it seems that students with a higher level of baseline knowledge are likely to score better in the post-lecture test as well as in the university examinations. Other studies have also shown that students' scores in formative assessments correlated well with their scores in the summative examinations (Geist & Soehren, 1997;Lukic et al., 2001;Krasne et al., 2006). One of these studies however showed the effects of formative assessment on long term retention of the knowledge, after the end of the course were not significant (Geist and Soehren, 1997).
A limitation of the present study was that long term gains, if any, from the lecture classes was not assessed. Another limitation of the present study was the pattern of questions in the university being different from those in the class tests. While the class tests consisted only of single response MCQs', the university exams had descriptive essay type questions. The essay questions test not only knowledge, but also the writing skills of the student. Also, many of the essay questions are not structured, leading to less objectivity in grading. This could be one of the reasons why the correlation between the class test marks and the university exams was not higher. Another aspect to be considered is that the lecture class may not be the preferred way of learning for some students.
These students probably gained their knowledge outside the classroom using other methods like self-study or group discussions. Such students would have performed poorly in the class tests and better in the university examinations. Application of a regression model that takes into account other variables such as time spent preparing for an examination, could have given more insight into the relative importance of the factors involved in performance at the university theory examination.

Conclusion
The present study demonstrates short term gains in students' knowledge of anatomy from didactic lectures. Their performance in class tests based on the material covered in the lecture showed a significant positive correlation with their performance in the theory component of the university summative examinations. No significant correlation was demonstrated between the perceived usefulness of the class tests and the students' performance in these tests. Tests incorporated into didactic lectures may be a useful method not only to improve students' benefit from lectures, but also to predict their performance in the theory component of the university examination, with certain limitations as mentioned.