Feasibility of Voice Recording and Transcription for Classroom Dynamics Assessment

Introduction: Classroom dynamics is important for promoting student‟s cognitive domains but has not been empirically assessed. Voice recordings in a classroom by students are common and could offer unique information potentially useful for assessing classroom discussion. Objectives: To explore the feasibility of voice recording and transcription as a tool to assess classroom discussion and to identify pitfalls and/or opportunities for improvement. Methods: Five graduate students in the Health Systems and Universal Coverage course were asked to do voice recording while attending the course (two students were randomly assigned for each session). They then perform detailed transcription to be quantitatively analyzed using character counts. Focus group discussion with the students and instructors was also performed to identify pitfalls and/or opportunities for improvement. Thematic content analysis was used for qualitative data analysis. Results: On average, each lecture session resulted in 20.57 pages (95% CI 17.72 to 23.43) and 49,863 characters (95% CI 42,516 to 57,210) of voice transcription files. There were no statistically significantly different numbers of pages and characters between the two transcribers (p=0.866 and 0.8021). The median percentages of class discussion domination by the instructors based on number of pages and characters were 94.74% and 97.44%, respectively. Conclusion: Voice transcription is an easy and useful data source for assessing classroom dynamics; however, without appropriate technology, the burden of transcribing might limit the usefulness of this approach.

The goal to increase students" cognitive ability can be accessed from both technical and interpersonal quality perspectives.While technical quality is easily justified from the subject contents, assessment of interpersonal quality is relatively more complicated.
The limited amount of class time must be efficiently used for each of the cognitive domains.
While promoting classroom dynamics in a lecture-based course with extensive amount of new content is challenging, even more difficult is feasible approach for assessing the dynamics of each classroom.
In healthcare industry, Donabedian"s structureprocess-outcome quality assessment framework has been well recognized Vol. 12, no. 2, 2018 (Donabedian, 2003).When applied to classroom-based session, structure comprises all inputs to the classroom such as human (instructor, learner, assistant), equipment, as well as the classroom environment, which affect the teaching and learning process during the session.While these inputs are essential for the classroom, students" cognitive ability would remain unchanged without the interaction between the instructor and the students.
This classroom dynamics required both students" participation and instructor"s skills.
Students" participation can be divided into academic and social engagement.Academic engagement is behavior related directly to the learning process whereas social engagement is the nature of students" interaction with instructors or peers (Finn et al., 2003).
Classroom discussion is one of the most common types of social engagement.Only about 6% of the time was devoted to classroom discussion and only 25% of the students took part in these discussions in college courses (Nunn, 1996).
A number of approaches have been used to assess classroom dynamics.Direct observation has been commonly used either for research or quality assurance purposes in the published literature.Significant amount of time and effort of the assessor is required.The findings could be reported based on either subjective description or discussion of the class session or objective quantification of some indicators such as percentage of time spent in participation and percentage of students participating (Nunn, 1996).
As findings from direct observation could be affected if the instructor and students are aware of the presence of assessor (Kline et al., 1990), some "offline" techniques have been used.In one study, classrooms were photographed and shown to groups of professors and students, each of whom evaluated the friendliness of and their preference for all the classrooms (Douglas & Gifford, 2001).The findings identified three most preferred classroom properties (view to outdoors, seating comfort and seating arrangement) but whether or not these opinions actually affect classroom dynamics remains questionable.
Students have commonly used portable voice recorders or mobile equipment with such functionality as adjunct to note taking to ensure the availability of the content and discussion after class adjourns.While the lecture content could be found in class materials and notes, the voice recordings offer unique information potentially useful for assessing classroom discussion.

Objectives
The objectives of this study were to explore the feasibility of voice recording and transcription as a tool to assess classroom discussion and to identify pitfalls and/or opportunities for improvement.

Course Characteristics
The study population comprised five graduate students (3 doctoral, 2 master degree) who registered to the course Health Systems and Universal Coverage (3014745) in the second semester of 2013.The objectives of the course were to understand concepts of health system and its components and to be able to comparatively analyze health systems across countries.Eleven of 16 sessions (68.75%) were lectures whereas the rest sessions were allocated to student presentations.Grading structure comprised three assignments (80%) and class participation (20%).The first assignment (20%) was called "health system challenges," in which each student selected one challenge from WHO"s framework for action, read 1-2 relevant journal articles, then presented and discussed in class.For the second assignment (25%), students applied the WHO"s Health System Assessment Approach and assessed one key health system function of an interesting country.
The third assignment (40%) required the student to compare characteristics, health indicators and health insurance systems of Thailand and the selected country.The main instructors received their doctoral education on health systems, health services and health economics from major institutions in Europe and USA.The lead instructor also had research and teaching experience on qualitative, quantitative and mixed methods research.
Each of the students received a small portable voice recorder with playback function through internal speaker and USB connectivity.Two randomly assigned students simultaneously did voice recordings for each of the sessions then transcribed into electronic document in Microsoft Word format named A and B. Focus group discussion with the students and instructors was also performed to identify pitfalls and/or opportunities for improvement.
The transcription was quantitatively analyzed using descriptive statistics of the number of pages and characters, instead of word frequency, because Thai language does not use "space" to denote each word.Student"s paired t-test was used to compare the numbers of page and character between the two transcribers of each session.The extent to which the instructor and the students speak was assessed using transcription B.
The transcribed files were imported into Atlas.ti qualitative data analysis software for the analysis.Thematic analysis was used for qualitative exploration of the transcription.The investigator read the transcription and listened to the voice recordings over and over again until he became familiar with the content to its entirety.

General characteristics
All five students were female with average age of 26 years.Three students attended this course to fulfill their doctoral requirement whereas two master-degree students electively took this course.This course was taught by three male instructors, all with doctoral degree in health systems, health services, health economics, health financing, or human resource for health.Also involved were three guest lecturers with either doctoral or medical degree.All instructors had taught for more than three years with good student feedback.
All of the course sessions have been conducted smoothly throughout the semester with some slight changes of the schedule and rooms.Three out of 16 sessions missed one student who was not responsible for the voice recording of the session.
Fifteen minutes of the first session were spent on explanation about logistics of this study before getting into the introductory content of the course.Because of the delayed procurement of the voice recorders, the first two students had to use their own mobile phone to record the class lecture and discussion.
Overall quality of the voice recordings was acceptable but occasionally suboptimal mainly because of some interference to the recording process.

Quantitative Findings
On average, each lecture session resulted in 20.57pages (95% CI 17.72 to 23.43) and 49,863 characters (95% CI 42,516 to 57,210) of voice transcription files.There were no statistically significantly different numbers of pages and characters between the two transcribers (p=0.866 and 0.8021; Student"s paired t-test, Table 1).

Qualitative Findings
Three main themes emerged from the transcribed recordings.First, all students shared their difficult time doing the transcription, especially the time and effort they spend on the transcription.On average, two hours were needed to transcribe a onehour session of class.Second, the quality of the recorded voice seemed to be the major determinant of transcription difficulty.Nonetheless, two out of five stated that the transcription process was a useful review of class session.Third, from the instructor"s perspective, the voice transcription could be modified to become narrative lecture materials.However, the more dynamic the class discussion is, the more difficult such benefit can be achieved.

Discussion
The findings from this study suggested potential benefits of voice recording and its transcription as a data source for assessing classroom discussion.With page and character count functionality of currently available software; relative percentage of class discussion time contributed by the instructor and students was easily calculated and comparable with that of other methods (Nunn, 1996).
The students not only studied in the class but also helped to collect data for class discussion assessment.Although the assumption that students loved to take notes and record voice was valid, the voice transcription was not so.Students" reaction on excessive class burden was inevitable; however, it suggested critical opportunity for improvement of this technique.More advanced technologies such as speech recognition and real-time transcription are potentially helpful for minimizing transcription time.
The small sample size limited generalizability of the findings to other settings.As all of the students in the present study were female, it would be difficult to imagine how male students would react to this technique.Another key issue is quantitative assessment of the voice transcription in Thai language.This technique might be less beneficial when voice recording is not possible.As quality of voice recorder has been less questionable, the environment of the classroom becomes relatively more important.