QUICK GUIDE
- Key Words: Video, Testing
- Learner English Level: intermediate and higher
- Learner Maturity Level: College/University, Adult
- Preparation Time: one hour or more to prepare video and handout
- Activity Time: up to 30 minutes for student prep/15-minute test
Introduction
One difficulty in testing conversation ability, especially at higher levels, is deciding what information students should be responsible for. Even with a common subject, such as vacations, we can not assume that all students will be able to talk about their vacations simply because they all have had vacations. The student who has just returned from a one week cycle tour around Hokkaido will probably have more to say than the student who has stayed at home during the same week. Although there are subjects that we can reasonably assume students can discuss, these are usually so generic that only the best students can make them into something interesting. Avoiding this problem by giving students topics in advance can often lead to recitations that have less to do with conversational ability and more to do with memorization skills.
The Hokudai Oral Proficiency Test (HOPT): Parts I and II
That was the problem we faced in the Hokudai Oral Proficiency Test (HOPT). Although a complete description is outside of the scope of this article, it is useful to mention two key points of the test. (For a complete description of the test, please see Brown, Glick, Holst, & Tomei, 1997.) The first is that the HOPT is an instrument designed to test large numbers (400+) of students with a few (5 to 6) interviewers in the space of one day. To do this, the fifteen-minute test seeks to measure oral proficiency through the conversational interaction within a group of three candidates. The interviewer takes no active role in the test. His or her role is to watch the conversation as it develops.
The second point is that the test is divided into two parts: Part 1 is a three-way discussion on one of six starting topics (family and friends, food and drink, out-of-class activities, sports, end of term vacation, and hometowns), and Part II is a discussion on four travel destinations. Part II serves the purposes of (a) giving additional "warming up" to students who may need it due to nervousness, and (b) assigning a new task to give these students a fresh starting point in the test. Part II is also conceived to be a greater challenge, requiring higher level skills than the first section. While Part I emphasizes the exchange of information among the three participants, Part II asks for skills such as speculation and asserting oneｹs opinions.
Thus, when we begin to ask the students to express more advanced skills in Part II, we need to make sure that all students can attain the same level of preparedness. Note that this does not mean an equivalent amount of preparation time. One student may need to work intensely for 30 minutes while another student may require only five, so a framework must be created to allow for this. Additionally, the framework should aim to avoid rewarding memorization, yet give all students access to the same set of facts. Had we presented these facts in written form, we would have been grading students partially on their reading ability rather than on their conversation skills. Had we informed the students of the facts when they entered the interview test, some students may have learned of the topic and prepared in advance, encouraging students to postpone their test until the last possible moment. Our solution for this was to use a video to present the Part II information of the HOPT on the day of administration.
During the registration for the HOPT on the day of the interview, the students were asked to view a five minute video and take notes using a supplied form (see Figure 1). The video was a fictional travel agent's presentation of four August vacation packages to Hawaii, London, Okinawa, and Tokyo. The travel agent detailed price, meals, and some sight-seeing opportunities for each. The students were then expected to note the information, using the form provided. This form was to serve as an aide-memoire for the students, helping them to argue for their own choices and against other choices. There were no "correct" answers as the purpose of this section was to have students express and support their own opinions and learn to challenge the opinions of others. The students watched the video in a separate viewing room, where the video was playing continuously, before moving to the intake room to be assigned to a group and an interviewer.
An analysis of the notes taken by the students gives some indication of the success of the format. About 70% (288 of the 406 note forms collected) were in English, while 84 note forms, or approximately 20% were in Japanese, and 34 note forms or less than 9% of the total were mixed. There were no errors in the vacation destination in the closed section of the form (i.e., destination, days and nights, included meals, and prices).
Answering Logistical Problems
Although space is too short to give even a brief bibliography of the work dealing with research on using video in the foreign language class, Gruba (1997), using the term "video media," gives an excellent bibliography of previous research as well as presenting a research agenda for the role of video media. However, one additional reason for using video is its ability to solve logistical problems. In the case of the HOPT, where between 450-600 students are processed by only six examiners, the video presentation not only provided students with information in a way which de-emphasized memorization, but also solved several logistical concerns.
As one of the key aspects of the HOPT is to group three students who do not share the same class with a teacher who is not their classroom teacher, a fixed schedule (with students assigned to specific times and specific examiners) would be a recipe for disaster. One student arriving late or forgetting his or her interview time would create a domino effect that would impinge on other students' efforts. The solution was to assign students to half-hour time slots and assign them to interview groups at the interview site, so that students would have to wait no more than fifteen minutes for their interview. By having the video play continuously in a separate room, students could arrive, watch the video until they were ready for the test, and move to the intake room where they would be assigned to a group. Students who wanted to watch the video multiple times could be accommodated, as could those students who only needed to watch the video once. Rewarding memorization, however, was avoided. Even if students were able to memorize the entire monologue verbatim, it would be little help because the task was to present their opinion as to which is the best choice and respond to the opinions of others. Furthermore, a video tape, with its presentation of images and sound, was able to create an atmosphere where students could concentrate on the information in a way that an audio recording could not have.
Conclusion
Because we wanted to test the conversational ability of large numbers of students, video offered not only a way to present the information so that no student weak in other skill areas was at a disadvantage, but also as a solution to several logistical problems we faced. Although many of the components of any large-scale testing program are specific to the institution, from our experience, video can play a key part.
References
Brown, C., Glick, C., Holst, M., & Tomei, J. (1997). The Hokudai oral proficiency test: Large-scale oral testing made possible. In K. Hartmann and T. Christensen (Eds.), The 1997 JALT Hokkaido Proceedings, (pp. 25-32). Sapporo: JALT Hokkaido.
Gruba, P. (1997). The role of video media in listening assessment. System, 25(3), 335-345.