The Language Teacher
August 2003
Building a Successful TOEFL Program:
A Case Study
Neil Heffernan
Ritsumeikan University
Introduction
The Test of English as a Foreign Language (TOEFL) is regarded as a major test for assessing EFL/ESL (English as a Foreign Language/English as a Second Language) proficiency around the world. It measures the English language aptitude of foreign students who apply for higher education in the United States and Canada or other countries where English is the language of instruction. The purpose of this paper is to outline how a regimented and well-organized TOEFL program for Japanese students can lead to student success, which is measured in terms of achievement on the actual TOEFL test.
Educators and students alike are aware of the need for TOEFL education. This need is evident in the volume of resources Japan has invested in improving English proficiency in the past decade (Itoko, 2002). The nature of an increasingly globalized world has placed great importance on learning English for professional purposes, whether in business, sports, science, or technology. Most students realize that a score of five hundred or higher will allow them to enter a university overseas (Obermeier, 2000), which may in turn grant them access to a brighter future. Furthermore, the same score, or slightly higher, allows students to apply for Monbukagakusho grants, which would essentially allow them to study abroad for 12 months with government paid tuition (Monbukagakusho, 2001).
The tasks on the TOEFL tests skills that have been deemed to be relevant and important for completing coursework in all subjects at 4,300 North American universities and colleges at both the graduate and undergraduate levels (Abraham, 1990). Thus, not only does preparing/ studying for the TOEFL supply learners with academic and linguistic skills necessary to enter a foreign university, it also grants them the opportunity to greatly improve their English ability for other future endeavors. Indeed, with Japanese universities relaxing their admission standards so as to sustain enrollment levels (Mulvey, 2001), more students are looking to the ryuugaku experience for a more inclusive English-intensive education. This experience is undoubtedly a valuable one that can only brighten the future of university-aged students.
It is with these goals in mind that the Center for Language Acquisition (CLA) at Ritsumeikan University implemented a ten-week TOEFL program to be run twice a year. The course was conceived by a handful of professors on the board of the CLA who were conscious of the need for students at the university to not only become more proficient in English, but to score extremely well on the TOEFL test. Programs of this nature are still relatively rare in Japan, with the majority of universities having no intensive TOEFL preparation program at all. For the reasons mentioned above, the CLA was intent on providing a meaningful experience to its students; the majority of whom were interested in studying abroad, and thus needed a high TOEFL score.
This study was performed during the period of October - December, 2002. The 165 undergraduate students enrolled in this non-credit course (for which they had to pay an extra fee) were divided into three proficiency levels based on a pre-course diagnostic test. Teachers were selected in-house, giving highest priority to those with extensive TOEFL teaching experience. Teachers applied to teach the course and were paid by the CLA.
The TOEFL consists of four sections: listening, structure/written expression, reading, and writing. The test is given in two types of environments. The first is in TOEFL test centers found in numerous countries around the world. The second is in institutions where students are currently studying. For students taking the Institutional Test Program (ITP), the writing section is omitted. The ITP gives schools, colleges and universities, English language institutes, and other agencies throughout the world the opportunity to administer the TOEFL ITP test locally to their own students. Both forms of TOEFL are official and acceptable for student use. However, the ITP has a limited scope since it is often used only as an indicator of a learner's English proficiency between partner universities (Ritsumeikan has such a partnership with the University of British Columbia). Since TOEFL ITP scores are more or less interchangeable with TOEFL scores obtained from an official TOEFL institution, they were deemed appropriate for this study.
Historically, Japanese college age students have displayed poor performance on the TOEFL test (Sawa, 1999; Honna, 1995). Further, overall English abilities in Japan seem to be in decline (see Mulvey, 2001; "Misu-machi", 1999; "Daigaku", 2001; Monbukagakusho, 2002). With this in mind, Monbukagakusho developed a tactical plan to foster English abilities among Japanese. This plan included improving university entrance examinations, boosting the motivation of learners, revising educational content, and upgrading the teaching system (Monbukagakusho, 2002).
A successful TOEFL program should effectively prepare students for the test, with the ultimate goal of increasing any previous score they may have achieved. For the purposes of the program described here, the administration, professors and coordinators of this TOEFL program set a 50-point increase in score as the goal of the course. This study will describe the course of action taken by the teachers involved in the teaching of the 10-week program.
Methods
Following a diagnostic test, students were placed into three levels; intermediate, high-intermediate and pre-advance, based on their score. However, students who had a pre-existing TOEFL (score approximately 20) were not required to take the diagnostic test because this score was deemed to be equal to the diagnostic scores. Similarly, students took an ITP test at the end of the course, which served as their program exit score. The numbers of students were as follows: intermediate 69, high-intermediate 67, pre-advanced 29.
Classes were 3 hours in length, and were held twice a week over a ten-week period. Teachers (both Japanese and native English speakers) were assigned two sections of the test to cover each class. One class was reserved for listening and structure/written expression, and the other was devoted to the reading and writing sections. It was decided to include a writing class in the curriculum because it would provide students with appropriate training for taking the test with an official TOEFL institution. Moreover, since most students in the program were interested in the ryuugaku experience, no effort was spared to ensure they could achieve this goal.
Seeing as a course of this nature is uncommon in most Japanese universities, it is worth noting that this particular program worked because it took place in a Japanese setting: The teachers, syllabus, and design of the program were specifically selected for Japanese students, with their needs and, more specifically, weaknesses in mind. The syllabus was planned around the Longman Complete Course for the TOEFL Test textbook (Philips, 2001), and designed by the course coordinators. Classes focused on the four skills in the test: grammatical skills such as recognition of and the ability to manipulate verb forms, pronouns, nouns, clauses, parallel structure, conjunctions, and comparisons; listening skills such as listening for detail, questions, idioms, and anticipation of the topic; reading skills concentrated on things such as vocabulary, stated and unstated detail, drawing conclusions and determining the tone and purpose of a passage; and writing skills such as developing ideas, sentences, vocabulary, and paragraphs into meaningful prose. It was here that the teachers' familiarity with the TOEFL test--and with their learners' individual needs--was employed. Further, the intensive nature of the class schedule enabled teachers to closely monitor student progress: progress that was clearly perceptible by the end of the course. Instructors taught techniques such as memorization skills, shortcuts to answering questions, time saving methods, proper writing techniques, vocabulary building, looking for keywords, and deciphering idioms.
A teacher's workshop, led by the two coordinators, was held two months before the starting date of the course. At the workshop, the objectives of the course were covered, and the teaching material distributed. The workshop served as a general introduction to the 10-week program. In addition to the workshop, the coordinators of the program were in continuous contact with the teachers to address their concerns. A post-course teacher survey found the workshop and coordinator communiqués to be extremely helpful and informative, allowing teachers to fully understand what was expected of both teachers and students during the course. Moreover, teachers were given ample time to review and study the materials so they would be sufficiently prepared for the course.
Student Attitudes and Progress
In order to gauge student attitudes on their progress and satisfaction with the course, pre-course, mid-term and final questionnaires in Japanese were distributed, and revealed some noteworthy responses. Specifically, the questionnaire asked about student attitudes toward the textbook, syllabus, teachers and their explanation of the concepts in the textbook, difficulty of the lessons, atmosphere in the classroom, class level, speed of lessons, and overall satisfaction with the course. The questionnaire gave students a three or four point scale to record their responses. The four point scale consisted of: very good; good; satisfactory; and, not satisfactory. The three-point scale was utilized for the class level, speed of the lessons, and explanations categories. These points were: too high or fast; just right; and, too low or slow. The data from the questionnaires was then gathered, transcribed, and translated into English for the English-speaking teachers. The coordinators of the course passed the information onto the teachers, who then used any student comments and suggestions to improve their teaching style (see the results section).
With the purpose of monitoring student progress during the course, a Personal Development File (PDF) was put together. The PDF was a logbook that allowed students and teachers alike to assess student progress. Aside from having a questionnaire at the beginning to record basic student information such as sex, previous TOEFL scores/diagnostic test score, and reason for taking the course, the PDF consisted of charts listing the exercises in the textbook. Students added their scores to the PDF as they completed in-class exercises and homework. Furthermore, students were given a weekly writing assignment, which was then graded with detailed comments according to the TOEFL 1-6 scale for writing (TOEFL Test and Score Manual, 1990-91). Students also wrote their writing scores in the PDF. Teachers collected the PDF at the end of every class so they could monitor student achievement by noting student scores on exercises completed in-class and for homework. Teachers were able to ascertain whether or not students were making progress by studying the results of their work.
The rationale behind creating the PDF was to allow teachers to observe student strengths and weaknesses, with the intent of addressing student concerns each class. The PDF also acted as a barometer for the students themselves: by keeping track of their work for 10 weeks, they were encouraged to fully commit themselves to the learning process. This was reflected in student comments at the end of the course. These comments show that students thought the PDF acted as a motivator for them to improve their daily and weekly work. Indeed, over the 10 weeks, teachers noticed a substantial improvement in student PDF scores on the textbook exercises. This success can be attributed to hard work on the part of the students, and consistently skillful teaching by the instructors. Students tended to put a great deal of work into the course at home. The majority of students reported doing 12 to 15 hours of homework per week on tasks such as extra reading exercises (articles, short stories), grammar exercises, writing short essays on a given topic, and compiling reports in English on news stories seen on television. This augured well for both the students and the course: Ultimately, results would be the arbiter of the success of the program.
Results
The pre-course questionnaire was completed by 87% of the students in the course (145/ 165). Of the 145 students, 81 (56%) stated that their purpose for taking the course was to study abroad, and 57 (37%) claimed their motivation was to improve their English skills.
The mid-term and final questionnaires showed that, overall, students were pleased with the quality of the classes and the teachers. However, the mid-term survey showed a large number of students stating that they were unsure of how to rate the course in general, mostly because it was too early to ascertain if it was effective. In spite of this, detailed written comments by many students allowed some mid-term adjustments to teaching styles, such as; the pacing of the classes, providing more in-depth explanation of complicated grammar, and repetition of listening tasks. Student comments were essential factors for teachers at this stage of the course; teacher's reported tweaking their teaching styles to better fit their learners' needs. Student comments on the mid-term and end-of-term questionnaires will also assist in improving the next 10-week course conducted at this university.
The end-of-course questionnaire, given on the final day of class, showed a marked difference in student attitudes toward the program. In total, 107 (65% of the total). Generally, students expressed satisfaction with the outcome of the course, with 78% claiming they were either satisfied or very satisfied with the course (for an outline of responses to the final questionnaire, see Figure 1). Attitudes toward teachers also improved, with the middle ground seen in the mid-term questionnaire all but disappearing. The comments made at the end of the course sounded encouraging notes for future TOEFL programs at this university.
Figure 1: Final Questionnaire Results
| Question/Category | Response | % of students |
| Textbook Speed of class Atmosphere of class Level of class Teacher explanations Syllabus General satisfaction |
good/very good just right just right just right good/very good good/very good satisfied/very satisfied |
73% 92% 80% 94% 84% 94% 78% |
Note: Total number of students answering the final questionnaire = 107.
Since the pre-course scores were so diverse, there was plenty of room for improvement for the majority of students entering the program. The lowest pre-course score was 296, while the highest was 563. The breakdown of scores was as follows (a more comprehensive list of pre-course and end-of-course scores on the ITP are in Figure 2): Twenty-three students (14%) scored between 300-350 while thirty-five students (21%) achieved between 351-400. Fifty-six students (34%) scored between 401-450, while thirty-one students (19%) garnered scores between 451-500. Finally, nineteen students (11%) scored 501, or better.
Figure 2: Pre-Course and End-of-Course ITP Results
| Intermediate (pre-course scores of 296-420) | ||
| Pre-Post /Increase | Pre-Post/ Increase | Pre-Post/ Increase |
| 350-450 / 100 | 383-490 / 107 | 360-507 / 147 |
| 296-397 / 101 | 406-486 / 80 | 352-452 / 100 |
| 410-517 / 107 | 420-510 / 90 | 393-486 / 93 |
| 306-417 / 111 | 360-497 / 137 | 400-443 / 43 |
| 346-426 / 80 | 423-457 / 34 | 410-447 / 37 |
| 320-456 / 136 | 350-456 / 106 | 356-477 / 121 |
| 346-460 / 114 | 393-456 / 63 | 403-483 / 83 |
| 313-427 / 114 | 420-493 / 73 | 413-473 / 60 |
| 400-487 / 87 | 313-432 / 119 | 416-470 / 54 |
| 383-483 / 100 | 310-423 / 113 | 343-496 / 53 |
| 357-403 / 46 | 406-433 / 27 | 376-393 / 17 |
| 390-487 / 97 | 367-430 / 63 | 380-470 / 90 |
| 400-463 / 63 | 410-450 / 40 | 416-470 / 54 |
| 343-496 / 53 | 406-467 / 61 | 413-423 / 10 |
| 416-410 / -6 | 320-417 / 97 | 386-497 / 111 |
| 356-460 / 104 | 320-433 / 113 | 403-450 / 47 |
| High-Intermediate (pre-course scores of 420-500) | ||
| Pre-Post/ Increase | Pre-Post/ Increase | Pre-Post/ Increase |
| 433-463 / 30 | 443-467 / 24 | 436-463 / 27 |
| 443-450 / 7 | 433-487 / 50 | 486-450 / -36 |
| 433-487 / 54 | 433-487 / 40 | 460-520 / 60 |
| 440-450 / 10 | 443-460 / 17 | 486-536 / 50 |
| 473-487 / 54 | 486-530 / 44 | 486-467 / -11 |
| 440-513 / 73 | 473-510 / 63 | 456-487 / 54 |
| 460-500 / 40 | 476-523 / 47 | 453-500 / 47 |
| 450-497 / 47 | 470-473 / 57 | 470-490 / 20 |
| 473-490 / 17 | 450-521 / 71 | 453-507 / 54 |
| 423-450 / 27 | 423-450 / 27 | 470-490 / 20 |
| 433-483 / 50 | 463-480 / 17 | 443-433 / -10 |
| 453-507 / 54 | 450-521 / 71 | 473-490 / 17 |
| Pre-Advanced (pre-course scores of 500-563) | ||
| Pre-Post/ Increase | Pre-Post/ Increase | Pre-Post/ Increase |
| 520-507 / -13 | 530-540 / 10 | 536-500 / -36 |
| 563-583 / 20 | 533-580 / 47 | 506-520 / 14 |
| 500-553 / 53 | 506-517 / 11 | 503-527 / 14 |
| 516-500 / -16 | 510-533 / 23 | 500-543 / 43 |
| 500-513 / 13 | 520-573 / 53 | |
Note: Pre = pre-course score; Post = end-of-course ITP; Increase = the increase or decrease in score from the pre-course ITP to the post-course ITP. The pre-course test was administered four days before the course began (some students did not take it, due to a pre-existing TOEFL score). The post-course ITP was on December 7th, 2002. Ninety-eight students took the final ITP test.
From the outset, the final ITP was determined to be the litmus test for the whole program. However, not all students took the test. The results are based on scores achieved by students who took the exam. Ninety-eight students (59% of the initial enrollment) took the final ITP. The breakdown of their scores, in accordance with pre-course groupings, is as follows: Of the 23 students with a pre-course score between 300-350, 12 took the ITP. Of these students, 10 (83%) increased by 100 points or more, while two (17%) increased by 90-99 points. Of the 35 students who had a pre-course score of between 351-400, 20 took the ITP. Of these students, seven (35%) increased by 100 points or higher; six (30%) increased by 80-99 points; three (15%) increased by 60-79 points; and two (10%) increased by 40-59 points.
Of the 56 students who had achieved pre-course scores between 401-450, 30 took the ITP. Of these learners, one (3%) increased by 100 points or higher; two (7%) increased by 80-99 points; six (20%) increased by 60-79 points; and 10 (33% of the total) increased by 40-59 points.
Of the 31 students who had a pre-course score of 451-500, 22 took the ITP. In this group one student (4%) increased by 71 points. Further, seven (32%) increased by between 50-60 points; and five (23%) increased by 40-49 points. Finally, of the 19 students who scored 501 or better on pre-course tests, 14 took the ITP. In this group, two (14%) increased their score by 50-55 points; two (14% of the total) increased by 40-49 points; two (14%) increased by 20-25 points; and five (36%) increased their score by 10-19 points. Clearly, for this last group of students it was more difficult for them to raise their scores significantly, seeing as their starting point was much higher than the other students.
Conclusion
The improvement exhibited by the learners in this study demonstrates that a clearly thought out, well-taught TOEFL program can produce tangible results. Specific goals were set from the beginning of the program, then a plan was formulated and implemented throughout. The instructors involved in the course were experienced and aware of the needs and goals of the students facilitating a substantial increase in TOEFL scores. The course itself came at an important time for this university and others, as there is a palpable need for intensive TOEFL programs of this type in Japan. The students, who completed the course, were highly motivated and did the assignments. These factors all contributed to the success of this program. There was no secret formula involved here: Hard work by everyone involved from the planning stage to the final ITP ensured an increase in TOEFL scores for the majority of students in the program.
In total, 60% of the students who took the final ITP test obtained a score of 50 or more above their original score. Furthermore, 93% of the students who took the final test had some increase in their TOEFL score. Conversely, seven students actually saw a decrease in their final score. This could be the result of a number of factors, including lack of attendance and participation, or incomplete homework (when the student log was examined, this was the case for some students). Some pre-advanced students had a decrease in score, which can be explained in the intrinsic difficulty in raising ones TOEFL score at this level. However, seeing as the majority of students reached the pre-determined 50-point increase deemed to be the goal of the course, it can be stated that the course was a success. Future programs of this nature will want to build upon this success and reach a higher percentage of students achieving the stated goal. The program described here could be easily replicated at any university or college in Japan with similar results. However, at present, only a few offer programs of this nature. Why is this? It is evident that there is a need for productive TOEFL instruction in Japanese universities. More needs to be done to create programs of this type so that Japanese university students will improve their English ability, and obtain the necessary scores to enter foreign universities.
Much of the success of this program can be attributed to its uniqueness. This involved using highly skilled teachers, reliable teaching methods, good materials, an interactive student-teacher worksheet (the PDF), a teacher-training session, and regularly distributing questionnaires to the students. Students in this course dramatically improved their scores because of the nature of the program: It was tailor made for Japanese university students and their specific needs in each area of the TOEFL test.
References
Abraham, P. F. (1990). The reading proficiency of non-native English speaking applicants to American undergraduate education: How is it assessed? How should it be assessed? Unpublished manuscript, Harvard University School of Education. Cambridge, MA: Harvard University Press.
Daigaku yurugasu "2006 nen mondai" [The university-shaking "Year 2006 Problem"]. (2001, January 11). Asahi Shinbun, p.19.
Honna, N. (1995). English in Japanese society: Language within language. In J.C. Maher & K. Yashiro (Eds.), Multilingual Japan, 45-62. Philadelphia: Multilingual Matters, Ltd.
Itoko, F. (2002). English language education in graduate programs: A key strategy in the era of globalization. JACET Bulletin, 35, 57-62.
Misu machi nyuugaku: Kaisho gimon daigaku to koukou Chuukyoushin ga chuukan houkoku. [Mismatched university admissions: Doubts about a solution an interim report from the Central Committee on Education]. (1999, November 2). Yomiuri Shinbun, p. 3.
Monbukagakusho. (2002). Developing a strategic plan to cultivate Japanese with English abilities. [Online] Available: www.mext.go.jp/english/news/2002/07/020901.html.
Monbukagakusho. (2001). The objectives of student exchange. [Online] Available: www.mext.go.jp/english/news/2000/05/kk000501.html.
Mulvey, B. (2001). The role and influence of Japan's university entrance exams: A reassessment. The Language Teacher, 25(7), 11-17.
Obermeier, A. (2000). Listening training for the TOEFL. Electronic version. The Language Teacher, 24(3). [Online] Available: jalt-publications.org/tlt/articles/2000/03/obermeier.
Phillips, D. (2001). Longman complete course for the TOEFL test. White Plains NY: Addison-Wesley Longman, Inc.
Sawa, T. (1999, October 18). Cramming cripples Japan. The Japan Times, p. 20.
TOEFL Test and Score Manual. (1990-91). Princeton, NJ: Educational Testing Service.
Neil Heffernan is a lecturer at Ritsumeikan University. He obtained his PhD in July, 2002. His research interests include TOEFL, CALL and sociolinguistics.
All materials on this site are copyright © by JALT and their respective authors.
For more information on JALT, visit the JALT National Website