Informational Seminar on the Computer-Based TOEFL

Page No.: 
2
Writer(s): 
Robert L. Brock, Kokugakuin University

Starting from October 2000, the Test of English as a Foreign Language
(TOEFL) will be available in Japan as a new computer-based test, the TOEFL-CBT
(see the advertisement in TLT May 2000 pp. 31-32). Whereas the pencil and
paper test has been held only a few times a year, the computer-based test
can be taken by appointment at a designated center, at a date and time to
suit the candidate. One of the major changes is that every test will include
an essay which is included in the score. The essay can be typed or hand
written. Up to now the TOEFL essay (Test of Written English-TWE)
has been optional and scored separately. Another change is that the TOEFL-CBT
is computer-adaptive, which means that the computer program selects the
level of difficulty of the next test item according to how many of the previous
questions the candidate has answered correctly. The program is thus continually
estimating the candidate's level and refining the accuracy of its estimate.
The candidate's unofficial scores, except for the essay, are displayed immediately
on the computer screen at the end of the test.

To introduce and explain these changes, Temple University in Tokyo hosted
an informational seminar on May 20 with presentations by the Japan Language
Testing Association
(JLTA) and Educational Testing Service (ETS).
Participants were welcomed by the Dean of Temple University Japan, Professor
Richard Joslyn, who explained that Temple will be one of three testing centers
in Tokyo. About 100 people attended, the great majority of whom were Japanese.

To inform participants of the theory behind TOEFL-CBT, Professors
Randy Thrasher and Youichi Nakamura of JLTA gave a lecture in Japanese on
"Basic Concepts in Language Testing." This was supplemented by
both a booklet and slides in English. They first reviewed classical test
theory and then went on to explain how item response theory can be used
to yield sample-free calibration, where the estimate of a candidate's level
of ability does not depend on the particular items he or she was tested
on. This forms the basis of TOEFL-CBT's computeradaptive testing,
where the program selects questions suitable to the apparent level of the
candidate from a large pool of test items.

In the second part of the seminar, Gena Netten and Phil Everson of ETS
reviewed the history of the TOEFL and explained what TOEFL-CBT
now entails. With the assistance of Professor Yoshida of Sophia University,
they also discussed how to prepare students for the test.

In future computer-testing centers, candidates will sit at individual
workstations, each with volume-adjustable headphones. They first work through
a tutorial on the use of the computer. Then the test commences with the
listening section. During this part of the test the computer displays pictures
which provide a context for the questions. While most questions remain multiple
choice, some now require more than one answer. There are also new question
types where candidates have to identify items depicted in a picture on the
computer screen, select the correct order for a process, or sort items into
categories. The structure section follows the listening section and both
of them are computer-adaptive. After a five-minute break, candidates do
the non-adaptive reading section and finish the test with their essay.

The TOEFL-CBTwas introduced in some countries in 1998. A comparison
test on subjects taking both the pencil and paper test and the computer
test showed no difference in group performance. The pencil and paper test
will remain through the 20001 testing year and will continue to be administered
in three remote centers in Japan. This test will include a compulsory TWE
essay from July 2000. As the two versions of the TOEFL are concurrent
and scores remain valid for two years, the two versions of the test have
non-overlapping score ranges to differentiate them. However, results on
one version can be matched with the other using concordance tables provided
by ETS.

The room where the seminar was held will contain the computer testing
center, which has not yet been installed. While Mr. Everson demonstrated
how to answer questions on a screen projected from his laptop, participants
could not get any hands-on experience themselves. This would have been useful
-one person announced she would take the first available computer test in
order to explain the procedure to her students.

Further information on TOEFL-CBT is available from the TOEFL
website http://www.toefl.org, and sample
material, the computer tutorial, and practice tests are available on CD-ROMs
from ETS.