International TESOL Encourages Assessment Literacy Among Test-Makers

Page No.: 
7
Writer(s): 
Tim Murphey, Nanzan University

The "TESOL (Teachers of English to Speakers of Other Languages) Resolution on English Entrance Exams at Schools and Universities" was approved at the organization's March 17, 2000 annual business meeting (see full text below). The business meeting was held near the end of their annual convention in Vancouver, Canada, which was attended by over 7,700 teachers internationally. As original drafter of the resolution (it went through many hands and drafts subsequently), I was allowed to speak for a few minutes in support of it. Below you will first find my opening statement, and then the full text of the resolution. Following my opening remarks there were several people who expressed reservations about the resolution. However, two past TESOL presidents and representatives from other countries expressed support and said it would address issues in many contexts internationally. While the resolution is not perfect, it is an encouragement to professional educators worldwide to develop more assessment literacy that will, in turn, also have an impact on what and how teachers teach.

Support for the Resolution (opening remarks by Tim Murphey)

The purpose of this resolution is to encourage the development of what Michael Fullan calls assessment literacy in professional educators who are involved in testmaking for entrance into schools and universities. It sets no standards, does not give precise instructions, and does not blame anyone. It simply encourages continual study and development with consideration for relevant research. At present these indicate a concern for reliability and validity and multiple forms of assessment. Note that these are scientific and ethical concepts, used internationally in many fields to enhance professionalization. They are not western, nor political, nor commercial -- they are professional.

My own education concerning these matters comes from my ten-year career in Japan and my communications with others in other parts of the world. In many institutions in the world, entrance to schools and universities for many students are decided by entrance exams that are created by members of these individual institutions who have had little or no training in assessment. These exams are often the SOLE criteria for entrance and use English as one of the main components. This resolution simply seeks to encourage educators who are in the position of exam-making to become more assessment literate.

Let me now cite a few leaders in education from my own environment.

In Japan, the Nobel Prize laureate Prof. Sawa Takamitsu of Kyoto University stated (Daily Yomiuri, p. 12, Dec. 14 1999) "I believe that postwar education, which has focused on entrance examinations, has ruined society."

David Nunan, present TESOL president, made it clear the exams need changing in a Japan Times article Oct 23, 1993 (p. 3). I quote: "'Harmony between curricula and examination methods is very important to motivate students,' Nunan said. 'Even if schools promote speaking ability, for instance, their efforts will not be successful if examinations only test grammar because students focus on learning what examinations require,' he said."

In a Jan 27 article from this year (Daily Yomiuri, p. 2), it says that the present Japanese minister of education Nakasone (and I quote) "has set up a private advisory panel to investigate why Japanese, who study English in middle school, high school, and at university--10 years in all--nevertheless cannot communicate well in the language."

The testing specialist J.D. Brown has pointed out that researchers in Japan also use reliability and validity concepts in their regular scientific research. So these are not foreign concepts in Japan or to Japanese culture. They have just not been traditionally part of the entrance exam picture. I quote from one interview with J.D. Brown published in JALT''s The Language Teacher in March of 1998, p. 26 in which he says "Why is it that Japan has 300 exams or more? These exams are being made by people who don't know what they're doing, who say they don't know what they're doing. They are doing the best they can, but ultimately, they don't know what they are doing. They are preparing tests that are haphazard and of unknown reliability and validity. The sad thing is that these tests are then used to make very, very important decisions about peoples' lives. All of this wouldn't bother me so much if the people making the tests were looking at them in an effort to improve them."

TESOL Resolution on English Entrance Exams at Schools and Universities

Whereas

Assessment practices play an important role in access to educational and employment opportunities and

Whereas

When such exams are not controlled for validity and reliability and are used as the sole criteria for entrance into educational programs, they may not accurately reflect students' English language abilities; and

Whereas

Professional responsibility requires attention to these issues; therefore, be it

RESOLVED,

That the membership of TESOL recommend that the Board of Directors address the issues of valid and reliable testing and the use of multiple forms of evaluation for purposes of entrance to schools and universities, by

a) Requesting that writers of entrance exams make available to examinees and independent researchers evidence that measures are being taken to evaluate and improve the reliability and validity of their exams;

b) Lobbying for more than one form of entrance evaluation (e.g., tests, interviews, essays, recommendations, projects, school transcripts) that can be developed and used when possible;

c) Educating TESOL members regarding these issues through print and electronic media;

d) Facilitating discussion of these issues through various means, such as panels and forums.