Student Evaluation of Teachers
Stephen M. Ryan
Eichi University |
The evaluation of university teachers by their students, a well-established
procedure in many North American and European universities, will be introduced
into all Japanese universities in the near future. This fact should be of
particular concern to foreign teachers.
Culturally-determined expectations may lead Japanese students to judge
their teachers against standards that are literally "foreign"
to their native-speaker teachers. However well these teachers teach in their
own terms, they may not live up to their students' image of a " good
teacher."
Although many foreign teachers have been aware of this discrepancy for
some time, the matter is given greater urgency by new Ministry of Education
proposals to link student evaluations of teachers directly to negotiations
on employment contracts.
To illustrate the seriousness of this situation, this article will briefly
review several recent studies of what Japanese university students mean
by a "good teacher" and make some suggestions for further research.
Pressure for Evaluation
Pressure for quality control in university teaching has been building
at least since the Ministry of Education introduced its new "Guidelines
for Establishing New Universities" in 1991 (Simmons, Christensen, Yonally,
& Shiozawa, 1996). The Guidelines call for "Self-Check and Evaluation"
within universities (Monbusho, 1991). This has led many universities to
set up self-evaluation committees. In some institutions, these committees
meet once a year to report that all is well. In others though, committees
have been working hard to construct mechanisms for self-evaluation, usually
involving questionnaires to be filled out by students.
These moves were given fresh impetus by a report to the Ministry of Education
from the University Council recommending that tenure be replaced by a contract
system for all university teachers, and that one of the factors involved
in decisions about contracts should be evaluations of teachers carried out
by their students at the end of each semester ("Proposed," 1996).
The Ministry responded by saying that it intended to introduce a bill in
the next regular diet session to carry out these suggestions.
The mechanism of student evaluations was not detailed in the report but
it seems reasonable to assume that it will involve an extension of the questionnaires
already being used by self-evaluation committees.
Since students will presumably evaluate their teachers by comparing them
with their idea of an "ideal teacher," studies which seek to analyse
this concept take on a new relevance. There have been several such studies
in recent years, each conducted in different ways but all leading to the
same conclusion: Japanese students' ideas about good teachers differ from
those of students in the countries that many of their foreign teachers come
from.
"Good Teacher" Studies
Hadley and Yoshioka Hadley (1996) surveyed Japanese university students
with the question, "What is a good teacher?" Students were asked
to list attributes which best described a good teacher. The most frequent
answers were "kind," "friendly," "impartial,"
"understandable," "cheerful," "punctual,"
"fun," "enthusiastic," and "humorous." There
seems to be little here that is surprising: Surely, few teachers anywhere
would deliberately set out to be "unkind," "unfriendly,"
or "partial."
Shimizu (1995) focussed specifically on English teachers, asking university
students to say which "qualities and attributes they felt important
in their foreign and Japanese English teachers" (p.6). The most popular
responses were: "reliable," "intelligent," "easy
to get acquainted with," "not show favoritism," and "not
treat students as idiots." Again, there are few surprises here.
A recent study I was involved in (Makarova & Ryan, 1997) produced
similar results by asking Japanese university students a similar, but not
identical, question: "What, in your opinion, makes a good foreign language
teacher at a university?" The most frequent answers were: "easy
to understand," "enjoyable lessons," and "good communication."
The "Japanese-ness" of these answers only becomes apparent
when they are compared with answers to similar questions from students in
other countries. The survey I was involved in asked the same question to
students in Russia, using the Werner-Campbell back-translation method (Werner
& Campbell, 1970) to ensure, as far as possible, equivalence between
versions of the questionnaire in the two languages (Russian and Japanese).
The top three answers by the Russian students were: "knowledge of subject-matter,"
"being demanding," and "professionalism," items which
did not appear in the Japanese responses at all. Some Russian respondents,
like their Japanese counterparts, did mention "kindness," "sense
of humor," and "objectivity," but these answers were less
frequent, indicating that they were less important to the Russian students.
Clearly, expectations of a good teacher vary in some respects from culture
to culture. While teachers from outside Japan may not necessarily disagree
with the expectations of their Japanese students, they may have different
priorities in conceptualising the kind of teacher they would like to be.
A study comparing the expectations of Japanese students and their foreign
teachers in this area would clearly be enlightening. However, the difficulties
of assembling a large enough sample of foreign teachers have so far proved
insurmountable for those who conduct such research.
A Comparative Study
I have, however, been involved in a detailed study which compared the
expectations about a good university teacher held by Japanese and Australian
university students (Durham & Ryan, 1992). While the detailed findings
of such a study are relevant only to the specific cultural contexts from
which data were collected, they are, in outline, suggestive of the kinds
of differences in expectations likely to occur between foreign teachers
and their Japanese students.
Our sample consisted of science and humanities students from the two
countries, roughly 110 from each country, well-balanced in terms of gender.
Most of these students were in their second or third year at university.
We deliberately avoided including first-year students, who might not yet
be acculturated to university life. Questionnaires used the respondents'
native language, and back-translation was again used to ensure the equivalence
of Japanese and English versions. After an initial questionnaire, in which
we simply invited students to respond to the question, "What is a good
university teacher?" we took answers given frequently by students in
either country and listed them together, asking students in both countries
to rank the items on the list in order of importance. To make the task manageable,
we divided our original question into four sub-questions, which were suggested
by the responses to our original question: "What kind of personality
does a good teacher have?"; "What does a good teacher know?";
"What can a good teacher do well?"; and "How does a good
teacher treat students?"
The percentage of respondents ranking each response first was calculated
for each country. The responses and the percentages calculated are shown
in the following figures (Figures 1-4). Detailed exploration of these findings
as they apply to Japanese and Australian contexts can be found in Durham
and Ryan (1992). The intention here is to indicate the kinds of differences
that were found.
What kind of personality does a good teacher have?
Figure 1 shows that there were very few differences between Japanese
and Australian responses to this question. In fact, the only statistically
significant difference here is the Australian respondents' preference over
the Japanese for a patient teacher.
Since personality traits can be very difficult to fake, this is good
news for Australian teachers planning to work in Japanese universities or
vice versa, and suggests that we should look elsewhere for the major differences
in culturally determined expectations.
Figure 1. What kind of personality does a good
teacher have?
What does a good teacher know?
The responses in Figure 2 suggest that Japanese students are much less
concerned about the subject-mastery of their teachers than are Australian
students. Instead, they are eager to have a teacher with a wealth of knowledge
about life in general, a fund of jokes and funny stories, and wisdom in
the art of teaching.
Figure 2: What does a good teacher know?
What can a good teacher do well?
Figure 3 shows that whilst agreeing that a good teacher needs to be good
at motivating students and explaining things, the two groups differ on the
other items. For the Japanese students, "helping students to think"
and "correcting mistakes in a friendly way" are more important
than for the Australians, who are more concerned with the imparting of knowledge
than are the Japanese.
Figure 3: What can a good teacher do well?
How does a good teacher treat students?
From Figure 4 it is clear that neither group wants to have a strict teacher.
For the Australians, having an "impartial" teacher is most important,
whereas in Japan there is concern that the teacher should also listen to
students and get to know them.
Figure 4: How does a good teacher treat students?
Implications
Speculation about why these differences exist is an interesting exercise
(one in which we have indulged elsewhere: see Durham & Ryan, 1992) but
here my point is that there is ample research evidence of significant differences
in the images that Japanese and Australian students have of their ideal
teacher: the image which they presumably call on when evaluating a particular
teacher.
The implication for foreign teachers in Japanese universities is clear:
Being "the best darned teacher you know how to be" may not be
enough to get you a glowing evaluation from your students if they are using
criteria to judge you which differ significantly from your own.
What About Foreign Teachers?
The situation is further complicated by suggestions that Japanese students
may apply different standards to Japanese and non-Japanese teachers. Shimizu's
study (1995), mentioned above, gave students the chance to say whether an
attribute was important for Japanese teachers, foreign teachers, both, or
neither, and concluded:
The responses imply that Japanese students evaluate Japanese and foreign
instructors by different standards. The results suggest that foreigners
are not seen as serious teachers. Qualities such as intelligence and being
knowledgeable did not seem as important for foreign teachers as for Japanese.
(p.8)
In one of the few studies in English of teacher evaluation procedures
at Japanese universities, Redfield (1991), having administered his evaluation
questionnaire to classes taught by both Japanese and foreign teachers, concluded,
"Native English teachers with academic background and current interest
in foreign language teaching and Applied Linguistics were evaluated in a
much better light than colleagues who lacked these characteristics"
(p. 99).
Conclusion
The evaluation of teachers by their students in Japan is an area where
further research is urgently required. We need to know more about how Japanese
students evaluate their teachers and whether they do indeed apply different
standards to foreign and Japanese teachers. Faced with the prospect of job-threatening
student evaluations of their teaching in the next few years, teachers at
Japanese universities, especially foreign teachers, need to arm themselves
with as much information about their students' expectations as possible.
References
Durham, M., & Ryan, S. M.(1992). What kind of teacher
do students want? A cross-cultural comparison. Speech Communication Education,
5, 70-99.
Hadley, G., & Yoshioka Hadley, H. (1996). The culture
of learning and the good teacher in Japan: An analysis of student views.
The Language Teacher, 20(9), 53-55.
Makarova, V., & Ryan, S. M. (1997). The language teacher
through the students' looking glass and what you find there: Preliminary
results. Speech Communication Education, 10, 12 -154.
Monbusho. (1991). Shin daigaku setchi kijun. Tokyo:
Ministry of Education, Science and Culture.
Proposed teacher reviews draw fire. (1996, October 30).The
Japan Times, p. 2.
Redfield. M. (1991). Teacher evaluation: A pilot study.
Osaka Keidai Ronshu, 42 (3), 85-100.
Shimizu, K. (1995). Japanese college student attitudes
towards English teachers: A survey. The Language Teacher, 19(10),5-8.
Simmons, T. L., Christensen, T., Yonally, D., & Shiozawa,
T. (1996). Change in education: Historical and social perspectives. In G.
van Troyer, S. Cornwell, & H. Morikawa (Eds.), On JALT 95: Curriculum
and evaluation (pp. 6-12). Tokyo: Japan Association for Language Teaching.
Werner, O., & Campbell, D.T. (1970). Translating, working
through interpreters and the problem of decentering. In R. Naroll &
R. Cohen (Eds.), A handbook of method in cultural anthropology (pp.
398-420). New York: The Natural History Press.
Article copyright
© 1998 by the author.
Document URL: http://www.jalt-publications.org/tlt/files/98/sept/ryan.html
Last modified: August 31, 1998
Site maintained by TLT
Online Editor
|