The pedagogic term active learning frequently appears in syllabi and textbooks used by instructors in Japan. This method was emphasized in a 2012–2015 national project entitled Improving Higher Education for Industrial Needs funded by the Ministry of Education, Culture, Sports, Science, and Technology (Mizokami, 2014). Since then, university budgets have allowed for the design and building of many active learning rooms, centers and even campuses. These developments can bewilder some foreign language teachers who for years have been conducting lessons seemingly identical to what today is defined as active learning. Active learning means to directly involve students in the learning process by having them brainstorm ideas to solve a problem, collaborate on projects in groups, and freely access information, before reporting on what they discovered to the entire class. However, change in the way midterm and final examinations are performed has lagged behind active learning methodology. The evaluation of how students actively learn continues to be tested by ubiquitous paper tests, handing-in of written reports, or at best a one-to-one conversation test.
In this issue’s Teaching Assistance, I interview Yuta Kawamura, a graduate student of English Education, who is writing a Master’s thesis related to active learning in the hospitality industry. Kawamura is particularly interested in the testing of language skills for applied purposes and assessment of tourist-oriented service skills. He has catalogued various kinds of evaluation methods utilized by human resources practitioners at tourism training institutes in Kyushu and Taiwan. He conducted interviews with trainees and trainers, and observed how evaluations were conducted. Often called upon to be a TA, assist interns in Taipei, and give lectures on this research, the master’s degree candidate plans to continue his studies in a doctoral program (see Figure 1).
David McMurray: How did you become interested in the tourism and hospitality field?
Yuta Kawamura: I’ve been paying my way through university by working part-time in restaurants for the past 6 years. I recall watching television on Sept. 7, 2013 when the announcer and news presenter Christel Takigawa caught the attention of the International Olympic Committee by slowly enunciating the honorific phrase o-mo-te-na-shi and using gestures. Everyone started talking about hospitality. Other members of her Cool Japan Team who were spearheading efforts to bring the Olympics to Tokyo spoke with passion in Rio de Janeiro in the English and French languages. Since that time, government and trade associations have increased their interest in defining what hospitality means.
DM: And what does it mean?
YK: Well, motenashi refers to giving hospitality, treatment, reception, and service. Hospitality is not unique to Japan, but it is an ability to be proud of having. Most hotels, restaurants, and shops in Japan encourage their staff to follow manuals on service and etiquette.
DM: What other developments in the tourism industry have caught your eye?
YK: The Ministry of Agriculture, Forestry, and Fisheries (2016) adopted a certification system for chefs of Japanese cuisine, washoku, to guarantee quality at overseas establishments that serve Japanese fare. And the way to make it was subsequently added to UNESCO’s Intangible Cultural Heritage list. Since then, tourism organizations have studied the importance of training activities to reach a desired level of high quality customer service (Kyodo, 2016).
DM: What is your research about?
YK: I am interested in the ways non-Japanese servers and chefs are actively trained in groups to provide hospitality and cook Japanese cuisine. To add to the vibrant debate on what motenashi and washoku really are, I designed new research questions to measure the techniques identified by the Japan Education Center for the Hotel Industry (2015) that are currently used to evaluate hospitality learning outcomes. I identified the advantages and disadvantages of the methods used to evaluate training activities. My goal is to suggest which specific assessment tools are best in determining how well a trainee in the classroom is learning and when she is ready to intern outside of Japan. I focused on establishments located in Taipei with employees who speak English and cater washoku dishes to customers utilizing motenashi techniques.
DM: Who were you able to interview in Taiwan?
YK: Participants were based in hotels and at a university in Taipei. I interviewed three Taiwanese chefs at a Japanese restaurant in a five star hotel in Taipei. At a university where all subjects were taught in English and Chinese, I was able to observe trainees for four months. In addition, information was gathered from a training organization for hotel and restaurant staff in Fukuoka.
DM: What kinds of replies did your interviewees based in Taiwan give?
YK: A Taiwanese chef at a Japanese restaurant in Taipei explained that every year his human resource division invites chefs from Tokyo to train staff. These chefs speak in English using lots of Japanese terminology. For a few hours prior to the start of kitchen work, they demonstrate the basics of how to cleanly remove scales and entrails from fish, how to prepare dashi and how to present the dish beautifully. The invited trainers don’t give any evaluation of the trainees. The head chef of the Taiwanese restaurant evaluates his own staff through observations during the course and at work while carrying out daily activities in the kitchen for customers.
DM: What kinds of measuring techniques evaluate learning outcomes?
YK: There are 14 different tests currently in use: A paper and pencil test; a language memorization test that includes recognition and recall of culinary vocabulary; written reports and essays; recognition and recall examinations; practical skills tests and hands-on testing; targeted behavioural observations; tests that are co-developed by trainer and trainee; employee discussions; customer comment cards; making video clips of structured situational interviews; unlimited time power tests; speed tests to evaluate how fast trainees can complete a project; affective self-reporting that reveals a trainee’s feelings; and the performance of secondary tasks by trainees who are asked to substitute content other than what the teacher demonstrated in class.
DM: Did your research instruments and observations match what other researchers have found?
YK: From an analysis of questionnaires, Anh Ho (2012) suggested that observation, discussion with employees, and guest comment cards were the three most important criteria for managers to use. She discovered that “one of the most important ways for managers to evaluate trainee’s reaction, learning, and behaviors on the job” while a test given after training “was rated as the least important method to evaluate learning acquired from training” (p. 35). Suzuki (2015) filmed training and discussions taking place at world-renowned Japanese restaurants and recorded a chef’s interview that supports Anh Ho’s (2012) research.
DM: What kinds of problems arise in deciding on how to test trainees in the hospitality sector?
YK: Evaluation methods for the hospitality industry must make it easy to see when a skill is being performed competently. But evaluators need a long enough period to draw valid inferences. Non-verbal behavior can also provide valuable information, but it could be open to misinterpretation. Tipping customs vary by country, but that is one form of measurement. Discussions with personnel can be negated by dishonesty, future-oriented promises, reliability, and verifiability.
DM: Which active testing do you think works best?
YK: It is difficult to answer which specific assessment tool should be applied to determine how well a trainee is actively learning, but I have designed a model that allows us to focus on which tests can measure the trainee’s affect, cognition, and skill level (see Figure 2). For customer-service training, training is primarily conducted on-the-job rather than in a classroom setting. This setting makes it difficult to use paper-based or computer-based methods such as an evaluation form or test. Evaluation should be simple enough for practitioners to utilize constantly on a daily basis. Most managers seem to observe and immediately react and intervene when trainees underperform.
DM: Well, I hope those managers might read and learn from your research.
References
Ho, A. (2012). How are hotel managers utilizing the training evaluation tools available to them? Graduate theses and dissertations, Iowa State University. Retrieved from http://lib.dr.iastate.edu/etd/12806
Japan Education Center for the Hotel Industry. (2015). The beginner’s guide to Washoku Kentei. Tokyo: The Japan Education Centre for the Hotel Industry.
Kraiger, K, Ford, J., and Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78(2), 311-328.
Kyodo. (2016, Jan. 18). Japan to set up certification program for ‘washoku’ chefs overseas. Japan Times. Retrieved from https://japantimes.co.jp/news/2016/01/18/national/japan-set-certificatio...
Ministry of Agriculture, Forestry, and Fisheries. (2016). Guidelines for certification of cooking skills for Japanese cuisine in foreign countries. Retrieved from <http://www.maff.go.jp/e/policies/market/attach/pdf/index-4.pdf>
Mizokami, S. (2014). Active learning to kyoju gakushu paradigm no tankan [Active learning and the transition of the teaching/learning paradigm]. Tokyo: Toshindo.
Suzuki, J. (2015). Wa-Shoku dream. Retrieved from http://washokudream.jp