An Investigation into the Effect of Raw Scores in Determining Grades in a Public Examination of Writing

Page No.: 
David Coniam



This article examines the effect on the grades assigned to test takers either directly through the use of raters’ raw scores, or through the use of measures obtained through multifaceted Rasch measurement (MFRM). Using data from the Hong Kong 2005 public examination of writing, the current study examines how test takers’ grades differ by comparing the results of grades from “lenient” raters against those of “severe” raters on the two systems for assigning grades–raw band scores and MFRM-derived scores. Examination of the results of a pair of raters indicates that the use of raw scores may produce widely different results from those obtained via MFRM, with test takers potentially disadvantaged by being rated by a severe rather than a lenient rater. In the Hong Kong English language public examination system from 2007 onwards, band scales will be used extensively, as indeed they already are in many Asian countries. The article therefore concludes with a call for consideration to be given to how test takers’ final grades may be derived from raw scores.