It is important for teachers and researchers to be able to assess L2 learners’ proficiency through their performance. The measures of complexity, accuracy, and fluency (CAF) have been used for over 30 years in L2 research to analyze language performance (Ellis & Barkhuizen, 2005; Housen & Kuiken, 2009; Wolfe-Quintero, Inagaki, & Kim, 1998). However, there remain unanswered questions about CAF measures. For example, the length measure (number of words per syntactic unit) needs to be investigated because it has been used inconsistently: Some researchers have used it as a syntactic complexity measure, while others have used it as a fluency measure. Koizumi (2005) pointed out this discrepancy as a serious problem because the interpretation of a single measure varies depending on researchers’ orientations. In addition, the results of factor analysis across some studies (e.g., Sheppard, 2004; Tavakoli & Skehan, 2005) showed that the measures of fluency could be divided into two types: speed and disfluency. These discrepancies in the construct are key issues pertaining to the measures and need to be investigated. Moreover, although CAF measures have often been investigated in Indo-European languages, they have not been sufficiently investigated in other languages; thus, it is important to determine whether CAF measures can be applied to the Japanese language in the same way.
Accordingly, this study examined the construct validity of CAF measures in Japanese as a second language (JSL) from the following three perspectives.
- As expected, do the measures represent distinct factors of the three CAF dimensions?
- Does the length measure (the number of words per syntactic unit) have the same construct as the syntactic complexity measure does, rather than having that of the fluency measure?
- Can the speed measure (the number of words per minute) and the disfluency measure (the number of disfluency markers per minute) be explained as one construct?
To investigate these research questions, 10 general measures were calculated from the narrative production of 113 university-level students learning JSL. Their Japanese language proficiency level ranged from intermediate to advanced on the ACTFL–OPI (American Council on the Teaching of Foreign Languages–Oral Proficiency Interview). Subsequently, a factor analysis was conducted to investigate the construct validity of CAF measures. The initial solution was extracted using the principal factor method, followed by Promax rotation. A three-factor solution was adopted using the Kaiser criterion of eigenvalues greater than one.
The results of this factor analysis indicated the following.
- The validity of CAF measures was partially demonstrated in terms of syntactic complexity measures (number of clauses per Analysis of Speech unit [AS-unit], and number of subordinate clauses per AS-unit) and accuracy measures (percentage of error-free AS-units, number of errors per AS-unit, and number of errors per clause) but was not demonstrated in terms of lexical complexity measures (number of word types per 100 words, and the Guiraud index) or fluency measures (number of words per minute, and number of disfluency markers per minute).
- The length measure indicates syntactic complexity because of the high loading (.83) on the same factor with the general syntactic complexity measures.
- The speed measure and the disfluency measure did not have the same factor as one construct of the fluency, which in turn supports the findings of previous studies (Sheppard, 2004; Tavakoli & Skehan, 2005). The results of this study suggest that further research must be conducted to establish the validity of the fluency measure and the validity of the lexical complexity measure, especially for the Japanese language, which has an agglutinating morphology.
近年、複雑さ（complexity）、正確さ（accuracy）、流暢さ（fluency）の側面から言語運用を分析する指標（以下、CAF指標）が多くの研究で用いられているが、研究者によって独自の測定方法が用いられており、その妥当性は明らかではない。そこで、本研究では10種類の測定方法について、CAF指標の構成概念妥当性を検証するため、日本語学習者113名の発話データを用いて探索的因子分析を行った。
その結果、構文的複雑さ、正確さについては共通因子による妥当性が示されたが、流暢さは測定方法のタイプ（速さと非流暢さ）によって同じ構成概念を示すとは言えないことが分かった。さらに、構文的な単位に含まれる語数（長さ）については先行研究で解釈が割れていたが、分析結果では流暢さではなく構文的複雑さを示した。本研究の結果から、流暢さの指標と、日本語の語彙指標に対する検証が今後の課題として得られた。
