|
The Language Teacher | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Return to The Language Teacher Online Introduction The focus of this research is the affective factors involved in a computer-based simultaneous reading and listening process. The study included measurement of student computer experience, student opinion of technology and technology's potential, and enjoyment of the specific multimedia features used in the reading system. It was hypothesized that given a multimedia reading session or a text only experience, students would claim to enjoy the multimedia experience more. It was further suspected that students would agree that they understood more (this was operationalized as understanding story content and vocabulary) in the multimedia reading than students in a text-only environment. Computers offer a different kind of simultaneous reading and listening experience than do traditional tapes and tapescripts. Instant access to recorded text without cueing and reviewing, high quality digitized sound, and the possibilities of adding a variety of multimedia enhancements, make computers a potentially powerful medium for reading and listening. The study was designed to contribute to three kinds of knowledge about computer assisted language learning (CALL). First, it was hoped that the results would provide evidence and theoretical support for the use of sound in computer-based reading. Second, that it would support an approach to CALL research characterized by attention to the effects of discrete components of multimedia (e.g., sound, animation, video, etc.). Third, and particularly relevant to CALL practitioners, the paper should offer advice in the design and construction of computer-based reading materials. There is a fair body of research on the affective factors in computer use. Stevens (1991), Todman and Lawrence (1992), and McInerney, McInerney, and Sinclair (1994), discuss fear of computers (computer anxiety) in the case of self access labs, teacher trainees, and primary school and university students, respectively. Massoud (1991), is a good example of the examination of age, gender, and anxiety and Crable, Brodzinski, Scherer, and Jones (1994) offers a discussion of individual differences in computer anxiety. Enjoyment, on the other hand, is not as well researched. Few researchers have asked students directly if they enjoyed a specific kind of computer use and fewer still have broken down computer use into discrete elements as is done here. However, outside the field of CALL, Green (1993) asked students to rate both their enjoyment and the effectiveness of classroom activities. To some extent, the present study follows Green's model. The questionnaire used here asks students to rate their enjoyment on a Likert scale and to estimate their comprehension in percentage terms. DesignThe design of this study was fairly conventional. Japanese university students, studying at Edinburgh University, were divided into control (N=23) and experimental groups (N=20), given a short story to read on the computer for 30 minutes, then given a 20 item questionnaire. The control group read the story without sound and the experimental group were able, by clicking on the first letter of a sentence, to hear a digitally recorded version of the sentence. Students in the experimental group wore headphones. Both groups were given a short training period (10 minutes) on an unrelated text and both groups were given the same amount of time to read the story. The story appeared on screen with a large black font on a white background. The only nontextual interface features were two buttons: one to turn the page ahead and one to turn it back. The questionnaire items were also conventional. I used a 5-point Likert scale (strongly agree to strongly disagree) and had it translated into Japanese by a Japanese colleague and then checked by another Japanese teacher. I included 3 open-ended questions in the questionnaire (see Appendix 1). Questionnaire ItemsSee Appendix 2 for the complete list of questionnaire items. Three items on the questionnaire were examined with regard to enjoyment. These items asked students to agree or disagree (on a five-point scale) with:
Two other items were included to assess students' self-reported comprehension of the reading exercise. These items were:
ResultsThe following section describes the responses of students in both groups to specific items mentioned above. Appendix 1 shows response distribution and Chi Square results. Desire to use the computer again: Of control group students, 85% agreed with this statement and 83% of experimental groups students agreed (see Appendix 1). The difference between the groups is not significant. However, all students were overwhelmingly positive about the reading‹whether with sound or without. Using the computer to read was a waste of time: We see a similarly clear response in both groups with regard to this item. Of all participants, 96% disagreed or disagreed strongly with this statement, and there was no significant difference between the two groups. Generally speaking, students did not dislike the exercise but, as is seen in the item on enjoyment, seemed to discriminate between the value and the pleasure of the activity. I enjoyed using the computer to read the story: In the control group, 50% agreed with this item. In the experimental group, however, 83% agreed that they enjoyed using the computer to read the story. The difference between groups is significant. Students in experimental group clearly claimed to enjoy the exercise more than those in the control group. Story plot and content comprehension: Students were asked to rate their comprehension in percentage terms. It was decided that a satisfactory cut-off could be made at 50% claimed comprehension (i.e., between those who claimed more than or less than 50% comprehension). The data suggests that the control group students thought they understood more than 50% and the control group less than 50%. Using a Chi-square distribution, the differences were statistically significant Vocabulary comprehension: Using the same method of combining categories, there was no significant difference between groups with regard to reported vocabulary comprehension. If the two groups are combined, 49% said they understood less than half and 51% reported that they understood more than half of the vocabulary. Previous computer use: Only 26% of all the students reported that they had used a computer often or very often, while 40% of the students indicated little or no previous computer experience. DiscussionEnjoyment: It was immediately obvious from informal observations that those involved in listening were concentrating well. On several occasions, other teachers who happened to be in the same room while the experiment took place noted how absorbed the experimental group students were. On other occasions, when the experimental group students were interrupted and told to stop, they were visibly surprised, as if startled from deep concentration. The questionnaire results showed that the experimental group enjoyed the reading activity more than the control group. However, there are several other factors worth considering including the possibility of a "novelty effect" and responses to the open-ended questionnaire items. Novelty effect: As other researchers have noted, computer-based materials are prone to producing a "novelty effect" (e.g., Krendl and Brohier 1992). We would expect, because of the subjects' relative inexperience, that a "novelty effect" affected the results to some degree. However, the strength of the response to this question cannot be accounted for fully by the newness of the exercise. One would expect, for example, that if there were a novelty effect influencing the response to item 9, then there might be a corresponding effect positively influencing perceived comprehension (items 19 and 20 on the questionnaire). In fact, it was the control group who reported significantly greater plot comprehension (with the 50% division) than the experimental group. Enjoyment of the activity does not necessarily correspond to whether or not students felt they understood more than 50% of the story's plot. In fact using all response categories, the Pearson correlation showed no significant correlation between items 9 and 19, suggesting that students were not judging their enjoyment by how much they understood the text. In other words, it is possible for students to enjoy this kind of reading activity without feeling that they understood it well. It may be possible to claim that the addition of the audio context made the reading more enjoyable for the experimental group. It may also be true, therefore, that reports of enjoyment are not entirely due to a novelty effect. Open ended items: Having said this, the notion of a positive novelty effect on item 9 results should not be discarded entirely. Open-ended responses from the control group produced some positive comments about the reading exercise. Three students in the control group noted that they would not change the exercise at all and that they enjoyed it immensely. It is difficult to imagine even the most enthusiastic proponent of CALL thinking that reading a story, consisting of nothing more than text on a computer screen, was "fine as it is." Interestingly, not a single control group student commented that they could have just as easily read the story in a book (which seems obvious to the researcher) and only one student noted that reading in books is easier than reading on a computer. There was apparently either some novelty effect in the control group or several enthusiastic technophiles among their number. Students did enjoy reading with sound‹despite the fact that many found it difficult. Student responses in the open-ended questionnaire from the experimental group included the following observations:
The positive comments here confirm that the activity was generally well-liked. The negative comments, however, suggest that the computer screen is still not the ideal medium for reading and that students were well aware of the "limited" multimedia features available. It is interesting, as an aside, that despite the group of students being relatively inexperienced, they still had fairly high expectations of what the computer might offer them. Comprehension (plot and vocabulary): We assume that despite the collapsing of the comprehension responses into two categories, the results still give some indication of students' perceived understanding. Therefore, since the control group indicated a higher level of plot comprehension than did the experimental group there appear to be at least two implications:
Difficulty (plot and vocabulary): The data from items 5 and 6 (difficulty of plot and vocabulary) suggests that the addition of sound to the reading exercise did not make the words or plot easier for the experimental group. Indeed both groups considered the plot and vocabulary to be difficult. This goes against the researcher's intuition and is possibly explained by the fact that we did not predict the large percentage of novice computer users in the questionnaire sample. Indeed it was expected that most students would have at least passing familiarity with a mouse and that the training period would be simply a re-familiarization and a means of initiating students to the specific way in which the mouse would be used (pointing at the beginning of sentences and looking for linked words). It was optimistic to expect complete beginners to master the use of the mouse in the short training period provided. Perhaps more importantly, however, the correlation between experience and difficulty in the experimental group supports what was suspected after analysis of the post-test sample‹that in novice users the addition of sound produces an adverse effect on the learner in terms of their perception of context/lexical difficulty of the reading text. The mouse could have distracted students from an overall, story-level comprehension. By focusing on sentence-by-sentence listening, students may have been unwittingly directed to sentence-level comprehension (or indeed simply to trying to comprehend the interface) rather than focusing on more global processing. The Preferred Medium: Despite the comment by one student in the control group that he preferred the computer to a book as a reading medium, a significant number of students in both groups combined did not disagree that traditional reading materials are preferable. Since, on the whole, students enjoyed using the computer, this seems somewhat of a contradiction. However, most computer users would agree that reading on a computer is tiring for the eyes and not as pleasant as reading from paper. Several students made comments to this effect in the open-ended part of the questionnaire, suggesting that this factor contributed to the notion that for these students, reading in "normal" book is preferable Conclusion The relationship between enjoyment and effectiveness is not clearly understood. It is not known, for example, whether certain methods are effective because they are enjoyable or enjoyable because they are effective. Green (1993, p. 8) argues that the relationship between enjoyableness and effectiveness is "circular and mutually reinforcing...that enjoyableness enhances effectiveness, and that the belief that something is effective tends to make it more enjoyable than it would be otherwise." Furthermore, this is a slightly "rough and ready" approach. More could be done to clarify the meaning of "enjoyment" by eliciting more information from students (e.g., through interviews). Nevertheless, this research can be a good starting point for further study. One can also hope that enjoyment leads to repeated use and more extensive reading. With regard to computer-based materials design, if the required computer expertise increases, it is likely that the cognitive load on the user increases as well. Comprehension may suffer from computer complexity. At the very least, reported comprehension is lower. Moreover, it is possible that student-controlled, sentence-by-sentence listening distracts readers from overall comprehension. We would do well to compare different loci of control over listening (i.e., student control vs. computer control) and different interfaces for controlling listening materials. Multimedia research is often plagued by the large number of "routes" a user may take in the program. However, with a careful experimental design, the computer is a reliable research tool‹given the same commands, the computer will respond exactly the same way every time. The researcher can utilize this advantage by examining the key elements of multimedia one element at a time. We need to find both the best way of using each element and the best way of combining them. ReferencesBlattner, M. (1993). Sound in the multimedia interface. In H. Maurer (Ed.), International Multimedia and Hypermedia Annual Proceedings of Ed-Media '93 (pp. 76-82). World Conference on Educational Multimedia and Hypermedia in Orlando, FL. Crable, E., Brodzinsky, J., Scherer, R., & Jones, P. (1994). The impact of cognitive appraisal, locus of control, and level of exposure on the computer anxiety of novice computer users. Journal of Educational Computing Research, 10(4), 329-340. Green, J. (1993). Student attitudes toward communicative and non-communicative activities: Do enjoyment and effectiveness go together? The Modern Language Journal, 77(1), 1-10. Krendl, K. A., & Brohier, M. (1992). Student responses to computers: A longitudinal study. Journal of Educational Computing Research, 8(2), 215-227. Massoud, S. (1991). Computer attitudes and computer knowledge of adult students. Journal of Educational Computing Research, 7(3), 269-291. McInerney, V., McInnerney, D., & Sinclair, K. (1994). Student teachers, computer anxiety and computer experience. Journal of Educational Computing Research, 11(1), 27-50. Stevens, V. (1991). A study of student attitudes toward CALL in a self-access student resource centre. System, 19(3), 289-29. Todman, J., & Lawrence, H. (1992). Computer anxiety in primary school children and university students. British Educational Research Journal, 18(1), 63-72. Appendix 1 For all items, students were given a choice using a 5-point Likert scale. Where necessary and appropriate (for statistical purposes) the data were collapsed to 2 or 3 categories. In many cases, the expected frequencies in each cell in the resulting contingency tables were too low to allow for the use of all categories. 1. "I would like to try reading on the computer again"Agreement with Statement
Chi Square Results: "other" and "agree"
2. "Using the computer to read was a waste of time"Disagreement with Statement
Chi Square Results: "other" and "disagree"
3. "I enjoyed using the computer to read the story"Agreement with Statement
Chi Square Results: "other" and "agree"
4. Story plot and content comprehensionComprehension
Chi Square Results: 0-50% & 50-100%
5. Vocabulary comprehensionComprehension
Chi-SquareResults: 0-50% & 50-100%
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
