136 284 THS Date 0-7639 This is to certify that the thesis entitled PAPER-BASED AND COMPUTER-BASED VOCABULARY INSTRUCTION: A COMPARATIVE STUDY presented by Alina C. Pajtek has been accepted towards fulfillment of the requirements for M . A . degree in TESOL 0m 7;, my Major professor August 221 2002 MS U is an Affirmative Action/Equal Opportunity Institution LIBRARY Michigan State University PLACE IN RETURN BOX to remove this checkout from your record. To AVOID FINES return on or before date due. MAY BE RECALLED with earlier due date if requested. DATE DUE | DATE DUE - DATE DUE 1351 “it nn‘nnfl . 6 ‘0 4 JUN-H iiiiizi 6/01 cz/CIRC/DatODuep65-p. 15 COMPUTER-BASED VERSUS PAPER-BASED INSTRUCTION: EFFECTIVENESS AND MOTIVATION By Alina C. Pajtek A THESIS Submitted to Michigan State University In partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Linguistics and Germanic, Slavic, Asian and African Languages 2002 ABSTRACT COMPUTER-BASED VERSUS PAPER-BASED INSTRUCTION: EFFECTIVENESS AND MOTIVATION By Alina C. Pajtek The aim of the study was to analyze differences between computer-based versus paper-based activities in terms of effectiveness and motivation. The main research questions were: (a) What are the differences between computer and paper-based vocabulary instruction in terms of effectiveness measured by “guessing from context” exercises? (b) Are there differences in participants’ motivation between paper-based and computer-based vocabulary learning? The study was both qualitative and quantitative; the qualitative part was carried out through questionnaires and focused mainly on participants’ responses to the means of instruction and on their motivation and confidence in the usefulness of computer-based activities in the language classroom. The quantitative data were gathered through a pretest-posttest design measuring both the participants’ improvement in learning the target vocabulary from pretest to posttest, and the differences between paper-based and computer-based vocabulary instruction. The experimental results suggest that more complex computer-based vocabulary activities need to be used to be able to state that computer-generated activities represent a learning tool that is statistically more effective than paper-based activities. ACKNOWLEDGMENTS I would like to express my appreciation and gratitude to Professor Debra M. Hardison, my thesis supervisor, and to Professor Charlene Polio and to Dennie Hoopingamer, members of my thesis committee. I am also grateful to Anne Todd for drawing the pictures used in this study and for her support and patience. Many thanks to my husband for encouraging me to write this thesis and for his constant support and understanding. iii TABLE OF CONTENTS LIST OF TABLES ............................................................................. v KEY TO ABBREVIATIONS ................................................................ vi CHAPTER 1 INTRODUCTION .............................................................................. 1 CHAPTER 2 STUDY DESIGN .............................................................................. 13 Experiment 1 .............................................................................. 14 Method ............................................................................ 14 Results and Discussion .......................................................... 18 Experiment 2 ............................................................................... 20 Method ............................................................................. 20 Results and Discussion .......................................................... 21 CHAPTER 3: CONCLUSION .............................................................. 28 REFERENCES .................................................................................. 29 APPENDICES .................................................................................. 33 Appendix A: Handout Pretest ............................................................ 33 Appendix B: Handout Treatment ......................................................... 35 Appendix C: Handout Posttest ........................................................... 38 Appendix D: Questionnaire ............................................................... 39 iv LIST OF TABLES Table 1: Tests of Fixed Effects .......................................................... page 19 Table 2: Question number 1 ................................................................... 21 Table 3: Question number 2 ................................................................... 22 Table 4: Question number 3 .................................................................. 22 Table 5: Question number 4 ................................................................... 23 Table 6: Question number 5 ................................................................... 23 Table 7: Question number 6 ................................................................... 23 Table 8: Question number 7 ................................................................... 24 Table 9: Question number 8 ................................................................... 24 Table 10: Question number 9 .................................................................. 25 Table 11: Question number 10 ................................................................ 25 Table 12: Question number 11 ................................................................ 25 Table 13: Question number 12 ................................................................ 26 KEY TO ABBREVIATIONS CAI= Computer Assisted Instruction CALL= Computer-Assisted Language Learning CAT= Computer Adaptive Testing CAVOCA= Computer Assisted Vocabulary Acquisition CMC= Computer Mediated Communication ESL= English as a Second Language Gp= Experimental Group administered the paper-based activities Gc= Experimental Group administered the computer-based activities ICT= Information and Communication Technologies vi Chapter One Introduction Vocabulary learning theories have suggested that elaboration and motivation play an important role in vocabulary learning processes; however, the empirical literature on concepts of cognition and on motivation and need in this area has not progressed substantially (Laufer & Hulstijn, 2001). Ellis (1994a, 1994b) argued that some aspects of new words (e. g., pronunciation, stress) are learned implicitly as a result of frequent exposure, but that the words’ meanings are learned explicitly. Computer-Assisted Language Learning (CALL) can provide the tools necessary both for implicit vocabulary learning (frequent exposure), and for explicit vocabulary learning by offering access to online dictionaries and other learning software (Zhao, 2002). Therefore, the present study was motivated by the following questions: a) Are there differences in second language (L2) learners’ motivation between computer-based and paper-based vocabulary learning approaches? b) What are the differences between computer— and paper-based vocabulary instructional approaches in terms of effectiveness as measured by a pretest-posttest design? Different studies have shown that some tasks better facilitate incidental vocabulary learning than others do. For example, Hulstijn (1992) stated that learners better retain words when they are asked to guess their meaning from context followed by a multiple choice exercise than if they are asked to read a passage and are given a list of synonyms of the target words. The remark made by Hulstijn motivated my choice of the guessing-from-context exercise used in this study followed by a fill-in—the-gaps exercise to check participants’ understanding of the target words, and expose them again to the target vocabulary by giving them the chance to utilize the words in context. Newton (1995) argued that higher retention was achieved with unknown vocabulary items that were negotiated for meaning, than the ones that were not, even by passive participants to negotiation. The treatment part of the present study also included a section in which participants, after trying to guess the words’ meaning from context, needed to compare their answers in groups and negotiate the meaning of the target words. Kojic-Sabo and Lightbown (1999) argued that the level of success in language learning is strongly influenced by the number of learning strategies. Since all clusters of students who had higher scores on the tests that were administered had used both time and learner independence strategies, and since students who did not use them scored the lowest, these two seem to be the most important strategies that favor vocabulary learning. The outcomes of the study proposed by Kojic-Sabo and Lightbown (1999) emphasized that students of English as a second language (ESL) have more chances to engage in independent learning activities that facilitate vocabulary learning. Even though ESL students have more advantages regarding some leaming strategies, there are few differences between the way in which the two groups approach vocabulary learning. In both learning situations, dictionary use proved to be very helpful in the vocabulary learning process. The independence learner strategy for vocabulary learning proved to be one of the best strategies used in the study, showing that it is crucial for learners to become responsible for their own learning. One of the activities considered part of the independent learner strategy was “encountering and practicing new lexical items in outside classroom activities” and the learners preferred this for such activities as sources for vocabulary learning (Kojic-Sabo & Lightbown, 1999, p. 182.). Some learners also prefer to use CALL software at home compared to using it in a classroom situation (Pajtek, 2001). Used either at home or in class, CALL software and ICT (Information and Communication Technologies) in general can make a great contribution to the development of independent learner strategies and to the increase in students’ responsibility for their own learning. Second language learning processes are strongly influenced by motivation, which can play a crucial role in language learners’ success or failure. Motivation determines the choice of certain learning strategies, the amount of interaction with speakers of the target language, the performance in formal learning settings, and the level of overall learning proficiency (Oxford & Shearin, 1994). Although motivation for learning an L2 and implicitly for learning L2 vocabulary, is influenced by the learners’ educational background, past education-related success or failure and by socio-cultural factors, less is known about the mechanisms through which motivation influences attention and cognitive processes (Crookes & Schmidt, 1991). External incentives motivate students to learn more vocabulary words and of a higher degree of difficulty (Gardner & MacIntyre, 1991). Because of the associations some learners make between computer games and CALL (Pajtek, 2001), the use of computers in language learning can give students a feeling of competition or of playing while actually learning, and therefore can be considered a possible external incentive. Motivation became a research topic in itself after Gardner and Lambert (1972) classified motivation into integrative and instrumental. Integrative motivation is associated with a non-pragmatic interest in the target language and culture, the desire to communicate with speakers of the target language, and to identify oneself with the target cultural community. The integrative type of motivation is associated with interest in foreign languages, attitudes toward learning languages or towards the learning situation (Gardner, 1982). Instrumental motivation was associated with learning a foreign or a second language in order to have access to some pragmatic advantages of language proficiency, such as a good job or a higher salary (deyei, 1990). Most researchers supported the model of integrative-instrumental motivation and the validity of the theory, but some investigators argued that the model was simplistic and that it required further analysis. An interesting observation Domyei (1990) made in his study carried out in Hungary, was that up to the intermediate level, students’ motivation is of an instrumental type and after the intermediate level, more integrative motivation is needed to progress in proficiency. This observation strengthens the traditional view that integrative motivation is encountered with a higher level of language proficiency (Gardner, 1985; Krashen, 1981) Clement, Domyei, and Noels (1994) looked at how self-confidence and learning environments influence motivation for learning foreign languages. They proposed a “tricomponent motivational complex”, which included instrumental knowledge orientation, self-confidence and learning environment. Both self-confidence and the learning environment proved to influence language learners’ motivation. Self-confidence influences motivation directly and indirectly through students’ attitudes towards learning, and group cohesion results in a positive perception of the learning environment. These three factors (instrumental knowledge orientation, self-confidence, and learning environment) can influence both integrative and instrumental motivation. However, according to the results of the study, self-confidence did not seem to be associated with evaluation of the learning environment. Domyei (1994) proposed a framework for L2 motivation, which consisted of three levels: the language level, the learner level, and the learning situation level. The first level, the language level, is connected to the students’ attitudes towards the target language speaking community, language goals and expectations, and their attitudes towards the target culture. The second level, the learner level, is identified with students’ personal attitudes towards language learning. The third level is related to the learning situation level and group cohesion. One of the levels that make up the framework for L2 motivation, as mentioned above, is the learning situation level. Thus, we can conclude that the learning environment plays an important role in L2 learners’ motivation. One of the objectives of the present study was to analyze the way in which CALL environments influence ESL students’ motivation during the language learning process and to compare it to their motivation when learning vocabulary through a paper-based approach. When using educational software, ESL students have a higher level of motivation than when being taught through the paper-based approach because they associate learning with playing computer games (Pajtek, 2001). Dempsey, Lucassen, Haynes and Casey (1996) analyzed the use of computer games for educational purposes. Even though few studies have been done in this field contributing to some doubt as to the effectiveness of I! educational games, computer gaming is definitely a promising area of education that requires further investigation. Some of the areas of interest observed by Dempsey et al. were learning styles, the subjects’ preference of certain features of computer games, and the strategies they used to carry out the given tasks. The results of the study showed that in order to have educational value, a game should display characteristics such as speed, difficulty, clear and concise instructions, timing, and help functions. The most common learning strategy used by players was trial and error, which leads us to the conclusion that successful learning environments should promote “discovery learning” (p. 90). Both aesthetic features and participants’ confidence played an important role in maintaining interest in the game and continuing it. According to the study, another important quality of computer games was that they develop in participants problem solving and logical inference skills. In the last century, technological advances have had a tremendous influence on everyday life. These changes have had subsequent effects on learning in general, and on second language learning in particular. Technological resources have been incorporated as valid tools in the leaming/teaching methodology. The continuous and very rapid expansion of technological capabilities, especially in recent years, have led to the adoption of various technological tools in the second language classroom without an appropriate assessment of the pedagogical value of such tools. A tool that is an extraordinary advance from a technological point of view could have little pedagogical value to an untrained user (e. g. sound spectrograph). Salaberry (2001) pointed out that “most new technologies (radio, television, VCR, computers) may be revolutionary in the overall context of human interaction; however, it is not clear that they have achieved equal degrees of pedagogical benefit in the realm of second language teaching” (p. 50). Several authors caution teachers about the use of new technological tools in the L2 classroom without a proper assessment of the pedagogical value of such tools. They also emphasize that ignoring the viable technological capabilities that could be beneficial to the teaching/learning process would reduce the teaching/learning potential of the second language class. L Through technology, distance learning education has already been used for about seventy years, in the beginning through radio, and later through computers. Both researchers and practitioners are aware of its educational advantages and disadvantages. Cabarga (1937) noted that one of the major problems that distance learning raised was that of attendance. One of the most promising technological advances for the teaching/learning process in the second language class is without doubt the development and spread of personal computers and the World Wide Web. Although there is still a debate on the pedagogical relevance of Computer Assisted Instruction (CAI) in general, and CALL in particular, there are specific advantages that CALL could provide, in teaching, learning, and testing procedures used in the L2 classroom. One of the most widely accepted benefits of CA1 is its capacity of instant feedback, which allows the learner to immediately review his/her mistakes, facilitating correct learning. Another useful aspect of the capacity for immediate response from computers is the Computer Adaptive Testing (CAT) method, which proves to be beneficial by reducing testing times, the need for high numbers of tests, the boredom and frustration of testing, etc. Some of the disadvantages of CA1 mentioned by Salaberry (2001) are the high cost of computers and computer programs, the lack of technical support personnel, and the negative attitudes of some teachers towards the integration of CA1 in the curriculum. The first CALL programs were drill and practice exercises, which did not differ in principle fi'om paper—based exercises. The benefits that the computer-based environment can offer regarding drill and practice exercises are the use of interesting and stimulating designs and methods (e.g. matching exercises), the possibility of self-pacing (students can choose the type of exercises that they are interested in and the level of difficulty of those exercises), and the immediate feedback capability of computer applications. Another valuable aspect of CALL is the use of multimedia applications that expand and enhance the learning experience that is considered to be positively influential to the motivation of second language learners. The major criticism that language laboratories have elicited is the lack of feedback. Computers with their immediate feedback capability and the ease of use of computer multimedia applications could assume the role of language laboratories. Intelligent CALL is a promising way of using computers in learning. For instance, there are computer programs that use a natural language processor that is capable of analyzing students’ responses instead of just comparing them with a correct stored version. Pederson (1986) suggested that one of the greatest advantages of CALL is that it allows teachers to carry out certain pedagogical tasks that might be difficult to implement in other learning environments. Being able to deliver the content of a lesson in different ways (e.g., through color, graphics, and feedback) could be a crucial improvement in the learning environment. Computer Mediated Communication (CMC) is also a very positive aspect of computer use in second language learning. CMC not only gives learners the chance to use learning materials (many of which are free) via the World Wide Web, it also facilitates contact between L2 learners and native speakers of the target language or other L2 learners from around the world. Computers have also been used in assessment and testing. Dunkel (1990) argued that some of the benefits that CAT provide are a shorter testing time, less boredom and frustration, immediate feedback, and self-pacing. Groot (2000) presented a computer assisted word acquisition program that uses vocabulary items of interest to the target population. The purpose was to establish the efficiency of the program; therefore, he compared it with vocabulary learning methods in which bilingual vocabulary lists were used. The program took the L2 learners through the mental processing stages that L1 learners go through when learning vocabulary, even though these two populations do not have exactly the same characteristics in terms of language learning. The learning method through CAVOCA (Computer Assisted Vocabulary Acquisition) proved to be more time consuming and difficult than paired associate learning methods because participants were not acquainted with it. The aim of the four experiments was to determine which method was more efficient in terms of long-term retention; therefore, the scores on the tests administered immediately after treatment were not analyzed and taken into account. On the delayed tests, the experimental group scored higher on the cloze tests and the control group scored higher on the receptive tests. Groot (2000) argued that the results of the cloze tests showed that better retention was directly related to the depth of vocabulary processing; the higher scores on the delayed receptive tests by the control group were attributed to the fact that the subjects were advanced language learners and had already developed meta- cognitive strategies that helped them acquire new vocabulary out of context, through bilingual vocabulary lists. Groot (2000) concluded that a combination of the two methods is “most advisable”, since there are too many variables to take into account when deciding upon the best method of vocabulary learning. The variables identified in the study were: “age, subjects’ cognitive level, quantity and quality of the rehearsal practice, the intensity of processing, L1-L2 equivalence of the words to be learnt, etc.” (p.22.) ICT (Information and Communication Technologies) has great potential to drastically improve foreign and second language learning in general, and to facilitate second language vocabulary acquisition. Zhao (2002) proposed that ICT should be regarded as a “tool that enables the conditions for optimal language learning” (p. 7) and enumerates the roles technology can play in language education. According to him, technology cannot only “deliver content” but also “manipulate content”(i.e., “adjust it to their own linguistic, cultural, and cognitive levels”, p. 8). ICT can give language students access to meaningful and authentic communication, and can be adapted according to learners’ language levels, leaming objectives, and learning styles. He also mentioned feedback as one of the language learning areas that can benefit fiom ICT, and classified it into “immediate feedback”, “relevant feedback”, and “cumulative feedback” (p. 10). 10 Zhao (2002) pointed out that the areas of technology that can influence language education the most in the future are “multimedia computing, the Internet, especially the Web, and speech synthesis and recognition” (p. 16). In a qualitative and quantitative pilot study on differences between paper-based and computer-based vocabulary instruction, Pajtek (2001) argued that computer-based vocabulary exercises were not more effective than their paper-based equivalent, but that the level of motivation was significantly higher for computer-based exercises. In conclusion, CALL can influence students’ performance and motivation when learning a foreign or second language in the following ways. It can increase students’ instrumental and integrative motivation since learning environments can influence students’ motivation during the language learning process; improve learner independence strategies when learning second language vocabulary; deliver and manipulate content; give language learners access to meaningful communication and therefore, favor incidental vocabulary learning; be adapted according to learners’ language levels, learning objectives, and learning styles; give learners immediate feedback, relevant feedback, and cumulative feedback. The purpose of the present study was to demonstrate that CALL could achieve a higher motivational value and pedagogical benefit than the paper-based instruction in 11 terms of effectiveness. Previous research studies argued that even though “new” technologies have had a crucial influence on human interaction, they have not been shown to be as important in the field of second language teaching. To recap, the research questions were: 0 Are there differences in learners’ motivation between paper-based and computer-based vocabulary learning? 0 What are the differences between computer and paper-based vocabulary instruction in terms of effectiveness as measured by a pretest-posttest design? My hypotheses are that computer-based vocabulary activities will prove to be more effective than paper-based vocabulary activities, and that participants will show a higher level of motivation for the computer-based activities. 12 Chapter Two Study Design A total of 30 participants in this study were divided into two groups: one was given paper-based instruction, and the other one computer-based instruction. The participants were learners of English as a Second Language in an Intensive English Program at an American university. A quantitative and a qualitative experiment were conducted in this study. Even though the two experiments shared participants, the materials, procedure, results, and analyses were different. The quantitative experiment (Experiment 1) involved a pretest-posttest design administered to both the computer- and paper-based groups. The content of the activities used in the pretest, treatment and posttest sections of the paper-based and the computer- based approaches was exactly the same. The qualitative experiment (Experiment 2) consisted of a questionnaire administered after the posttest. There were sixteen questions that were exactly the same both for Gp (the group of participants administered the paper-based activities) and Go (the group of participants administered the computer-based activities). The questionnaire was paper-based both for Gc and Gp since the purpose of the qualitative study was not to compare the effectiveness of the two means of instruction, but to analyze the participants’ answers to a variety of questions referring to the two means of instruction: paper-based and computer-based. l3 Experiment 1 Method Participants There were 30 participants in this study, of which 15 were administered computer-based (Gc) activities and 15 paper-based (Gp) activities. In Go, most of the participants were Asian (Korean, Japanese, Taiwanese) plus one Mexican and one Arabian. The ratio of male to female participants was 1:1. They were enrolled in the pre-intermediate level of an Intensive English Program at a large university in the United States. Some of the participants were undergraduate students in their home countries; others had already graduated from college. About half of them studied English because they wanted to increase their chances of getting a better job when they returned to their countries; the other half needed to meet the language requirements to enter an undergraduate or graduate program at an American university. None of them was considering immigrating to the US. All participants were continuously exposed to the L2 outside the classroom because most of them had American roommates or lived in dormitories with students from all over the world. In Gp, the participants’ backgrounds and the language goals were the same. The participants in the two groups came from two different sections of the same general proficiency level in the same language program. At the time the study was conducted, I was not the instructor of the participants in the study. Each group was homogenous in terms of target language level, being made up of mostly pre-intermediate learners of English as a second language. None of the participants reported any vision or hearing problems. 14 Materials The vocabulary used in my study was exactly the same for Gc and Gp to permit an analysis of differences between these modes of instruction; therefore, the discussion below on the choice of the target words refers both to the computer-based and the paper- based approach. The target words I used in my study are fi'om the same semantic field - physical description, specifically expressing weight. I chose nine words: tubby, chubby, overweight, obese, large, flabby, scrawny, anorexic and underweight, all of which were presented in context. I chose these words because they are useful in describing physical appearance when referring to body weight, and because it was very unlikely that the participants would be able to either recognize or utilize them in context prior to instruction. The participants hadn’t been exposed to the words before in a formal setting at the language center where the study took place. The fact that the target vocabulary was appropriate for the participants’ level was also confirmed by a pilot study undertaken before the current study (Pajtek, 2001). The pilot study was conducted with learners of the same level and in the same program as the participants in the current study. The participants’ backgrounds in both studies were similar. A pretest consisting of a guessing-from-context exercise and a matching exercise, both targeting the same vocabulary as the current study had been administered in the pilot study. Even though those learners had recognized the target vocabulary, they had not been able to differentiate between semantically similar words. This finding determined my choice of the same target words in the current study. 15 The pretest consisted of a guessing-from-context exercise (see Appendix A). The materials used in the treatment part consisted of two pictures, one of a fat person and the other of a thin person, a handout with nine target words and their definitions and a cloze exercise (see Appendix B). The pretest and posttest were almost identical (see Appendix C). I tried to make sure that in both the pretest and the posttest “guessing-from-context” activities all sentences were equally explicit. I considered it important to use very similar exercises in the pretest and the posttest to ensure that some contexts or definitions were not more supportive than others. Moreover, participants were not given feedback after the pretest, so they did not know the meaning of the target words unless they checked them in a dictionary after the data collection session. They were not allowed to have dictionaries during the pretest, treatment or posttest. Procedure Computer-based Group. Participants were seated at computers and instructed to go to a web address indicated by me, read nine sentences, guess the meaning of the underlined words from context, and then type a synonym or the definition of the target words on the keyboard. Students were given both verbal and written directions, the latter ones being on the same webpage with the exercises, and were encouraged to write as much as possible when defining the target words. Participants were given 15 minutes for this task. They were not allowed to consult dictionaries or each other at any point during the pretest and were not given feedback at the end of the pretest. Also, their answers were anonymous. They typed what they thought the words meant and after that they had to click on the “submit” button. I used a program that stored all data collected, so I did not have to print their answers. 16 The treatment was administered one week after the pretest. In the treatment part, participants were told to go to a web site where there were two pictures — one of a fat person and the other of a thin person, and the target words above the pictures (see Appendix B). Participants compared their answers with each other after typing the target words under the picture that was closest in meaning with the target words. Next, they were asked to work in pairs (negotiation) and discuss the differences in meaning between the words under each of the two pictures, after which I checked the answers. I then took over control of the participants’ computers, and as I was explaining a target word, I would move the cursor over it on my computer and its definition would pop up on their screens. Similarly, learners “played with/ practiced” the target words on their computers by moving the cursor over the target words and seeing the words’ definitions pop up. The last activity in the treatment part was a cloze exercise. Participants had to do a cloze exercise with the target words. In the end, Gc had the chance to check their answers and the incorrect ones were automatically deleted. Participants were given feedback during treatment. The posttest was identical to the pretest in terms of procedure. It was administered one week after the treatment. Paper-based Group (Gp). Gp was administered the paper version of the pretest. They were also given 15 minutes and asked to read the sentences, and guess and write down the meaning of the underlined words or their synonyms. Their answers were anonymous. Students were given both verbal and written directions, the latter ones being on the same page with the exercises, and were encouraged to write as much as possible when defining the target words. Learners were not allowed to consult dictionaries or each 17 other at any point during the pretest and were not given feedback at the end of the pretest. After they finished writing their answers, I collected their handouts. The treatment was administered one week after the pretest. Participants were asked to arrange the target words into two columns — either under the picture representing a fat person, or under the picture representing a thin person. Participants checked their answers with each other (negotiation) and then I elicited the answers. I reinforced the meaning of the target words by pointing to the words’ definitions on an overhead projector transparency. Later, participants received a handout of the same definitions and were given a few minutes to read them. Next, Gp was given a fill-in-the-gaps exercise with the target words at the top of the handout. The differences between the cloze exercise the two groups were given is that Go had the chance to get a hint fi'om the computer if they didn’t know the word they needed, while Gp was only able to ask their partners or their instructor. The posttest was identical to the pretest in terms of procedure. It was administered one week afier the treatment. Results and Discussion An AN OVA was carried out to determine if there were statistically significant differences between the pretest and posttest within both instructional groups, and to determine if there were any differences between the two means of instruction. Even though the number of participants in the study was 30 at the beginning, because not all of them were present in one of the sessions, in the quantitative study I was able to use the results from only nine participants, who attended all sessions. The following methods were used: The Estimation Method, The Residual Variance Method, and Fixed Effects SE Method, and the Degrees of Freedom Method. 18 None of the methods of analysis used showed a statistically significant difference between the paper-based and the computer-based approaches, but there was a statistically significant difference between the pretests and the posttests of both the paper-based and the computer-based approaches. I will explain in detail the third method used, “Tests of Fixed Effects” (Table 1). The F value was calculated for each of the effects — approach (paper-based/computer- based), treatment (pretest/posttest), and the interaction of Approach x Treatment. The results of the statistical analysis showed that while there was a statistically significant improvement in participants’ performance when comparing pretests to posttests [F (87. 12)=0.0001 , p< 0.05], there was no statistically significant difference between the paper-based and the computer-based groups [F (0.57)= 0.4607, n.s.]. Also, the F value for the Approach x Treatment interaction was calculated [F (l .28)= 0.2746, n.s.] indicating that the improvement from pretest to posttest was not mediated by instructional approach. Table 1: Tests of Fixed Effects Effect F Value Pr>F Approach (paper- 0.57 0.4607 based/computer-based) Treatment (pretest/posttest) 87. 12 <.0001 Approach * Treatment 1.28 0.2746 (pretest/posttest) Given the results obtained in the quantitative study, there are two possible interpretations of the present analysis. The first is that there are no quantitative differences between the two means of instruction; that is, the computer-based and the 19 paper-based approaches have the same level of efficacy when used to teach ESL vocabulary. The second interpretation is that a difference does exist but would emerge only with materials and learning software that are more sophisticated and require a higher level of processing. Experiment 2 Method Participants Participants were the same as in Experiment 1. Materials On the same day as the posttest, both experimental groups were asked to complete a questionnaire that focused on the reasons the participants study English, their computer skills, and their preference for the paper-based or the computer-based approach when learning in general, and when learning languages (see Appendix D). The questionnaire had 14 questions, which were either open-ended, scalar ratings, or multiple-choice questions. In the open-ended questions, participants were asked to write as much as possible, and in the others, they had to circle the number that best reflected their opinions (e.g., they had to choose 1 out of 7 if they completely disagreed with a statement, or 7 if they completely agreed with it). Procedure The questionnaire was administered on paper to both groups because the purpose of this experiment was to obtain information about learners’ experience with the two instructional approaches (paper-based and computer-based). They had to answer the questions on the same day as they did the posttest. There was no time limit and 20 participants were encouraged to write as much as they could in answering each question. When they finished answering the questions, I collected the handouts. Results and Discussion The questionnaire focused mostly on the motivation factor in computer-based versus paper-based vocabulary instruction, and on participants’ preference for one of the two means of instruction. The major purpose of the qualitative portion of the study was to analyze participants’ motivation for using a certain means of instruction over the other. In analyzing each question, I usually looked at the percentage for the answers chosen by the most or the fewest participants in the study. In the following discussion, each question is presented in the order in which it appeared on the questionnaire. The number of participants selecting a given option is presented in the far right column. Table 2: Question Number 1 1. Why do you learn To get a better job when I go back to my country 10 English? Because I like it 7 To get into Graduate School 11 Because my parents/ my company sent me to the US to 3 study English Because I want to learn about other cultures 7 As shown in Table 2, it appears that most of these participants learned English in order to get into Graduate School or to get a better job when they went back to their home countries (see also Appendix D for the first question; learners had to choose all answers that applied). Also, a substantial number said they were studying English to learn about other cultures and because they liked it. Only three of them said they learned English 21 because their parents or company sent them to the US. to study English. Overall, the subjects’ motivation to learn English was both instrumental and integrative. The subjects’ motivation was integrative because they said they wanted to learn about other cultures and because they liked English, but was also instrumental because they wanted to learn English to get a better job when they went back to their countries and to get into Graduate SchooL Table 3: Question Number 2 2. How would you rate your ability in Ability Rating No. of Learners working with computers? Circle the right number. (1= the lowest, 7= the highest) 1 2 2 1 3 5 4 5 5 8 6 5 7 2 Table 4: Question Number 3 3. What computer system do PC 25 you prefer? Circle one. Macintosh 1 Doesn’t matter 2 Whereas most participants rated themselves between pre-intermediate and post- interrnediate in terms of their ability to work with computers, only 1 out of 13 preferred Macintosh to PC (Tables 3 and 4). 22 Table 5: Question Number 4 4. How much do you like to work with Rating No. of computers? Circle the right number. (1= the Learners lowest, 7= the highest) 1 1 2 l 3 5 4 4 5 4 6 6 7 5 Table 6: Question Number 5 5. How often do you use Frequency the Internet? Circle one. No. of Learners Several times a day 21 Once a day 3 Several times a week 3 Once a week 1 Never 0 On a 7-point scale, ratings were fairly evenly spread across the range of 3 to 7 in terms of how much they liked to work with computers. Also, on the question of how often they used the Internet, 75% said they used it several times a day, and the rest selected either once a day, several times a week, or once a week (Tables 5 and 6). Table 7: Question Number 6 6. How often do you use Frequency N o. of Learners or did you use learning software to study for subjects other than English? (e.g., Math, Chemistry, Physics) Several times a day 5 Once a day 2 Several times a week 4 Once a week 4 Never 13 23 Table 8: Question Number 7 7. How often do you use Frequency N o. of Learners learning software to improve your English? Several times a day 3 Once a day 4 Several times a week 11 Once a week 3 Never 5 It is interesting to note that whereas only 14% of the subjects said they used leaming software several times a week to study for subjects other than English, 42% said they studied English several times a week using learning software. Also, 46% said they never used learning software to study subjects other than English, and only 19% said they never used learning software to improve their English skills (Tables 7 and 8). Table 9: Question Number 8 8. How much do you like to work with Rating No. of computers to improve your English? Circle Learners the right number. (l= the lowest, 7= the highest) 1 3 2 2 3 2 4 5 5 6 6 6 7 2 About 46% of the participants circled 5 and 6 on the 7-point scale when asked how much they liked to work with computers to study English. When asked how helpful they considered computers were in improving their English, about 46% chose ratings 5 and 6, and 8% chose 1. About 15% said computers were extremely important in helping them improve their English (Tables 9 and 10). 24 Most participants stated that it was better to learn English using learning software at home; 57% said without the teacher’s help, and only 20% said they preferred to use such software during class time (Table 11). Table 10: Question Number 9 9. How helpful do you think computers are in Rating No. of Learners improving your English? Circle the right number. (l= the least useful, 7= the most useful) 1 2 2 1 3 3 4 4 5 6 6 6 7 4 Table 11: Question Number 10 10. Circle one answer. Do In class, with the teacher 7 you prefer to use learning being present software: Outside class, on my own 16 Outside class, with friends 5 Table 12: Question Number 11 11. How popular is learning software in your Rating No. of Learners country? Circle the right number. (1= the least popular, 7= the mostJropular) 1 3 2 3 3 3 4 5 5 3 6 5 7 4 Participants seemed to have different opinions concerning the popularity of learning software in their countries. From a 7-point scale, 12% chose the rating of 1, 25 19% chose 4, and 12% chose 5, and 15% chose 7. However, a higher number of them thought that learning software is popular in their countries. About half of them said they had bought learning software to learn a foreign language. The great majority thought that learning software was helpful in learning a foreign language (Tables 12 and 13). Table 13: Question Number 12 12. Have you ever bought learning Yes 12 software to learn a foreign , language? Circle one answer: No 13 Were they helpful? Yes 12 N o 4 I don’t know/ no 10 answer Among the advantages of using computers when learning English that were mentioned in the questionnaire, the following seemed to occur the most often: 0 It is comfortable, convenient, interesting, easy, one can repeat exercises 0 We enjoy working with the Internet, with computers, computer games 0 We can improve our English skills 0 It saves time and I can contact all of the world whenever I want 0 Easy to repeat and listen, clear voice 0 We can learn a lot quickly 0 It is easier to learn pronunciation 0 Practice English tests, search for information about American culture 0 I can learn by myself 0 Practice for the TOEFL test 26 Among the disadvantages of using computers when learning English that were mentioned in the questionnaire, the following were the most frequent: o It is difficult to learn how to spell 0 We just watch, do not get involved a I am tempted to chat with friends in Korean and I don’t use English 0 Takes more time, sometimes the computer does not work well 0 Easier for me to study English using the paper-based approach 0 There is no disadvantage o It is expensive 0 It is easy and comfortable for teaching 0 It makes me tired According to the participants’ responses, the advantages of using learning software to study English seem to surpass the disadvantages. However, I do not consider that, at present, language learners would benefit from a computer-based instructional approach only, but rather through a combination of the two. 27 Chapter Three Conclusion The results of the present study strengthen the hypothesis of previous research undertaken on the use of technology in education, that even though “new” technologies have had a crucial influence on human interaction, they have not been shown to be as important in the field of second language teaching. The computer-based vocabulary exercises used in this study did not prove to be more efficient than the paper-based exercises, but the learners showed a higher level of motivation towards the use of CALL rather than the paper-based approach. The limitation of the study lies in the choice of the computer-based vocabulary activities; if I had used more sophisticated ones, the results of the study might have been different and confirmed the hypothesis that computer-based activities can be more effective. However, my choice of these activities was determined by the fact that these types of computer-based vocabulary activities are currently utilized at the language center at which the study was conducted. The results do demonstrate that not all technology-based activities are more effective than the paper ones, and that closer attention has to be paid when choosing a computer approach over a paper-based approach. Further research needs to be done and computer-based activities need to be designed that require a higher level of processing in order to prove that computer-based activities are more effective than paper-based activities. 28 REFERENCES Cabarga, D. (1937). Teaching Spanish by radio. Modern Language Journal, 22, 189-200. Clement, R., Domyei, Z., & Noels, K.A. (1994). Motivation, self-confidence, and group cohesion in the foreign language classroom. Language Learning, 44, 417-448. Craik, F., & Lockhart, R. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 104, 268-294. Craik, F.I.M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104, 268-294. Crookes, G. & Schmidt, R., (1991) Motivation: Reopening the research agenda. Language Learning, 41, 469-512. Dempsey, J. V., Lucassen, B. A., Haynes, L. L., & Casey, M. S., (1996). Instructional applications of computer games. Paper presented at the 1996 annual meeting of the American Educational Research Association, NYC. deyei, Z. (1990). Conceptualizing motivation in foreign language learning. Language Learning, 40, 45-78. Domyei, Z. (1994). Motivation and motivating in the foreign language classroom. The Modern Language Journal 78, 273-84. Dunkel, P. (1990). The effectiveness research on computer-assisted instruction and computer-assisted language learning. In P. Dunkel (Ed.), Computer-assisted language learning and testing: Research issues and practice (pp. 5-36). New York: Newbury House. Ellis, N.C. (1994a). Consciousness in second language learning: Psychological perspectives on the role of conscious processes in vocabulary acquisition. AILA Review 11, 37-56. 29 Ellis, N.C. (1994b). Vocabulary acquisition: The implicit ins and outs of explicit cognitive mediation. In N. C. Ellis (Ed.), Implicit and Explicit Learning of Languages (pp. 211-282). London: Academic Press. Gardner, R. C. (1985). Social psychology and second language learning: The role of attitudes and motivation. London: Edward Arnold. Gardner, R. C. (2001). Integrative motivation and second language acquisition. In A. deyei & R. Schmidt (Eds), Motivation and second language acquisition (Technical Report #23, pp. 1-19). Honolulu: University of Hawai’i, Second Language Teaching and Curriculum Center. Gardner, R. & Lambert, W. (1972). Attitudes and motivation in second-language learning. Rowley, Massachusetts: Newbury House Publishers. Gardner, R.C. & MacIntyre, RD. (1991). An instrumental motivation in language study: Who says it isn't effective? Studies in Second Language Acquisition, 13, 266-272. Groot, P. J. M. (2000). Computer-assisted second language vocabulary acquisition. Language Learning and Technology, 4, 60-81. Hulstijn, J. (1992). Retention of inferred and given word meanings: Experiments in incidental vocabulary learning. In P. J. Amaud & H. Bejoint (Eds), Vocabulary and Applied Linguistics (pp. 113-125). London: Macmillan. J oe,A. (1995). Text-based tasks and incidental vocabulary learning: A case study. Second Language Research, I 1 , 149-58. J oe,A. (1998). What effects do text-based tasks promoting generation have on incidental vocabulary acquisition? Applied Linguistics, 19, 357-77. Kojic-Sabo, & I, Lightbown, RM. (1999). Students’ approaches to vocabulary learning and their relationship to success. Modern Language Journal, 83, 176-192. Krashen, S. (1981). Second language acquisition and learning. London: Pergamon. 3O Laufer, B. and Hulstijn, J. (2001). Incidental vocabulary acquisition in a second language: the construct of task-induced involvement. Applied Linguistics, 22, l- 26. McCarthy, M, O’Dell, F, Shaw, E. (1997). Vocabulary in use, upper intermediate. Cambridge: Cambridge University Press. The Newbury House dictionary of American English. (2000). Heinle & Heinle Publishers. Oxford advanced learner ’3 dictionary. (1995). Oxford: Oxford University Press. Oxford, R. and Shearin, J. (1994). Language learning motivation: Expanding the theoretical framework. The Modern Language Journal, 78, 12-28. Pajtek, A. (2001). Diflbrences between computer-based and paper-based vocabulary instruction. Project for course on Research Methods in Second Language Acquisition, Michigan State University. Pederson, K. (1986). An experiment in computer-assisted second-language learning. The Modern Language Journal, 70, 36-41. Peirce, B.N. (1995). Social identity, investment, and language learning. TESOL Quarterly, 29, 9-31. Redman, S., & Shaw, E. (1999). Vocabulary in use, intermediate. Cambridge: Cambridge University Press. Salaberry, R. (2001). The use of technology for second language learning and teaching: A Retrospective. The Modern Language Journal, 85, 39-56. Schreck, R. And Schreck, J. (1991). Computer-assisted language learning. In M. Celce- Murcia (Ed.), Teaching English as a Second or Foreign Language (pp. 472- 486). Heinle & Heinle Publishers. 31 Seal, B. (1991). Vocabulary learning and teaching. In M. Celce- Murcia (Ed.), Teaching English as a Second or Foreign Language (pp. 296- 310). Heinle & Heinle Publishers. Zhao, Y. (2002). E—Ianguage learning: practices, problems, and promises. A concept paper submitted to the United States Department of Education, Michigan State University, Department of Education. 32 Appendix A (For the computer-based version, please see http://clear.msu.edu/research/alina/) Exercise one: Guessing vocabulary from context Read the following sentences and write a synonym or a definition for the underlined words: My sister’s baby is very sweet: she loves to eat, she is chubby and has blond hair and big, brown eyes. chubby= Are you saying that you are scrawny? Do you watch Ally McBeal? I think “Ally” is almost scrawny, but not you! scrawny: Can you see that tubby little man? He looks like Santa Claus, but he’s my favorite uncle, always funny and in a good mood. tubby: 4. Because he has spent so much time in the hospital and hasn’t exercised in a while, his whole body is quite flabby. His doctor recommended that he take up a sport when he gets well and leaves the hospital. flabby= 5. I love my grandma- she is a large, gray-haired, 75-year-old woman, always ready to play practical jokes on people. large= When his doctor told him he was obese and needed to change his eating and exercise patterns, he was first offended, but later he realized the doctor had been right. obese= 7. Some teenage girls become anorexic because they think they are too fat. anorexic= 33 8. She thought she was overweight, and was so obsessed with her weight that she went to the doctor who told her she was actually underweight. overweight: underweight: 34 Appendix B (For the computer-based version, please see http://clear.msu.edu/research/alina/) 1. Put each word under the picture to which it is closest in meaning. tubby chubby overweight underweight obese scrawny flabby large anorexic :13 y, AVA ‘ 35 2. Discuss in pairs the differences between the words in the same column. 3. a. An adjective used in a friendly manner to describe people who are 1. Tubby short and round. b. Adjective used to describe a certain part of the body or children 2. Chubby (it is closer to “fat” than to “thin”.) c. The person suffering from this eating disorder refuses to eat. 3. Anorexic d. The most neutral term for “fat”, also used by physicians. 4. Overweight e. Physicians use this word to say that someone is unhealthily fat. 5. Obese f. Word used to describe flesh that is fat and loose. 6. F labby g. Negative adjective suggesting that a person or an animal is small 7. Scrawny or weak. h. This is a euphemism for fat. (A euphemism is a “nice” word) 8. Large 9. Underweight i. Adjective describing a thin person; it is usually utilized by physicians. Cloze exercise. Fill in the gaps with the words below: scrawny anorexic flabby underweight overweight chubby tubby obese large 1. Most people want to stay slim, but not as as that woman over there. 2. After visiting the doctor who told him he was , he decided to exercise everyday. 36 3. I think she was expecting to meet a tall, slender man, not a one! 4. Did you say he was ? He should eat three meals a day and work less. No, he said he was ! He should eat less. 5. When I was little, I was a little girl, with blond hair and big, black eyes. 6. She is hypochondriac, that is, she always thinks she has a disease. For example, now she is almost sure she is because she does not have an appetite anymore. 7. During winter people usually put on weight and their bodies become , but if they exercise, they are slender again. 8. My math teacher was a tall, guy. We were always afraid of him. 37 Appendix C (For the computer-based version, please see http://clear.msu.edu/research/alina/) Posttest 1. Exercise one: Guessing vocabulary from context Read the following sentences and write a synonym or a definition for the underlined words: She is not fat, but her legs are pretty chubby. chubby= The old man over there seems pretty scrawny- his clothes are far too large for him! scrawny: Even though she isia tubby little woman, her husband is tall and slender. tubby: 4. After she gave birth, her skin was a little flabby, so she had to start exercising. flabby: 5. Can you describe your fiiend so that I know whom to look for in the train station? He’s a tall, large man always wearing orange shorts and a green T-shirt. large= 6. Obese patients are advised to change their diet. obese= 7. Ifyou think you are anorexic, you should definitely see a doctor! anorexic: 8. She thought she was overweight, and was so obsessed with her weight that she went to the doctor who told her she was actually underweight. overweight= underweight: 38 Appendix D Questionnaire Circle the appropriate answer(s): 1. Why are you learning English? To get a better job when I go back to my country Because I like it To get into Graduate School Because my parents sent me to the US to study English Because I want to learn about other cultures How would you rate your ability to work with computers? Circle the number that applies. (1= the lowest, 7= the highest) 2 3 4 5 6 7 What computer system do you prefer? Circle one. PC Macintosh How much do you like to work with computers? Circle the number that applies. (1= the lowest, 7= the highest) 39 5. How often do you use the Internet? Circle one. Several times a day Once a day Several times a week Once a week Never How often do you use or did you use learning software to study for subjects other than English? (e.g., Math, Chemistry, Physics) Several times a day Once a day Several times a week Once a week Never How often do you use learning software to improve your English? Several times a day Once a day Several times a week Once a week Never 40 8. How much do you like to work with computers to improve your English? Circle the number that applies. (l= the lowest, 7= the highest) 9. How helpful do you think computers are in improving your English? Circle the number that applies. (1= the least useful, 7= the most useful) 10. Circle one answer. Do you prefer to use learning software: 0 In class, with the teacher being present 0 Outside class, on my own 0 Outside class, with friends 11. How popular is learning software in your country? Circle the number that applies. (l= the least popular, 7= the most popular) Country you are from: 41 12. Have you ever bought learning software to learn a foreign language? Circle one answer: 0 Yes 0 No Was it helpful? Circle one answer: 0 Yes 0No 13. What are some advantages of using computers when learning English? Please describe three: 1. 2. 3. 14. What are some disadvantages of using computers when learning English? Please describe three: 1. 2. 42 lot it . I. I... ‘l’-