INCORPORATING ORAL TESTS IN HIGH STAKES ENGLISH EXAMS IN BANGLADESH: A STUDY ON BANGLADESHI STUDENTS’ AND TEACHERS’ PERCEPTIONS OF ORAL TESTING DURING UNIVERSITY ADMISSION By Abeer Noor A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Teaching English to Speakers of Other Languages—Master of Arts 2018 ABSTRACT INCORPORATING ORAL TESTS IN HIGH STAKES ENGLISH EXAMS IN BANGLADESH: A STUDY ON BANGLADESHI STUDENTS’ AND TEACHERS’ PERCEPTIONS OF ORAL TESTING DURING UNIVERSITY ADMISSION By Abeer Noor The purpose of this study is to investigate students’ and teachers’ attitudes toward and perceptions of oral testing during university admission in Bangladesh. A mixed-methods approach was used to analyze quantitative and qualitative data collected from surveys and interviews. In total, 133 EFL (English as a Foreign Language) teachers and students participated in this study. Results of the study revealed that students and teachers shared many similarities in their attitudes toward oral testing. Both groups believed that oral testing has positive effects on developing learners’ English speaking skill. The main challenges to oral testing during admission, according to them, were students’ lack of familiarity with oral tests and their anxiety about L2 speaking. Their thoughts and beliefs, as presented in this study, have implications for developing the policy on oral assessment during university admission in Bangladesh. ACKNOWLEDGEMENTS I would like to express my sincere gratitude to my thesis director, Dr. Paula Winke, who constantly supported and encouraged me in completing this research project. Dr. Winke’s advice and guidance helped me incredibly from the initial to the final phases of my master’s thesis. I was lucky to attend her course on language assessment, which sparked my interest in investigating the English language assessment system in Bangladesh. I am grateful to my thesis committee member, Dr. Debra M. Hardison, for her invaluable feedback and comments, which helped me improve my paper. Thanks must go to my MA TESOL classmates, especially Khoa Truong and Ghulam Rasool, for their time and enthusiasm for discussing various research issues with me. I am also grateful to my Bangladeshi colleagues, Ashik Istiak and Abduallh Al Muktadir, who assisted me to contact with my participants. Finally, I would like to thank all the EFL teachers and students who participated in this study and made the online survey a success. iii TABLE OF CONTENTS LIST OF TABLES ......................................................................................................................... vi INTRODUCTION .......................................................................................................................... 1 LITERATURE REVIEW ............................................................................................................... 5 Research on Language Testing in the Bangladeshi Context ....................................................... 5 Absence of listening and speaking tests. ................................................................................. 6 Non-communicative nature of the test-items ........................................................................... 7 Lack of uniform grading criteria ............................................................................................. 7 Test centeredness ..................................................................................................................... 7 Grade inflation ......................................................................................................................... 8 Miscellaneous problems .......................................................................................................... 9 Research on English Language Tests in the Asian Contexts ...................................................... 9 Asian countries that have changed their assessment systems based on CLTA ..................... 10 Asian countries that have not yet changed their assessment systems.................................... 16 Research Questions ................................................................................................................... 18 METHODS ................................................................................................................................... 19 Design of the Study ................................................................................................................... 19 Participants ................................................................................................................................ 19 Instruments ................................................................................................................................ 19 Procedure ................................................................................................................................... 20 Data Analysis ............................................................................................................................ 21 RESULTS ..................................................................................................................................... 23 RQ1: With What Types of Oral Tests are Bangladeshi Students Familiar? ............................. 23 RQ2: What Attitudes Do Teachers and Students Have Toward Oral English Testing During University Admissions? ............................................................................................................ 27 RQ3: What are the Challenges and Barriers to Oral Testing During University Admission? .. 37 DISCUSSION ............................................................................................................................... 52 Predominance of Individual Presentation as an Oral Test Component ..................................... 52 Students’ Lack of Familiarity with Paired and Group Orals ..................................................... 53 Relationship Between Students’ Levels of Education and Test Familiarity ............................. 55 Similarities Between Students and Teachers’ Attitudes Toward Oral Testing ......................... 58 Some Differences in Students and Teachers’ Attitudes Toward Oral Testing.......................... 60 Major Barriers to Oral Testing as Perceived by Teachers and Students ................................... 61 Differences in Students and Teachers’ Perceptions of the Barriers to Oral Testing ................. 64 CONCLUSION ............................................................................................................................. 65 Implications ............................................................................................................................... 65 Limitations of the Study ............................................................................................................ 66 iv APPENDICES .............................................................................................................................. 68 APPENDIX A: Interview Questions ......................................................................................... 69 APPENDIX B: Survey .............................................................................................................. 70 REFERENCES ............................................................................................................................. 76 v LIST OF TABLES Table 1: Students’ Experience Taking Oral Tests ........................................................................ 23 Table 2: Students’ Overall Familiarity with Different Types of Oral Tests ................................. 24 Table 3: Students’ Experience Taking Oral Tests at Undergraduate Level .................................. 25 Table 4: Students’ Experience Taking Oral Tests at Higher Secondary Level ............................ 26 Table 5: Students’ Experience in Taking Oral Tests in Admission Exam ................................... 26 Table 6: Reliability of the Scales Based on Students’ Responses ................................................ 28 Table 7: Students’ Perceptions About Oral Testing During Admission ....................................... 28 Table 8: Students’ Responses to Individual Items ........................................................................ 29 Table 9: Reliability of the Scales Based on Teachers’ Responses................................................ 32 Table 10: Teachers’ Perceptions About Oral Testing During Admission .................................... 32 Table 11: Teachers’ Responses to Individual Items ..................................................................... 33 Table 12: Comparison Between Teachers’ and Students’ Attitudes ............................................ 37 Table 13: Reliability of the Scales Based on Students’ Responses .............................................. 38 Table 14: Challenges and Barriers to Oral Testing as Viewed by Students ................................. 38 Table 15: Students’ Responses to Individual Items ...................................................................... 39 Table 16: Reliability of the Scales Based on Teachers’ Responses.............................................. 44 Table 17: Challenges and Barriers to Oral Testing as Viewed by Teachers ................................ 44 Table 18: Teachers’ Responses to Individual Items ..................................................................... 45 Table 19: Comparison Between Students And Teachers’ Responses .......................................... 50 vi INTRODUCTION Developing learners’ English communicative competence is not just a requirement of the language curriculum in Bangladesh, but also a prerequisite for their career opportunities and success. For English as a Foreign Language (EFL) teachers in Bangladesh, there is a pressing need for improving learners’ English language performance in real-life communicative events in real time. During my experience as an EFL teacher, I have always felt that our overarching classroom and curriculum goal of developing learners’ communicative competence remains unfulfilled because of a major mismatch between the English language curriculum and the mainstream assessment system. From the primary to the higher secondary level, English language learners are tested only on their reading and writing skills. A very similar pattern of English tests can be observed in the majority of the public and private universities admission exams, which do not assess learners’ proficiency in English listening and speaking. Lack of exposure to oral tests makes it very difficult for them to perform well in the undergraduate studies in which the majority of the students at Bangladeshi universities are required to take part in oral presentation or viva voce as part of the course requirements. Therefore, it was a common thing for me to observe many of the first year undergraduate students struggling to improve their spoken English performance by the time they reach the end of their first year at university. As an English teacher in Bangladesh, I have always been interested in looking for ways in which speaking tests could be integrated into the final assessment of English courses. I found that my students were very much eager to improve their spoken English because speaking and writing good English is seen as essential in securing better jobs in Bangladesh. While there were plenty of options for me to assess their written English performance, I often found fewer opportunities to assess their spoken English performance because the English course final exams 1 were mainly written. Although students’ performance in oral presentation or other speaking activities were assessed in class, the total points these spoken components added to the course final grade were fewer than what can be covered by the reading and writing sections. The emphasis on written components in exams naturally encourages students to focus more on English writing than speaking, and this was true for my students too. The fact that many university students do not have to go through any spoken test during university admission makes the situation even worse. After admission to various university programs, which often includes one or two core English courses, students with low proficiency in spoken English may end up competing with students having high spoken proficiency. Lack of any systematic spoken assessment also creates difficulties in placing these students at their right levels, and designing lessons that best meet their learning needs. Therefore, as a teacher, I have always felt the necessity to address the dichotomy between speaking and writing assessment in Bangladesh. Previous research in English language testing in Bangladesh addressed the negative washback of the current English language assessment system. However, the absence of a spoken English section in university admission, and the main reasons for this absence remain largely unexplored. Against the backdrop of this conspicuous absence of oral tests in the English language assessment system in Bangladesh, I decided to turn to current English language students and teachers to investigate their perceptions of the challenges and barriers to incorporating oral tests during university admission. My aim for this study was to find out how much they felt it necessary to incorporate a speaking section in the admission exam, and what major difficulties they have faced or expect to face while administering any oral tests. In addition to these, I also attempted to know about the types of oral tests they would prefer during university admission, 2 and the suggestions they have in overcoming the major barriers to oral testing during the admission. In this thesis, I started with a literature review, in which I explored the previous research on English language testing in Bangladesh, with a focus on the problems or inconsistencies identified by the researchers. Although I attempted to look specifically into the research on the challenges and barriers to oral testing in Bangladesh, I could not retrieve any information from any peer-reviewed articles. I also reviewed some studies on English language assessment in other Asian countries. Most of these studies detailed the positive effects of administering oral tests and how oral tests were administered in different Asian countries, and a few of these also provided some information about the possible challenges faced in administered oral tests. Reviewing these studies helped me compare other Asian EFL testing situations with those of Bangladesh and see what types of oral tests they run and how they run these during university or college admission, or as part of the course final assessment. The information I could gather by reviewing the way oral tests are administered in other Asian countries like China, Japan, and Singapore, can give some insight into the ways of overcoming possible barriers to oral testing. I conducted this study by surveying and interviewing students and teachers at two different Bangladeshi universities. In the survey, participants responded to 60 Likert scale statements related firstly, to their attitudes to oral testing, and secondly, to the barriers to including oral tests during university admission. I used descriptive statistics to analyze and report the survey data. Throughout my analysis, I kept students’ and teachers’ data separate, and finally compared them to see whether the two groups’ responses differed significantly or not. To have a better understanding of the survey data, I interviewed some of the teachers. From these 3 interviews, I gathered information on what they viewed as the possible solution to overcome the challenges to oral testing in Bangladesh. 4 LITERATURE REVIEW During the past few decades, a number of Asian countries have reformed their English language assessment policies to make them consistent with the communicative language teaching approach (CLTA), which places equal emphasis on all four language skills (e.g. Rahmat, Sungif, and Yusup, 2015; Qian, 2008; Zheng & Cheng, 2008). One key area of the reformation was the inclusion of an oral test section in high-stakes English examinations. In response to this change, many Asian EFL/ESL (English as a Foreign/Second Language) researchers investigated the reliability and validity of different types of oral tests, and reported various problems and challenges related to oral testing (Loh & Shih, 2016; Sidhu, Fook, & Mohamad, 2011; Zhang & Elder, 2009;). In Bangladesh, the English language curriculum was reformed to align it with the communicative approach but oral English tests are still not a part of the major English examinations including the university admission tests. The absence of oral tests has a strong negative washback on the teaching of speaking skill in Bangladesh (Maniruzzaman & Hoque, 2010). To address this issue, I reviewed some recent studies on oral testing in Bangladesh and nine other Asian countries with a view to finding information about the current status of oral English tests in these countries, the problems related to oral testing, and some possible solutions. Research on Language Testing in the Bangladeshi Context In 1997, the National Curriculum and Textbook Board (NCTB) in Bangladesh revised the secondary English textbook (English for Today) based on the communicative approach in order to replace the traditional grammar-based English teaching practices (Khan, 2010). This initiative sparked a lot of discussion among teachers and researchers who weighed the pros and cons of the CLT-based English curriculum, revealing at the same time the inconsistencies that exist between 5 the principles of CLT and the actual classroom practices in Bangladesh (e.g., Chowdhury & Ha, 2008; Haider & Chowdhury, 2012; Rahman, 2015). However, little research has been done to investigate how much the current language assessment system aligns with the CLT-based English curriculum in Bangladesh. When I searched for resources for my current study in the Linguistics and Language Behavior Abstracts (LLBA) and Google Scholar databases, I found only four articles (e.g. Das, Shaheen, Shrestha, Rahman, and Khan, 2014; Khan, 2010; Maniruzzaman & Hoque, 2010; Tahereen, 2014) that dealt specifically with the condition of Bangladeshi EFL tests in relation to the current EFL curriculum. Although these studies did not elaborate on the oral testing situation in Bangladesh, they unanimously revealed a mismatch between the CLT-based English curriculum and the present assessment system. Some of the most recurrent issues of these studies, which I found relevant to my present study, are discussed below. Absence of listening and speaking tests. Khan (2010) interviewed Bangladeshi teachers and examiners to know about their perceptions of the current English language assessment system at the secondary and higher secondary level. Most of the teachers she interviewed reported that the summative assessment system at the secondary and higher secondary level did not have any place for testing speaking and listening, although these two constitute the fundamental skills of a language. Similar viewpoints were expressed in research done by Tahereen (2014), who claimed that listening and speaking were not included in language tests, nor in lesson plans, and even the language of instruction in these classes was mostly Bengali (the students’ L1). Although some teachers indicated that they assessed listening and speaking in the classroom, as reported in the study by Das et al. (2014), the oral tasks mainly involved reading aloud and answering questions. 6 Non-communicative nature of the test-items. Most of the teachers and examiners who participated in Khan’s (2010) study reported that test-contents were not suitable for assessing learners’ communicative competence that their ESL syllabi emphasize. First, there was a total absence of listening and speaking in the tests. Second, the reading components like re-ordering sentences were making additional demands on the test-takers without really assessing their communicative language. This was also confirmed by Das et al. (2014) who analyzed the question papers of the Secondary School Certificate (SSC) examination. They pointed out that “some of the questions are basically testing knowledge of forms and there is very little focus on eliciting learner language” (p. 339). Lack of uniform grading criteria. Teachers who participated in Khan’s (2010) study reported that there were “no specific criteria for marking” the reading and writing sections of the language exams at SSC and HSC levels (p. 138). The details of how the examiners graded the students’ written papers revealed how varied they were in terms of grading criteria. Some examiners viewed grammatical accuracy and correct spelling to be important. Some looked at the use of appropriate vocabulary, while some preferred the use of standard language. Das et al. (2014) reported that teachers graded the scripts in terms of writing skills, grammatical rules and spelling, creativity of the answers, and other criteria. What was common across all these criteria was the obvious focus on discrete grammatical items. Although criteria like writing skills and creativity were mentioned by some, there were no details about how the examiners really perceived these constructs. A significant lack of consistency and uniformity in the grading criteria used by examiners were evident in these studies. Test centeredness. The research on assessment in Bangladesh seems to point out that testing is central to and driving instruction, rather than the instruction and classroom content 7 driving what gets assessed. That is, there is strong washback from testing onto the curriculum. Among negative washback discussed by Maniruzzaman and Hoque (2010), and Tahreen (2014), one major concern was the tendency to teach to the test. The authors reported in these articles that the tests were not aligned with the syllabus. Rather the classroom practices appropriated the syllabus according to the test. Therefore, little attention is given to build learners’ communicative competence because this was not the main purpose of the language tests. Thus, as pointed out by Bangladeshi researchers, teaching and learning of speaking did not get priority in the language classrooms (Khan, 2010; Maniruzzaman & Hoque, 2010). Learners relied heavily on test preparation guides; the contents of the syllabus and the learners’ goals were overlooked (Das et al., 2014). Tahereen (2014) further wrote that English teaching in Bangladesh focused on grammatical rules, translations, and vocabulary memorization. Topics for assessing written compositions were so predictable and over-used that they barely attempted to test students’ communicative competence. Grade inflation. An implicit and explicit emphasis on achieving high scores or grades was also discussed in these articles. Some participants confided to Khan (2010) that the current testing system made it easy to score and grade high but the actual language competencies of the students were not increasing. Some complained, “The brilliant students are not getting their due” (Khan, 2010, p. 135). In Das et al. (2014), examiners mentioned candidly that some examiners try not to fail students. As a Bangladeshi EFL learner and teacher this information is not new to me. As the Secondary School Certificate (SSC) and Higher Secondary School Certificate (HSC) results come out each year, language teachers and researchers in Bangladesh can see grade inflation, but little competence in English, and this issue lingers in the media for months and months. 8 Miscellaneous problems. Tahereen (2014) wrote that the language test system in Bangladesh lacked validity because it did not assess learners’ ability to use English in real life contexts and situations, such as expressing thoughts and feelings orally. Khan (2010) similarly questioned the validity and reliability of the HSC English exam, which did not include the assessment of two fundamental skills, listening and speaking. She also identified the lack of examiners’ training and their lack of understanding of the test specifications as few of the major weaknesses of the English exam at the higher secondary level. In addition to the lack of listening and speaking tests, as pointed out by Das et al. (2014), the prevailing language system was not communicative in nature because it encouraged students to memorize language components for getting good grades. Research on Bangladeshi EFL exams, thus, revealed a huge discrepancy among the language teaching curriculum, the assessment system, and the classroom practices. However, it is hopeful to note that most of the teachers and students, in the studies of Khan (2010) and Das et al. (2014), acknowledged the need to assess all four skills of language. Even though oral assessment was absent at the national level, the classroom observation data in Das et al.’s (2014) study showed the evidence of oral assessment as part of the classroom practices at some schools. Research on English Language Tests in the Asian Contexts Asian countries like Japan, China, South Korea, India, Malaysia, and Thailand have reformed their traditional English teaching curricula and introduced the communicative approach for teaching English (e.g., Kamal, Siraj, Alias, & Attaran, 2013; He & Dai, 2006; Sasaki, 2008; Qian, 2008; Zheng & Cheng, 2008). Some of these Asian countries (e.g. China, India, Malaysia, Singapore, Japan, and Hong Kong) have changed their language assessment systems to make them complementary to CLTA curricula (e.g., Kamal, et al., 2013; Ramanathan, 2008; Sasaki, 9 2008; Qian, 2008; Zheng & Cheng, 2008). However, some inconsistencies between curricula and assessment types still prevail in countries like Pakistan, South Korea, and Thailand (Choi, 2008; Jilani, 2009; Prapphal, 2008). In the following details, I have pointed out some notable oral English tests and several key features of the English testing systems of these countries. Asian countries that have changed their assessment systems based on CLTA. a) China: In the Chinese EFL context, English language assessment including oral tests is a widely researched topic. Recent studies in this area provided evidence for a number of oral English tests administered publicly throughout the country (Zhan & Elder, 2009; Zheng & Cheng, 2008). An analysis of these studies showed that the oral test sections were added to the mainstream English exams with a view to promoting communicative English teaching and learning. A description of the high-stakes English exams in China is as follows. College English Test (CET) is one of the nationwide English language tests in China. It is administered by National College English Testing Committee (NCETC) to assess English proficiency of Chinese undergraduate students who are studying at Chinese universities or colleges and whose major is not English (Zhan & Elder, 2009; Zheng & Cheng, 2008). Zheng and Cheng (2008) reported that the number of students taking CET in 2006 was 13,000,000, which made it “the largest English as a foreign language test in the world” (p. 410). This test consists of three separate tests and the CET-Spoken English Test (CET-SET) is one of them. CET-SET, introduced first by NCETC in 1999, is a 20-minute test that is held twice a year at authorized test centers in China (He & Dai, 2006; Zhan & Elder, 2009; Zheng & Cheng, 2008). Students who successfully pass the other two CET tests (e.g. CET-4 and CET-6) are eligible to take the CET-SET. This oral test includes three parts: a) a warm-up session: interaction between the candidate and the examiner; b) an individual presentation and a group discussion: an 10 individual presentation based on visual prompts and interaction among candidates in group discussion; c) Further questions between the examiner and candidates (He & Dai, 2006; Zhan & Elder, 2009; Zheng & Cheng, 2008). Chinese undergraduate students who are majoring in English language and literature take the Test for English Majors (TEM), administered by the National Advisory Committee for Foreign Language Teaching (NACFLT) (Jin & Fan, 2011). The TEM consists of four tests, two of which are oral tests (e.g. TEM4 Oral and TEM8 Oral). The length of each oral test is about 25 minutes and the two tests are held at two different times in a year. The TEM4 Oral includes three tasks: a) retelling a story after listening to it once, b) a presentation on a topic related to the story, and c) interaction between two candidates. The TEM8 Oral includes mainly two types of tasks: a) interpretation of a speech in 150 or 200 words, and b) a presentation on a given topic (Jin & Fan, 2011). Another important oral test, which is administered nationwide in China, is the oral subset of the Matriculation English Test (MET) (Sun & Henrichsen, 2010; Xiajou & Yi, 2000). The MET is taken by students for university enrolment. The task involved in this oral test is basically the candidate’s interaction with two examiners on the basis of some prompts (Xiajou & Yi, 2000). Research on oral English tests in China also gives evidence of other types of assessment that are practiced at various places in China. For example, the Computer Based English Listening and Speaking Test (CELST) is administered in Guandong province in China (Zhan & Wan, 2016). Besides these nation-wide English tests, Lin (2014) studied the advantages of large-scale computer-assisted oral English testing at a Chinese university. She investigated the reliability of the test in terms of its content, measurement, and scoring. The standardized reliability coefficient of this test was 1.26 and 1.68. 11 All these studies reveal the vibrant and dynamic nature of English tests in China. Some of the key issues that researchers in these studies dealt with were the validity and reliability of the tests, and the washback and impact of these tests on the curriculum and syllabi. For example, a number of researchers pointed out that the introduction of the Spoken English Test in CET has a positive washback on the curriculum because it promotes students’ communicative competence (Zhang & Elder, 2009; Zheng & Cheng, 2008). Xiajou and Li (2000) wrote about the validity of the oral subset of the Matriculation English Test, reporting that the test “has attained validity on the strength of its being a message-based test of the interactive and contextualized use of spoken English” (p. 1). Besides, some of these studies directly dealt with the use of computer technology in administering the oral tests (Lin, 2014; Zhan & Wan, 2016). b) Singapore: According to the official website of the Singapore Admissions and Assessment Board, the major public exams in Singapore include Primary School Leaving Examination (PSLE), GCE ‘N’ level, GCE ‘O’ level, and GCE ‘E’ level (National examinations, 2017). Based on their results in the PSLE exams, students are placed either in GCE ‘O’ (for Express course) or GCE ‘N’ Level (for Normal course) (Secondary, 2017). All these exams include speaking tests. The PSLE English Exam, for example, has a ten-minute speaking section which carries 15% of the total score and there are two oral tasks in this test: a) reading aloud, which is designed for evaluating examinee’s pronunciation and articulation, and b) a conversation based on visual stimulus (Loh & Shih, 2016). In their review of the PSLE English Exam (PSLE EL), Loh and Shih (2016) investigated into the test fairness framework and reported that the improvements in test design, especially the inclusion of a conversation component in the speaking section, have increased the validity of the test. Although Loh and Shih (2016) could not 12 make any conclusive remarks about the presence or absence of bias in this test, an investigation (Noor, Muniandy, Krishnan, & Mathai, 2010) into teachers’ perceptions of the PSLE EL oral test revealed some problems and challenges related to this test, including rater’s bias, lack of understanding of the scoring rubric, and culturally irrelevant conversation topics or pictures. However, among the 10 teachers interviewed in this study, all believed that oral communication is an important part of language education. Ang-Aw and Goh (2011) studied the ‘O’ level English examination, another high-stakes exam which includes an oral test. This examination is designed for secondary level students who want to gain access to tertiary education. The oral part of the test consists of three sections: a) reading aloud, b) picture discussion, and c) conversation. The conversation part is designed to evaluate a candidate’s communicative competence (Ang-Aw & Goh, 2011). c) Malaysia: There have been a considerable number of studies on the School Based Oral English Assessment (SBOEA), which was first implemented in 2002 in Malaysian schools. Kamal et al. (2013) reviewed 15 articles on SBOEA in Malaysia and found the following research topics: perceptions, analyzing factors, knowledge and understanding, concerns, washback effects and miscellaneous. Unlike any exit or entrance tests, SBOEA is integrated with the regular English teaching/learning process at school and the students’ performances in this test are assessed by their classroom teachers (Sidhu, Fook, & Sidhu, 2011). Under the SBOEA system, a student is assessed three times over the course of two years. The tasks involved in this test are individual presentations with/without prompts, group orals, and paired orals. Sidhu, Fook, and Mohamad (2011) studied teachers’ perceptions of the SBOEA. The challenges teachers faced in implementing the test includes lack of understanding the grading criteria, huge number of examiners, time constraints, and lack of resource materials. On the other hand, the 13 challenges of taking SBOEA as identified by students include factors like test taking anxiety, test takers’ negative attitudes toward the test, and their lack of proficiency in English (Fook, Sidhu, Rani, & Aziz, 2011). Another high-stakes oral English test in Malaysia is the oral component of the Malaysian University English Test (MUET). Rahmat, et al. (2015) wrote that the MUET was introduced by the Malaysian Education System in 1999. MUET measures all the four skills of a language including speaking. They studied 225 undergraduate students at a Malaysian university and found that the scores of MUET positively correlate with the scores of the students’ academic achievement. d) Hong Kong: According to Qian (2008), English language tests in Hong Kong exist in the form of public examinations and school-based assessment at primary and secondary levels. Qian (2008) summarized that the Use of English (UE) exam, which is a part of Hong Kong Advanced Level Examinations (HKALE), is administered to assess students’ English language proficiency for university education as well as future employment. One of the five sections that the test has is an oral English test. This section involves a group interaction task for about 20 minutes. It carries 18% of the total score. Like China and Malaysia, this section was added by the Hong Kong examination authority to create a positive washback on the school English curricula. e) Japan: One of the major English tests in Japan is known as EIKEN which stands for Test in Practical English Proficiency (EIKEN speaking test, n.d.). According to the information given in EIKEN speaking test (n.d.), this test has two stages: stage 1 consists of listening, reading and writing. Successful completion of this stage leads to the second stage which tests students’ speaking proficiency through a face-to-face interview. Piggin (2011) mentioned that the face-to- face interview with two examiners may reveal the validity of the test, but the overall research 14 into the EIKEN oral test component, its scoring scale, and other details seemed to be scarce. Sasaki (2008) wrote that the Japanese Ministry of Education included subjects on oral communication in 1989 for high school students. He also pointed out that the Japanese government’s increased emphasis on developing students’ practical English proficiency led some senior high schools and universities to replace part of their admission test by the TOEFL (Test of English as a Foreign language) and TOEIC (Test of English for International Communication). f) India: In India, the English language tests are going through a process of reformation. As explained by Ramanathan (2008), many institutions are still adhering to the old system where only reading and writing are assessed, except a few Boards of Education and the state of Tamil Nadu which have ventured to make their tests more communicative by introducing listening and speaking sections. Although the author did not elaborate on the types of oral tasks involved in these tests, he wrote about the problems of conducting listening and speaking tests, such as large class size, students of varying proficiency levels in the same class, the lack of validity of the grading process, emphasizing public speaking skills over communicative skills, and the lack of access to technological resources. In short, research on oral language testing in the countries reviewed above provides evidence of existing oral English tests which are administered publicly by the authorized body of the government. Studies on these types of oral English tests focused on the types of oral tasks, their validity and reliability, and their washback effects on the curriculum and classroom teaching practices. These tests include a varied number of task types. Some of the most common oral tasks in these tests are individual presentations, peer interaction, and oral proficiency interviews. A number of researchers in these Asian countries investigated whether the oral tasks involved in the tests are communicative enough or not (Noor et al, 2010; Zhan & Wan, 2016). 15 Some of the studies also dealt with identifying oral testing problems and challenges, such as raters’ bias or lack of understanding of grading criteria (Zhang & Elder, 2009; Sidhu, Fook, & Mohamad, 2011). Asian countries that have not yet changed their assessment systems. Unlike the Asian countries discussed above, oral assessment is still not a direct part of the public English language examinations in Pakistan, Korea, and Thailand. A description of the English assessment systems of these countries is as follows. a) Pakistan: In Pakistan, the Higher Secondary Examination (HSC) includes a test on English language but this only measures students’ performance in grammar, translation, and literature but there is no speaking component (Adnan & Mahmood, 2014; Jilani, 2009). Adnan and Mahmood (2014) surveyed fifty teachers teaching at the higher secondary level to investigate possible washback of the HSC exam. They reported that 86% of the teachers emphasized only the contents that are important for the exam. They pointed out that students’ competence in listening and speaking is very low. These skills are not the focus in their language classes because these are not tested in the final exam. b) South Korea: Choi (2008) reported that the Korean Scholastic Aptitude Test (KSAT) is a high-stakes exam which has an English section with a strong washback effect. According to Choi (2008), ‘The most serious problem is that the KSAT does not include speaking and writing components, thus leading to very little, if any, teaching of speaking and writing at high school’ (p. 41). c) Thailand: The language test scenario at the post-secondary level in Thailand is a little bit different. In 2014, 86 universities in Thailand implemented the Central University Admission System (CUAS) (Cherngchawano & Jaturapitakkul, 2014). The CUAS requires candidates to 16 pass the General Aptitude Test (GAT) as one of the undergraduate admission requirements. One of the sections of this test is the English speaking section, which is basically composed of multiple-choice questions (General Aptitude Test [GAT], 2017). However, Prapphal (2008) explained that standardized language tests like IELTS, TOEFL, and TOEIC are becoming increasingly popular for university admission as many of the universities are running international programs. He also reported that few graduate schools in Thailand have included TOEFL in their admission requirements for doctoral programs. To sum up, the Asian language testing situations as discussed above indicate that Asian countries are now noticeably divided between two groups: a) countries having an established form of oral tests in their language assessment system and b) countries where there are no publicly administered oral tests. Among the countries that administer public oral tests, there exist a number of variations in terms of the types of oral tasks, the length and weight of the tests, the amount of interaction involved, the levels of education at which these are administered, and the problems and challenges involved in administering these tests. Research focus of the two groups of Asian countries as discussed above is also different. On one hand, researchers of the countries that lack publicly administered oral tests emphasized the necessity of incorporating oral tests in the major school/college exams, pointing out that the absence of any oral tests has a strong negative washback on the curriculum and classroom practices. On the other hand, researchers from China, Singapore, and Malaysia investigated various issues related to their existing oral tests with a view to recommending further improvements. In spite of these differences, the importance of administering oral English tests has been widely acknowledged by many Asian researchers (e.g. Jilani, 2009; Khan, 2010; Qian, 2008; Ramanathan, 2008; Sasaki, 2008; Zheng & Cheng, 2008). It can be assumed that studies on the assessment policies of the countries that 17 have existing oral assessment systems can help lead to possible ways of introducing oral assessment in countries like Bangladesh, Pakistan, South Korea, and to some extent India. While all these studies, as reviewed in this chapter, contribute to our understanding of the current oral English assessment systems in the Asian context, they offer little information about the challenges and barriers to oral testing, especially in countries like Bangladesh, India, Pakistan, and South Korea, where oral tests are not part of the mainstream English language exams. This gap in the previous research has been one reason why I embarked on the present study, in which I devoted my attention to identifying the major obstacles to oral testing during university admission at the public and private universities in Bangladesh. The following four questions guided my investigation in this study: Research Questions 1. With what types of oral English tests are Bangladeshi students familiar? 2. What attitudes do students have toward taking an oral English test for university admission? 3. What are the challenges and barriers to oral testing during university admission? 18 Design of the Study METHODS In this study, I used a mixed-methods approach in which I involved both quantitative and qualitative data. To investigate the three research questions, I used quantitative data collected from an online survey. To get further details into the students’ and teachers’ attitudes toward oral tests and their perceptions of the challenges and barriers to oral testing during admission, I interviewed some teachers who participated in the survey. I used the interview data to supplement the quantitative results of the survey. Participants Eleven EFL teachers and 122 EFL students from two Bangladeshi universities responded to the survey. One of these universities was a privately owned university located in Dhaka, the capital and largest urban area of Bangladesh. The other one was a public university located in a relatively less developed part of Mymansingh in Bangladesh. All of the students had learnt English as a foreign language for about 12 years. At the time of the survey, they were enrolled in English language and literature programs at the undergraduate level. Among the 11 teachers I surveyed, five teachers were further interviewed. Instruments I used two instruments for this study: a questionnaire and a set of interview questions. The questionnaire consisted of three sections in accordance with the three research questions: a) students’ familiarity with oral tests, b) students and teachers’ attitudes toward incorporating oral tests during admission, and c) their perceptions of the challenges and barriers to oral testing. In the first section of the survey, students were specifically asked to select the types of oral tests they took at three different stages: the higher secondary school exam, the university admission 19 exams, and the English course final exams at university. In the second section of the survey, both students and teachers were asked to mark their level of agreement on 18 Likert scale statements about the washback effects of oral testing during admission and the importance of incorporating oral tests into admission exams. In the third section of the survey, 41 Likert scale statements were included to learn about participants’ understanding of the challenges and barriers to oral testing during university admission (Appendix B). I used 7-point Likert scales for the survey because some researchers (Cicchetti, Showalter, & Tyrer, 1985; Oaster, 1989; Preston & Colman, 2000) claimed that a 7-point scale can show the highest test reliability. For section two and three, both students and teachers responded to the same version of the questionnaire. The first section of the survey was designed only for the students, so no teachers took it. To get some details of the participants’ perceptions and attitudes toward oral testing, I conducted semi-structured interviews with five teachers (Appendix A). Mackey and Gass (2005) wrote that in a semi structured interview researchers can use “a written list of questions as a guide, while still having the freedom to digress and probe for more information” (p.173). Hence, my main purpose for choosing a semi-structured interview type was to ensure enough participation from the interviewees. Both the survey and the interview that I used for this study was conducted online. I used Qualtrics to design and distribute the survey and conducted my follow-up interviews via Facebook and Whatsup. Procedure After getting the IRB approval for collecting data from students and teachers, I contacted my participants via Facebook and email. From each of the two universities, I identified and contacted two EFL teachers and provided them with details on how I would conduct the survey and interview online. Based on their responses, I created a closed group on Facebook (English 20 Exams in Bangladesh) where 184 interested participants joined as members. I posted the survey link generated by Qualtrics on the Facebook page and finally 133 participants took part in the survey. Later, five teachers were interviewed via Facebook messenger and Whatsup. Before the survey, the participants were asked for their consent to participate. The Likert scale statements were randomized to ensure the reliability of the questionnaire. Data Analysis For this study, I used descriptive statistics in SPSS to analyze and present the quantitative data I collected from the survey. For the first part of the survey, which included data on students’ familiarity with oral tests, I analyzed participants’ responses using frequency and percentage statistics. The second and third parts of the survey consisted of 59 Likert-scale items, which I included to measure 14 separate scales. I considered examining these 14 scales in this study because I thought these would be highly relevant to Bangladeshi oral testing situation. As I could not find any recent study that included a survey on students’ and teachers’ attitudes toward and perceptions of oral testing during admission, I had to rely on my language teaching and testing experience while determining the scales and items of the survey used in this study. I estimated the reliabilities of these scales in SPSS using Cronbach’s alpha to measure the internal consistency of the items in each scale. I also checked the item total correlations and the alpha value of the scale if each item were deleted. For the ordinal data collected through each Likert-scale item of the survey, I analyzed the central tendency as well as the dispersion of the data by five statistical measures: mean (M), median (Mdn), standard deviation (SD), minimum (min) and maximum (max). I conducted the nonparametric Mann-Whitney U test to check 21 whether any significant differences existed between students’ and teachers’ responses to the Likert-scale statements. The interview data were analyzed qualitatively by interpreting the participants’ responses to interview questions. I sorted out the recurrent themes emerging from the interview data and used those to supplement my discussion on the quantitative analysis. 22 RESULTS RQ1: With What Types of Oral Tests are Bangladeshi Students Familiar? To learn about student participants’ familiarity with oral tests, they were asked about the types of oral tests they took before and the spoken English courses they had completed so far. Among 122 student participants, 118 students across four undergraduate years responded to these survey questions. First, students’ overall experience in taking oral tests in relation to their familiarity with spoken English courses was analyzed using descriptive statistics. Table 1 shows the frequencies and percentages of participants’ experience in taking oral tests at the higher secondary school (HSC) exam, during university admission, and as part of university courses. Table 1: Students’ Experience Taking Oral Tests Study Level Student (N) Familiarity with Spoken English courses Oral test Oral test Oral test taken taken at HSC taken during admission as part of university courses Year 1 Year 2 Year 3 Year 4 All 22 71 10 16 118 n 11 55 08 12 86 % 50 78.6 80 75.1 72.9 n 10 38 5 4 57 % 45.5 52.9 50 25 48.3 n 7 51 4 4 66 % 31.8 71.4 40 25 55.9 n 16 66 10 16 108 % 72.7 94.3 100 100 91.5 The majority (72.9%) of the students from Year 1 to Year 4 reported having familiarity with spoken English courses. According to their reports, the 3rd year undergraduate students were most familiar with spoken English courses, whereas half of the first year students had no experience with these types of courses. With regard to their oral test experiences, the majority of the students (91.5%) took oral tests at the university. Even though the first year students were less experienced than other three groups, 72.7% of these students took an oral test at the university. This was different from what all of the students reported about their higher secondary 23 school exam and university admission exam. On average, only 48.3% of the total student participants took oral tests at HSC and 55.9% during their university admission. Data on students’ experience with oral tests were further analyzed to see what types of oral tests they were mostly familiar with. In the survey, I specifically asked them how many tests they took among the following four: individual presentation (IP), oral proficiency interview (OPI), group oral (GO), and paired oral (PO). There was a blank for them to list any other tests they took apart from these four types but no one reported any. The frequency distribution of the data is shown in Table 2. Table 2: Students’ Overall Familiarity with Different Types of Oral Tests Study Level Student Past Past Past (N) experience of taking experience of experience taking GO of taking PO IP n % 16 72.7 94.3 66 100 10 16 100 22 70 10 16 118 108 91.5 Year 1 Year 2 Year 3 Year 4 All n 6 26 5 11 48 % 27.3 37.1 50 31.3 40.7 n 8 16 5 11 40 % 36.4 22.9 50 31.3 33.9 Past experience of taking OPI n % 3 13.6 62.9 44 50 05 08 50 No Experience n % 5 22.7 1.4 1 0 0 0 0 60 49.2 6 5.1 Based on participants’ responses in Table 2, the most common type of oral test they took was individual presentation (91.5%), followed by oral proficiency interview (49.2%) previously. Group orals and paired orals were the least common among the oral tests they took, with group oral taken by only 40.7% and paired orals by 33.9% of the students. Another important finding was the difference between first-year students with the other three groups (in the 2nd, 3rd, and 4th year). About 22.7% of the first year students reported that they had no experience taking any oral tests, whereas almost everyone between the undergraduate year 2 and year 4 had some sort of exposure to oral testing. 24 Further details of students’ familiarity with different types of oral tests at three different stages (undergraduate level, higher secondary school level, and during university admission) of their education are given in Tables 3-5. Table 3: Students’ Experience Taking Oral Tests at Undergraduate Level Study Level Student Past Past Past Past No (N) experience of taking experience of taking experience of taking experience of taking Experience Year 1 Year 2 Year 3 Year 4 All 22 70 10 16 118 OPI GO PO IP n % 9.1 2 40 28 4 40 37.5 6 40 33.9 n % 27.3 6 27.1 19 5 50 68.8 11 41 34.7 n % 18.2 4 15.7 11 5 50 56.3 9 29 24.6 n % 63.6 14 84.5 60 10 100 100 16 100 84.7 n % 27.3 6 5.7 4 0 0 0 0 10 8.5 Among the four types of tests, individual presentations were given by the majority of the students (84.7%), but paired orals were taken by only 24.6% of the students. The other two types, group orals and oral proficiency interviews, were taken by about 34.7% and 33.9% respectively. The findings related to students’ overall experience with the types of oral tests as presented in Table 2 differ slightly with regard to group orals and oral proficiency interviews results in Table 3. At undergraduate level, students’ experience taking group orals was slightly higher than that of oral proficiency interviews (Table 3), but when all three stages (i.e., HSC, admission, and undergraduate level) were considered, their experience taking group orals was slightly lower than their experience with oral proficiency interviews (Table 2). 25 Table 4: Students’ Experience Taking Oral Tests at Higher Secondary Level Past Study Level Student Past Past Past (N) experience of taking experience of taking experience of taking experience of taking No Experience Year 1 Year 2 Year 3 Year 4 All 22 70 10 16 118 OPI GO PO IP n % 9.1 2 13 18.6 10 1 0 0 16 13.6 n 2 15 1 3 21 % 9.1 21.4 10 18.8 17.8 n 6 9 3 1 19 % 27.3 12.9 30 6.3 16.1 n % 31.8 7 24 34.3 30 3 6.3 1 35 29.7 n % 54.5 12 32 45.7 40 5 68.8 12 61 51.7 More than half of the participants (51.7%) reported that they had no experience taking any oral tests at higher secondary level, which is different from what they reported about their experience with oral tests at undergraduate level (Table 3), and during admission exams (Table 5). Among the tests they took at this level, individual presentations were the most common type (29.7%), and the least common were oral proficiency interviews. Table 5: Students’ Experience in Taking Oral Tests in Admission Exam Study Level Student Past Past Past Past No (N) experience of taking experience of taking experience of taking experience of taking Experience Year 1 Year 2 Year 3 Year 4 All 22 70 10 16 118 OPI GO PO IP n 1 32 2 2 37 % 4.5% 45.7 20 12.5 31.4 n 2 9 1 0 12 % 9.1 12.9 10 0 10.2 n 2 4 1 2 9 % 9.1 5.7 10 12.5 7.6 n % 6 27.3 42.9 30 30 3 2 12.5 34.7 41 n % 15 68.2 27.1 19 60 6 12 75 44.1 52 Figures in Table 5 show that 44.1% students reported to have no experience with any oral tests during university admission. Although some students had experience with oral proficiency interviews (31.4%) and individual presentations (34.7%) to some extent, their experience with group or paired orals was very low, with group orals taken by only 10.2% and paired orals by 7.6% of the students. With regard to oral proficiency interviews, there is a difference between students’ responses for HSC level and admission exams. Tables 4 and 5 show that more students 26 (31.4%) reported having taken oral proficiency interviews during university admission than they did at higher secondary level in which only 13.6% of the students took oral proficiency interviews. However, students’ rate of taking individual presentations and oral proficiency interviews during university admission is consistent with their overall experience across three different stages (i.e., HSC, admission, and undergraduate level) as presented in Table 2. RQ2: What Attitudes Do Teachers and Students Have Toward Oral English Testing During University Admissions? A 7-point Likert scale questionnaire was used to measure respondents’ level of agreement or disagreement on 18 statements related to the second research question (i.e., what attitudes do students’ and teachers’ have toward oral testing during university admission?). I included these 18 statements in this section to gauge participants’ opinions about four different issues or scales in Table 6. The seven choices and their numeric values were coded as: Strongly Agree (= 7), Agree (= 6), Somewhat Agree (5), Neither Agree nor Disagree (4), Somewhat Disagree (3), Disagree (2), Strongly Disagree (1). To understand how strongly the participants agreed or disagreed to the statements of these four scales, I calculated the central tendency of the data using two measures, mean and median. I measured the standard deviation, minimum, and maximum to see how much their responses were spread out. Using these two sets of measures, I interpreted the strength and importance of each scale in discussing participants’ attitudes toward incorporating oral tests in university admission exams. Responses of students and teachers were processed separately for this section. Among 122 student participants, 100 responded to the items. Their responses were included in the statistical analysis to find out the internal consistency of the scales using SPSS. Distribution of the items per scale and their internal consistency are presented in Table 6 below. 27 Table 6: Reliability of the Scales Based on Students’ Responses Sl. Scales Cronbach’s Alpha No. of items 1. Oral tests as an important component of admission exams 2. Positive effects of including oral tests in admission exam 3. Oral test scores’ importance to admission decisions 4. Negative effects of excluding oral tests in admission exam 7 4 4 3 .755 .733 .657 .622 All four scales, in general, had a moderate level of internal consistency, and the values of Cronbach’s alpha are presented in the table in the order of high to low. The most reliable among the four is the first scale (7 items, p = .755). The mean (M), median (Mdn), standard deviation (SD), minimum (min), and maximum (max) of these four scales are shown in Table 7. Data have been arranged chronologically in the table in the order of high to low mean scores. Table 7: Students’ Perceptions About Oral Testing During Admission Sl. Scales M Mdn No. of items SD Min Max 1. Positive effects of including oral tests in admission exam 2. Negative effects of excluding oral tests in admission exam 3. Oral tests as an important component of admission exams 4. Oral test scores’ importance to admission decisions 4 3 7 3 6.31 6.25 0.65 4.00 7.00 5.89 6.00 0.94 2.00 7.00 5.59 5.86 0.97 2.67 7.00 5.30 5.50 1.11 2.00 7.00 As shown in Table 7, the mean score of 6.31 for Scale 1 suggests that the participants had a relatively high level of agreement to all four statements related to the positive effects of including oral tests. The second highest level of agreement was reached for Scale 2 (negative effects of excluding oral tests) with a mean score of 5.89. They had a moderate degree of agreement on Scale 3 (M = 5.30, SD = 1.11) and Scale 4 (M = 5.59, SD = 0.97), with scores ranging between 2.00 (disagree) to 7.00 (strongly disagree). An item-by-item analysis as shown in Table 8 can further help expand the analysis presented in Table 7. 28 Table 8: Students’ Responses to Individual Items Sl. Scale Item ID Item Description Q16_4 Oral test during 1. Positive effects of including oral tests in admission exam M 6.39 Mdn SD Min Max 6.00 7.00 2.00 0.77 Q16_5 Q16_7 admission can make students practice speaking in English more than they do now. If oral tests are administered, students will focus on developing their oral communication skills. If students have to take oral tests, they will be motivated to speak in English. 6.37 7.00 0.89 2.00 7.00 6.26 6.00 0.93 2.00 7.00 Q16_3 Oral test during 6.13 6.00 0.87 2.00 7.00 admission can help diagnose students' oral proficiency. 2. Negative effects Q16_10 Students are reluctant 6.02 6.00 1.09 2.00 7.00 of excluding oral tests in admission exam to practice speaking English because they do not have to take any oral tests. Q16_13 Absence of oral test has 6.02 6.00 0.96 2.00 7.00 negative influence on developing EFL students' communicative competence. Q16_9 Absence of oral test has 5.57 6.00 1.49 1.00 7.00 5.94 6.00 1.20 2.00 7.00 3. Oral tests as an Q10_2 important component of admission exams negative effects on developing students' oral English proficiency. For university admission, English speaking test is as important as English reading and writing. 29 Table 8 (cont’d) Q10_4 During university 5.75 6.00 1.37 2.00 7.00 Q10_3 Q10_1 Q10_6 admission, both public and private universities should administer an English speaking test. For admission to any undergraduate programs, English speaking test is equally important as other sections like mathematics, general knowledge, and Bengali. It is important for a student to take an English oral test before getting admission to any university. It is important to test students' English speaking proficiency before offering admission to undergraduate English language programs. 5.71 6.00 1.52 1.00 7.00 5.60 6.00 1.51 1.00 7.00 5.41 6.00 1.47 2.00 7.00 Q10_5 Oral English tests 5.40 6.00 1.70 1.00 7.00 should be made compulsory during university admission exam. Q10_7 Universities should measure candidates' proficiency in spoken English before offering admission to any undergraduate programs. 5.40 6.00 1.63 1.00 7.00 30 Table 8 (cont’d) 4. Oral test scores’ importance to admission decisions Q10_12 Candidates' 5.77 6.00 1.31 1.00 7.00 performance in oral English tests should carry points in university admission exams. Q10_13 Students securing high 5.23 6.00 1.72 1.00 7.00 scores in oral test during admission should get priority in receiving admission offer. Q10_15 Students' performance 5.17 6.00 1.65 1.00 7.00 in oral test should be considered while offering tuition award during admission test. Q10_14 Universities should 5.05 6.00 1.71 1.00 7.00 require candidates to pass an oral test for admission. The mean and median scores of the 18 items listed in Table 8 reveal that students, on average, agreed to all 18 statements at varying degrees. Of the 18 items, the highest mean score (6.39) can be observed for item no. Q16.4 (i.e., oral test during admission can make students practice speaking in English more than they do now), with a standard deviation of .77 suggesting that most students had similar ratings for this statement. The lowest mean (5.05) was for item Q10_14. (i.e., universities should require candidates to pass an oral test for admission). The standard deviation (1.71) for this item also suggests that students’ ratings were slightly spread out between a minimum of 1.00 (strongly disagree) to the maximum of 7.00 (strongly agree). It can also be observed from Table 8 that the students’ overall level of agreement on the first two scales (i.e., the positive effects of including oral tests, and the negative effects of excluding oral tests in admission exam) was higher than on the last two scales (i.e., oral tests as 31 an important component of admission exams, and oral test scores’ importance to admission decisions). Among the 11 teachers who participated in this study, 10 teachers responded to the second part of the survey, which related to the second research question. The reliability of the scales based on their responses is shown the following table. Table 9: Reliability of the Scales Based on Teachers’ Responses Sl. Scales Cronbach’s Alpha No. of items 1. Negative effects of excluding oral tests in admission exam 2. Oral tests as an important component of admission exams 3. Oral test scores’ importance to admission decisions 4. Positive effects of including oral tests in admission exam 3 7 4 4 .836 .768 .247 .245 Although the reliabilities of the first two scales, as presented in Table 9, were high (.836 and .768), the α coefficients of the last two scales (.247 and .245) in the table showed low internal consistency, which was different from the trend observed in students’ data for the same part of the survey (see Table 1). One possible reason for this low reliability is the small sample size. Only ten teachers took part in the survey and this might have affected the α values. Since taking out any items form these two scales did not increase their reliabilities, no items were deleted. The central tendency of the data set, as well as its dispersion is presented in Table 10. Table 10: Teachers’ Perceptions About Oral Testing During Admission Sl. Scales M No. of items Mdn SD Min Max 1. Positive effects of including oral tests in admission exam 2. Negative effects of excluding oral tests in admission exam 3. Oral tests as an important component of admission exams 4. Oral test scores’ importance to admission decisions 4 3 7 3 6.55 6.75 0.47 5.75 7.00 6.30 6.50 0.92 4.00 7.00 6.20 6.21 0.57 5.00 7.00 6.05 6.13 0.66 5.00 7.00 32 Table 10 shows that teachers, on average, reached a moderate to high level of agreement in responding to the items in all four scales. Mean scores of the first two scales (i.e., positive effects of including, and negative effects of excluding oral tests during admission) were relatively higher than the latter two scales (i.e., oral tests as an important component of admission exams, and oral test scores’ importance to admission decisions). This finding is similar to the students’ responses as presented in Table 7. The standard deviations of the scales, ranging from 0.47 to 0.92, indicate that teachers’ responses did not deviate too much from each other. Among the four scales, the lowest minimum score (4.00 = neither agree nor disagree) was detected in one scale, and the maximum score was the same for all four (7.00 = strongly agree). An item-by-item analysis is presented in Table 11. Table 11: Teachers’ Responses to Individual Items Sl. Scale Item Description 1. Positive effects of including oral tests in admission exam Item ID Q16_5 Q16_7 M Mdn SD Min Max 6.90 7.00 0.32 6 7 6.90 7.00 0.32 6 7 If oral tests are administered, students will focus on developing their oral communication skills. If students have to take oral tests, they will be motivated to speak in English. Q16_4 Oral test during 6.60 7.00 0.70 5 7 admission can make students practice speaking in English more than they do now. Q16_3 Oral test during 5.80 6.00 1.48 2 7 admission can help diagnose students' oral proficiency. 33 Table 11 (cont’d) 2. Negative effects of excluding oral tests in admission exam Q16_9 Absence of oral test 6.30 6.50 0.95 4 7 has negative effects on developing students' oral proficiency. Q16_10 Students are reluctant 6.60 7.00 0.52 6 7 to practice speaking English because they do not have to take any oral tests. Q16_13 Absence of oral test 6.00 6.00 1.49 2 7 3. Oral tests as an Q10_1 important component of admission exams Q10_6 has negative influence on developing EFL students' communicative competence. It is important for a student to take an English oral test before getting admission to any university. It is important to test students' English speaking proficiency before offering admission to undergraduate English language programs. 6.40 6.50 0.70 5 7 6.30 6.50 0.95 4 7 Q10_4 During university 6.60 7.00 0.52 6 7 admission, both public and private universities should administer an English speaking test. 34 Table 11 (cont’d) 4. Q10_3 6.20 6.00 0.79 5 7 For admission to any undergraduate programs, English speaking test is equally important as other sections like mathematics, general knowledge, and Bengali. Q10_5 Oral English tests 6.20 6.00 0.79 5 7 Q10_2 should be made compulsory during university admission exam. For university admission, English speaking test is as important as English reading and writing. Q10_7 Universities should measure candidates' proficiency in spoken English before offering admission to any undergraduate programs. 5.90 6.00 1.37 3 7 5.80 6.00 0.79 4 7 5. Oral test scores’ Q10_12 Candidates' 6.50 6.50 0.53 6 7 importance to admission decisions performance in oral English should carry points in university admission exams. Q10_15 Students' performance 6.10 6.00 0.57 5 7 in oral test should be considered while offering tuition award during admission. Q10_14 Universities should 6.00 6.00 1.25 3 7 require candidates to pass an oral test for admission. Q10_13 Students securing high 5.60 6.50 1.90 2 7 scores in oral test in admission should get priority in receiving admission offer. 35 The descriptive analyses of individual items in Table 11 show that two items, Q16_5 and Q16_7, have the highest mean score (6.90) with a standard deviation of 0.32, and the lowest mean score (5.60) can be seen for Q10_13, which has a standard deviation of 1.90. For the first scale (positive effects of including oral tests), teachers’ responses ranged between high and moderate level of agreement for the following three statements: Q16_5. If oral tests are administered, students will focus on developing their oral communication skills (M = 6.90, Mdn = 7), Q16_7. If students have to take oral tests, they will be motivated to speak in English (M = 6.90, Mdn = 7), and Q16_4. Oral test during admission can make students practice speaking in English more than they do now (M = 6.60, Mdn = 7). Their agreement level was relatively low for the last statement of this scale, i.e. Q16_3. Oral test during admission can help diagnose students' oral proficiency (M = 5.80, Mdn = 6). Teachers’ ratings for this statement varied between a minimum score of 2 (Disagree) to a maximum of 7 (Strongly Agree). Among the three items in the second scale (negative effects of excluding oral tests from admission), teachers agreed more on item Q16_10, i.e. students are reluctant to practice speaking English because they do not have to take any oral tests (M = 6.60, SD = 0.52, min = 6, max = 7). A similar level of agreement was seen for item Q10_4 (during university admission, both public and private universities should administer an English speaking test) in the third scale (oral tests as an important component of admission exams). In the fourth scale (oral test scores importance for admission decisions), mean scores of three items (Q10_12, Q10_14, and Q10_15) were at or above 6.00. This is slightly higher than the mean score of 5.60 as seen for the statement: Q10_13. Students securing high scores in oral test during admission should get priority in receiving an admission offer. Teachers’ responses were spread between a minimum score of 2 (Disagree) to a maximum of 7 (Strongly Agree). 36 To compare students’ and teachers’ attitudes towards oral testing during admission, the Mann-Whitney U test was conducted because the study involved data from two independent samples (students and teachers). The central tendency of the two data sets indicated that the data were not normally distributed, and therefore, I chose to run a nonparametric test. The test was run in SPSS and its results are presented in Table 12. Table 12: Comparison Between Teachers’ and Students’ Attitudes Sl U Mean Rank Scales 1. Positive effects of including oral tests in admission exam Student Teacher 58.98 71.15 z p r 433.5 -1.088 0.277 -.099 2. Negative effects of excluding oral 57.28 77.45 350.5 -1.827 0.068 -.168 tests in admission exam 3. Oral tests as an important 58.66 80.75 347.5 -1.929 0.054 -.176 component of admission exams 4. Oral test scores’ importance in admission decisions 57.58 80.2 333 -2.012 0.044 -.185 Results of the first two scales in Table 12 did not show any significant difference between teachers’ and students’ attitudes. Difference between students’ and teachers’ attitudes toward the third scale in Table 12 was marginally significant (U= 347.5, p = 0.054). The difference between these two groups’ attitudes toward the fourth scale (i.e., oral test scores importance in admission decisions), as presented in Table 12, was statistically significant (U= 333, p = 0.044) although the effect size was small (r = .185). RQ3: What are the Challenges and Barriers to Oral Testing During University Admission? To find out the participants’ perceptions of the challenges and barriers to oral testing during university admission, I used SPSS to analyze the participants’ responses to the third section of the survey consisting of 41 statements (Appendix B). The participants responded to each statement on a 7-point Likert scale which was coded as: Strongly Agree = 7, Agree = 6, Somewhat Agree = 5, Neither Agree nor Disagree = 4, Somewhat Disagree = 3, Disagree = 2, 37 Strongly Disagree = 1. The 41 items of this section represented 10 different scales as listed in Table 13. The reliabilities of these scales were measured separately for both students and teachers. Table 13 presents the α coefficients of the scales based on students’ responses. No. of items Cronbach’s Alpha Table 13: Reliability of the Scales Based on Students’ Responses Sl. Scales 1. Examiners’ lack of training 2. Lack of funds 3. Huge number of students 4. Student’s anxiety about L2 speaking 5. Lack of uniform grading policy 6. Lack of program/university policy 7. Lack of staff 8. Lack of administrative support 9. Students’ lack of familiarity with oral tests 10. Lack of tech support 4 5 3 5 5 3 3 4 5 4 .872 .844 .839 .814 .781 .732 .721 .707 .681 .664 In total, 122 students took part in the survey, although there were a number of missing values which were omitted by SPSS while analyzing the items. Items in all ten scales revealed certain degrees of internal consistency, which can be understood from the chronological listing of the scales from high to low in Table 13. The first four scales turned out to be the most reliable ones with α coefficients between .872 and .814. The descriptive statistics of the scales and the items are presented in Tables 14 and 15. Table 14: Challenges and Barriers to Oral Testing as Viewed by Students Sl. Scales M Mdn No. of Items SD Min Max Student’s anxiety about L2 speaking 1. Students’ lack of familiarity with oral tests 2. Lack of tech support 3. Lack of uniform criteria 4. huge number of examinees 5. Lack of program policy 6. Examiners’ lack of training 7. Lack of administrative support 8. Lack of staff 9. 10. Lack of funds 5 5 4 5 3 3 4 4 3 5 5.91 5.80 5.52 5.23 5.19 5.16 5.05 4.95 4.67 4.52 6 6 6 5.40 5.67 5.67 5.50 5 4.67 4.60 38 2 0.861 2.60 0.823 2.60 1.014 1.100 1.60 1.345 1.387 1.33 1.403 1.50 1.225 1.25 1.491 1.33 1.405 1.60 1 7 7 7 7 7 7 7 7 7 7 The mean scores of the scales as presented in Table 14 reveal that students’ responses, on average, clustered more around the middle points (Neither Agree nor Disagree = 4, and Somewhat Agree = 5) of the Likert scale. The highest mean (5.91) was observed for the first scale (i.e., student’s anxiety about L2 speaking) and the lowest (4.42) for the last scale in the list (i.e. lack of funds). Although the standard deviations of the scales are quite similar, the table shows that the data of the 9th scale (Lack of staff) was relatively more spread out than the other (SD = 1.49, min = 1.33, max = 7). The maximum value in all scales was 7 while the minimum ranged between 1 and 2.6. Further details of the individual scales are presented in Table 15. Table 15: Students’ Responses to Individual Items Sl. Scale Item Description Item ID Q15_13 Oral tests are difficult for 1 Students’ anxiety about L2 speaking students because they are shy of speaking in English. Q15_10 Oral tests are difficult for students because they feel nervous in taking oral tests. Q15_14 Oral tests are difficult for students because they lack confidence in speaking in English. M Mdn SD Min Max 6.09 6 0.76 3 7 6.02 6 1.07 2 7 5.93 6 1.19 2 7 Q15_11 Oral tests are difficult for 5.89 6 1.07 2 7 students because they often feel a high level of anxiety in taking oral tests. Q15_7 Q15_12 Oral tests are difficult for students because they are afraid of failing the test. It is difficult for students to take speaking tests because they are more accustomed to taking reading and writing tests only Students' lack of familiarity with oral tests 2 5.87 6 1.12 2 7 6.11 6 0.98 2 7 39 Table 15 (cont’d) Q15_9 Q15_8 Q15_5 Q15_6 Q15_2 Q15_4 Q15_3 Q15_1 Q12_21 Q12_23 3 Lack of tech support 4 Lack of uniform grading criteria Speaking test is difficult for students because they never practice or prepare for any oral test in their undergraduate English classes. Speaking test during university admission is challenging to the students because they did not take any oral tests during the SSC or the HSC exam. It is difficult to do well in the oral test because students are not familiar with different types of speaking tests. It is not easy to take a speaking test because students never practiced it in their English classes in school. It is not possible to administer speaking test using computers in our universities. It is difficult to rate huge number of test takers because there is no use of automated scoring. It is difficult to find technological support in recording students oral test data It is difficult to administer a speaking test because there is a lack of technological support. It is difficult to rate oral tests appropriately because there is a lot of variation among raters’ judgments. It is difficult to rate oral tests because there is no fixed policy for rating. 5.97 6 1.02 2 7 5.95 6 1.04 2 7 5.77 6 1.16 2 7 5.41 6 1.55 1 7 5.84 6 1.25 1 7 5.57 6 1.33 1 7 5.55 6 1.38 2 7 5.48 6 1.39 1 7 5.59 6 1.32 1 7 5.47 6 1.35 1 7 40 Table 15 (cont’d) Q12_24 It is difficult to evaluate speaking test because there is no clear evaluation criteria for different types of oral tests. 5.4 6 1.33 1 7 Q12_20 Rating oral tests can be 5.11 6 1.51 1 7 difficult because it is influenced by raters’ backgrounds and personal preferences It is difficult to rate oral tests because the ratings of oral tests can be subjective. It is difficult to arrange any speaking test by the teachers or examiners because there are a lot of test takers. The number of candidates is too large to be tested orally by examiners during university admission. I think the most significant barrier to oral testing is the huge number of candidates. Q12_22 Q12_4 Q12_5 Q12_6 5.11 6 1.47 2 7 5.34 6 1.51 1 7 5.2 6 1.61 1 7 5.18 6 1.55 1 7 Q15_15 Students do not prefer to take 5.22 6 1.71 2 7 any speaking test because their programs do not require that. Q15_17 English teachers do not take any oral test because it is not required by the departmental/university policy. 5.16 6 1.72 1 7 5 High number of examinees 6 Lack of program policy Q15_16 Students do not feel any 5.1 6 1.72 1 7 interest in taking oral tests because these do not add any points to their final grade. Q12_12 Oral testing is difficult because 5.3 6 1.57 1 7 many examiners do not have enough training on it Q12_14 Oral testing is difficult because 5.22 6 1.51 1 7 examiners never took the test when they were students. 7 Examiners' lack of training 41 Table 15 (cont’d) Q12_13 Oral testing is difficult because 5.14 6 1.67 1 7 many examiners do not know which types of oral tests they should administer for different purposes. Q12_15 Oral testing is difficult because 5.11 6 1.65 1 7 examiners did not receive any training on rating different oral tests. Q12_16 Oral testing is not easy because 5.52 6 1.45 1 7 8 Lack of administra tive support Q12_17 Q12_19 Q12_18 9 Lack of staff Q12_ 3 Q12_1 Q12_2 5.18 6 1.57 1 7 4.98 6 1.59 1 7 4.48 5 1.95 1 7 5.26 6 1.62 1 7 4.68 5 1.87 1 7 4.39 5 1.91 1 7 there is no university/department policy for administering the test. It is difficult to administer oral test because it takes a lot of time. I think there is a lack of logistic support in administering any oral test during admission. It is difficult to administer oral test because there is not enough rooms or spaces for that. It is not feasible to include a speaking test in our university admission exam because it is not possible to test all candidates by a limited number of teachers. It is difficult to arrange a speaking test during university admission because the number of examiners in my Department is not sufficient. It is not easy to arrange any speaking test during our university admission because there is a lack of staff for administering the test. 42 Table 15 (cont’d) 10 Lack of Q12_10 funds Q12_9 Q12_11 Q12_8 Q12_7 It is not easy to arrange a speaking test because there is no allotment of funds for administering such a test. It is difficult to administer any large-scale speaking tests because the funds are not enough to train testers or raters. I think the lack of fund is the most challenging aspect of administering the oral test. It is difficult to administer any speaking tests because the funds are not enough to pay examiners. It is difficult to arrange any speaking tests during university admission because it is costly. 4.91 6 1.63 1 7 4.74 6 1.74 1 7 4.72 5 1.74 1 7 4.51 5 1.75 1 7 4.38 5 1.84 1 7 Descriptive statistics of individual items in Table 15 show that statement Q12_7 (it is difficult to arrange any speaking tests during university admission because it is costly) in Scale 10 (Lack of funds) had the lowest mean score (M = 4.38), which indicate that students, on average, were more neutral about this statement. However, the standard deviation of the item was 1.84, which is slightly higher than all other items in Scale 10 (Lack of funds). The statement on which they seemed to agree the most was Scale 2, Q15_7 (It is difficult for students to take speaking tests because they are more accustomed to taking reading and writing tests only), which had a mean score of 6.11 and a standard deviation of 0.99. Among the 41 statements, the median values were 6 for 36 statements, and 5 for other six statements. Although students’ responses ranged between 1 (Strongly Disagree) and 7 (Strongly Agree), the standard deviation of the 41 items did not reveal any significant dispersion of their responses to these statements, which suggest that their views were similar in nature. The lowest 43 standard deviation (0.76) was detected in Q15_13 (Oral tests are difficult for students because they are shy of speaking in English) in Scale 1 (Students’ anxiety about L2 speaking), with data spread between a minimum of 3 and a maximum of 7. Statement Q12_18 (It is difficult to administer oral test because there is not enough rooms or spaces for that) in Scale 8 (Lack of administrative support) had the highest standard deviation (1.95). No. of items Cronbach’s Alpha Table 16: Reliability of the Scales Based on Teachers’ Responses Sl. Scales 1. Lack of funds 2. Huge number of examinees 3. Student’s anxiety about L2 speaking 4. Lack of program policy 5. Students’ lack of familiarity with oral tests 6. Lack of staff 7. Lack of tech support 8. Examiners’ lack of training 9. Lack of uniformity in grading policy 10. Lack of administrative support * One item was deleted. 5 3 5 3 5 2* 4 4 5 3* .923 .921 .885 .790 .758 .697 .694 .685 .657 .647 Reliability of the items based on the teachers’ responses as presented in Table 16 show that all the scales have internal consistency (p > .50), with the first three in the list being the most reliable ones. Scale 6 (with 3 items) and 10 (with 4 items) initially showed low reliability. This initial reliability estimate of the two scales might have been affected by the small sample size (10 teacher participants). However, deleting a particular item from each scale helped increase its reliability and therefore I excluded those two items from further analysis. Table 17: Challenges and Barriers to Oral Testing as Viewed by Teachers Sl. Scales M Mdn No. of items SD Min Max 1. Students' lack of familiarity with oral tests 2. Students' anxiety about L2 speaking 3. Lack of program policy 4. Lack of tech support 5. Huge number of examinees 6. Lack of uniformity in grading policy 5 5 3 4 3 5 6.28 6.18 6.07 6.05 5.62 5.54 6.40 6.20 6.33 6.13 6 5.80 0.719 0.727 1.004 0.856 1.343 0.854 5.20 5 3.67 4.50 3.67 4.40 7 7 7 7 7 7 44 Table 17 (cont’d) 7. Lack of staff 8. Examiners' lack of training 9. Lack of admin support 10. Lack of funds 2 4 3 5 5.30 5.28 5.27 5.10 5.50 5.13 5 5 1.494 1.121 1.153 1.476 2 3.50 3.67 3 7 7 7 7 Table 17 presents the mean scores of all ten scales in the order of high to low. Scale 1 (Students’ lack of familiarity with oral tests) in the table has the highest mean score (M = 6.28) and Scale 10 (Lack of funds) has the lowest (M = 5.10). The standard deviations of the data in these scales did not reveal any significant dispersion. The mean and the median values of these scales as presented in Table 17 suggest that teachers, on average, tended to have a moderate to high level of agreement with the statements of these scales. Further details can be observed in Table 18. Table 18: Teachers’ Responses to Individual Items Sl. Scale Item ID Item Description Q15_7 M Mdn 6.63 7 SD Min Max 0.52 6 7 6.63 7 0.74 5 7 6.5 7 0.76 5 7 6.13 6.5 1.36 3 7 1 Students' lack of familiarity with oral tests Q15_8 Q15_9 Q15_6 It is difficult for students to take speaking tests because they are more accustomed to taking reading and writing tests. Speaking test during university admission is challenging to the students because they did not take any oral tests during the SSC or the HSC exam. Speaking test is difficult for students because they never practice or prepare for any oral test in their undergrad English classes. It is not easy to take a speaking test because students never practiced it in English classes in school. 45 Table 18 (cont’d) Q15_5 It is difficult to do well in the oral test because students are not familiar with different types of speaking tests. 5.75 6 1.39 3 7 2 Students' anxiety about L2 speaking Q15_11 Oral tests are difficult for 6.63 7 0.52 6 7 students because they often feel a high level of anxiety in taking oral tests. Q15_10 Oral tests are difficult for students because they feel nervous in taking oral tests. 6.25 6.5 1.04 4 7 Q15_13 Oral tests are difficult for 6.38 6.5 0.74 5 7 students because they are shy of speaking in English. Q15_14 Oral tests are difficult for students because they lack confidence in speaking in English. Q15_12 Oral tests are difficult for students because they are afraid of failing the test. 6.25 6.5 0.89 5 7 6.25 6 0.71 5 7 3 Lack of program policy Q15_15 Students do not prefer to take 6.25 6 0.71 5 7 any speaking test because their programs do not require that. Q15_16 Students do not feel any 6.25 7 1.39 3 7 interest in taking oral tests because these do not add any points to their final grade. Q15_2 Q15_3 Q15_4 Q15_17 English teachers do not take any oral test because it is not required by the departmental/university policy. It is not possible to administer speaking test using computers in our universities. It is difficult rate huge number of test takers because there is no use of automated scoring. It is difficult to find technological support in recording students oral test data 4 Lack of tech support 5.88 6 1.64 2 7 6.5 7 0.76 5 7 6.38 6.5 0.74 5 7 6.13 7 1.46 3 7 46 Table 18 (cont’d) Q15_1 5 High Q12_6 number of examinees Q12_4 It is difficult to administer speaking test because there is a lack of technological support. I think the most significant barrier to oral testing is the huge number of candidates. It is difficult to arrange any speaking test by the teachers or examiners because there are many test takers. Q12_1 Q12_5 The number of candidates is too large to be tested orally during university admission. It is difficult to arrange a speaking test during university admission because the number of examiners in my Department is not sufficient. It is not easy to arrange any speaking test during our university admission because there is a lack of staff for administering the test. Q12_2 5.63 6 1.69 2 7 5.75 6.5 1.75 3 7 6 6 1.07 4 7 5.38 6 1.51 3 7 5.5 6 1.93 1 7 5 5.5 1.85 2 7 7 Lack of uniform grading criteria Q12_23 It is difficult to rate oral tests 6.13 6.5 1.13 4 7 because there is no fixed policy for rating. Q12_24 It is difficult to evaluate 6.13 6.5 1.13 4 7 speaking test because there is no clear evaluation criteria for different types of oral tests. Q12_21 It is difficult to rate oral tests appropriately because there is a lot of variation among raters’ judgments. 5.25 5.5 1.91 1 7 Q12_20 Rating oral tests can be 5.25 5 1.04 4 7 difficult because it is influenced by raters’ backgrounds and personal preferences 47 Table 18 (cont’d) Q12_22 It is difficult to rate oral tests 5 5 1.51 2 7 8 Lack of Q12_16 Oral testing is not easy because 6.13 6 0.84 5 7 because the ratings of oral tests can be subjective. administra tive support there is no university/department policy for administering the test. Q12_19 I think there is a lack of 5.5 6 1.77 2 7 logistic support in administering any oral test during admission. Q12_18 It is difficult to administer oral 4.25 3.5 2.12 2 7 test because there is not enough rooms or spaces for that. 9 Examiners’ Q12_15 Oral testing is difficult because 5.88 6 1.13 4 7 lack of training examiners did not receive any training on rating different oral tests Q12_12 Oral testing is difficult because 5.5 6 1.41 3 7 many examiners do not have enough training on it Q12_13 Oral testing is difficult because 5.38 5.5 1.77 2 7 many examiners do not know which types of oral tests they should administer for different purposes. Q12_14 Oral testing is difficult because 4.75 5.5 2.25 1 7 examiners never took the test when they were students. Q12_10 It is not easy to arrange a 5.88 6.5 1.36 4 7 Q12_9 speaking test because there is no allotment of funds for administering such a test. It is difficult to administer any large-scale speaking tests because the funds are not enough to train testers or raters. 5.38 6 1.85 2 7 10 Lack of funds 48 Table 18 (cont’d) Q12_11 I think the lack of fund is the 5.38 5.5 1.77 2 7 Q12_8 Q12_7 most challenging aspect of administering the oral test. It is difficult to administer any speaking tests because the funds are not enough to pay examiners. It is difficult to arrange any speaking tests during university admission because it is costly. 4.88 5.5 2.23 2 7 4.38 4.5 1.92 2 7 Table 18 shows that items under Scale 1 (Students’ lack of familiarity with oral tests) and Scale 2 (Student’s anxiety about L2 speaking) have relatively higher mean scores than the items in other scales. The highest mean score (6.63) was observed for the following three items: Q15_7, Q15_8, and Q15_11. For Q15_7 (it is difficult for students to take speaking tests because they are more accustomed to taking reading and writing tests only) the standard deviation was 0.52 and the minimum and maximum values ranged between 6 and 7. This was same for Q15_11 (Oral tests are difficult for students because they often feel a high level of anxiety in taking oral tests). The central tendency of these items suggests that teachers had a moderate to high level of agreement about these statements. The median, minimum, and maximum values of other items of Scale 1 and 2 indicate that no teachers strongly disagree to any of these statements. Items which had relatively low mean scores included Q12_18 (It is difficult to administer oral test because there are not enough rooms or spaces for that), Q12_14 (Oral testing is difficult because examiners never took any oral English test when they were students), and Q12_8 (It is difficult to administer any speaking tests because the funds are not enough to pay examiners). The standard deviations of these three items were higher than other items. Q12_18 had the lowest mean score (4.25), which suggest that teachers, on average, were more or less neutral about this statement. Their responses were slightly spread out for this item (SD = 2.12, min = 2, 49 max = 7). Item Q12_14 had the highest standard deviation (2.25). The mean score of this item was lowest (4.75) among other items in the ninth scale (i.e., examiners lack of training) in Table 18. The minimum (1 = Strongly Disagree) and maximum (7 = Strongly Agree) values for this item indicated that teachers had varying opinion about this statement. Based on the statistics of this item it would be wrong to think that Bangladeshi EFL teachers were not familiar with oral tests when studying English. Teachers’ responses implied that it was less important than other issues for the discussion of the barriers to oral testing. Descriptive statistics for individual items in Scale 8 (Lack of administrative support) reveal that teachers’ responses were quite varied. For the item Q12_16 (Oral testing is not easy because there is no university/department policy for administering the test) teachers had a relatively high level of agreement. However, for Q12_18 (It is difficult to administer any speaking tests because the funds are not enough to pay examiners), the teachers’ agreement level was relatively low. Items in this scale also had the lowest internal consistency as presented in Table 16. On the other hand, in Scale 2, teachers did not have any strong disagreement about any of the five statements. Overall, their level of agreement was high, with the mean scores ranging between 6.63 and 6.25. Table 19: Comparison Between Students And Teachers’ Responses Sl. Scales Mean Rank U 50 N 116 115 116 118 116 116 1. High number of examinees 2. 3. 4. Lack of funds Examiners’ lack of training Lack of uniform grading policy Lack of tech support Students’ lack of familiarity with oral tests Student’s anxiety about L2 speaking 5. 6. 7. Student Teacher 57.33 56.97 58.64 58.87 70.90 68.85 57.05 66.35 z r p 406.000 416.500 515.500 471.500 .213 .280 .886 .505 -1.245 -1.079 -0.144 -0.667 -.115 -.100 -.013 -.061 56.97 56.83 74.80 76.25 367.000 352.500 .104 .078 -1.626 -1.760 -.150 -.163 116 57.42 69.95 415.500 .254 -1.142 -.106 Table 19 (cont’d) 8. 9. 10. Lack of administrative Lack of program policy Lack of staff support 114 114 116 55.24 55.95 57.86 81.00 73.60 65.30 285.000 359.000 462.000 .017 -2.380 .102 -1.633 .500 -0.674 -.222 -.152 -.062 The students and teachers’ responses about the problems and challenges of oral testing during admission were compared by the non-parametric Mann-Whitney U test in SPSS. I chose the non-parametric option because the comparison involved two independent samples and the data were not normally distributed. The Mann-Whitney test results showed that there was a statistically significant difference in their responses for Scale 10 (Lack of program policy) (U = 285, p = .017). Their mean ranks revealed that teachers’ viewed the lack of program policy as more problematic than the students’ did. However, the effect size was small (r = -.222). For the rest of the scales in Table 19, no significant differences were observed. 51 DISCUSSION Predominance of Individual Presentation as an Oral Test Component Findings on students’ experiences taking oral tests suggest that most of the university students in Bangladesh are familiar with some forms of oral tests, although the extent of their familiarity varied considerably based on their undergraduate level and the types of oral tests they took. In this study, I asked the students to respond (by choosing yes or no) whether they took any of the following four types of oral tests: oral proficiency interview, group orals, paired orals, and individual presentation. It becomes clear from the data that among the four types of oral tests, the most familiar one was individual presentation, which was taken by 100% of the 2nd and 3rd year students. Although the percentage of 1st year students (72%) who took IP was lower than other three groups (Year 2, Year 3, & Year 4), the report suggests that they were more familiar with individual presentation than the other three test types. While individual presentation was the most familiar oral test category for the 118 respondents of this survey, paired and group orals seemed to be the least familiar ones. A possible reason why an individual presentation predominates as an oral test component at Bangladeshi universities is that almost all university programs require students to participate in individual presentation in class. Therefore, the English language programs at these universities also attempt to prepare students for this type of oral activity. The fact that 84.7% of the respondents reported to having participated in individual presentations at the undergraduate level further highlights the presence of this test type at Bangladeshi universities. Moreover, administering individual presentation in class by the course teachers saves the use of additional funds, logistic support, or administrative staff. In my interview with the teachers, some of them 52 expressed ideas that favored this argument. The following two interview excerpts can exemplify this: Excerpt 1, Male Teacher 1: Individual presentation is a better option…. If it’s about individual presentations, we don’t have to face any kind of bureaucratic process. We don’t need any permission from anyone. I can do it myself. I can arrange it in the classroom itself. Excerpt 2, Male Teacher 2: I prefer individual presentation as it is more convenient for me to administer such test keeping in mind the number of students in per class. While I am not fully aware of the reason why they have similar sort of preference, my colleagues also prefer individual presentation over other tests. Similar to the Bangladeshi context, some Asian EFL tests that I reviewed for this study had individual presentations as a common test type. Some high-stakes tests like CET-SET, TEM4, and TEM8 in China, include individual presentations based on some prompts (Jin & Fan, 2011, Zhang & Elder, 2009). In Malaysia, MUET and SBOEA also had individual presentations in the speaking test modules (Rethinasamy & Chuah, 2011; Shidu, Fook, & Shidu, 2011). I noted that in the speaking tests in these countries, an individual presentation was not the only criteria for judging learners’ oral competence. Leaners’ performances in conversational tasks were also assessed. Students’ Lack of Familiarity with Paired and Group Orals Results of the study revealed that Bangladeshi students’ were less experienced than other Asian EFL students were in taking paired orals and group orals as parts of the speaking tests. Among the four types of oral tests that I questioned about, I found that paired orals were the least 53 common type of oral tests that they took in all three stages investigated this study (Tables 3-5). Even though students reported having taken more oral tests at undergraduate level, only 29% of the students took a paired oral at the university. Among the 122 students I surveyed, some had experienced taking oral proficiency interviews and individual presentations during university admission but only nine students took a paired oral at this stage. Based on the teachers’ responses in the interviews, I think that paired or group orals were more common in language classrooms than in exams. Three of the five teachers I interviewed told me that students in their classes engage in group discussion or oral interaction in pairs. However, when I asked what they preferred for the university admission, some teachers talked about their preference toward oral proficiency interviews: Excerpt 3, Male Teacher 2: I think the present viva voce examination (which is a mere formality) is appropriate enough for the students who want to get admission. The content might include such preliminary discussion like introducing oneself or talking about a hobby in English. If the points earned during viva voce examination are made mandatory for the final admission, the evaluation process will be more appropriate and the students will be able to make right choices while selecting any particular discipline. Excerpt 4, Male Teacher 3: During university admission, we can ask students to introduce, for the basic information, themselves to get the idea of their competency levels. Administrators should adopt a group of other questions which is related some interesting aspects like food, important archaeological spot, journey, favorite genre of books and music. 54 Whereas the Bangladeshi students I surveyed reported having little experience with paired or group orals in high-stakes exams, a number of EFL exams in China, Singapore, Malaysia, and Hong Kong included either group or paired interaction (Qian, 2008; Sidhu, Fook, & Sidhu, 2011; Loh & Shih, 2016; Zheng & Cheng, 2008) as part of assessing learners’ oral proficiency. In reviewing some empirical studies on language tests between 2004 to 2014, Sandlund, Sundqvist, and Nyroos (2016) wrote “there was an increase of paired/group studies over the last few years, possibly mirroring the social turn within the broad field of second- language acquisition research.” Inclusion of paired or group orals in oral tests was seen by many test reviewers as effective in assessing learners’ communicative competence (He & Dai, 2006; Sandlund, et al., 2016). He and Dai (2006) wrote that an important reason for including group discussion in CET-SET in China was to assess learners’ use of a range of interactional language functions (IFLs) such as agreeing or disagreeing, challenging, supporting, negotiating meaning, and asking for information while interacting. In this regard, the scope of oral proficiency interviews in assessing learners’ communicative competence was seen as restricted by many researchers because the two interlocutors (i.e., the examiner and the candidate) in an OPI do hot hold equal positions, and most often the allocation and distribution of turn taking is controlled by the examiner. Thus, many of Asian countries, where oral assessment is present, have included paired or group interactional tasks in the oral tests because the communicative functions that can be achieved by these tasks. Relationship Between Students’ Levels of Education and Test Familiarity Findings of the study showed that none of the 3rd and 4th year students was unfamiliar with oral tests, whereas few from the 1st and 2nd year group reported never having taken any oral tests (Table 2). This suggests that students’ level of familiarity with oral tests is likely to increase 55 as they move from a lower level to an upper level in an undergraduate program. Findings related to students’ familiarity with oral tests at the HSC level further support this idea. Overall, these students reported having a higher degree of familiarity with all four types of oral tests at the undergraduate level than at the HSC level. It was interesting to note that for the 4th year students, almost 75% never took any oral tests at the HSC exam (Table 4) but 100% of them had to take an oral test in the form of an individual presentation at the undergraduate level (Table 3). This reconfirms the fact that by the time students reach the 3rd and 4th year at university, they are already familiar with individual presentation because this oral test type is a course requirement for many university programs. The emphasis on oral testing at higher levels of education, as observed in the Bangladeshi EFL context, was consistent with that of the Chinese EFL context. Most of the high-stakes tests in China included an oral subset to ensure that learners’ pay equal attention to speaking as they do for reading and writing. Unlike Singapore, Malaysia, and Hong Kong, where oral tests are a part of English language assessment system at school, results of this study indicate that a large number of students in Bangladesh take no oral tests at the higher secondary level in school. Although it is true that respondents at their advanced levels in the university were more familiar with oral tests, it would be a mistake to think that they were equally familiar with all four types of oral tests. Figures in Table 2 show that the 1st and 2nd year students were more familiar with Group Orals and Paired Orals than the 2nd and 3rd year students were. For Oral Proficiency Interviews, the statistics in Table 2 were different. The 1st year group showed the least degree of familiarity, while the 2nd year group showed the highest degree of familiarity. Half of the 3rd and 4th year students reported never having taken Oral Proficiency Interview. It is clear that for Individual Presentation only, students’ level of familiarity was higher among the 56 upper level students than the lower ones. Variation in three other test types only indicates that there is no consistency in administering these test types at various stages of English assessment in Bangladesh. Students responses as presented in Tables 2-5 made it clear that the oral tests they took at three different stages (i.e. HSC, admission, and undergraduate level) did not ensure an equal distribution of different oral test components like individual presentation, paired oral, group oral, and oral proficiency interview. This is different from the trends observed in EFL oral tests in China, Malaysia, and Singapore. In these countries, the English tests like CET-SET, MUET, and SBOEA included at least two to three separate oral test components in assessing learners’ oral proficiency (Qian, 2008; Sidhu, Fook, & Sidhu, 2011; Loh & Shih, 2016; Zheng & Cheng, 2008). A good mix of different types of oral components in an oral test can increase its validity as He and Dai (2006) indicated that it is difficult to infer about a learner’s oral proficiency from a single speaking task because different tasks may relate to different linguistic features. Findings based on students’ responses in this study further revealed a conspicuous absence of oral tests in university admission exams (Table 5). More than half of the respondents in Years 1, 3, and 4 reported that they took no oral tests during admission. Interestingly, a large number of students in Year 2 reported that they gave individual presentations and had taken oral proficiency interviews during admission. One possible explanation for this could be the fact that at one of the universities where I surveyed students, students have to take an interview before admission. However, teachers who I interviewed from this university mentioned that those interviews were not meant to be language proficiency tests: “They [university authority] arrange interviews during admission but that’s only for choosing the subjects or departments…not that sort of test actually” (Male Teacher 1). 57 Similarities Between Students and Teachers’ Attitudes Toward Oral Testing Participants’ responses to the Likert-scale survey in this study revealed striking similarities between students’ and teachers’ attitudes toward oral tests in admission exams (Table 12). On average, both groups reached a relatively high level of agreement that oral testing during university admission can have positive effects, and excluding it can be negative. This resonates with what Khan (2010) reported in her study on English course final exams at the higher secondary level in Bangladesh. Most of the teachers she interviewed expressed their opinions in favor of administering listening and speaking tests because these tests can have positive washback on the curriculum. This study added to Khan’s (2010) findings by showing that students were equally positive about taking English oral tests in high-stakes examinations. Research on English speaking tests in Chinese and Malaysian contexts that I reviewed for this study also presented similar findings. Wenyuan (2017), for example, reported that 65% of the total respondents in his study found CET-SET in China as essential or very essential. Malaysian researchers like Rahmat, et al. (2015) also wrote that the oral subset in English exams at the university and school level was perceived as beneficial by the respondents in their studies. To my understanding, the main reason a majority of the participants in all these studies showed positive attitudes toward speaking tests was not because they were favoring any specific test formats, but for the overall implications those oral tests had on the language curriculum and teaching. Zhen & Cheng (2008), for example, viewed the introduction of the CET-SET as a strong supplement to the test battery that helped promote students’ communicative competence. Responses of the participants in this study also revealed their concerns about the absence of oral tests, especially in high-stakes exams. They perceived this absence an obstacle to the practice of teaching and learning English speaking in language classrooms. Both groups that I 58 surveyed in this study agreed highly that students are reluctant to practice speaking English because they do not have to take any oral tests (see Q16_10 in Table 8 & 11). In reviewing the existing English language assessment systems in Bangladesh, researchers clearly pointed out that the lack of any listening and speaking components was partly responsible for the lack of listening and speaking practices in language classrooms (Maniruzzaman & Hoque, 2010; Tahereen, 2014).This types of negative washback effects of excluding oral tests might be one reason why participants in the present study agreed, although to a lesser extent, that oral test is an important component of an admission exam, and should influence the admission decision (Tables 7 &10). Researchers from the Asian countries where oral tests are not a part of the mainstream EFL assessment system voiced similar concerns about the negative washback effects of the assessment system. Some Pakistan researchers (Adnan & Mahmoud, 2014; Jilani, 2009) considered the exiting English assessment system at higher secondary school level faulty because it assessed learners reading and writing abilities only. Since the EFL context in Pakistan is heavily test-centered, the lack of any speaking tests in the high-stakes exams resulted in a lack of opportunities to practice this skill in class. Jilani (2009) commented, “The graduates of the HSC exam lack even the intermediate level of proficiency in English. Therefore, when they enter the professional life, they feel the need of learning to speak, read and write English” (p.181). Similar situation exists in South Korea where speaking and writing skills are not assessed in at the higher secondary level. Choi (2008) mentioned that the majority of the EFL students in Korean schools invested their time and energy in learning test-taking strategies and bothered little about improving their productive skills in English. He further wrote, “Many responsible stakeholders, not only educators but parents and students as well, contend that it is high time that drastic measures be taken to incorporate speaking and writing skills in the KSAT or college entrance 59 exam” (p.41). The participants that I surveyed and interviewed in this study expressed similar opinions. Some Differences in Students and Teachers’ Attitudes Toward Oral Testing A comparison between students and teachers responses showed no significant difference in their levels of agreement on the following three scales: a) positive effects of oral testing during admission, b) negative effects of excluding oral tests during admission, and c) oral tests as an important component in admission exams. However, there was a statistically significant difference in their opinions on how much oral testing should influence the admission decision. Although both groups agreed that oral tests should influence admission decisions, the figures in Table 12 showed that teachers’ favored the view more than the students did. Analysis of the individual statements in Scale 4 (oral tests should influence admission decision) also showed that there were some differences in how students and teachers rated these statements. Whereas the teachers largely agreed that universities should require students to pass an oral test for admission (O10_14), the students’ level of agreement on this statement was minimum (M = 5.05). The extent of agreement on statement Q10_12 (candidates performance in oral tests should carry points in university admission) was relatively high among both teachers and students. All these findings suggest that students and teachers supported the inclusion of oral tests in admission, but the students did not have any strong opinion on how much their oral test scores should influence the ultimate admission decisions. When I considered how students and teachers responded to the individual statements in three other scales in Table 8 and 11, I found that none of the teachers reported to have any strong disagreement on any of the statements. The standard deviations also suggest that within the group the teachers’ responses did not vary too much. The students’ responses, on the other hand, 60 were more dispersed. They fluctuated between Strongly Disagree and Strongly Agree for most of the statements. One possible reason of this variation could be the differences in their study levels and university environment. This survey was taken by students from all four undergraduate years, whose familiarity with oral English tests differed from each other. The other reason could be the small sample size for the teachers. Since the number of teachers (11) and students (122) in this survey was very different, no strong conclusion can be drawn from the statistical difference presented in Table 12. The small effect size (r = .185) also indicates that the difference is only trivial. Major Barriers to Oral Testing as Perceived by Teachers and Students Just like Part 2 of the questionnaire, there was a significant overlap between students’ and teachers’ perceptions about the challenges and barriers to oral testing during admission. In the third section of the survey, the students and teachers rated statements related to ten types of barriers as listed in Table 13. Among the ten types of barriers, both groups identified students’ lack of familiarity with oral tests and their anxiety about L2 speaking as the top most challenges. Bangladeshi students’ perception of the first scale in Table 15 (i.e., students’ anxiety about L2 speaking) was similar to Malaysian students’ perception of taking oral tests. While investigating the challenges of taking SBOEA by the Malaysian EFL learners, Fook, et al. (2011) reported that the learners perceived some psychological factors like nervousness, shyness, and low self- confidence in speaking English as major challenges, and these had a negative impact on their test scores. Students who I surveyed in this study also agreed highly that oral tests are difficult for students because they are shy or nervous about taking oral tests (see Q15_13 and Q15_10 in Table 15). One of the EFL teachers I interviewed told that because of students’ anxiety about oral tests, “we don't get cooperation from the students though some of them are very good in 61 speaking” (Female Teacher 1). Another female EFL teacher mentioned that the main problem she experienced was that her students were afraid of taking oral tests, so she often had to motivate them a lot in participating in oral tests. Students’ lack of familiarity with oral tests was identified by the participants as another major challenge to oral testing in Bangladesh. This is relevant to what I reported on the demographic information of the students who took this survey. A large number of students indicated that they had not taken any types of oral tests at the higher secondary level. Previous studies on the Bangladeshi EFL assessment system indicated that the absence of oral testing in the high-stakes public exams in Bangladeshi schools only meant there would be less focus on speaking in class (Das et al., 2014; Khan, 2010; Maniruzzaman & Hoque, 2010; Tahereen, 2014). I assume that many Bangladeshi universities do not administer any oral tests because most of the students, who pass their higher secondary level without taking any speaking test, are simply not prepared for taking one when their admission is due. In their interview with me, the teachers also voiced their concern about students’ lack of familiarity with oral tests. Among the ten barriers to oral testing during admission, both students and teachers found any lack of funds, staff, or administrative support as less problematic issues (Tables 14 & 17). In terms of financial support from the universities, data presented in Tables 15 and 18 revealed that there were conflicting views within both groups on whether their universities had enough funds to run a speaking test or not. For example, the mean score of the item 12_17 (it is difficult to administer oral test during admission because it is costly) was 4.38 for both groups (Tables 15 & 18), suggesting that, on average, neither group had any strong opinion about it. Many of the teachers and students, in fact, disagreed with this idea. One reason for these conflicting views could be the differences in the financial condition of public and private universities. An analysis 62 of the data in terms of public and private universities could have provided better information on this. The survey results in Table 17 revealed that teachers level of agreement on the eighth scale (i.e., lack of examiners’ training) was relatively low (M = 5.28, Mdn = 5.13). This was different from what I found in my interviews with five teachers. None of them received any training in assessing oral tests, and only one teacher told that he attended a workshop on designing oral tests. When interviewed, all these teachers also noted that there was a lack of uniform grading policy in assessing students’ oral performance. Although the teachers’ survey responses in this study did not reveal any strong opinion about the lack of uniformity in grading criteria, the following interview excerpt can help explain the challenges in rating oral tests in Bangladesh. Excerpt 5, Male Teacher 2: As the contents of the oral test vary, so the task of rating it becomes challenging. In place of making small grids for putting appropriate points against their [students] responses, the rating relies solely on the judgment of the course teacher….There is no university policy for rating any sort of speaking test. The lack of grading policy was seen as problematic by Asian researchers from Bangladesh, Malaysia, and India. Researchers like Khan (2010), Ramanathan (2008), and Sidhu, Fook, and Mohamad (2011) found teachers’ lack of understanding the grading criteria and the lack of validity of grading process as some major problems in oral assessment. As a matter of fact, problems related to rating students’ oral performance is a much-explored issue in second language testing. Many researchers investigated problems related to rater effects and 63 recommended rater training programs to overcome various sources of bias (Winke, Gass, & Myford, 2012; Zhang & Elder, 2014). Differences in Students and Teachers’ Perceptions of the Barriers to Oral Testing Although students and teachers’ perceptions of the barriers to oral testing during admission were very similar to each other, the Mann-Whitney test (in Table 19) revealed a statistically significant difference in terms of their opinions about the lack of program policy. Whereas teachers highly agreed that there is a lack of program policy (M = 6.07), the students’ level of agreement was lower than that (M = 5.17). One reason teachers had a rather stronger opinion about the lack of a program policy may be that they are more informed about the program or department policy than the students are. However, the effect size of this difference is small (r = .222). As mentioned earlier in the discussion, the number of teachers (n = 11) who participated in this survey was small so it is difficult to draw any generalizations from their data. A close look at Tables 15 and 18 show that the students’ and teachers’ opinions about the three scales, lack of administrative support, lack of staff, and lack of funds, were slightly different. The mean scores of the teachers’ responses suggest that they moderately agreed that the lack of staff, administrative support, or funds posed obstacles to oral testing. However, students’ responses, on average, revealed a more neutral stance on these issues. Although these small differences had no statistical significance, I assume that teachers’ opinions were close to each other because they are likely to be more experienced in terms of the financial or administrative limitations of a university than the students are. The fact that no teachers had any strong disagreement (min = 3, max = 7) on these issues confirms this argument. 64 CONCLUSION The overarching goal of my study was to address the absence of oral testing in high- stakes English exams in Bangladesh. I attempted to find out what students and teachers thought about including English oral tests in university admission exams. Both groups strongly agreed that including oral tests is essential for ensuring the development of speaking proficiency in EFL learners. The responses I gathered by surveying and interviewing my participants provide insight into the major problems and barriers to oral testing during admission in Bangladeshi public and private universities. Students’ lack of familiarity, their anxiety about L2 speaking, the lack of technological support, and the huge number of examinees were seen as some major barriers to oral testing. A comparison between students’ and teachers’ overall responses showed that both groups shared similar attitudes toward and perceptions of oral testing during admission. In the following two instances, their opinions seemed to differ: a) oral tests should influence admission decisions, and b) the lack of program policy posed any major obstacle to oral testing during admission. In these cases, teachers seemed to have stronger opinions than the students had. Implications The results of this study can be useful in making a future policy of assessing learners’ English language proficiency during admission. It can be understood from the study that both students and teachers shared a positive attitude toward including oral tests in admission, considering its washback effect on the entire language teaching and learning practices. Results of this study can have implications for developing a coherent English language assessment system across various levels of education in Bangladesh. Survey results revealed that these students faced an increased demand to participate in oral tests as they moved along the upper levels of their undergraduate years, although they only had a few opportunities in practicing oral tests 65 during their school years. A proper balance between the oral assessments at various levels of education is necessary to ensure students’ consistent progress in oral proficiency. In this study, I also addressed the level of familiarity that students had in taking different types of oral tests. It was found that the majority of the students were more familiar with individual presentations, compared to other interactive formats like group orals or paired orals. I believe that any oral testing policy should consider such information before implementing anything into the existing system because it is important to familiarize students with different oral test formats before they can be tested on them. Limitations of the Study One major limitation of the study was the small sample size. Only 11 teachers and 122 students participated in the survey. A larger sample size would reveal a more accurate picture of students’ and teachers’ attitudes and perceptions. In this study, I involved participants from only two universities. If more universities were covered, it would be easier to make more generalizations about the problems and challenges to oral testing. Although I included a public and a private university in this study, I did not compare participants’ attitudes based on this distinction. Lastly, for my research, I could only interview five EFL teachers. Interviews with a larger number of teachers and students would lead to a more in-depth understanding of their choices as revealed in the quantitative analysis. These interviews could also provide participants’ suggestions for possible solutions to the problems and barriers explored in this study. To conclude, this study was an attempt to address the gap in research on oral testing in Bangladesh. I believe that identifying and understanding the major barriers to oral testing can be the first step to removing any inconsistencies between the English assessment system and the communicative language teaching curriculum in Bangladesh. In spite of some limitations, I 66 believe this study can help raise EFL teachers’, students’, and policy makers’ awareness of the challenges and barriers to oral testing in Bangladesh. 67 APPENDICES 68 APPENDIX A: Interview Questions 1. I am investigating oral testing. Can you tell me about oral testing at your university and in your classes? আিম ইংের(cid:8)জ (cid:10)মৗিখক পরী(cid:16)া িবষেয় গেবষণা করিছ। আপনার িব(cid:26)িবদ(cid:28)ালয় বা আপনার (cid:30)ােসর (cid:10)মৗিখক পরী(cid:16)া িবষেয় িকছ বলুন। 2. What types of oral test are administered during university admission? িব(cid:26)িবদ(cid:28)ালয়েয় ভিত$র সময় ইংের(cid:8)জেত (cid:10)কান ধরেণর (cid:10)মৗিখক পরী(cid:16)া (cid:10)নয়া হয়? 3. What types of oral test do you administer in your class? আপনার (cid:30)ােস আপিন িক িক (cid:10)মৗিখক পরী(cid:16)া (cid:10)নন? 4. Did you receive any training or workshop for assessing speaking proficiency? আপিন িক ইংের(cid:8)জ (cid:10)মৗিখক পরী(cid:16)া যাচাইেয়র জন(cid:28) (cid:10)কান )িশ(cid:16)ণ +হণ কেরেছন? 5. Did you receive any training or workshop for designing speaking tests? আপিন িক ইংের(cid:8)জ (cid:10)মৗিখক পরী(cid:16)া িডজাইন করার উপর (cid:10)কান )িশ(cid:16)ণ িনেয়েছন? 6. What challenges did you face in designing oral testing as a teacher? একজন িশ(cid:16)ক িহেসেব (cid:10)মৗিখক পরী(cid:16)া িডজাইেনর (cid:10)(cid:16)ে0 আপিন িক িক অসুিবধা (cid:10)বাধ কেরন? 7. What challenges did you face in rating oral tests? (cid:10)মৗিখক পরী(cid:16)া যাচাইেয়র (cid:10)(cid:16)ে0 আপিন িক ি◌ অসুিবধা অনুভব কেরন? 8. What challenges did you face in administering oral tests? ((cid:10)মৗিখক পরী(cid:16)া পিরচালনার (cid:10)(cid:16)ে0 আনিপ িক িক অসুিবধা অনুভব কেরন?) 69 Survey Part 1 (Students Only) APPENDIX B: Survey 1. Which undergraduate level are you enrolled in? (বত$মােন িব(cid:26)িবদ(cid:28)ালেয় আপিন (cid:10)কান 3ের অধ(cid:28)য়নরত?) a) First year ()থম বষ $) b) Second year (ি5তীয় বষ $) c) Third year (তৃ তীয় বষ $) d) Fourth year (চত থ $ বষ $) 2. Did you take any undergraduate English course on Spoken English? (আপিন িক িব(cid:26)িবদ(cid:28)ালেয় (cid:10)মৗিখক ইংের(cid:8)জ িবষেয় (cid:10)কান (cid:10)কাস $ কেরেছন?) a) Yes (হ(cid:28)াঁ) b) No (না) 3. Did you take any undergraduate English course that included Spoken English? (আপিন িক িব(cid:26)িবদ(cid:28)ালয়েয় এমন (cid:10)কান ইংের(cid:8)জ (cid:10)কাস $ কেরেছন (cid:10)যখােন (cid:10)মৗিখক ইংের(cid:8)জ িশ(cid:16)া অ8ভ 9 িছল?) a) Yes (হ(cid:28)াঁ) b) No (না) 4. What types of oral English test did you take in the final exams of your undergraduate English language courses? (Check all that apply.) (:াতক পয $ােয় অনু;<ত ইংের(cid:8)জ িবষয়=েলার পরী(cid:16)ায় আপিন িন>9 (cid:10)কান (cid:10)কান (cid:10)মৗিখক পরী(cid:16)া িদেয়েছন? যত=েলা )েযাজ(cid:28) তার সব=েলা ;টক িদন।) a) Individual presentation (একক উপ@াপনা) b) Group oral (গ্BপিভিCক কেথাপকথন) c) Paired oral (দুইজেনর কেথাপকথন) d) Oral proficiency interview (ইংের(cid:8)জেত ইDািভ$উ) e) None of the above ((cid:10)কান;টই নয়) f) Other (please specify) (অন(cid:28)ান(cid:28))_____________ 5. What types of oral English test did you take at your higher secondary level? (Check all that apply.) (মাধ(cid:28)িমক পয $ােয় অনু;<ত ইংের(cid:8)জ িবষয়=েলার পরী(cid:16)ায় আপিন িন>9 (cid:10)কান (cid:10)কান (cid:10)মৗিখক পরী(cid:16)া িদেয়েছন? যত=েলা )েযাজ(cid:28) তার সব=েলা ;টক িদন।) a) Individual presentation (একক উপ@াপনা) b) Group oral (গ্BপিভিCক কেথাপকথন) c) Paired oral (দুইজেনর কেথাপকথন) d) Oral proficiency interview (ইংের(cid:8)জেত ইDািভ$উ) 70 e) None of the above ((cid:10)কান;টই নয়) f) Other (please specify) (অন(cid:28)ান(cid:28))_____________ 6. What types of oral English test did you take during admission to your university program? (Check all that apply). (িব(cid:26)িবদ(cid:28)ালয়েয় ভিত$ পরী(cid:16)ায় আপিন ইংের(cid:8)জেত িন>9 (cid:10)কান (cid:10)কান (cid:10)মৗিখক পরী(cid:16)া িদেয়েছন? যত=েলা )েযাজ(cid:28) তার সব=েলা ;টক িদন।) a) Individual presentation (একক উপ@াপনা) b) Group oral (গ্BপিভিCক কেথাপকথন) c) Paired oral (দুইজেনর কেথাপকথন) d) Oral proficiency interview (ইংের(cid:8)জেত ইDািভ$উ) e) None of the above ((cid:10)কান;টই নয়) f) Other (please specify) (অন(cid:28)ান(cid:28))____________ Survey Part 2 The following statements deal with oral English tests during university admission at Bangladeshi universities. Please indicate how much you agree or disagree to these statements: Item ID Statements y l g n o r t S e e r g A e e r g A t a h w e m o S e e r g A r e h t i e N t a h w e m o S e e r g a s i D e e r g a s i D y l g n o r t S e e r g a s i D Q16_3 Oral test during admission can help diagnose students' oral proficiency. Q16_4 Oral test during admission can Q16_5 Q16_7 make students practice speaking in English more than they do now. If oral tests are administered, students will focus on developing their oral communication skills. If students have to take oral tests, they will be motivated to speak in English. Q16_9 Absence of oral test has negative effects on developing students' oral English proficiency. Q16_10 Students are reluctant to practice speaking English because they do not have to take any oral tests. Q16_13 Absence of oral test has negative influence on developing EFL 71 Q10_1 Q10_2 students' communicative competence. It is important for a student to take an English oral test before getting admission to any university. For university admission, English speaking test is as important as English reading and writing. For admission to any undergraduate programs, English speaking test is equally important as other sections like mathematics, general knowledge, and Bengali. Q10_4 During university admission, both Q10_3 public and private universities should administer an English speaking test. Q10_5 Oral English tests should be made Q10_6 compulsory during university admission exam. It is important to test students' English speaking proficiency before offering admission to undergraduate English language programs. Q10_7 Universities should measure candidates' proficiency in spoken English before offering admission to any undergraduate programs. Q10_12 Candidates' performance in oral English tests should carry points in university admission exams. Q10_13 Students securing high scores in oral test during admission should get priority in receiving admission offer Q10_14 Universities should require candidates to pass an oral test for admission. Q10_15 Students' performance in oral test should be considered while offering tuition award during admission test. 72 Survey Part 3 The following statements deal with oral English tests during university admission at Bangladeshi universities. Please indicate how much you agree or disagree to these statements: Item ID Statements y l g n o r t S e e r g A e e r g A t a h w e m o S e e r g A r e h t i e N t a h w e m o S e e r g a s i D e e r g a s i D y l g n o r t S e e r g a s i D Q12_1 Q12_2 Q12_ 3 Q12_4 It is difficult to arrange a speaking test during university admission because the number of examiners in my Department is not sufficient. It is not easy to arrange any speaking test during our university admission because there is a lack of staff for administering the test. It is not feasible to include a speaking test in our university admission exam because it is not possible to test all candidates by a limited number of teachers. It is difficult to arrange any speaking test by the teachers or examiners because there are many test takers. Q12_5 The number of candidates is too large Q12_6 Q12_7 Q12_8 Q12_9 to be tested orally by examiners during university admission. I think the most significant barrier to oral testing is the huge number of candidates. It is difficult to arrange any speaking tests during university admission because it is costly. It is difficult to administer any speaking tests because the funds are not enough to pay examiners. It is difficult to administer any large- scale speaking tests because the funds are not enough to train raters. Q12_10 It is not easy to arrange a speaking test because there is no allotment of funds for administering such a test. Q12_11 I think the lack of fund is the most challenging aspect of administering the oral test. 73 Q12_12 Oral testing is difficult because many examiners do not have enough training on it Q12_13 Oral testing is difficult because many examiners do not know which types of oral tests they should administer for different purposes. Q12_14 Oral testing is difficult because examiners never took the test when they were students. Q12_15 Oral testing is difficult because examiners did not receive any training on rating different oral tests. Q12_16 Oral testing is not easy because there is no university/department policy for administering the test. Q12_17 It is difficult to administer oral test because it takes a lot of time. Q12_18 It is difficult to administer oral test because there is not enough rooms or spaces for that. Q12_19 I think there is a lack of logistic support in administering any oral test during admission. Q12_20 Rating oral tests can be difficult because it is influenced by raters’ backgrounds and personal preferences Q12_21 It is difficult to rate oral tests appropriately because there is a lot of variation among raters’ judgments. Q12_22 It is difficult to rate oral tests because the ratings of oral tests can be subjective. Q12_23 It is difficult to rate oral tests because there is no fixed policy for rating. Q12_24 It is difficult to evaluate speaking test Q15_1 Q15_2 Q15_3 because there is no clear evaluation criteria for different types of oral tests. It is difficult to administer a speaking test because there is a lack of technological support. It is not possible to administer speaking test using computers in our universities. It is difficult to find technological support in recording the oral tests. 74 Q15_4 Q15_5 Q15_6 Q15_7 Q15_8 Q15_9 It is difficult to rate huge number of test takers because there is no use of automated scoring. It is difficult to do well in the oral test because students are not familiar with different types of speaking tests. It is not easy to take a speaking test because students never practiced it in their English classes in school. It is difficult for students to take speaking tests because they are more accustomed to taking reading and writing tests only Speaking test during university admission is challenging to the students because they did not take any oral tests during the HSC exam. Speaking test is difficult for students because they never practice or prepare for any oral test in their undergraduate English classes. Q15_10 Oral tests are difficult for students because they feel nervous in taking oral tests. Q15_11 Oral tests are difficult for students because they often feel a high level of anxiety in taking oral tests. Q15_12 Oral tests are difficult for students because they are afraid of failing the tests. Q15_13 Oral tests are difficult for students because they are shy of speaking in English. Q15_14 Oral tests are difficult for students because they lack confidence in speaking in English. Q15_15 Students do not prefer to take any speaking test because their programs do not require that. Q15_17 English teachers do not take any oral test because it is not required by the departmental/university policy. Q15_16 Students do not feel any interest in taking oral tests because these do not add any points to their final grade. 75 REFERENCES 76 REFERENCES Adnan, U., & Mahmood, M. A. (2014). Impact of public examination on teaching of English: A washback perspective. Journal of Education and Practice, 5(2), 132-139. Retrieved from http://www.iiste.org/Journals/index.php/JEP/article/view/10649 Ang-Aw, H. T. & Goh, C. C. M. (2011). Understanding discrepancies in rater judgment on national-level oral examination tasks. RELC Journal: A Journal of Language Teaching and Research, 42(1), 31-51. Cicchetti, D. V., Showalter, D., & Tyrer, P. J. (1985). The effect of number of rating scale categories on levels of inter-rater reliability: a Monte-Carlo investigation. Applied Psychological Measurement, 9, 31-36. Cherngchawano, W., & Jaturapitakkul, N. (2014). Lexical profiles of Thailand university admission tests. PASAA: Journal of Language Teaching and Learning in Thailand, 48, 1- 27. Choi, I. (2008). The impact of EFL testing on EFL education in Korea. Language Testing, 25(1), 39-62. Chowdhury, R., & Ha, P. L. (2008). Reflecting on western TESOL training and communicative language teaching: Bangladeshi teachers' voices. Asia Pacific Journal of Education, 28(3), 305-316. Das, S., Shaheen, R., Shrestha, P., Rahman, A., & Khan, R. (2014). Policy versus ground reality: secondary English language assessment system in Bangladesh. The Curriculum Journal, 25(3), 326-343. EIKEN speaking test. (n.d.). In Eiken Foundation of Japan. Retrieved from http://www.eiken.or.jp/eiken/en/eiken-tests/speaking-tests/ Fook, C. Y., Sidhu, G. K., Rani, N., & Aziz, N. A. (2011). Analyzing factors associated with students’ oral test performance. The International Journal of Educational and Psychological Assessment, 9(1), 27-47. General Aptitude Test [GAT]. (2017). In Interantional House Bangkok. Retrieved from http://www.ihbangkok.com/exam-preparation/university-entrance/gat/ Haider, Z. M., & Chowdhury, T. A. (2012). Promoting CLT within a computer assisted learning environment: A survey of the communicative English course of FLTC. English Language Teaching, 5(8), 91-102. 77 He, L., & Dai, Y. (2006). A corpus-based investigation into the validity of the CET-SET group discussion. Language Testing, 23(3), 370-401. Jilani, R. (2009). Problematizing high school certificate exam in Pakistan: A washback perspective. Reading Matrix: An International Online Journal, 9(2), 175-183. Jin, Y., & Fan, J. (2011). Test for English majors (TEM) in china. Language Testing, 28(4), 589- 596. Kamal, N. D. M., Siraj, S., Alias, N., & Attaran, M. (2013). Research and trends in the studies of school-based oral English assessment from 2003 to 2011: A review of selected journals. Malaysian Online Journal of Educational Sciences, 1(3), 41-51. Khan, R. (2010). English language assessment in Bangladesh: Developments and challenges. In Y. Moon & B. Spolsky (Eds.), Language assessment in Asia: Local, regional or global? (pp. 121–157). South Korea: Asia TEFL. Lin, L. (2014). A data-based investigation into reliability and validity of computer-assisted oral English test. Applied Mechanics and Materials, 543-547, 4494-4497. Loh, T. L., & Shih, C. (2016). The English language test of the Singapore primary school leaving examination. Language Assessment Quarterly, 13(2), 156-166. Mackey, A., & Gass, S. M., (2005). Second language research: Methodology and design. Mahwah, NJ : Erlbaum. Maniruzzaman, M. & Hoque, M. E. (2010). How does washback work on the EFL syllabus and curriculum? A case study at the HSC level in Bangladesh. Language in India, 10(12), 49- 88. National Examinations. (2017). In Singapore Examinations and Assessment Board. Retrieved from http://www.seab.gov.sg/ Noor, N. B. M., Muniandy, M. K., Shanmugan, S. K. K., & Mathai, E. J. (2010). Upper primary teachers' perceptions of PSLE English oral assessment. English Language Teaching, 3(4), 142-151. Oaster, T. R. F. (1989). Number of alternatives per choice point and stability of Likert-type scales. Perceptual and Motor Skills, 68, 549-550. Piggin, G. (2011). An Evaluative Commentary of the Grade 1 EIKEN Test. Language Testing in Asia,1(4), 144-167. Prapphal, K. (2008). Issues and trends in language testing and assessment in Thailand. Language Testing, 25(1), 127-143. 78 Preston, C.C., & Colman, A.M. (2000). Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104, 1-15. Qian, D. D. (2008). English language assessment in Hong Kong: A survey of practices, developments and issues. Language Testing, 25(1), 85-110. Rahman, M. S. (2015). Implementing CLT at higher secondary level in Bangladesh: A review of change management. Journal of Education and Practice, 6(2), 93-102. Rahmat, N., Min, L. S., Sungif, N. A. M., & Yusup, F. N. M. (2015). English language proficiency tests and academic achievement: A study on the Malaysian University English Test as a predictor of technical programme undergraduates academic achievement. Advances in Language and Literary Studies, 6(1), 114-119. Ramanathan, H. (2008). Testing of English in India: A developing concept. Language Testing, 25(1), 111-126. Rethinasamy, S. & Chuah, K. M. (2011). The Malaysian University English Test (MUET) and its Use for Placement Purposes: A Predictive Validity Study. Electronic Journal of Foreign Language Teaching, 8(2), 234-245. Sandlund, E., Sundqvist, P., & Nyroos, L. (2016). Testing L2 talk: A review of empirical studies on second-language oral proficiency testing. Language and Linguistics Compass, 10(1), 14-29. Sasaki, M. (2008). The 150-year history of English language assessment in Japanese education. Language Testing, 25(1), 63-83. Secondary. (2017). In Ministry of Education Singapore. Retrieved from https://www.moe.gov.sg/education/secondary Sidhu, G. K., Fook, C. Y., & Mohamad, A. (2011). Teachers' knowledge and understanding of the Malaysian school-based oral English assessment. Malaysian Journal of Learning and Instruction, 8, 93-115. Sidhu, G. K., Fook, C. Y., & Sidhu, S. K. (2011). Students' reactions to school based oral assessment: Bridging the gap in Malaysia. Asian EFL Journal, 13(4), 300-327. Sun, C., & Henrichsen, L. (2011). Major university English tests in china: Their importance, nature, and development. TESL Reporter, 44(1-2), 1-24. Tahereen, T. (2014). Testing Based Language Teaching in Bangladesh: Does It Promote or Impede Learning?. TESOL International Journal, 9(1), 148-178. 79 Wenyuan, Z. (2017). The washback effect of CET spoken English test upon college English Teaching. Canadian Social Science,13(1), 62-68. Winke, P., & Gass, S. (2013). The influence of second language experience and accent familiarity on oral proficiency rating: A qualitative investigation. TESOL Quarterly, 47(4), 762-789. Xiajou, L., & Li, W. (2000). Testing oral English on a mass scale: Is it feasible? The oral component of the MET in china. Hong Kong Journal of Applied Linguistics, 5(1), 160- 186. Zhan, Y., & Wan, Z. H. (2016). Test takers' beliefs and experiences of a high-stakes computer- based English listening and speaking test. Regional Language Centre. RELC Journal, 47(3), 363-376. Zhang, Y., & Elder, C. (2009). Measuring the speaking proficiency of advanced EFL learners in China: The CET-SET solution. Language Assessment Quarterly, 6(4), 298-314. Zhang, Y., & Elder, C. (2014). Investigating native and non-native English-speaking teacher raters' judgments of oral proficiency in the college English test-spoken English test (CET-SET). Assessment in Education: Principles, Policy & Practice, 21(3), 306-325. Zheng, Y., & Cheng, L. (2008). Review of college English test (CET) in china. Language Testing, 25(3), 408-417. 80