Zambia (J 984/5). XII.RESEARCH REPORTFIRST-YEAR READING LEVELS AT THE UNIVERSITY OF ZIMBABWEK. McGINLEYCommunication Skills Centre, University of ZimbabweIN A FAST-CHANGING educational situation as in Zimbabwe in recent years, thereis much talk of declining standards; but little evidence has been adduced insupport of such assertions. Much the same can be said about reading levels at theUniversity of Zimbabwe Š a matter of vital concern to the CommunicationSkills Centre, the University's English language/study skills unit. Much of itseffort centres on the comprehension of texts and it is clearly important to knowwhether or not there is a significant gap between assumed and real reading levels:whether it is useful to use university-level texts as the basis for teachingapproaches to comprehension or whether special materials should be selected orproduced.In theory, all incoming students have an acceptable level of languagecompetence, having passed O level English Language, a reputable publicexamination. In practice, the Communication Skills Centre has to run specialcourses in English language and language-related skills, and demand for suchcourses originated in University departments. In addition, the CommunicationSkills Centre runs its own test to identify those in need. A recent comparison bythis writer (McGinley 1985) of the skills tested at O level and those needed atuniversity revealed significant differences.In an attempt to elicit further information about reading-levels, scripts from theCommunication Skills Centre's past entry-tests were examined. These revealedevidence of low reading-comprehension of texts well below university level. Thesituation therefore suggested the need to investigate reading levels among first-year students in a systematic way.It was considered important both to get a large sample size and to haveoptimum testing conditions. These were satisfied by administering the specialreading-level test to all incoming first-year students as an addition to theCommunication Skills Centre's entry-test, which students take very seriously asfailure to pass this test results in their having to attend a course at the Centre. TheCommunication Skills Centre's test was planned to take about one and a halfhours; a further half-hour was given for the reading test and this was adequate.TEXTS CHOSENTwo texts were carefully chosen, one from a Cambridge O level EnglishLanguage past paper. They were chosen so as not to advantage or disadvantageany group of students on account of, for example, specialist content. Theuniversity-level text was from Sociology, a subject not commonly taught atsecondary level. Because the difficulty of texts can vary so much around the levelschosen, and because it was considered important to space the texts effectively, allpossible texts were subjected to a number of text-difficulty indices. (These give anobjective measure of difficulty which is usually expressed in reading age Š anindex of 12, for example, being a text that an average twelve-year old should beable to manage,) Four indices were used and the results from the chosen texts are7576tabulated in Table 1. The average reading age for the O level text was 17.1 andthat for the university text was 19.55. Assuming students write O level at around16-17 years and enter university at around 18-19 years, the<lifference in the levelof the texts was considered to be about right. In fact, a university text with anindex of 19.55 is relatively easy as the average, for such texts is usually higher.Table 1INDEX DIFFICULTY OF CHOSEN TEXTSReadability testsFOGO Level text 18University text 20.20Fry1621Flesch17.3019Text was too short to calculate this indexAPPROACHSMOG*18Average17.1019.55Even though the time available for the test was limited to half an hour, it wasnevertheless necessary to use a test which was both valid and reliable: testing whatit was supposed to be testing and doing so consistently through time. Clozeprocedure seemed the most appropriate. Oiler (1979, p. 357) says: 'It has beendemonstrated many times over that cloze scores are extremely sensitive measuresof reading ability.' He also quotes (p. 63) from research by Swain, Lapkin andBarik who conclude that 'the cloze technique has been shown to be a valid andreliable means of measuring second language proficiency.' Harrison (1980,pp. 106, 107) says: 'Cloze procedure correlates highly with formal reading tests.In this sense, it has a high validity...In large scale research cloze results are morereliable [than other tests].' Alderson (1979, p. 220) says: 'Since Taylor (1953) thegeneral consensus of studies into and with the procedure has been that it is areliable and valid measure of readability and reading comprehension for nativespeakers of English.' For more on the validity and reliability of cloze procedure,see, for example, Anderson (1971) and Gilliland (1980).The origins of cloze procedure as a testing approach are relatively recent,around 1953. It was a technique used in testing comprehension among LIspeakers (those to whom English is a first language) and it is now in common usefor testing in L2 contexts (where the user's first language is one other thanEnglish). It may be contrasted with more traditional language-testing approacheswhich often test discrete points of usage; cloze procedure tests comprehension ofsentences as a whole and, in some cases, inter-sentence and supra-sentenceelements within paragraphs. It works simply by deleting every nth word in aparagraph; the reader completes the blanks with appropriate words (these neednot be the exact ones). The rate of deletion is often taken as every fifth word (seeMacGintie 1961), with 20 deletions in a 100-word passage: below that it becomes77too difficult to predict; above that, some research shows that the reader is notnecessarily helped (see Harrison 1980 and Harris 1976, who also consider theissue of frequency). Correlation of results with other forms of testing in LI is high.What evidence there is in L2 context suggests that correlation is good (seeAlderson 1971, who quotes six authors in this connection). There is now a vastbibliography on the use of cloze procedure.STUDENTS TESTEDIn order to get as full a picture as possible of changes in reading levels amongstudents coming to the University, the testing was extended to some school-levelgroups and to some second-year University students. Two 'feeder' schools wereidentified (those supplying students to the University) and the tests wereadministered to likely university aspirants ('A' stream O level students). The testswere also given to beginning second-year students in two departments to ascertainthe extent to which reading levels improved after one year at the University. Asummary of results from all groups together with sample sizes is shown in Table 2.Table 2AVERAGE SCORES ON READINGFirst-year studentsAgricultureCivil EngineeringMedicineScienceArtsSocial ScienceSchool studentsSchool ASchool BSecond-year studentsDepartment ADepartment BScores (outO leveltext10.9310.6010.5010.309.929.709.708.4011.0610.40of 20)Universitytext8.608.098.407.757.647.188.907.4010.609.67TESTSSampleat random30/7150/11450/6249/99100/30468/14230/3035/3539/3929/60Note: A score of 8/20 (40%) = minimum comprehensionA score of 12/20 (60%) = independent reading level78RESULTS AND CONCLUSIONSThe distribution of scores for each group is presented in the Appendix, with rawscores on the horizontal axis and frequency on the other; this clearly shows thenumber of students getting a certain score level. Two levels of comprehension areidentified: that of 40 per cent called 'instructional level' (or the level of minimumcomprehension) at which the reader needs the help of a teacher to attain fullercomprehension; and that of 60 per cent at which the reader is assumed to be at the'independent reading level'. The figure of 40 per cent is a well-researched one,according to Harrison (1980); the figure of 60 per cent is less well so.From these results, the following conclusions may be drawn:O level text: If 60 per cent is the independent reading level, then a minority offirst-year students approach that level on the O level text. A small minority do notattain minimum comprehension (40 per cent) and are reading at what might bedescribed as 'frustration level'.University text Most students are at or just below minimum comprehension.Testing at Schools; Some feeder schools may have a higher average readingcomprehension than incoming first-year university students.Testing of 2nd-year University students: By the beginning of the second year atuniversity most students approach assumed first-year reading level.In general, then, it would appear that because most incoming students havebelow O level reading comprehension and take one full year at university to attainassumed first-year reading level, there is a three-year gap between the assumedand real reading levels of incoming first-year students.ReferencesALDERSON, J.C. 1979 'The cloze procedure and proficiency in English as aforeign language', TESOL Quarterly, XIII, 219-27.ANDERSON,!. 1971 'A technique for measuring reading comprehension andreadability', English Language Teaching, XXV, 178-82.GILLILAND, J. 1980 'Readability', in W.A. Gatherer and R.B. Jeffs (eds.),Language Skills through the Secondary Curriculum (Edinburgh, HolmesMcDougall), 178-87.HARRIS, D. 1976 'Testing reading comprehension in ESL/EFL: Backgroundand current state of the art', in E.M. Anthony and J.C. Richards (eds.),Reading: Insights and Approaches (Singapore, RELC).HARRISON, C. 1980 Readability in the Classroom (Cambridge, CambridgeUniv. Press).McGlNLEY, K. 1985 'Preparing for university: Adequacy/inadequacy of 'O'level English Language competence'. Bulletin of the Associate College Centre[University of Zimbabwe], XXI, ii, 1-11.MACGINTIE, W.H. 1961 'Contextual constraint in English prose paragraphs',Journal of Psychology, LI, 121-30.OLLER, J.W. 1979 Language Tests at School: A Pragmatic Approach(London, Longman).TAYLOR, W. 1953 'Cloze procedure: A new tool for measuring readability',Journalism Quarterly, XXX, 415-33.79APPENDIX: RESEARCH RESULTSKey: Vertical axis: frequencyHorizontal axis: scoreInstructional level: 8 out of 20Independent reading level: 12 out of 20FIRST-YEAR STUDENTSAgricultureO level text (average 10.93)X X X X X X Xxxxxxxxx101520University text (average 8.60)"X XXX XXxxxxxxxxxxxXX X0 5* 1 paper not attemptedCivil EngineeringO level text (average 10.60)*101520XXXXXXXX XXXXX XXXXXXXX XX XX XXXXXXXXXXXXXXXXXXXXXXXX0 5* 1 paper not attempted10152080Civil Engineering (cont.)University text (average 8.09)*x xx xxXX XXX XX XXX XXX X XX X X X Xxxxxxxxxxxxxx0 5*2 papers not attempted101520MedicineO level text (average 10.60)xX XX XXX XXXX XX X X X XX X X X XX X X X Xxxxxxx xxxxxxxxx0 5 10University text (average 8.43)*1520XXXX XX XXXXXXXXXxxxxxxxxxxxxx_l 1XX X0 54 papers not attempted10152081ScienceO level text (average 10.30)XX XXXX XXXXXXXXXXXXXXX XX XX X X X XXXXX X X X X X XXXX1X XXXXXX101520University text (average 7.75)"xX XX XXXX XX X X X X XXXX X XXXXXxxxxxxxxxxxxXX X0 5* 4 papers not attempted101520ArtsO level text (average 9.92)XX XXXX X X XXXX X X XXXX X X XX XXX XXX X XX XXX XXX X XXX XX XXX XXXXX XXXXX XXXXXXXXX XXX10152082Arts (cont.)University text (average 7.64)*XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX X X XX X X X X X X X X X X X X X X X X X X X X X0 5* 18 papers not attemptedSocial SciencesO level text (average 9.70)1020XXX XX XX XXXXXXXXXXXXXXXXXXX XX XX X X XX X X XXXXXXXXXXXXXXXXXXXXXXXXXX XX X X X X0 5 10University text (average 7.18)*1520XXXX X X XX X X XX XX XX X X X XX X X X Xxxxxxxxxxxxxxxxxxxxxxxxx0 57 papers not attempted101520SCHOOL STUDENTS83School AO level text (average 9.70)X XXXXX XXXX X X X XX X X X XI05101520University text (average 9.0)x xxxxx x x x x xxxxxxxxx0 5SchoolBO level text (average 8.40)101520x xX XX XXXX XXXXXX XXX0 5 10University text (average 7,40)1520XXXX XXXXXXXXXXX XX XX XX XXXXX X X XX X X X X X10152084SECOND-YEAR STUDENTSDepartment AO level text (average 11.06)xX XXXXX X X X X X XXXX XXXXxxxxxxxxxxxxxxx0 5 10University text (average 10.60)1520xX XXXXXX XX Xxxxxxxxxxxxx() 5Department BO level text (averageX10.40)XXXX X10X XX XXX X X XXXXXX XX XX X15 2XXXX0 5 10University text (average 9.67)1520xXXXX XX X X X X XXXXXXXXX X101520