What Silent Reading Tests Alone Can't Tell You: Two Case Studies in Adult Reading Differences

by John Strucker
Before joining NCSALL last fall as a researcher, I worked as a reading teacher in adult basic education (ABE) for 11 years at the Community Learning Center in Cambridge, MA. When I began in 1985, our center relied primarily on "quick-and-dirty" silent reading tests to place students in class.

Over the years, however, my colleagues and I began to realize that sensitive, multi-component reading assessment would provide more useful diagnostic and teaching information than silent reading comprehension tests alone. In addition to silent reading, we began doing word analysis (phonics), word recognition, spelling, oral reading, and oral vocabulary assessments with all learners who scored below grade equivalent (GE) 8 on our old test and with any learner reporting a history of serious childhood reading problems.

As we began to use multi-component testing, we noticed that most of our adult basic education (ABE) readers presented very mixed, uneven patterns of strengths and needs across the various components of reading. Some of their reading skills were fairly well-developed, but often many important skills were not. We also began to notice recurring patterns of strengths and needs, and we began to identify typical reader profiles. At the Harvard Adult Literacy Initiative, Professor Jeanne S. Chall had also begun to identify and describe distinctive patterns of strengths and needs among the ABE learners served in her adult reading laboratory (Chall, 1991).

My subsequent research (Strucker, 1995, see the box on page 14) confirmed this: unlike normally progressing young readers, who, by definition, have relatively even reading profiles-- e.g., an "average" fourth grader usually has approximately fourth grade skills across the components of reading-- ABE readers tend to have very uneven reading profiles. To put it another way, as Chall (1991) noted, many ABE readers' profiles resembled those of children who have been diagnosed with reading difficulties.

Why are there so many uneven profiles in ABE? This is a complicated question, but let me suggest a few reasons: Most of our native speakers-- up to 78% according to my preliminary research-- report they had serious reading problems when they were children. Therefore, their reading profiles may have begun to develop unevenly in childhood and remained uneven into adulthood. Second-language speakers in ABE classes generally have acceptable print skills, but usually they have not developed commensurate vocabulary levels in English. Moreover, some may not have had sufficient native language education to have developed these concepts in their native languages.

Why is the "unevenness" of ABE readers' profiles important? Let's back up for a moment to talk about the reading process. The "print aspects" of reading, like word recognition, and the "meaning aspects" of reading, like comprehension and vocabulary, are thought to support each other interactively (Adams, M.J., 1994). But the converse is also true: significant difficulties in one or more components not only hinder one's current reading, they may also impede future progress, for adults or children (Curtis, M.E., in press; Roswell & Chall, 1994). For example, if word recognition is slow and inaccurate, the effortless processing of text that enables comprehension to take place may be impaired, despite a reader's background knowledge, vocabulary, and analytic ability (Perfetti, 1985).

Below, I present case studies of two typical adult learners to illustrate what this notion of "uneven reading profiles" can mean in concrete terms. Both students scored an identical grade equivalent (GE) 4 in silent reading. But, they are very different readers, with very different instructional needs. Their stories highlight two important issues: the value of thorough diagnostic testing that goes beyond silent reading comprehension, and the value of a wider variety of classroom placements than many ABE centers are currently able to offer.

"Richard"

Born in a city near Boston, Richard was 24 when I met him. He enrolled in our center to earn a high school degree in order to enlist in the military. His K-12 schooling featured many interruptions because his family moved frequently during his childhood: "I was never in kindergarten at all and during first, second, and third grade we moved all the time. [Teachers] didn't really deal with my reading problems because by the time they noticed them, we had moved....I'm still very hurt to this day....If I'd had an education, I could have done anything."

Richard's teachers eventually did notice his reading problems, and he was placed in special education classes from middle school on. In high school he was a popular, outgoing student, earning varsity letters in football and basketball. Because he was bright and well-spoken, his friends assumed he would go on to college, perhaps even with an athletic scholarship. In reality, however, Richard's reading had remained stalled at primary school levels.

In the middle of his junior year, his mother moved the family to Florida. Richard re-enrolled in school there, but he began to work long hours after school to help support the family. He soon dropped out to take on a 40-hour-per-week schedule in a fast food restaurant.

A year later, he returned to the Boston area. He has worked in a number of jobs since then, including security guard, restaurant worker and cook, and clothing salesman.

*GE 3 is the highest extrapolated score possible for word analysis

Above are his intake scores on the Diagnostic Assessment of Reading (DAR) (Roswell & Chall, 1992). Richard's grade equivalent (GE) scores should not automatically be interpreted to mean that he is "identical to a first-grader in word analysis" or "identical to a sixth-grader" in oral vocabulary. The miscue patterns of an adult such as Richard and those of a child can often be very different. In vocabulary, for example, Richard probably knows many words, learned through his work experience, that a sixth grader might not know, while a sixth grader might know many school-based words that Richard may have had trouble reading as a child. This doesn't mean that grade equivalents are meaningless. If analyzed together, GE's can serve to indicate areas of relative strength and weakness. Notice, for example, that Richard's "print skills"-- word analysis, word recognition, and spelling -- were much weaker than his "meaning-related skills"-- oral reading, comprehension, and oral vocabulary.

Looking within each test tells us more. My test notes reveal that Richard's word analysis skills were spotty and uncertain: he was able to produce all of the consonant sounds, but many only with great difficulty and some in a distorted, guttural form. He was unable to isolate short vowel sounds and unable to read unfamiliar short vowel words, silent "e", double vowel, and r-controlled vowel words accurately. Richard's spelling miscues paralleled his word analysis errors: a few single vowel words were spelled correctly, but those with double vowels (trian for train; chier for chair) were not.

His word recognition miscues involved guesses based on the first few letters of a word and its overall shape, again with much uncertainty about vowels: witch for watch, courage for carriage, nicest for notice. Several times during testing I reminded Richard to take his time, but he persisted in attempting to read rapidly, even at the sacrifice of accuracy.

Richard's oral reading miscues were similar to those in word recognition: midnight for middle, old for odd. He was able to use the context to monitor and self-correct some of his mistakes. His self-corrections did not affect his scored mastery level, but they did slow down his reading and make it appear very labored. Although he mastered GE 4, even his GE 2 oral reading was not fluent; it contained several self-corrections, hesitations, and repetitions.

Silent reading comprehension was an area of relative strength for Richard, but he took more than ten minutes to read and answer four questions on the 100-word GE 4 passage. Oral vocabulary at GE 6 was Richard's strongest skill. Some responses, however, reflected his word analysis and phonological difficulties: for console-- "When you put something where you can't see it...." while others were vague and imprecise: for environment -- "A place you like...." It is important to measure vocabulary orally; written vocabulary tests may conflate vocabulary with word recognition when used with people who have decoding problems.

Richard's silent reading and vocabulary scores taken alone might have led to his placement in an intermediate reading class that would have concentrated on silent reading comprehension, vocabulary, and basic expository writing. Instead, Richard's severe difficulties with decoding and spelling (as shown in the DAR word analysis, word recognition, spelling and oral reading tests) led to his placement in a class which focused on helping students develop reading fluency and accuracy. This class covered the decoding and spelling of double-vowel syllables and polysyllabic words, and it included lots of opportunities for the oral reading of connected texts-- especially stories, poems, and plays, which Richard particularly enjoyed.

Even though silent reading comprehension skills were not emphasized in this class, after five months Richard began to score at or above GE 6 in silent reading on the TABE (Test of Adult Basic Education) and ABLE (Adult Basic Learning Exam), if they were not timed. Both the TABE and ABLE were normed on ABE adults. They employ the familiar format of short passages followed by multiple-choice questions. The TABE is timed and the ABLE is not timed. (For more on reading tests, see the box on page 16) It appeared that his modest progress in the ‘print aspects' of reading had begun to help Richard unlock his strengths in the ‘meaning aspects' of reading.

The happy ending to Richard's story has yet to be written. After a year in our center, he began to work two jobs to help support his mother when she became ill. Reluctantly, he had to drop his ABE classes. As in childhood, Richard's education had again been interrupted, but at least he had proved to himself that he could make significant progress.

"Vanessa"

When I met Vanessa she was 24 and the mother of a three-year-old. She had been referred by the state's welfare-to-work program to brush up her academic skills so she could go on to job training. Born in Lima, Peru, Vanessa remembered knowing how to read before she entered school, "...because my mom showed me." She reported no trouble with reading or any other school subjects throughout her nine years of schooling in Peru. In Lima she even studied "basic English," but, she recalled, "...whatever they taught us there, it was nothing like real English here [in the US]." When she was 15, her family moved to Massachusetts, and Vanessa was immediately placed, at her father's insistence, in regular, as opposed to ESOL (English for Speakers of Other Languages) or bilingual, ninth grade classes in an urban high school. "That first year...I got no tutoring or anything. Lucky for me there were other Spanish-speaking kids in the class, from Puerto Rico and the Dominican Republic. They explained things and translated, but that first year I just picked up English by listening to people and watching TV."

The next year, she enrolled in a different high school, where "...I got ESOL classes for three years, and it really helped." At the same time she was taking and passing commercial courses in English, but becoming increasingly bored with school: "I quit when I was a senior, with only three months to go... [because] my boyfriend decided to quit, so I followed. I went to a beauty academy to be a beautician. They got me loans to pay for tuition. But they just think of the money. I finished the course, but I couldn't pass the written licensing test in English. Now I still owe them $9,000! Then I got pregnant with my daughter and couldn't work anymore."

 

*GE 3 is the highest extrapolated score possible for word analysis

Above is Vanessa's reading profile, based on the same DAR tests admin-istered to Richard. We notice immediately that even though Vanessa and Richard had identical comprehen-sion scores at GE 4, their profiles are nearly the reverse of each other. Vanessa's reading was relatively stronger in the "print aspects" as compared to the "meaning aspects," while Richard's strengths lay in the "meaning aspects" as compared to the "print aspects." The graph (below) of Richard's and Vanessa's reading profiles superimposed on each other shows how different two readers can be, even when they have identical silent reading comprehension scores.

Vanessa's word analysis skills, while somewhat rusty, seemed relatively intact. Her word recognition score almost hit the GE 6 level, with most of her miscues involving the use of Spanish pronunciation rules on English words: fahvorahblay for favorable and streaking for striking. Her oral reading errors followed this pattern closely. In contrast to Richard, whose oral reading lacked fluency well below mastery level, Vanessa's oral reading remained fluent even above her mastery level. Vanessa's own analysis of her miscues made sense: she explained that since leaving high school she had spoken mostly Spanish at home, watched Spanish-language TV, and read mostly Spanish newspapers and magazines. Her English reading had suffered for lack of practice.

Vanessa's silent reading comprehension at GE 4 -- which she mastered -- and GE 5--which she almost mastered -- only took a few minutes, compared to Richard's ten. She lamented that she couldn't use a Spanish/English dictionary. Her oral vocabulary, also at GE 4, suggested that a dictionary might have helped. As the English words on the test got harder, Vanessa's definitions grew vaguer, even when they were counted as correct: environment -- "What's going on in the world, like smoke...."

Vanessa's profile led us to place her in a different class from Richard. She was enrolled in an intermediate reading class which concentrated on advanced decoding skills, writing, vocabulary, and silent reading comprehension. In addition to this class, Vanessa and other non-native speakers of English received one class per week taught by an ESOL (English for Speakers of Other Languages) specialist. This class focused on the specific needs of people like Vanessa. These learners are fluent speakers of English, but they often need special instruction in the vocabulary and syntax of written English -- e.g., uses of signal words like therefore, despite, however, although-- and in how to transfer metalinguistic knowledge acquired in their native language to English-- e.g., that the English suffixes -tion, -ed and -ly correspond to the Spanish suffixes -cion, -ado(a), and -mente, respectively.

Vanessa's story has a happy ending. She made rapid progress in our center, largely because she was able to regain and enhance her neglected English reading and writing skills. Within five months she had transferred to a combined office-skills/GED program, and, following that, to a prestigious secretarial school. Last summer I met Vanessa on the street and learned more good news: she and her boyfriend have married, they have a second child, and he has landed a good job with benefits. And, with obvious pride, Vanessa reported that she has used her combined Spanish and English literacy skills to obtain her "dream job" as a bilingual medical secretary.

---

Patterns of Adult Reading

In Patterns of Reading in ABE (1995), John Strucker tested and interviewed a sample of 120 ABE readers designed to resemble the learners in the Massachusetts ABE system as a whole. Students were tested with the Diagnostic Assessments of Reading (DAR). The 120 DAR component profile scores were subjected to computer cluster analysis. Nine instructionally- meaningful clusters of ABE learners emerged, ranging from beginners all the way up to those at GED levels.

Here are some of Strucker's findings in brief.

• The ABE readers were quite diverse, especially at intermediate levels (GE 4 - 7), with five distinct clusters identified.
• Most learners had relatively "uneven" profiles of strengths and needs, with only about five percent of the learners displaying the relatively "even" profiles associated with normally developing young readers.
• Native speakers tended to have relatively stronger "meaning-based skills" as compared to "print-based skills," while non-native speakers exhibited the opposite pattern. Chall (1991) reported similar findings.
• The number of native speakers with reading difficulties in childhood was surprisingly high. Seventy eight percent reported formal recognition of their problems by school authorities and subsequent placement in either remedial reading or learning disabilities classes.
• Many native speakers at all levels tended to have difficulty with oral reading fluency, even below their eventual levels of mastery, suggesting that they were having difficulty processing text efficiently and effortlessly.
• Many second-language speakers in ABE classes had surprisingly low levels of oral vocabulary in English (GE 2 to GE 4), despite their fluent levels of conversational English. Similarly low levels of oral vocabulary occurred among some inner-city young adults who were native speakers.

---

In Conclusion

These brief case studies highlight a number of inter-related points for ABE teachers, administrators, and policy makers to consider.

Given that ABE readers are so diverse and their profiles are so uneven, shouldn't sensitive, multi-component diagnostic testing be done with all learners? This testing does not need to be time-consuming, expensive, or burdensome for the learners. The DAR, for example, takes about 40 minutes to administer, and most teachers can learn to use it with just a few hours of training. Most students enjoy the one-on-one attention and instant feedback which tests like the DAR provide.

Does our current array of classes allow us to offer very different readers, like Richard and Vanessa, the different kinds of help they need? Like other ABE teachers, I have struggled to teach learners with very widely divergent needs in the same class. It can be done if the teacher recognizes who those learners are and what their needs are, but it entails a terrible sacrifice of their limited and precious instructional time. To put it another way, attempting to teach "Richards" and "Vanessas" at the same time involves cutting in half the instructional time available to each type of learner.

What can we do about this situation? More money to offer a wider range of classes would certainly help. But we may want to explore some organizational changes as well. In urban and suburban areas, small programs might consider merging to create larger, more versatile centers. Or, they might consider a division of responsibilities in which each small center might specialize in a certain level type of learner, and then refer readers of other types and levels to cooperating centers which specialized in teaching those learners.

Richard and Vanessa represent only two typical ABE reading profiles, but there may be as many as ten to 12 instructionally-relevant reading profiles in the ABE learner population as a whole. And we know even less about the reading profiles of ESOL learners. Just as reading teachers need to know more about each student we teach, the field as a whole needs to know more about the different types readers who come to our centers. Only then will we be able to match our teaching and class placements to their needs. To that end, my colleagues and I at NCSALL, in partnership with practitioners around the country, will be giving basic diagnostic assessments, including the DAR, to a national sample of about 500 learners to create composite portraits of the various kinds of readers we meet in ABE.

These diagnostic portraits of ABE readers can then be used to inform the work of fine-tuning and, where necessary, redesigning our instructional approaches and class placements. But while we're doing this, we can't afford to neglect our traditional commitment to developing curriculum that is relevant, culturally inclusive, and mindful of adult experience and cognitive skills. It's an exciting time for practitioners and researchers in ABE reading, a time when the field will be moving forward on many fronts simultaneously.

---

Tests of Reading

The Diagnostic Assessment of Reading (DAR) measures word analysis (phonics), word recognition (graded word lists), spelling, oral reading (graded short passages), comprehension (short graded passages followed by questions and an oral summary) and oral vocabulary. It is administered one-on-one with ample opportunity for feedback and discussion with the learner. It is criterion-referenced in that learners are given opportunities to master increasingly harder material until they reach their highest mastery level. The DAR reading comprehension tests are not timed.

The DAR is very "user-friendly" for teachers and students because of its clear directions and convenient format. Teachers can measure the same components of reading using other batteries, such as the Woodcock-Johnson Language Proficiency Battery Revised (Riverside, 1990). Or, they can assemble their own diagnostic batteries from tests they already own; e.g., using the Wide Range Achievement Test (WRAT) for word recognition and spelling, the Gray Oral Reading Test for oral reading, a standardized silent reading test like the TABE, ABLE, or Nelson for comprehension, and an oral vocabulary measure such as the ABLE 1, or the Peabody Picture Vocabulary Test. See Chall, J.S. & Curtis, M.E. (1990) and Roswell, F. and Chall, J.S. (1994) for more on diagnostic achievement testing in reading.

Tests

Brown, J., Bennett, J.M. & Hanna, G. (1978) The Nelson Reading Test. Chicago: Riverside Publishing Co.

Dunn, L.M. & Dunn, L.M. (1981) Peabody Picture Vocabulary Test (PPVT). Circle Pines, MN: American Guidance Service.

Karlsen, B. & Gardner, E.F. (1986) Adult Basic Learning Exam, Levels 1,2&3 (ABLE). New York: The Psychological Corporation, Harcourt, Brace, Jovanovich. (Note: Only the ABLE Level 1 measures vocabulary orally.)

Roswell, F. & Chall, J.S. (1991) Diagnostic Assessments of Reading and Trial Teaching Strategies (DAR). Chicago: Riverside Publishing Co.

Test of Adult Basic Education (TABE). (1987) Monterey, CA: CTB/McGraw-Hill.

Woodcock, R. (1991) Woodcock Language Proficiency Battery-Revise (WLPB-R). Chicago: Riverside Publishing Co.

Wilkinson, G.S. (1993) Wide Range Achievement Test - Revised (WRAT-3). Chicago: Riverside Publishing Co.

Wiederholt, J., & Bryant, B.R. (1992) Gray Oral Reading Test, 3rd Edition (GORT-3). Chicago: Riverside Publishing Co.

---

References

Adams, M.J., (1994). Beginning to Read, Thinking and Learning About Print. Cambridge, MA: MIT Press.

Chall, J.S. (1991) "Patterns of Adult Reading." Paper presented at the conference on Literacy and Adults with Learning Disabilities, Teacher's College, Columbia University, October 2-4.

Chall, J.S. and Curtis, M.E. (1990) "Diagnostic Achievement Testing in Reading." In Reynolds and Kamphaus (eds.), The Handbook of Psychological and Educational Assessment of Children. V. I. New York: Guilford Press.

Curtis, M.E. (In press) "Teaching Reading to Children, Adolescents, and Adults: Similarities and Differences." To appear in L.R. Putnam (ed.), Readings in Language and Literacy. Cambridge, MA: Brookline Books.

Perfetti, C.A. (1984) Reading Ability. New York: Oxford.

Roswell, F. & Chall, J.S. (1994) Creating Successful Readers. Chicago: Riverside Press.

Strucker, J. (1995). Patterns of Reading in Adult Basic Education. Unpublished doctoral dissertation, Harvard University Graduate School of Education, Gutman Library, Cambridge, MA 02138.