[AAACE-NLA] Adult Literacy Assessments: Helpful or Harmful?

[tsticht@znet.com]

May 29, 2007

The Assessment of Adult Literacy Without Adequate Opportunity for Adult
Literacy Education Should be Forsaken

Tom Sticht

The National Assessment of Adult Literacy (NAAL) report released in 2006
indicated that almost half the nation's adults were so poorly literate as
to be unable to function well in our contemporary society. This problem was
exacerbated when talking about Hispanics and Black Americans. They fell well
below the White adults in their literacy skills. All this lead some to
suggest that our nation's international competitiveness is at risk because
of the lack of functionality of our workforce. But is this all true? Some
background and additional research information provides a basis for
questioning these results and inferences from the NAAL.

Background

The Young Adult Literacy Survey (YALS)of 1985 was the forerunner to the
National Adult Literacy Survey (NALS), International Adult Literacy Survey
(IALS), Adult Literacy and Life Skills (ALL) survey, and the National
Assessment of Adult Literacy (NAAL) of 2003. The YALS provided the basic
methodology for all these assessments in scaling literacy on Prose,
Document and Quantitative scales. These assessments all used Item Response
Theory (IRT) to scale the literacy abilities of adults and the difficulty
levels of the test items. This produces some interesting comparisons.

For instance, on the 1985 YALS, in the Document literacy assessment, 73
percent of the tasks demanded 300 level skills or lower, while 57.2 percent
of young adults possessed 300 level skills (about the middle of Level 3 on
the later NALS)  or higher. Thus, the Document tasks tended to be skewed
toward the easy end of literacy task difficulty. Overall, the average
percent correct for Document literacy tasks was 83.3. Whites scored 85.9,
Hispanics 77.6, and Blacks 71.8 percent correct on the average for Document
tasks. Thus, using percentage of items correct, all ethnic groups appeared
fairly capable, even though there are clear ethnic differences.

However, while 65.4 percent of Whites scored at the 300 skill level or
higher, only 37.0 percent of Hispanics and 19.8 percent of Blacks scored at
the 300 skill level or higher. Note that, if one focuses on the fact that
only one in five Blacks were at the 300 skill level or above on the
Document scale, one might infer a very low performance level for Blacks on
Document tasks.  Yet, overall, Blacks performed over 70 percent of the
Document tasks correctly. This apparent contradiction results from the fact
that to be at the 300 level of skill requires that people possess an 80%
probability of being able to perform tasks that are at that level of
difficulty. But people with lower levels of skill have a greater than zero
probability of being able to correctly perform 300 level tasks. When the
latter are taken into consideration, as in calculating the overall average
percent correct, then a much greater percentage of the population may be
seen to be able to perform Document tasks across the full range of
difficulty levels, from easy to hard, than are able to perform tasks at the
300 level of difficulty or above.

In the construction of the YALS assessment it was thought that the materials
and tasks selected were representative of "real world" tasks that adults
would encounter. If that were so, and if Hispanics and Blacks scored over
70 percent correct, then one might think that overall these young adults
were fairly capable in their abilities to perform representative "real
world" tasks. But the scaling procedure, that is the IRT scaling
methodology, provided a different perspective.

How "real world" the tasks were on the YALS, or are on the other assessments
that came later is questionable. For instance, on the YALS and the NALS one
Prose literacy item was as follows: The following poem was read: "The
pedigree of honey does not concern the bee-a clover, any time, to him is
aristocracy." The task was then to answer the question "What is the poet
trying to express in this poem?"

As it turns out, this was one of the most difficult of all questions, with a
difficulty level of 387 placing it in NALS level 5. But the question is, how
"real world" is this task for most adults?

Some Research

My concerns for how valid such assessments are for telling us how well
people might succeed in the "real world" was aroused when during the
Vietnam war over 350,000 young men were inducted who had literacy scores on
average below the 6th grade level. They had all been excluded as unfit for
service due to low cognitive ability. However research showed that later on
some 85 percent performed their jobs well and completed their military
service. In one study even though Black soldiers scored 20 percentile
points lower than Whites on the cognitive/literacy tests, they performed
about as well as White soldiers on job knowledge and job performance tests.
In some studies Blacks actually outperformed whites.

Later, as a member of the National Commission on Testing and Public Policy,
I argued strongly for positions that were included in the recommendations
of the Commission, including "#1. Testing policies and practices must be
reoriented to promote the development of all human talent. We must
reevaluate how we judge the quality of tests, the names we give them, the
ways we report results, and the ways we use them. No testing program should
be tolerated that classifies people as unable to learn; potentially negative
classification in school or the workplace should be accompanied by learning
opportunities."

And "#4. The more test scores disproportionately deny opportunities to
minorities, the greater the need to show that the tests measure
characteristics relevant to the opportunities being allocated."

Clearly, today the results of the NAAL paint a disproportionately negative
picture of the literacy skills of Hispanics and Blacks.  They are
classified as being unable to adequately perform literacy tasks needed for
education and work. But even though this denigrates the skills of these
minority groups, designating them as inferior in our educational and
workforce activities, there has been no major commitments to providing
accessible, sustainable, life-relevant educational opportunities for them
in the Adult Education and Literacy System (AELS) of the United States.
Indeed, from fiscal year 2000 to fiscal year 2004 there was a decline of
12.8 percent (78,986) for Blacks enrolled in the AELS.

In my view, it is entirely possible that our adult literacy assessments and
their classification of some half of adults, and even more who are
Hispanics or Blacks, as able to meet only the lowest demands for literacy
for learning, working, and earning, may be doing more harm to our
international competitiveness than the actual skills of our workforce. How
many international corporations will want to work in a country with half
the workforce declared unfit for productive work in contemporary times?

Degradation by classification,  without adequate opportunities for
education,  should not be undertaken.

Thomas G. Sticht
International Consultant in Adult Education
2062 Valley View Blvd.
El Cajon, CA 92019-2059
Tel/fax: (619) 444-9133