[AAACE-NLA] Assessing Adult Literacy

[tsticht at znet.com]

April 12, 2007

Strategies for Evaluating the Effectiveness of Adult Literacy Programs:
An Update From 1990

Tom Sticht
International Consultant in Adult Education

Over a decade and a half ago, in April of 1990,  I participated in a
National Conference on State Literacy Initiatives in Harrisburg,
Pennsylvania. At the conference, which was co-sponsored by the National
Governor's Association, the U. S. Department of Education, Project Literacy
U.S. and three organizations in Pennsylvania, I was asked to present a
plenary session on the topic of "Strategies for Evaluating Effectiveness of
Programs."

In my presentation I discussed three areas of assessment of adult literacy.
The first was at the national level, such as what we have today in the
National Assessment of Adult Literacy (NAAL). These types of assessments
are aimed at giving us an idea of the literacy skills of adults regardless
of how or where they may have acquired these skills.

A second arena for assessment is in the evaluation of programs. Here, we
evaluate programs to see if the programs are meeting standards, to see if
they are maintaining their quality, and to see if programs are improving in
their effectiveness. My presentation focused on this second area of
assessment of literacy.

The third arena for assessment that I discussed was the assessment of
individuals for instructional decision making. I noted that individuals are
assessed to see where they should be placed in an instructional program, to
diagnose their specific needs, to determine instructional materials and
methods, and to certify that learning has taken place.

Today, all of these types of assessment of adult literacy skills are
widespread. At the national level since 1990 we have had two national
assessments of adult literacy, the National Adult Literacy Survey (NALS) of
1993 and a decade later the National Assessment of Adult Literacy (NAAL) of
2003. Despite the repeated reporting of 30, 60, or 90 million adults with
literacy skills judged inadequate by the U. S. Department of Education and
the National Governor's Association in both 1993 and 2003, some 95 percent
of adults reported that they did not think they have a literacy problem and
federal policies in the management of adult literacy programs appear to have
lead to a loss of some one million enrollments in programs since 1993.
Additionally, investments in the Adult Education and Literacy System (AELS)
today barely exceed $200 per enrollee, less than 3 percent of the $6500
provided for each child in the federal Head Start program. All this suggest
that we need to rethink the nature of our national assessment of adult
literacy and what our national and state policies ought to be in the light
of what new assessments reveal.

Today, in keeping with my second use of assessment, that is, to see if
programs are meeting standards, to see if they are maintaining their
quality, and to see if programs are improving in their effectiveness, there
is a National Reporting System (NRS) that collects data on learning gains
and several outcomes such as secondary degrees/certificates awarded,
transition into further education/training, employment and so forth.
However, there are not yet national standards for quality nor comparable
national assessment tools that would make valid comparisons of how well
different programs and different states are performing in both maintaining
acceptable quality standards and monitoring their programs so that drops in
performance can be corrected and steps can be taken to improve performance.
Further, the NRS data on learning do not include pre- and post-test scores
so that  gains in learning can be estimated more accurately. Resources also
need to be made available that would permit the numerous programs to return
that dropped out of the AELS, taking some million enrollments with them,
when the NRS demands for data collection were imposed across the nation.

The third arena for assessment that I discussed in 1990 was the assessment
of individuals for instructional decision making. Today there is a
wide-ranging discourse on various discussion lists on formative assessment
for instructional decision making, assessment for learning disabilities,
diagnostics in assessing components of reading ability, and other aspects
of instructional decision making. Whether or not this discussion and
dialogue is actually improving instructional practices in the AELS is not
known to me. However, if classroom decision making is supposed to lead to
improvements in learning which are then captured in program assessments
using pre- and post-test data on standardized tests, as the present NRS
calls for, and if the latter is to lead to improved scores by adults on the
national assessments, then the recent NAAL data suggest that any
improvements in the AELS classrooms over the decade from 1993 to 2003 did
not bubble-up to the national level. There were as many adults in the
lowest level of literacy in 2003 as there were in 1993.

To repeat my conclusion from the 1990 conference, "As pedantic as it may
sound, more research is needed to develop strategies for the assessment of
programs that can improve adult literacy in the coming decade."

Thomas G. Sticht
International Consultant in Adult Education
2062 Valley View Blvd.
El Cajon, CA 92019-2059
Tel/fax: (619) 444-9133
Email: tsticht at aznet.net