[AAACE-NLA] Delusion of Accountability in Adult Education (longer)
tsticht at znet.com
Fri May 21 19:35:43 EDT 2004
Research Note 21 May 2004
The Delusion of Accountability in the Adult Education
and Literacy System (AELS) of the United States
International Consultant in Adult Education
Definition: Delusion: (1) A false belief or opinion. (2) a persistent
false belief that is a symptom or form of madness-Oxford American
The Workforce Investment Act, Title 2: Adult Education and Family Literacy
Act (WIA/AEFLA) was enacted by the U. S. Congress in 1998. It introduced
requirements for accountability including indicators of (1) gains in
learning, (2) placement in, retention in, or completion of, post-secondary
education, training, unsubsidized employment, or career advancement; and
(3) receipt of a secondary school diploma or a recognized equivalent [P.L.
105-220 Section 212(b)(2)].
To define and implement the accountability requirements of the WIA/AEFLA
the U. S. Department of Education established the National Reporting
System (NRS). This system collects data about each of the three categories
of accountability indicators from the states, consolidates it, and reports
it in an annual report to Congress. The most recent report is: "U. S.
Department of Education (2003). The Adult Education and Family Literacy
Act: Program Year 2001-2002: Report to Congress on State Performance.
Office of Vocational and Adult Education."
Reading this report I was struck by the delusional beliefs it propagates.
First is the belief that the Congress can and should somehow use the
information to hold states and local programs accountable for spending in
a responsible way the funds that are distributed in the State Grants. But
this is not possible nor desirable because the information is almost
completely useless. It is based on totally arbitrary "benchmarks" that
have been set at different levels, using different methods, with different
indicators at the state and federal levels. For instance. to provide
information to Congress about gains in learning, the NRS constructed six
levels of learning for adult basic and secondary education and six for
English Language learning. States are required to report "Percent of
enrolled adults who acquired the level of basic skills needed to complete
at least one education level (minimum Grade Level Equivalent 2 years)."
The report presents a graph on page 11 showing that in Program Year (PY)
2000-2001 36 percent of adults enrolled in ABE/ASE acquired the skills
needed to complete at least one education level. Then in PY 2001-2002 this
rose to 37 percent, but was below the federal Education Departments (ED)
performance goal of 40 percent.
However, there is no rationale given in the report as to why the federal
ED goal was set at 40 percent achieving the skills to move up one or more
levels nor is there any indication of what the different state goals were,
why they were set that way, or how progress was measured in each state.
However, we are told that different states used different tests, with
differing amounts of time between pre- and post-tests to assess growth in
learning. But we are not told how many students went from just a raw
score point or two below one level to just barely above the next level, a
result that could occur from the unreliability of the tests used and not
from learning at all. Similar problems hold for the data presented for
English Language acquisition.
The report cautions that "The comparison of educational functioning levels
and level gains across states is thus complicated by this lack of
comparability." (p. 6). But this is a gross understatement, because in
fact this "lack of comparability" in measurement tools and their
administration, coupled with the "lack of comparability" in the methods of
setting benchmark goals in each state and at the federal level, renders
these data totally meaningless and useless to Congress (or anyone else for
that matter) in deciding whether or not states are using their State
Grant funds responsibly and productively. In fact, if any member of
Congress or their staff members, or anyone else did use these data for
holding a state accountable for their State Grant funds then they did so
The delusional nature of the beliefs about the NRS accountability system
is illustrated by statements in the PY2001-2002 report that accept the
faults identified above while disparaging the judgments of teachers. One
statement says that "Assessment systems have greatly improved, with more
consistent and widespread use of standardized, psychometrically sound
assessments and abandonment of subjective assessments or teacher judgments
that do not accurately measure student learning."(pp. 8-9) This indicates
the unsubstantiated belief that the standardized assessments are accurate
and valid measures of student learning while teacher judgments are not. No
evidence is cited in support of this belief nor is any method suggested as
to just how one might "accurately measure student learning."
Another delusional belief regarding the NRS stated in the report says that
"Never before have states and local programs had the ability to make
data-driven decisions to help them design more effective programs and meet
students needs."(p.9) This is an astonishing statement suggesting
ignorance of the more than 35 years of human impact data available for the
State Grant program that supports the Adult Education and Literacy System
(AELS) of the United States.
Furthermore, if meeting the needs of learners is indicated in part by the
number of adults who seek out the AELS for services, then the "old" data
may have been more effective in managing programs than the "new" NRS data.
This is indicated by the fact that for the years from 1966 to 1998
enrollments in the AELS grew at a rate of almost 100,000 a year. In 1998
there were 4,020,550 enrollments in the AELS. After the WIA/AEFLA of 1998
was passed, which introduced new accountability requirements emphasizing
the use of "objective" methods like those of the NRS, enrollments fell to
3,616,391 in 1999, to 2,891,895 in 2000, to 2,673,692 in 2001, and then
rose slightly to 2,787,414 in 2002. This represents a decline of 1,233,136
(30.6 percent) enrollments in the first four years of the new
accountability system now operated as the National Reporting System (NRS).
While it may not represent a very sophisticated approach to program
accountability, it seems clear that if adults do not enroll in the AELS,
it isnt going to help them very much.
A final, and particularly distressing indication of delusional beliefs
regarding the NRS accountability system is indicated by a statement that
deprecates the opinions of the adults in the programs and claims that
"Programs have replaced measures such as self-esteem and student
appreciation of the classes with objective measures of student literacy
This extraordinary statement dismisses the importance of the values that
students place upon their programs and their improved feelings of worth
due to participation in these programs. It reveals the false belief that
standardized tests are "objective measures of student literacy gains" as
though subjective decisions about what content and procedures go into the
make-up and administration of the tests have not taken place. Worse yet,
it suggests that such tests are not only more objective than students
judgments, but also more valid indicators of what has been learned by
students in the programs. But remember, these students are adults, not
children. They are tax payers just like other adults. Who is better
equipped and more responsible than they to hold programs accountable for
meeting their learning needs and for determining whether or not they have
learned useful knowledge and developed better skills?
If anything, teacher and student judgments might prove a much more useful
approach than that of the (mis)use of standardized test data for insuring
that funds for the AELS are meeting student needs for learning. Rather
than working with experts in measurement, psychometrics, and testing,
accountability for Congress might be better served by engaging adult
learner groups such as VALUE to work on devising methods for letting
teachers and adult students in Congressional districts across the nation
better determine whether or not programs are meeting their needs.
But placing trust in the teachers and adult learners in the thousands of
programs in the AELS would require the overcoming of delusional beliefs
about assessment and accountability. And nothing is more resistant to
change than "a persistent false belief that is a symptom or form of
Thomas G. Sticht
International Consultant in Adult Education
2062 Valley View Blvd.
El Cajon, CA 92019-2059
Tel/fax: (619) 444-9133
Email: tsticht at aznet.net
More information about the AAACE-NLA