[AAACE-NLA] Standardized tests in adult education

[Thomas Sticht]

Aaace-nla colleagues: Paul Rogers asks what issues are discussed on the
list. Often on the nla list we have discussed the egregious misuse of
standardized tests by those striving for accountability of adult literacy
education programs. Recently I prepared a paper for the Basic Skills
Agency in the United Kingdom that illustrates the improper use of
standardized tests by a government agency in the UK aiming to support its
initiatives. Here is a brief extract from the paper that you may find of
interest.  Contact me if you want the complete paper. Tom Sticht

21 December 2003

Have the Literacy Skills of Adults in England Improved Since 1997?
A Critique of the Skills for Life Literacy Survey of 2003. 1

Thomas G. Sticht
International Consultant in Adult Literacy

In 2003 Her Majesty’s Stationery Office published a report entitled The
Skills for Life survey: A national needs and impact survey of literacy,
numeracy, and ICT skills. The report was prepared for the Department for
Education and Skills by the BRMB Social Research part of BRMB
International Limited.

As stated in the report, the aim of the Skills for Life (SFL) survey was
to produce a national profile of adult literacy, numeracy, ESOL (English
for Speakers of Other Languages) , and information and communications
technology (ICT) skills.

However, while the production of profiles for basic skills was the stated
aim of the survey, the Department for Education and Skills drew additional
inferences from the survey provided in a news release posted 3 November
2003 on its internet web page
(http://www.dfes.gov.uk/readwriteplus/news.cfm?page=168&region=1 )  The
news release entitled "Improving Adult Basic Skills, Addressing a
Generation of Neglect" includes the statement, "The survey demonstrates
early success in the Government’s drive to increase the numbers of adults
with basic skills in literacy and numeracy. The proportion of adults aged
16-65 that have literacy skills below the ‘Skills for Life’ baseline -
Level 1 (a D-G grade GCSE) - has fallen from the 7 million estimated in
1997 to 5.2 million adults now. "

According to the Department for Education and Skills news release, the
1997 estimate of 7 million adults with poor literacy skills was contained
in the Moser report of 1999 and was based primarily upon data from the
International Adult Literacy Survey (IALS) as reported by the Organization
for Economic Co-operation and Development (see for instance: Literacy
Skills for the Knowledge Society, OECD, 1997). Unfortunately, as discussed
below, there are major differences between the IALS tests used to
determine the estimate of 7 million adults with poor literacy skills in
1997 and the estimate of 5.2 million adults with poor literacy skills in
the SFL of 2003 which render comparisons of the results of the two surveys
invalid.

Focus on Literacy

In the present report attention is focussed upon the SFL profiles of
literacy in order to address issues comparable to those raised by the
IALS. Among these issues are (1) the construct validity of the SFL
literacy assessment, i.e., the question of just what it is that  the
SFL/Literacy tests measure, (2) the differences between the standardized
test estimates of adults with literacy problems and the adults own
self-assessed literacy skills, and (3) the problem of comparing IALS and
SFL survey results to reach conclusions about changes in the percentages
of adults with literacy problems from1997 to 2003.
*******
Summary and Conclusion

The Skills for Life survey has limited value as a measure of the literacy
skills of the adult population 16 to 65 years old in England. It lacks
construct validity, meaning that it is not certain what skills and
knowledge the survey is assessing. It is inconsistent with the adults’ own
perceptions of the adequacy of their literacy skills for meeting everyday
needs, suggesting that most adults whom the government declares are
lacking in literacy skills are not likely to believe that they have
literacy problems and will hence be unlikely to enroll in literacy
programs. Finally, due to considerable differences in the measurement
methods and scales of the IALS and SFL survey, it is not valid to use the
results of these assessments to calculate the change in the numbers of
adults who are lacking in literacy skills from 1997 to 2003.

For the complete paper contact:

Thomas G. Sticht
International Consultant in Adult Education
2062 Valley View Blvd.
El Cajon, CA 92019-2059
Tel/fax: (619) 444-9133
Email: tsticht at aznet.net