[AAACE-NLA] Literacy Testing Debacle

[Andrea Wilder]

Tom--

In regards this debacle, do you have any suggestions for resolving 
these testing dilemmas?  What do you suggest?  You have seen the scene 
for many years, and your perspective would be welcome.  Do you see 
anything that works AT ALL?

Thanks.

Andrea


On Oct 15, 2007, at 6:55 PM, tsticht at znet.com wrote:

>
> 12 October 2007
>
> The Great Literacy Testing Debacle in the United States
>
> Tom Sticht
> International Consultant in Adult Education
>
> Definition: Debacle: n. A total, often ludicrous failure. Online 
> dictionary
> at
> www.answers.com/topic/debacle
>
> The United States seems to be caught up in measurement mania when it 
> comes
> to literacy. The No Child Left Behind law calls for extensive testing 
> of
> children's reading abilities in different grade levels. For adults, the
> U.S. Education Department has developed adult literacy tests, and 
> Title 2:
> The Adult Education and Family Literacy Act of the Workforce 
> Investment Act
> of 1998 calls for accountability measures which the U. S. Education
> Department has implemented in a National Reporting System that makes
> extensive use of adult literacy tests. But in all these cases, the 
> actual
> measurement instruments and procedures for measuring reading/literacy 
> and
> comparing states suffer from major flaws. They all follow different
> procedures in their development, which renders them incomparable, and 
> hence
> interpretations of the data produced by comparing the findings of the
> various tests are essentially meaningless.
>
> Testing Children's Learning of Reading
>
> On page 39 of the June 4, 2007 issue of Time magazine a graph is 
> presented
> showing differences between the percentage of fourth graders in each 
> state
> who are deemed "proficient" in reading based on each state's different
> standardized test. The graph also shows the percentage deemed 
> "proficient"
> on the National Assessment of Educational Progress (NAEP) which are
> standardized tests given in all states in the nation. There are some 
> very
> large differences between the state and national test results. For
> instance, Mississippi reports that close to 90 percent of 
> fourth-graders
> are proficient in reading on the state developed test, while on the 
> NAEP
> only about 19 percent score as proficient. This is a whopping 71 
> percentage
> points difference in the numbers of fourth graders in Mississippi who 
> are
> considered proficient in reading.
>
> The Time article provides data indicating that using state test data 
> the
> average percentage of fourth graders considered proficient is 70 
> percent.
> Using the national NAEP tests only 30 percent of U.S. fourth graders 
> score
> as proficient. This is a 40 point average gap between state and 
> national
> estimates of fourth grade reading proficiency. The state and national 
> tests
> use different procedures to determine if children are proficient 
> readers,
> and are hence incommensurate. This raises the question of which tests
> should be considered as the valid indicators of the reading abilities 
> of
> the nation's fourth grade children, the state or the federal tests. …or
> perhaps neither.
>
> Testing Adult's Literacy Ability
>
> Jumping ahead to when fourth graders have grown up, the 2003 National
> Assessment of Adult Literacy (NAAL) presents data for Prose and 
> Document
> literacy which indicate that in 1992 15 percent of adults over the age 
> of
> 16 scored as proficient on these tests while in 2003 13 percent of 
> adults
> scored as proficient, a drop of 2 percent during the decade. 
> Surprisingly,
> only 30 percent of adult college graduates scored as proficient in
> literacy.
>
> Although there are clearly differences between the NAEP reading tests 
> for
> fourth graders and the adult literacy tests, again rendering them
> incommensurate, nonetheless they both attempt to portray how many of 
> their
> target groups are "proficient" in literacy. The data indicate that 
> there
> are fewer than half as many adults (13 percent) who are proficient in
> literacy as there are fourth-graders (30 percent) who are proficient 
> using
> the federal-made NAEP, and there are only a fifth as many proficient 
> adults
> as there are proficient fourth graders (70 percent), if the average of 
> the
> state-made tests are used. This suggests a tremendous loss of 
> proficiency
> as children grow into adulthood!
>
> Measuring Literacy For Accountability in Adult Literacy Education
>
> The problems of assessing literacy also show up in the accountability 
> system
> of the nation's Adult Education and Literacy System (AELS), which is 
> made up
> of the some 3,000 programs that are funded jointly by federal money 
> from
> Title 2 of the Workforce Investment Act of 1998 and state and local 
> funds.
> The National Reporting System (NRS) which prepares reports on how well
> adults are learning literacy in the AELS has acknowledged that 
> different
> states use different standardized tests, with differing amounts of time
> between pre- and post-tests to assess growth in literacy learning. The 
> NRS
> has also indicated that the comparison of educational functioning 
> levels
> and level gains across states is complicated by this lack of 
> comparability.
>
> But despite the acknowledged lack of comparability in the tests and
> procedures used in various states the NRS goes ahead and computes 
> averages
> of the percentage of adults making learning gains across the fifty 
> states.
> Of course, the lack of comparability in measurement tools and their
> administration renders these data totally meaningless and useless to
> Congress (or anyone else for that matter) in deciding whether or not 
> states
> are using their federal funds responsibly and productively.
>
> The Debacle of Testing Literacy Ability
>
> Despite all these faults of testing for literacy skills, there is 
> apparently
> no hesitancy in using the test results to reward some educators and 
> punish
> others for what they are doing to teach literacy, whether to children 
> or
> adults. Despite extensive use of standardized tests of various sorts 
> by the
> fifty states, thirty- year reading trend data with the NAEP show no
> meaningful improvement for 9, 13, or 17 year old children since the 
> early
> 1970s. Further, the testing of adult literacy in 1992 and again in 2003
> shows little or no improvement in literacy at the lowest levels and a
> decline at the highest levels.
>
> To date, then, the great literacy testing debacle has cost hundreds of
> millions of dollars, threatened teachers and administrators, subjected
> children to hours of drill and practice in test taking rather than 
> engaging
> in learning important content and skills, and cast aspersions on the
> literacy skills of America's workforce, thus advertising to the world 
> that
> the U. S. workforce is incompetent. This cannot be good for the health 
> and
> welfare of the nation nor its international competitiveness in the 
> global
> economy.
>
> Even if we could get the testing of literacy right, which we have not 
> done
> up to now, there is no way we can test ourselves out of the serious
> educational problems that afflict our K-12 and adult literacy education
> systems. There is a word for the obsessive repetition of utterly 
> foolish,
> unreasonable, and failed practices: insanity.
>
> Thomas G. Sticht
> International Consultant in Adult Education
> 2062 Valley View Blvd.
> El Cajon, CA 92019-2059
> Tel/fax: (619) 444-9133
> Email: tsticht at aznet.net
>
>
>
> _______________________________________________
> AAACE-NLA mailing list: AAACE-NLA at lists.literacytent.org
> http://lists.literacytent.org/mailman/listinfo/aaace-nla
> LiteracyTent: web hosting, news, community and goodies for literacy
> http://literacytent.org
>