[NLA] Government Misinformation - Summing Up

[Catherine B. King]

Colleagues:

It was explained to Tom by me that mediating to
the public is not statistical reporting and that both
are valid discourses--and in a note from John
Comings, director of NCSL, that the WebPage
is "meant to interest a non-research audience
in what we do."

But in "summing up," Tom still continues his argument
that implies that the only valid discourse is in statistical
reports shorn of mediating orders of thought, or other
kinds of research on human data, and makes conclusive
and critical judgments "summing up" without answering
the questions raised about his claim or without including
a reference to reasonable explanations. What
happened to collaboration and answering reasonable
questions from the field, or to listening to a reasonable
explanation?

This non-listening, one-size-fits-all scientism portrayed
as mono-validity is an intrusion into other kinds of discourse;
it serves to erase a good part of what is essential to the
data and by doing so it radically distorts the whole of
education with its philosophical provincialism--on the
faulty ground that statistical science cannot access these
important aspects of the data in principle--data that NCSL
is rightly looking into with a view towards the EFF
framework.

Scientism has always been problematic, but when turned
into policy about human beings, it is extremely dangerous.

Statistical science makes great contributions to education
and to much of research; but out of place, and as the ONLY
discourse, research field, or critical science worth listening
to by the public or by policy makers, it becomes an over-
simplified monster.  It does so by portraying itself as king
of the sciences, by fitting the too-small shirt of one science
onto the large and complex body of data; and by ruling out
other essential discourses (to the data) that focus on
"higher goals than being employed," etc.
As Tom's summing up note implies, that erasure would
include a mediation into common discourse among
teachers and the public, and concomitantly seriously
limiting and distorting a view of the whole.

The whole thing does adult education, or any education for
that matter, no good.

Regards,

Catherine King
Adjunct Instructor
Department of Education
National University
San Diego, CA



----- Original Message -----
From: Thomas Sticht <tsticht at znet.com>
To: <nla at lists.literacytent.org>
Sent: Tuesday, November 12, 2002 11:17 AM
Subject: [NLA] Government Misinformation - Summing Up


> Research Note 12 November 2002
>
> Tom Sticht
> International Consultant in Adult Education
>
> NALS-Based Misinformation From the Government: Summing Up
>
> On  November 1st I posted the first of three Research Notes pointing to
> misinformation being disseminated by government agencies or government
> funded agencies based on faulty data from the 1993 National Adult Literacy
> Survey (NALS).
>
> Research Note #1 (1 Nov 02).
>
> In this note I claimed that there was  NALS-Based Misinformation at
> USED/OVAE/DAEL. I noted that the USED web site for OVAE/DAEL states that:
>
> Quote: "The National Adult Literacy Survey estimates that about 90 million
> adults in the United States may lack the literacy skills needed to succeed
> in the economy of the future. End Quote
>
> I then pointed out that this was not correct on two counts. First, in the
> original 1993 report on the NALS, on page xviii the NALS researchers
> rhetorically asked whether the findings answered the critical question,
> "Are the literacy skills of America's adults adequate . to ensure
> individual opportunities for all adults, to increase worker productivity
> or to strengthen America's competitiveness around the world?" End Quote.
> The answer given by the NALS researchers: Quote:"Because it is impossible
> to say precisely what literacy skills are essential for individuals to
> succeed in this or any other society, the results of the National Adult
> Literacy Survey provide no firm answers to such questions." End Quote.
> Hence there is no way that the NALS could estimate that about 90 million
> adults in the U. S. may lack the literacy skills needed to succeed in the
> economy of the future. It could not even say whether the literacy skills
> met the needs of the present (that is, 1993).
>
> Second, and even more troubling with the OVAE/DAEL misinformation, is that
> it occurs even though it has been known to ED officials for some time that
> the final NALS Technical Report of January 2001 includes chapters by Dr.
> Andrew Kolstad, the former director of the NALS project at the NCES, which
> show that the NALS used arbitrary statistical methods which greatly
> overstated the numbers of adults with poorly developed literacy skills. In
> fact, using the new statistical methods he recommends, because they
> produce the most valid estimates of skills, the percentages of adults in
> the lowest two levels of literacy that the NALS uses fall by more than
> half! Thus the OVAE/DAEL statement of adults with "at risk" literacy
> skills would drop from 90 to less than 45 million adults.
>
> The Response From USED/OVAE/DAEL: None
>
> Research Note #2 (4 Nov 02).
>
> Here I called attention to NALS-Based Misinformation at the NIFL:
> Partnership For Reading pages at the NIFL web home page. I noted that the
> report on reading comprehension contained information that is not correct.
> Under a section at
>
(www.nifl.gov/partnershipforreading/adult_reading/comprehension/compassess5.
html)
> called Quick Summary of the Research, the report states:
>
> Quote: "A recent, large-scale study of adults' reading comprehension (the
> National Adult Literacy Assessment (sic), or NALS) provides information
> about adults' reading comprehension that is more reliable than the
> information we have about adults' fluency and vocabulary. Results from the
> NALS indicate that most ABE learners will have difficulty integrating and
> synthesizing information from any but the simplest texts.." End Quote
>
> I then noted that the problem with the foregoing is that, using the
> Partnership's definition of reading comprehension, Ouote"Reading
> comprehension can be described as understanding a text that is read, or
> the process of constructing meaning from a text (National Reading Panel,
> p. 4-5). " End Quote, the NALS did not assess reading comprehension, even
> in the so-called Prose comprehension scale. Instead, it assessed the
> performance of tasks ranging in cognitive complexity from low to high, all
> of which required the use of two or more "skills," thereby rendering it
> impossible to say for certain just what was being measured, a point made a
> decade ago (May 1992 Matching literacy testing with social policy.
> National Center on Adult Literacy) about such "real world," functional
> literacy tasks by Richard Venezky, a recent recipient of a NIFL grant to
> study reading. Nor is it possible to say why it was that adults could not
> perform the tasks, e. g, whether they could not construct meaning
> (comprehend) from the language used in the tasks, or if their information
> processing abilities in other skills posed problems for them.
>
> I also called attention to other research by the former director of the
> NALS project at the National Center for Education Statistics (NCES) and
> others that questioned the construct and standards (i.e. response
> probability)  validity of the NALS.
>
> The Response From NIFL: None
>
> Research Note #3 (7 Nov 02).
>
> In this note I called attention to misinformation on the home web page of
> the federally-funded National Center for the Study of Adult Learning and
> Literacy (NCSALL) which stated: Quote" More than 40 percent of working-age
> adults in the United States lack the skills and education needed to
> succeed in family, work, and community life today. By strengthening
> practice and policy, NCSALL helps these adults gain an opportunity to
> achieve their full potential." End Quote
>
> I asked rhetorically, Why is this statement misinformation? Then I
> answered, Because there is no known methodology for determining what the
> skills and education needed to succeed in family, work, and community life
> across the nation are [recall from Research Note #1 that the NALS
> researchers had made this point themselves] . It is even more difficult to
> identify what it means to "succeed" in family, work, and community life.
>
> I concluded that, clearly, the claim that Quote "More than 40 percent of
> working-age adults in the United States lack the skills and education
> needed to succeed in family, work, and community life today"  cannot be
> substantiated and represents yet another example of federally-funded
> misinformation about the skills of America's adults.
>
> The Response From NCSALL: Dr. John Comings, Director of NCSALL,  confirmed
> my claim of the dissemination of  misinformation by acknowledging that the
> NCSALL research team knew all along about the validity problems of the
> NALS but they ignored these problems and used the NALS data anyway in
> forming their estimate of 40 percent of the working age adults who "lack
> the skills and education needed to succeed in family, work, and community
> life today." But since this claim is based on invalid data from the NALS,
> it cannot be considered accurate. It is misinformation.
>
> Comment
>
> Taken together, these three Research Notes point to a continued use by the
> government and government-funded agencies of the results of a major survey
> which is now acknowledged by experts in psychometrics, literacy, and
> measurement to lack construct, standards (i.e., response probability), and
> use or consequential validity. Why this continued use is so is not known
> to me, but I suspect that it reflects a reluctance to admit the
> expenditure of millions of dollars on such a colossal blunder by a number
> of "experts," i.e., advisory panel members, researchers, reading
> specialists, psychometrists, and government officials.
>
> My guess is that this reluctance to acknowledge the problems of the costly
> NALS is compounded by the fact that the government officials of the U.S.
> and contractors to the government convinced other nations to use the NALS
> methods and to conduct the International Adult Literacy Survey (IALS).
> Consequently, now these other nations must defend their expenditures of
> funds on what is more and more recognized as an invalid approach to
> assessing adult literacy.
>
> So through the power that these government officials possess by virtue of
> their oversight of millions of taxpayers dollars, and their ability to
> send those dollars to those who support them, they are able to keep the
> misinformation flowing. I think that contributing to this continued flow
> of misinformation is the seeming lack of any indication from the adult
> literacy education field that anyone cares whether the NALS or IALS are
> valid indicators of the literacy abilities of adults. What does it matter,
> anyhow?
>
>
>
>
>
> _______________________________________________
> NLA mailing list: NLA at lists.literacytent.org
> http://lists.literacytent.org/mailman/listinfo/nla
> LiteracyTent: web hosting, news, community and goodies for literacy
> http://literacytent.org

_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org