[NLA] Government Misinformation-summing Up (long)

[Thomas Sticht]

This is a response from Tom Sticht to Catherine King’s several posts
commenting on my research note re NCSALL. In particular it responds to a
recent message made by Catherine in response to Andrea in which Andrea
posed questions about statements I had made earlier on the NLA list about 
what I called misinformation on the NCSALL home web page. Catherine said
about Andreas’s questions:  Quote: "Yes, these are indeed reasonable
questions, etc., but from the note Tom wrote (below), he's not asking what
they are.  He's inferring we cannot even get there because there is no
know methodology.   Any use of these terms, then-- with the assumption
that you, or I or the general public, may have some qualified idea about
what they mean in general discourse, is "misinformation." End Quote

But this is wrong. What I said in my original post was

Quote: But there, on the home web page of the National Center for the
Study of Adult Learning and Literacy (NCSALL) is the misinformation that:
Quote" More than 40 percent of working-age adults in the United States
lack the skills and education needed to succeed in family, work, and
community life today. By strengthening practice and policy, NCSALL helps
these adults gain an opportunity to achieve their full potential." End
Quote

I then went on to asking and answering a number of questions indicating
why I thought the statement was ‘misinformation" the first of which was:

Quote: "Why is this statement misinformation? Because there is no known
methodology for determining what the skills and education needed to
succeed in family, work, and community life across the nation are. It is
even more difficult to identify what it means to "succeed" in family,
work, and community life. "

Apparently Catherine thinks that I made a broad generalization that there
is today and will forever in the future never be a method for determining
what the skills and education needed to succeed in family, work, and
community life across the nation are and that it will forever  be
difficult to identify what it means to "succeed" in family, work, and
community life. But I did not make this generalization I said that today
there is no known methodology for accomplishing these analytical feats.
And I know of none.

But Catherine thinks otherwise. In an earlier post she provides us with a
method when she states: Quote" Tom is right that there "is a lack of
methods for measuring what it means to 'succeed' in family, work, and
community life," --if you limit his context to **statistical science.**
However, in each community, and for each adult-person, that notion of what
"success" means can certainly be explored **meaningfully** and quite
precisely in the context of a creative dialogue between researchers,
teachers and community members--adult students."End Quote

But notice that in my original comment I did not say that there is a lack
of methods for "measuring" what it means to ‘succeed’ in family, work, and
community life. I did not mention "measurement" and did not restrict my
comments to statistical science. Instead I said only that there is no
known methodology etc. period.

So let’s look at Catherine’s methodology. She provides Andrea and the rest
of us with the "creative dialogue" method and states: Quote: "The big
oversight of the NALS researchers and Tom's own statement is that we, as
you suggest, (1) ask reasonable questions about what those [statements]
mean (that's the methodology, but it is not statistical method); further,
(2) we dialogue with those whose lives are to be affected by our programs
about what such things as "success" mean and (3) we put forth our findings
to the community of researchers, theoreticians, practitioners, etc., in an
ongoing, developmental, and collaborative effort to understand further the
changing meanings of such commonly used terms." End Quote

Presumably Catherine is not aware that the methodology she has proposed is
precisely what has been tried in numerous studies, including the EFF, and,
surprise, in the NALS development, too. But the problem is that if one is
ever to do anything based on this kind of intensive dialogue, there are a
large number of methodological problems that have to be addressed, and at
some point one has to stop to try to figure out what has been learned, and
going back to re-do the dialogue from time to time is pretty resource
intensive.

These methodological problems become overwhelming when, like the NALS or
NCSALL study, one is trying to establish on a national basis what the
concept of "success’ in the family etc means, as was stated on the NCSALL
web site.  It is also difficult to do on the type of local community
basis, and one individual person at a time basis that Catherine provides
us as a methodology. For instance, in San Diego, there are some 2.5
million people in round numbers. Now how do we go about carrying on an
extended dialogue about success with 2.5 million people. Should we just
restrict ourselves to adults? OK. Who is an adult?  A 10, 16, 18, 21 ? ?
year old. Lets say 16 and over (but others may disagree). Now assume that
narrows our population to be interviewed to just 2 million adults. Can we
talk with them all one-to-one or even 10 in 200,000 focus groups? Or will
we have to rely on sampling? Darn! If we do that we will have to bring in
the statistical scientists. They will need to know things like how many
dialoguers do we need and have? Will they all ask the same questions or
just carry on free-wheeling discussions? How much time will the interview
take? Will you use different language speakers? How many different ones?
Will they translate the information? And so on and on.

If we don’t go to sampling then if each dialogue with each of the
2,000,000 adults takes 15 minutes (remember, this is intensive dialogue),
the dialogues will take 30,000,000 minutes, that is 500,000 hours, or
62,500 8 hour work days, or over 227 work years. Then all the dialogue
material must be recorded and read. If each record contains just 50 words,
that will give us 100,000,000 words of dialogue to comprehend. What will
we do to comprehend this much information?  Reading at 300 words per
minute will require over 333,333 minutes.That’s 5555 hours and that is
694, 8 hour work days or over three work years. How will we keep track of
what we are reading? And what if we end up with a lot of different ideas
about what these adults think a family is (is it two men living together
in an apartment as roommates? How about one person whose spouse has died
and is now all alone? Etc), or what counts as success in the family (Sure
a husband who runs away from his family but still supports them and loves
them is a success at family life, says he. Is he right?).

What if we get even more divergent ideas about what it means to possess
the skills and education needed to succeed in family life, work and in the
community? What if we come up with some ideas like, Needs to read with
understanding or he/she cannot function successfully in family, etc. Then
we discover someone in a literacy program who cannot read but has had what
he thinks is a successful family life, good work and is a contributing
member of the community (remember John Corcoran?).

As I hope everyone, and especially Catherine, can grasp, actually doing
what she says at the national level is exceedingly difficult, and leads to
contentious conclusions for a number of reasons, not the least of which is
simply because some 200 million adults or even a sample of 26,000 adults
do not agree on what is meant by success in family, work, or community and
never have and in my opinion, as a citizen not a scientist, never will.

But perhaps I am all wrong here. Perhaps Catherine could, like Albert
Einstein was wont to do, give us a thought experiment in which she
dialogues with herself and tells us what she thinks an adult is, what it
means to be a success in family, work, and community contexts, what the
skills and education are that are needed to achieve this success, and then
how we might extrapolate her methods to the hundreds of thousands of
communities across the nation and obtain non-statistical information to
inform the public using common language that "More than 40 percent of
working-age adults in the United States lack the skills and education
needed to succeed in family, work, and community life today." I will be
particularly interested in how she manages to do this without using some
form of statistical analysis.

Turning now to Catherine’s  repetitious comment in her notes that my
"statistical science" work consists of "one size fits all," let me say
that my scientific training is in psychology, not statistical science, and
I have worked in clinical, industrial, educational, personnel, and
organizational psychology, as well as in cognitive science, neuroscience, 
optics, acoustics, haptics, and biophysics, using methods from
anthropology, sociology, artificial intelligence, communications science,
experimental psychology, audiology, electronics, psychometrics, political
science, and other fields of study.  I have developed adult literacy
programs and standardized tests of listening, reading and mathematics for
the Army, Navy, and Air Force and various civilian organizations, and I
worked on the early phase of the EFF project. So I have used many
different kinds of methods to fit many different issues, problems and
educational programs. If you are truly interested in learning more about
how I think and go about solving problems in various contexts several
reports of mine, with additional references to my work,  can be found on
the web at www.nald.ca under full text documents searched by author using
S for Sticht.

I would appreciate information as to where I can find reports of your
research in adult literacy education.





_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org