[AAACE-NLA] Getting Hype Not Help

Fri Sep 24 07:15:53 EDT 2004

Not only that, Brother Thomas, but the basis of the parameters of what counts as reliable science needs much more critical analysis than current governmental assertions provide.  To define the "gold standard" as experimental design as it is in current govt supported circles, is to create a fetish out of a methodology, which may or may not be applicable to any given problem focus.  The scientific philosopher, Karl Popper, provides the following six-step process of investigation, which makes a lot of sense to me.

(1)   A clear exposition of the problem—, or if the problem may be assumed to be well known, a clear reference to it and to an exposition of it…[including any clarifications of] the always shifting problem situation (italics in original).
(2)   A more detailed survey of the relevant hypotheses bearing on the problem (and of the experiments bearing on the hypotheses, indicating the degree to which these are able to contribute to the appraisal of the hypotheses).
(3)   A more specific statement of the hypothesis (or hypotheses) which the author intends to propose, or to discuss, or to test experimentally.
(4)   A description of the experiments and the results. (Experimental research is not defined here in the narrow way as defined by the US Dept. of Education).
(5)   An evaluation:  whether the problem situation has changed; and if so, how.
(6)   Suggestions for further work arising from the work reported (Realism and the Aim of Science, pp. 50-51).

Note the emphasis on the problem situation as setting the framework for the investigation.  Relevant methodologies are drawn upon accordingly in which hypothesis formation along with data collection and analysis plays a significant role in moving the investigation forward.

Does education fall within the realm of the human sciences or the natural sciences? That is a question of no minor importance.

George Demetrion
Adult Literacy Theorist/Philosopher/Researcher/Practitioner

----- Original Message -----
From: Thomas Sticht
Sent: Thursday, September 23, 2004 10:57 PM
To: aaace-nla at lists.literacytent.org
Subject: [AAACE-NLA] Getting Hype Not Help

September 23, 2004

Is the Federal Government Practicing Hype
and Not Help in Adult Literacy Education?

Tom Sticht
International Consultant in Adult Education

The What Works Clearinghouse (WWC) (http://w-w-c.org ) was created in 2002
by the U.S. Department of Education's Institute of Education Sciences
(IES) to provide educators, policymakers, researchers, and the public with
a central and trusted source of scientific evidence of what works in
education. The internet web site of the What Works Clearinghouse boldly
proclaims that it is  "A trusted source of scientific evidence of what
works in education."

However, one wonders how the WWC can be "a trusted source" of anything
when it hasn’t done hardly anything? Who trusts it for what? The only
review now available on the web site deals with research on peer-assisted
learning. It asks:"Does peer-assisted learning work? " The site goes on to
define peer-assisted learning as "Peer-Assisted Learning (PAL) practices
are designed to improve academic outcomes by using students to teach one
another in pairs or small groups" .The site says it reviewed 191 studies
and only 15 met WWC evidence standards. Another 109 studies on peer
instruction are in the review process.

But now that someone knows that there are 15 studies that meet
scientifically sound standards about peer learning, so what? Does anyone
actually need research to know that a person’s peers can teach him or her
something? Haven’t we all  learned something from one of our friends who
was our peer?

The WWC web site also says "The WWC does not endorse any interventions nor
does it conduct field studies. Rather, we provide reports with a rating
system that gives you a sense of how much you can rely on research study
findings."  So if the WWC doesn’t endorse any interventions what is the
use of having such a clearinghouse? The important judgments of whether,
how, when, why to use peer instruction approaches are still left to
teachers or other instructional developers. And it would still be
necessary to evaluate whether the particular application of peer
instruction is effective in the particular context in which it is being
implemented.

All this raises the question of what is the value of government attempts
to determine "what works."

Another example of questionable U. S. government help in adult literacy
education comes from the National Institute for Literacy (NIFL) web site
(http://www.nifl.gov).

At this site there is a link
(http://www.nifl.gov/partnershipforreading/publications/adult.html)
to the Partnership for Reading with a report on reading in adult
instruction.  The linked page says, "Research Based Principles for Adult
Basic Education Reading Instruction represents the best information
available about how adults learn to read. This publication is a valuable
resource for educators and policy makers." But the latter may be a bit of
an overstatement.

The report itself deals with a number of aspects of reading instruction,
and gives a number of "principles" said to be derived from reviews of
research studies. However, looking at how these "principles"  are stated
leads me to wonder how anyone might make anything of them. For instance,
under Reading Assessments Profiles Principles there are a couple of
principles which state:
Principle: When measures of achievement are obtained for each crucial
aspect of reading instruction (alphabetics, fluency, vocabulary, and
comprehension), instructionally relevant patterns of scores, or profiles
of adults' strengths and needs in reading, may be observed. These profiles
suggest that ABE readers, including those in ESOL programs and those with
a reading disability, are very diverse and that any one measure of reading
achievement may not be sufficient to identify strengths and needs for
instruction. And while I’m sure that this *may* be true –so what?

Under Alphabetics a principle states: Participation in ABE programs may
lead to increases in adult beginning readers' word analysis abilities.
Another principle states: Phonemic awareness and/or word analysis
instruction may lead to increased achievement in other aspects of reading
for adult beginning readers.

Under Reading Comprehension a principle is given stating: Participation in
an adult literacy program may lead to an increase in reading comprehension
achievement.

So in all these cases, and others given at the web site, "principles"
derived from the literature are stated in terms of what "may" occur. So
what is one to make of this? Something "may" be helpful or, I suppose,  it
"may not". So what?

It seems to me that this type of "what works" or "evidence-based"
information is simply next to useless. It offers practically no help to a
teacher or program developer that a (very little) inquiry with experienced
adult literacy educators wouldn’t provide.

Unfortunately, we seem to be getting more and more of this type of "busy
work help" from the federal government and less and less of what is really
needed: funds for the Adult Education and Literacy System. By providing a
funding level of just some $220 per enrollment, while arguing that half
the adult population can’t meet the daily requirements for literacy that
our economy demands, it seems to me that the federal government is
providing hype for adult literacy education, not help.

Thomas G. Sticht
International Consultant in Adult Education
2062 Valley View Blvd.
El Cajon, CA 92019
Tel/fax: (619) 444-9133
Email: tsticht at aznet.net

_______________________________________________
AAACE-NLA mailing list: AAACE-NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/aaace-nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.literacytent.org/pipermail/aaace-nla/attachments/20040924/32d79143/attachment.html>