[AAACE-NLA] Getting Hype Not Help

[George demetrion]

The following is from the What Works Clearinghouse (http://w-w-c.org), cited in Tom's message:

What is scientifically based research?  
According to the Institute of Education Sciences, scientifically based research:  
employs systematic, empirical methods that draw on observation or experiment; involves data analyses that are adequate to support the general findings; relies on measurements or observational methods that provide reliable data; makes claims of causal relationships only in random-assignment experiments or other designs (to the extent such designs substantially eliminate plausible competing explanations for the obtained results);  
I find that to be a reasonable statement on the whole.  Note the qualifier in the parenthesis, especially the word "eliminate."  That's a very high threshold and well may beyond what can be substantially proven particularly when variables cannot be strictly delineated and measured, which is typically the case for complex human problems relatedto analyzing educational issues.
ensures that studies and methods are presented in sufficient detail and clarity to allow for replication or, at a minimum, to offer the opportunity to build systematically on the findings of the research;  
Sufficient detail and clarity--to be sure, as a regulative ideal, but what that means in any given study is up for some interpretation and perhaps multiple perspectives.  Detail and clarity are qualities of relative degrees.  The concept of replication denotes an ideal of exactitude.  The regulative ideals that these terms represent may not be squarable.  That doesn't mean "anything goes," but neither does it mean that some exactitude or certainty is necessarily achievable when speaking of qualities, and, depending on the nature of the study and/or problem, even desirable.
obtains acceptance by a peer-reviewed journal or approval by a panel of independent experts through a comparably rigorous, objective, and scientific review; and  
For a formal study, yes, I suppose, on the first half of the statement.  On the second part, I would substitute the word "competent" for "rigorous," in that the latter term as commonly used has metaphorical implications in terms of conveying an ideal of exactitude that is not necessarily accessible to a given study. The term, "competent inquiry," used by Dewey and referred to in Philipps & Burbules' Positivism and Educational Research, may also contain metaphorical implications, but the term is not as connotative as the term, rigorous.  Moreover, "competent inquiry" does not necessarily rely on "hard" data (another metaphor, right?), but on whatever is needed at any stage of an investigative process, which could include an imaginative hypothesis that sets the spotlight in a new direction.
uses research designs and methods appropriate to the research question posed.  
Exactly, yet that negates the claim that "experimental design" in its linkage to random sampling, is the "gold standard" of scientific research.
George Demetrion

(Snippet)
----- Original Message -----
From: Thomas Sticht
Sent: Thursday, September 23, 2004 10:57 PM
To: aaace-nla at lists.literacytent.org
Subject: [AAACE-NLA] Getting Hype Not Help

September 23, 2004

Is the Federal Government Practicing Hype
and Not Help in Adult Literacy Education?

Tom Sticht
International Consultant in Adult Education

The What Works Clearinghouse (WWC) (http://w-w-c.org ) was created in 2002
by the U.S. Department of Education's Institute of Education Sciences
(IES) to provide educators, policymakers, researchers, and the public with
a central and trusted source of scientific evidence of what works in
education. The internet web site of the What Works Clearinghouse boldly
proclaims that it is  "A trusted source of scientific evidence of what
works in education."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.literacytent.org/pipermail/aaace-nla/attachments/20040924/13e2df29/attachment.html>