[NLA] Searching for adult litercy experts

[KathleenBombach@aol.com]

I admit, I have not been reading the list for a while (too busy), but my 
attention was caught by Tom Sticht's message on 'research.' Now it appears 
that 'education research' is being held as the model for all of us.  The 
problem is that 'education research' is generally considered a joke by the 
other social sciences.  Let me list some of the reasons:

1.  Inability to set up true treatment/comparison groups;
2.  Use of elementary statistical analysis, rather than a rigorous analysis, 
reporting a simple but wrong conclusion, hence, lack of replicability;
3.  Results generally show weak correlations, which are reported as 'facts' 
way beyond their real importance;
4.  High levels of Type II errors, driven by 'publish or perish';
5.  Use of individual t tests, rather than more sophisticated multivariate 
tests, and lack of knowledge of the real likelihood of results reflecting 
reality rather than chance (Bonferroni Inequality);
6.  Changing the standard of reporting results from p=.05 to .10 and . 
15--when you combine that with the use of individual t tests and weak 
correlations, the result is make believe (but reported as research-proven 
fact);
7.  Lack of replicability, both because few results can be shown a second 
time, and because few in the field try to run independent tests of reported 
findings.
8.  In ability to distinguish between statistical significance and the actual 
import of a finding.
This is just a partial list.

The bottom line is that 'educational research' is the laughingstock of the 
hard and social sciences.  When I read educational research, I always come 
across the problems I have listed, including in some of the literacy research 
I have seen. These are some pretty basic errors.  So what is this great 
research that is out there that we need to emulate?  (I don't mean the 'we 
know that this works or that works', which is what the Department of 
Education is putting out.) Point me to that scientifically unassailable body 
of research please, because I cannot find it. 

Or maybe we should hurry up and produce some 'educational research' 
ourselves, muy pronto.  Fight schlock with schlock.

And I did not even get into any philosophical discussion--this is just nuts 
and bolts stuff.

Kathleen Bombach
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.literacytent.org/pipermail/nla-nifl-archive/attachments/20020506/6c1ec4ba/attachment.htm