[AAACE-NLA] Star Struck

[John Comings]

I do strongly support that we should base instructional decisions on the 
best available empirical evidence and professional wisdom, and I have 
argued that we should have a national jury (a concept I borrowed from David 
Rosen) to weight that evidence and provide evidence-based advice.

Experimental evidence would be better, but such evidence is difficult to 
gather when attrition rates are high and time-on-task are low.  This is the 
state of our services. So, I believe we should use our limited research 
funds, at this point, to explore how to support students to achieve 
sufficient persistence, intensity, and engagement in learning that they can 
make meaningful, measurable progress. Then, we could start testing 
instructional approaches, but they should be based on exploratory research 
that develops a theory that explains why literacy development in adults 
would be different than it is in children.

NCSALL was not ready for experimental research until we were into our 
second five years of the grant.  We needed to develop approaches to 
research and theories to test first.  Just as we started the experimental 
work, we were told we would not be refunded for a third five years. All is 
not lost. First, we are completing a piece of experimental research that 
looks at an intervention similar to STAR. Second, what we have learned is 
well-documented and so if the National Coalition for Literacy and the State 
ABE directors are successful at including funding for a national R&D center 
in the WIA reauthorization, then that new center can build on what we have 
done.

Write your Senators and Congress members to include $5 million a year of 
support for a national R&D center in WIA!

My criticism of OVAE is not that STAR has no experimental evidence to 
support it. We can't sit around and wait for experimental evidence to 
provide advice. My criticism is that OVAE did not go through an open, 
national process that established rules of evidence and then brought 
together a credible jury to weigh the evidence and make a finding. That 
jury should have looked not just at instructional approaches but at program 
designs that might better support retention of the sample and sufficient 
time-on-task.  Then, OVAE should have funded the research to assess if that 
advice is right, and if there are opposing theories that are credible, then 
test whether its advice is best or if the opposing theory provides better 
advice.

As a field, we should be advocating for an open, nonpartisan way to weigh 
the existing evidence to provide the best possible advice now and for a 
national R&D center that tests that advice and studies our students and our 
programs to develop theories that might lead to better advice.


John Comings, Director
National Center for the Study of Adult Learning and Literacy
Harvard Graduate School of Education
7 Appian Way
Cambridge MA 02138
(617) 496-0516, voice
(617) 495-4811, fax
(617) 335-9839, mobile
john_comings at harvard.edu
http://ncsall.gse.harvard.edu