[AAACE-NLA] [AAACE-NLA ]Literature Review - Benseman, Sutton and Lander

[Susan Reid]

Thank you Tom 
The literature review  Tom refers to can be found online 
http://www.minedu.govt.nz/web/downloadable/dl10603_v1/fl-literature-revi
ew-final-document-jan-2005-unis.doc
Also check out the New Zealand Literacy Portal 
www.nzliteracyportal.org.nz

Susan Reid 
Manager 
Consultancy Services,
Workbase the New Zealand Centre for Workforce Literacy 
www.workbase.org.nz
www.nzliteracyportal.org.nz


-----Original Message-----
From: aaace-nla-bounces at lists.literacytent.org
[mailto:aaace-nla-bounces at lists.literacytent.org] On Behalf Of
tsticht at znet.com
Sent: Friday, 7 April 2006 6:38 a.m.
To: aaace-nla at lists.literacytent.org
Subject: [AAACE-NLA] Curriculum Policy Proposals


David: regarding the Comprehensive Competencies Program (CCP), I wrote a
paper for the W. T Grant Foundation in 1987 indicating that the CCP did
not increase retention or gains in TABE scores over traditional
programs.

Regarding your call for a national effort on the alignment of content
standards, curriculum, and assessment as a means of improving programs
of the AELS, I noted that there have been the APL, CASAS, EFF, and NRS
which have tried to do (and the NRS is still trying) what you are
calling for and so far there is little evidence that any of these
national efforts has lead to notable improvements in the AELS. I
suggested trying to find out why these efforts have not been successful
in the past (not just a literature review; I agree with Marie that that
would not be terribly fruitful) and then determine how such an effort
that you are proposing would overcome the problems of the past and
present (i.e., the NRS).

You asked: "Are the standardized assessments the states are using to  
produce NRS data really well aligned to these new standards, or do   the
new standards cry out for new, better aligned assessments." This is one
of the questions I think should be investigated in the type of research
on past and present efforts on content/curriculum/assessment (cca)
alignment I mentioned in the preceding paragraph. Before one starts out
to do a new national effort to align cca it would be important to know
if improperly aligned assessments are being used, why better alignment
is not being used, and how better alignment might be achieved.

You also said: "I  would agree that if we were to have a national
curriculum model, or even one used by several states, it would be wise
to do systematic evaluation and/or research to find out if it had an
impact on student 
learning. "  But we DO have a national curriculum model, the NRS with
ABE, ASE, ESL as major content standards; we have thousands of published
materials aligned with the six levels of each of these content standards
in use across the nation, and we have assessments the field has agreed
to use to determine learning outcomes from this national curriculum.
There are also four years of data (NRS) now to determine if this system
has improved the AELS and if so in what ways.

On a state basis, we have a quarter century of data from the CASAS
project 
to align content, curriculum, and assessment in California. I mentioned
a decade of such data in my earlier post. So far I have not been able to
find data indicating that this cca project has improved participation,
retention, student goal achievement, or learning outcomes from the
numerous data-based reports available from the state of California.

Marie Cora posted a couple of messages on this topic. But contrary to
what Marie Cora said in her first post in response to this discussion,
Canada does not have national standards for ABE,ASE,ESL nor do they have
a national data collection system like the NRS. Each province runs its
own system. The federal government has produced reports with lists of
essential skills and levels of complexity for these skills, and there is
a project that has produced standards for ESL but these are not required
to be used across provinces.

The UK has a national curriculum framework but this lists content-free
"skills", like knowing how to use commas, for five levels of literacy,
entry level 1, entry level 2, entry level 3, level 1, level 2. The idea
is that the abstract, decontextualized  "skills"  can be taught by
embedding them in various content contexts of interests to adults. But
there are no data to show how adults actually progress through this
skills framework.

I agree with Marie in her second post calling for better understanding
of what is going on in the nation today. But I am not optimistic about
her call to try to find "best practices" this way. We have had numerous
attempts to find "what works" and "best practices" and so far these have
not lead to any notable improvements in student participation,
retention, goal achievement, or learning from the projects with which I
am familiar. I note that the What Works Clearinghouse of the federal
government has been looking for what works in adult literacy education
for over two years and so far has not produced any reports.

A recent report by Benseman, J., Sutton, A., & Lander, J. (2005)
illustrates some of the problems with much research in adult basic
education and considers that much of it is of poor quality and does not
lead to many clear cut conclusions, including research on best
practices. In the international literature review by Benseman et al,,
which includes over a dozen references to research in England's national
research center and a score of references to the U.S. national center,
the project summary
states:  "The purpose of this literature review is to provide a critical
evaluation of the available research evidence about effective practices
in literacy, numeracy and language (LNL) teaching and programme
provision in order to inform policy development within the broader arena
of foundation learning. ...Over the last three decades there has been a
considerable amount of research and writing in the sector, most of which
did not meet the criteria for this review. ... Despite the growing
recognition of the importance of LNL, there is still a dearth of
specific research relating to this area in New Zealand and the situation
is only marginally better overseas. ...Initially we were seeking
experimental research studies. However, the overall quality of the
studies located was not particularly high ... On the advice of the
project's technical advisor ...the scope of the research was expanded to
include case studies, observational studies and collections of
professional wisdom from practitioners, in order to ensure a reasonable
pool of studies to consider and to identify 'the best available
evidence'. "

In presenting their findings, Benseman, et al state, " The findings of
this review need to be considered tentative, due to the limitations of
the research base from which they are drawn."  The authors then go on to
present some factors that they "have some confidence ... are likely to
contribute to learner gain."  This includes factors like engaging
appropriately skilled teachers who can identify the strengths and
weaknesses learners have in speaking, reading, writing and numeracy;
deliberate and sustained acts of teaching, clearly focused on learners'
diagnosed needs;  a curriculum that is linked to the authentic literacy
events that learners experience in their lives; programs that allow for
high levels of participation, probably more than 100 hours of tuition;
and similar ideas which to a considerable extent appear to me to be
based as much or more on common sense than any research of a rigorous,
empirical nature. [Benseman, J., Sutton, A., & Lander, J. (2005,
January). Foundation learning
project: "Working in the light of evidence, as well as aspiration": A
literature review of the best available evidence about effective adult
literacy, numeracy, and language teaching Phase 5  report. Wellington,
NZ: Ministry of Education, Tertiary Education Learning Outcomes Policy
Group. ]

Up to now, then, the research community does not appear to have produced
much solid evidence that it knows how to improve the AELS. It might be
of use if instead of trying to put research into practice, we could put
researchers into practice. In this approach research could be conducted
in which the researchers went to some program that was in need of
improvement in some ways and then demonstrated in some scientifically
acceptable manner that they actually improved the program. While it is
always tempting to try to do research leading to generalizations that
one hopes will solve certain problems everywhere, it might be more
useful to demonstrate that one can accurately identify some problems
somewhere and then fix them. Putting researchers into practice and
letting them demonstrate that they know how to identify and solve
important problems of practice at some specific place, might prove a
more effective way of improving the field than trying to put research
into practice.

Tom Sticht



_______________________________________________
AAACE-NLA mailing list: AAACE-NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/aaace-nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org