NLA Discussion: Policy Recommendations from the NLA

[gdemetrion]

David, Jon and others:

I'd like to add another related policy recommendation, something for
thoughtful reflection, not necessarily requiring an immediate answer.  This
is part of a message I wrote about a year ago in a previous discussion on
assessment in an effort to move off the dime of having to have standardized
statistical information on the progress of every student.  Though needing
refinement, I think this would fit in with John Coming's call for sampling
and part of a viable response to John's point that:

"In fact, I believe that Congress is asking a simple question: 'What do we
get for our investment in ABE/ESL/GED
programs?'"

Part of the challenge in getting the dog to wag the tail instead of the tail
wagging the dog, is an assessment system that more accurately reflects how,
what, and why students learn as well as its significance for them, as well
as for what I have sometimes referred to as the "public good," which I view
as broader than current policy foci, yet grounded in the ethos of democracy,
this nation's founding ideal.  On the latter, I refer back to Catherine
King's last post, one of the most eloquent and to the point messages we've
seen on the centrality of democracy to the fabric of this republic and a
highly legitimate ground, which needs flushing out, to frame the purposes of
adult literacy education.

In a shift from a quantitative to a qualitative approach to assessment,
which draws on the former as a secondary source of information that then is
explained by the narrative report (below), I offered the following as a
framework to help think how things could be different.  The following is
based on the need to separate assessment criteria needed (and in part,
established) by students and instructors for informing and assessing
instruction, and that required for effective program evaluation.  The major
point is that for the latter, statistical (and highly limited) information
is not required for every student in order to answer Congress' basic
question, but it does require letting go of the dominance of the
quantitative metaphor if well structured representational forms of program
assessment/evaluation are to flourish as primary measures.

I believe that the success of this shift will prove a major factor in
determining the success of An Action Agenda for Literacy in the need to
garner the vital consensus to vigorously move the field forward.  Assuming
that the NCL is willing and able to carry forth on this, is the field
willing to publicly embrace the effort.  On this, I agree with Jon on the
need to ante up.

George Demetrion
Literacy Volunteers of Greater Hartford
Gdemetrion at msn.com
Gdemetrion at juno.com
Gdemetrion at lvgh.org



____________________________________________________________________________

At the end of each funding cycle the state office would have to issue a
narrative report on the programs that are funded by federal money.  The
report would be based on the narratives that the programs provided that then
would be summarized, synthesized and analyzed by the state offices of adult
education. This would be an ongoing progress report.  A broad range of
information would go into the report at both the local and state level and
there would be no need of a "one size fits all" approach."  May 1000 flowers
bloom.  Obviously, certain criteria would be needed, though the plurality of
the system in a given state would also be respected,  (Why should a town run
ABE program be judged on the same criteria as a community-based literacy
program?).

This scenario would include quantitative information, but more as support
(secondary measures) that would amplify or help explain the primary story
told through narrative. In this scenario, sampling, rather than focusing on
quantitative information on each student would be more pervasive.  Given the
narrative focus, there would be an emphasis on
development such as:

a)  This is where we've been
b)  This is what we attempted to achieve during this funding cycle
c)  This is what we actually accomplished, including unanticipated
    breakthroughs and a whole host of projects and initiatives that flowed
out of our work
d)  These are the problems that we still have-some of which we have the
capacity to improve on, some of which we don't given current resources
e)  This is where we'd like to go for the next funding cycle

This report could be developed by a representative team from both the field
and the state office with consultative support from a research institute.
The information would flow back to the programs as well as upward to the
federal government and would be a format to stimulate discussion, analysis,
and program development as well as "data" for national accounting.



----- Original Message -----
From: "David J Rosen" <DJRosen at world.std.com>
To: <nla at world.std.com>
Sent: Monday, February 26, 2001 1:44 PM
Subject: NLA Discussion: Policy Recommendations from the NLA


>
> Jon (Randall),
>
> Thanks for helping clarify what you mean by policy recommendations.  I
> would agree with David Heath that there have been some policy
> recommendations in our conversations on the NLA, but I think you are
> right that they are not always framed so that Congress would be able to
> deal with them.
>
> I would like to bring to your attention one, in particular, that makes a
> lot of sense to me -- and, I believe, to many others on the NLA list. In
> September, 1999, John Comings wrote:
>
> ========================================
> To: nla at world.std.com
> Subject: NLA Discussion: NRS
> From: John Comings <John_Comings at harvard.edu>
> Date: Mon, 20 Sep 1999 09:50:59 -0400 (Eastern Daylight Time)
>
>
> Congress may want a lot less than we think.  Our field has
> interpreted the call for accountability to mean that we
> must measure increases in skill and improvements in the lives of
> every adult student.  In fact, I believe that Congress is asking a
> simple question: "What do we get for our investment in ABE/ESL/GED
> programs?"  We can answer this question by looking at a small number
> of our students, but looking at them in-depth with assessment and
> evaluation tools that could uncover a wide range of impacts and
> outcomes.  Rather than collecting a little bit of data on all
> students, we would collect a lot of data on a few students.  This
> could be done well with the existing resources being put into
> accountability systems.  The results might not make us look very good,
> but if it was done right this approach would also identify ways
> to improve performance.  Funders, in Juliet Merrifield's way of
> thinking, would then be responsible for providing the resources
> needed to make those changes.
>
> We still need to be responsible to our students and that requires a
> different set of assessment tools to help them identify their
> strengths and weaknesses and measure their own progress.  I see this
> as a much more difficult assessment task than the accountability for
> funders.
>
> ======================================================================
>
> I see this as a recommendation to change the NRS regulations, to
> persuade Congress that a targeted, in-depth, follow-up study of a sample
> of programs makes more sense than the current shotgun approach of trying
> to collect these data for all programs.  I wonder if you could tell us if
> the NCL has considered this, and if so, what action(s) have been -- or
> might be -- taken.
>
> David J. Rosen
> <DJRosen at world.std.com>





---------------------------------------------------------------------------
To UNSUBSCRIBE FROM the NLA list, send an e-mail message to
majordomo at world.std.com
Skip the header. In the body of the message type (only) unsubscribe nla

To SUBSCRIBE TO the NLA list, send an e-mail to majordomo at world.std.com
Skip the header. In the body of the message type (only) subscribe nla
---------------------------------------------------------------------------