No subject

discussed that in their joint letter some months back. The separation
though is a bit disingenuous in that regardless of what assessment
system is used, it reflects ideology and a perception of adult literacy
education.  There is nothing "objective" about NRS standards.  Some of us
spent much of the fall and winter critiquing the positivistic philosophy,
the behaviorist psychology, and the deficit-oriented (get a job, get off
welfare, stay out of jail, vote) social policy upon which both the WIA
and NRS are based.  That discussion is documented in the NLA archives.

To gain consensus in the current climate, EFF developers would be required
to embrace much of the NRS framework, particularly the reporting system
and then impose that on an entire state if its rubrics are to count as
primary.  It is that ambition which I recommend the EFFers walk away from
and concentrate on what they can do best and where they are having an
effect:

a)  Keep focusing on linking the framework to instruction (We see lots of
good stuff on that from the EFF listserv)

b)  Keep focusing on linking EFF to curriculum development

c)  Take a more modest position in acknowledging that while EFF has a lot
to offer, it is *a* framework rather than *the framework for adult
literacy education

c)  Over time, and as part of the visioning effort, perhaps EFF can play
a role in helping to establish a more sound and humane social policy,
though that will require a national, rather than federal or
corporate-driven vision.

That's how I see it."

George responded again:

"Colleagues:

Perhaps the issue is not so much that the NRS data will prove much, but
that as a logical (one form of logic) system of statistical collection,
it will serve as a meta-discourse (called a data base) that then can
provide the illusion to support certain conclusions that will be
collected in some report, and appropriately filed.  In other terms, the
NRS stats may provide some illusion of control to justify (or not) the
spending of certain monies based upon a cost-benefits utilitarian
analysis of public investment.

That might not be so bad except for the costs to the field that such a
paean to the quantitative metaphor extracts.  What isn't getting funding
and what isn't getting legitimized as a result, what the religious
educator Maria Harris refers to as the "null curriculum?"

BTW:  One wonders how many small-to-mid sized programs are going to
receive (or even apply for) funding via the WIA for FY 2001.  Not to
fret, though.  We have fair and equitable access."

Catherine King replied to Tom Sticht and others:

"Colleagues:

(My note will seem longer than it is because I have quoted
Tom Sticht heavily and followed with comments.)

In his recent note, Tom Sticht says:

"Without a full specification of some content domain it is
not possible to assess whether or not adults or anyone
else have mastered the domain or even a specifiable portion
of it.

"There is a long history of this problem in public
education and psychometrics and so far as I know, there
has been no universally acceptable technical method found
to solve this problem. At the most, what is usually done
is that some understandably incomplete and only partially
satisfactory methods for saying someone has learned a little,
some more, or most of a somewhat loosely specified domain
are agreed upon and applied to judging a students learning."

The problem is: what do we understand as "the problem"?

If we have learned anything over the past 300 years (if not
in all of history) it is that knowledge is moving.  Fields
cannot be defined "once and for all" because not only is the
substance moving, so our questions are changing.

I am suggesting that a "universally acceptable technical
method" can only be found in the kinds and methods of
questions we ask, and cannot be had in finding the right
box or method to put all knowledge in.

Though we do rightly **seek** all there is to know--the
foundation of education itself--the expectation of
**finding** all knowledge, and subsequently fashioning
a "once and for all" test to measure for it all, is a
highly misguided adventure to say the least.
Improvement is a constant--and behind that constant is
the question we all have: "what is the better way?"

History tells us the "better" question is a part of our
conscious make up--or the method behind all methods. But
"what" is actually better has proven to be constantly on
the move--**that** happens to be a verified fact.

This brings us to Tom Sticht's comment below:

"Perhaps the Equipped for the Future project will be able
to cope with all the complexities of developing content
standards and measures that the rhetoric of accountability
calls for. However, given the fact that no one has ever
solved the problem of adequately defining any moderately
complex knowledge content domains and all their various
modes of representation and expression, it seems unlikely
that EFF will be able to accomplish this either. Without
a full specification of some content domain it is not
possible to assess whether or not adults or anyone else
have mastered the domain or even a specifiable portion
of it."

It seems to me Tom is the "pot calling the kettle black"
 here; but he has a truth I doubt he would want to
acknowledge.

The point is, he's right that it is "unlikely" complete
content will ever be mastered.  But his argument is wrong
to suggest that:  Therefore, EFF is unsuitable to the
needs and definition of adult education.

The point is, if it **is** indeed unlikely we will ever
master complete field content, as Tom says, perhaps
instead of denigrating EFF, we ask ourselves instead:

Are we asking the wrong questions? Perhaps our efforts
to "master content completely" are misguided, and perhaps
we are forcing something into a static and "logical"
mode that has proven itself to be dynamic, contrary,
and resistant to "inside the box" ordering, over a long
history of human activities?  The writers of EFF are not
afraid to ask these questions, nor are they afraid of
not being able to put answers in a drawer, as it were.

EFF has cheerfully broken the absolutist, shallow, and
closed-minded mode of thought that seeks to kill education's
creative side, however inadvertently.

This mode of thought is **absolutist** because it seeks
a "once and for all" standard by using the good name
of order, but taking it to its extreme in its facetized
mode.  It is dangerously **shallow** because it denigrates
a qualitative understanding of human education.
And it is **closed-minded** because it is extremely unhappy
with future oriented things it cannot already know or order
on a piece of paper with boxes to check.   EFF is none
of these things.

EFF directly recognizes creativity, movement, and a depth
of substantial human being and activities that is focused
on the individual's autonomous thought and faithful
self-direction of the adult, rather than the need for,
as Tom puts it, "figuring out how others can know what
the adults know and can do to certify . . . "

I think standards are essential.  Some, like knowing
the alphabet, and doing basic math, will probably never
change.  Though the field of math, for instance, is
never complete, it is essential that we standardize
what we do know about ordering our world.

But Tom's note implies with patriarchal patience that,
because teachers are not aware of some national logic,
they have therefore failed up to this point in educating
their students in the fuller sense that EFF speaks to,
as well as learning things that have been rightly
standardized?

Absolutism and a national order will not fix the problems
that the teachers are already working on and having much
success at.  The question is:  What really is better?
Tom says:

"In light of recent NLA discussions that suggest that
standards that link to curriculum are desirable, I wonder
how many programs are linking their curricula to the
NALS prose, document and quantitative scales. Also, how
many of the states that are using the CASAS assessment
system have linked their curriculum to the CASAS
assessments and from there to the NALS standards.
In an earlier post I noted how a large number of
teachers and administrators that I have been in touch
with in workshops over the last year have never even
seen a NALS item."

To Tom I would say:  It would be nice to have a clearing
house where all teachers could see what everyone else is
doing--where all of the tests are available--as **was**
part of the rhetoric last year?  This way the power of
education remains in the program director and the
teacher--and sometimes even in the adult student?
And we can all guess that they/we would choose what we
thought was really the best thing for our students?

***Or are we saying (Tom?)we can't trust teachers and
administrators to choose well, with the best interest
of their students in mind, given all the
collaborative choices developed by our colleagues?***

Forced standards based around competitive national
goals where withholding monies is the whip, is
paradoxically tantamount to pulling the rug out from
under national education, as well as taking a cannon
shot at the foundations of democracy, as we have
argued here before.

And, I don't know how a forced national standards
model can replace or retain the flexibility needed--
and already at work--at the level of delivery of
changing fields, **unless** we define a democracy
of educated people on the model of a factory of
material goods?   Forced national standards put us in
danger of becoming a "reality" that no one, not
even reality itself, can depart from."

Bob Bickerton, Massachusetts State Director of Adult Education, joined in:

"George, Regie, Tom, Catherine, et al,

I'd like to check in with some very basic questions:

1.      If WIA and the NRS are too narrow, shouldn't we be pushing for a
more complete vision of what our work is about at the state level?  WIA
provides states with the flexibility to do this.  Then the NRS could
simply become a sub-set of what's looked at from the broader perspective
of each state, i.e., nothing more than a report, unfortunately incomplete,
but without the power to narrow the important breadth of our work.  

2.      If quantitative measures alone are inadequate to describe the work
of our field and our students, shouldn't we be working to find consensus
across our field to articulate the qualitative dimensions?  WIA provides
states with the flexibility to do this.  I imagine the most difficult part
of this dialogue will be reaching consensus across diverse constituencies,
particularly when it comes to who may be able to make judgements about
what's credible and reliable.  But it seems to me it's well worth the
effort.

3.      If large cross-sections of students can articulate important
aspects of what they want to know and be able to do, shouldn't we be
working across our field and in partnership with our students to capture
this?  And if these turn into a rich, but necessarily incomplete set of
learning/content standards, shouldn't we at least honor this achievement?
Whether it's EFF or other similar efforts to surface such skills and
abilities sought after by students, shouldn't we find a way to agree to
make such 'learning/content standards' a part of the foundation of our
work (including articulating this foundation with valid and reliable
assessment processes) -- always acknowledging that this is a sub-set of
this universe and that we all need to continue to listen/hear and respond
with an even richer set of teaching and learning experiences?

I'm concerned that so much energy continues to go into finding fault with
everyone else's proposals that we're not making as much progress as our
students need and deserve from us -- to make our work truly accountable to
them!  So my final question:

4.      Are we afraid of any form of accountability with consequences,
including to those we profess to serve?

Here's hoping y'all are doing well during these changing and trying
times."

Replying to Bob, Regie Stites wrote:

"Bob and others:

I think the questions that Bob posed (or are they suggestions?) point us
in the right direction toward working out answers to concerns about
standards, accountability, and assessment systems.  What these
questions/suggestions imply is serious work on developing performance
assessments.  In an earlier post George Demetrion referred to the creation
of *rubrics* as a (poorly defined) step toward making *qualitative*
standards (like EFF) count as primary *quantitative* measures for the NRS.
Development of performance assessments (whether as guides for instruction
or for accountability measures) does involve creation of rubrics -- and
much else.  Rubrics are central because they are the mechanism for
translating detailed *qualitative* descriptions of performance goals into
*quantitative* measures of levels of performance.  At this point, the
qualitative/quantitative distinction begins to break down.  To make this
more concrete, a rubric is needed whenever we want to judge performance
that is more complicated than a set of correct/incorrect responses to test
questions. Teachers and students commonly develop and use rubrics to
evaluate the quality of writing, oral presentations, project work, etc.
If tasks are well-structured and the criteria for judging performance
(rubrics) are clear to learner and teacher then assessment gets folded
seamlessly into instruction. This is what I mean by performance assessment
and I believe that it can be made to work for accountability (probably
first at the state level as Bob suggests) as well as for instructional
purposes."

Kathleen Bombach added some concerns:

"I have several concerns.  One is that something like NRS standards
(really outcomes) have been tried before, under JTPA.  The result was that the 
population was creamed in order to meet the mandated outcomes.  The
expensive apparatus and process that developed around meeting these
outcomes grew immensely in order to weed out anyone who might not,
because of skill levels, English speaking ability, race and ethnicity,
gender, or personal/motivational factors, succeed in earning a GED or
getting a job. Less and less money actually flowed into direct
instruction and services for the participants who made it through the
maze.
 
The other side of this creaming was that low income people who could have 
striven for more were diverted into the JTPA system because they could 
provide quick positive results.  I remember a program we did under JTPA.
A number of the participants decided they wanted to go to college and got 
through the acceptance process, only to be told by their agency counselor 
that they were not allowed -- they had to go back for job placement or job 
training in one of the infamous 13-week programs.  (In one national study,
13 weeks was the average amount of time for JTPA vocational training
programs).

Since I did JTPA between 1983 and 1995, I observed every trick in the book
to make sure only "winners" were served.  In the beginning I bought into it
and participated unquestioningly.  After a year I began to question and try to 
make changes to address what I thought were the real needs of poor people, 
having been one most of my life until that point.  It didn't work -- the 
necessity to meet outcomes was too strong even when co-workers were 
sympathetic.  If the person was not going to result in a success in a 
relatively short period of time, they were diverted into someone else's 
program -- often the adult literacy programs.  Of course, no money for the 
learner went with the referral.

The second concern I have gets at the difference between outcome
measurements vs. accreditation requirements.  In an accreditation
process, one looks at the inputs, the activities, and the outcomes.  In a
system where the only things that matter are the outcomes, any way you
can get those outcomes becomes paramount, including not providing services
but pretending that you did. 

We have tried the outcomes based approach before and it failed to serve
most of the people who tried to enter the system it spawned. Now we are
stuck with it again because it is politically appealing and sounds so
good!  EFF is one way to keep the focus on the learner, not the outcome,
and to look at multiple aspects of providing a quality program defined as
meeting the needs of learners, not the state or the private sector.  If
we can unify behind an approach that may not be perfect to all, but has
substance and some earned momentum, as well as reflecting a lot of work
over a period of years by a lot of knowledgeable people, we may have a
long term chance of changing the dynamic from one of feeding the machine
with workers to a balance of home and family, citizenship, and work.  And
aesthetics, spirituality, and whatever else matters to individuals.

If we cannot push a broader agenda because we cannot agree on the nature
of our utopia, we will be stuck with recycling the same inadequate system
under a different name for the next hundred years.

However, we must continue our internal debates over the nature of
perfection."

Marie Hassett, from Boston, Massachusetts, joined the discussion:

"First, thank you to the people who have responded to this strand.  As one
of the developers of the Massachusetts ABE Curriculum Frameworks, I'm really 
benefiting from the knowledge and experience of the people who've been 
writing.

That said, I do wish to raise a concern:  I may be oversimplifying, but it 
strikes me that during our discussion of standards, a number of people
have treated the various standards/reporting systems (EFF, NRS, WIA et al) as
if they were necessarily prescriptive of what we're doing, and not
descriptive of a set of desired results.

I realize that standards can go a long way to determining what we choose
to teach and test, but I don't think any one set of standards can (or should) 
capture all of the outcomes all teachers/programs are hoping to achieve.  
Perhaps we need more investigation of the ways that standards accepted by 
programs or funders are translated into guidelines for classroom practice, 
and used by teachers.  Standards don't teach--teachers do."



Brenda Bell, replying to an earlier post by Andrea Wilder, about the issue
of meaning-making and standards:

"Andrea Wilder wrote:

'Andres,

To add a little more.  I am part of a highly educated group now trying to 
solve a very knotty problem.  It involves vocabulary and concept.  To my 
knowledge they, the rest of the group, don't have the vocabulary or
concepts to help them out of some dilemmas.  Potential damage to the group is 
possible--decline in trust, high levels of gossip, etc.  By a complete 
accident I have some of the vocabulary and the concepts.  Your post is
very helpful to me because it points me in a direction that heaven knows I
explore everyday, that of adult literacy. 

Is there anyone else out there who has framed the adult
education/literacy problem in this way?  i would be very interested in
learning more about how this has affected your teaching and your
students' learning.  It would at last interject some sanity, that is
MEANING, into the standards issue. People seem to forget that.  Meaning
somehow gets left out.  Isn't that why we all read?  Or am I missing
something?'


Meaning and purpose are integral to the EFF standards.  These standards
have been developed and are meant to be used within a larger framework -
one that includes adults' purposes and goals (the Four Purposes) and life
roles (Role Maps).  They are not separate from the purpose adults have for
acquiring/developing and using a skill.  

While written before the EFF standards were complete, the Transformational
Learning Project's article in the December 98 issue of Focus on Basics may
be useful in thinking more about the relationship of how we make
meaning/create understanding and the development of specific skills.  Of
particular use might be the matrix on page 26, which shows the EFF Skills
Categories (Communication, Decision-Making, Interpersonal, and Lifelong
Learning) and three ways of understanding (from TLP:  instrumental,
socializing, and self-authoring) - with examples based on perspectives on
working together with others."


Gloria Gillette, from Ohio, added:

"The NRS has done three very important things for  our state. It has
forced us to analyze:
          
	        Who  do we serve?
                How we do we serve them?
                How do we measure our effectiveness?

There has been a lot of flotsam and jetsam, but in the end I think it has
been a very good process of self reflection.

And I think in the end, we have developed a system that both embraces our 
goals and maintains the reality of the system in which we work, while 
respecting the integrity of the students we serve."

Replying to Bob Bickerton, Tom Sticht said:

"David: (longer message)  Recently Bob Bickerton addressed some questions
to several NLA list participants, including me. Here are my responses to
Bob's questions and comments. Thanks, Tom Sticht

Bob's Question 1.      If WIA and the NRS are too narrow, shouldn't we
be pushing for a more complete vision of what our work is about at the
state level?  WIA provides states with the flexibility to do this.  Then
the NRS could simply become a sub-set of what's looked at from the
broader perspective of each state, i.e., nothing more than a report,
unfortunately incomplete, but without the power to narrow the important
breadth of our work.  

Response: It did not take WIA or the NRS to provide states with the
flexibility of "pushing for a more complete vision of what our work is
about at the state level". States have always had that flexibility and
presumably have operated by some sort of vision over the last thirty
years. In realizing their various visions different states have made
different uses of different standardized tests.  I understand that in
New York, state-funded programs have had to use the TABE to measure
growth in learning; in California, Oregon, Washington, Connecticut, and
other states visions have been put into practice using the CASAS tests
to assess gains in learning; other states have included in the
realization of their visions other standardized tests in measuring parts
of their "visions." So as I understand it, then, the only thing new now
is that the NRS is systematizing the gathering of data using
standardized reporting forms and extending the need to provide learning
outcome data in some systematic way. Also, as I understand it, the NRS
has the blessing of the Council of State Directors of Adult Education.
It is, as you say, merely an accounting system for keeping track of
outcomes. It in no way narrows all the other types of information of
which states may wish to keep track. But I think that the idea that
funding may somehow be tied to the outcome data is a new thing under the
WIA. At the state directors level that may not be a problem because the
data are aggregated across local programs and the state is accountable
only for the aggregated data. It is highly unlikely that any state
personnel will lose their jobs because of failures to show learning
outcomes in line with their approved five year plans. But  folks in a 
given local program may be a little bit nervous if they can't meet state
goals. Some have told me that they worry that their jobs may be on the
line.

Bob's Question 2.      If quantitative measures alone are inadequate to
describe the work of our field and our students, shouldn't we be working
to find consensus across our field to articulate the qualitative
dimensions?  WIA provides states with the flexibility to do this.  I
imagine the most difficult part of this dialogue will be reaching
consensus across diverse constituencies, particularly when it comes to
who may be able to make judgements about what's credible and reliable. 
But it seems to me it's well worth the effort.

Response: States have always had the flexibility to articulate
"qualitative dimensions" of the work of our field and our students. WIA
does not prohibit such descriptions, either. It just does not include
them in the core indicators of learning. Instead the NRS discusses
implementing the WIA requirements for data on learning outcomes by
suggesting a number of standardized tests with standards for indicating
achievement at each of the six levels of ABE and ESOL. The NRS also
permits other sorts of standardized, quantitative indicators of learning
such as performance assessments that indicate learning at each of the
six levels of ABE or ESOL. Portfolios can be used, too, but there must
be some way of providing scores ranging from at least one to six so that
progress up through the six levels of learning can be indicated. This
may be done using various scoring guides, that is, rubrics, that permit
the assignment of ranks to performance indicators. Whether or not most
states will opt to use the more time consuming methodology of
performance/portfolio assessment over standardized tests is unknown by
me. But at the present time I think  most states have opted for
traditional standardized tests. I have not found any that  propose to
use the TALS, the commercial version of the NALS, perhaps  because it is
too time consuming (it uses performance tasks and hand scoring of
various responses using rubrics for scoring). Apparently, ease of
administration and scoring is a matter of some concern to programs. So
maybe the search for qualitative dimensions (note that anything with
dimensionality can be quantified!) may not be worth it to programs if
they are too much trouble and too costly to use. 

Bob's Question 3.      If large cross-sections of students can
articulate important aspects of what they want to know and be able to
do, shouldn't we be working across our field and in partnership with our
students to capture this?  And if these turn into a rich, but
necessarily incomplete set of learning/content standards, shouldn't we
at least honor this achievement?  Whether it's EFF or other similar
efforts to surface such skills and abilities sought after by students,
shouldn't we find a way to agree to make such "learning/content 
standards" a part of the foundation of our work (including articulating
this foundation with valid and reliable assessment processes)  always
acknowledging that this is a sub-set of this universe and that we all
need to continue to listen/hear and respond with an even richer set of
teaching and learning experiences?

Response: From what I have heard over the years talking with many
teachers and administrators almost everyone thinks that their program
and their teaching reflects what adults have said they want to know and
be able to do over the years. In short, they seem to think they have
been capturing what their adult learners have articulated as their
learning needs and desires. But when some adult learner says he or she
wants to be able to pass the drivers license test, they don't mean
drivers license tests in general, as some sort of general competency
statement or "content standard" such as "convey ideas in writing"   or "
solve problems and make decisions" or "completes application forms" but
rather as the specific vocabulary use, reading, writing, spelling,
problem solving, and  decision making they have to do to pass the
specific test they have to take in their state and locale to get their
drivers license.  But the sorts of things they have to learn in the
specific do not usually show up on the general assessment tools that are
based on broader competency or learning content standards, such as the
TABE, ABLE, AMES, NALS, TALS, CASAS or any other standardized tests
represent. Hence, though programs may  strive continuously  "to
listen/hear and respond with an even richer set of teaching and learning
experiences" they do not always make much happen that shows up on the
assessment devices that are used for accountability.  In my experience,
this appears to be what frustrates lots of teachers and learners about
various attempts to create generally applicable content standards and
the present stock of assessment tools. In going from the specific to the
general, most of the actual learning seems to get stripped away.

Bobs Question 4.      Are we afraid of any form of accountability with
consequences, including to those we profess to serve?

Response: Most of the teachers and administrators I have spoken with are
not afraid of accountability with consequences so long as the
accountability is based upon what they think they can do and are doing
as adult educators. They express some concern about being held
accountable for things like job placement and income earnings which they
see as beyond their professional responsibilities. They express concern
about the use of standardized tests like those included in the NRS
guidance papers because they do not seem to relate much to what they are
teaching and they think the tests do not accurately reflect what
learning does take place in their programs (see above). Regarding those
they serve, survey after survey has rewarded teachers and administrators
with positive feedback statements from those they serve. If funding was
based on the accountability reflected in the praise adult learners have
for their teachers and programs, the Adult Education and Literacy System
(AELS) in the United States would not be in the obscenely under funded
position it is presently in."

George Demetrion:

"Colleagues:

To reiterate, this is an excellent and highly relevant discussion on
standards.

Thanks to Regie Stites  for defining the term, "rubric."  I think I get
that now. Whether it's the GED writing test or EFF standards, various
criteria are assigned a number (or to use NRS terminology, a level) that
would allow a quantitative measure.  Regie feels that they could be made
to work at the state level for accountability as well as instruction,
though Tom Sticht argues that most states in the current NRS
requirements are opting for the easier to administer standardized
tests.  One wonders also, in light of the NRS mandate and time frame, how
many states are willing and perhaps able to incorporate the more
qualitative indicators of assessment that well constructed rubrics
(depending on what they're linked to) might begin to capture.  As Regie
points out, there's much more than rubrics involved in establishing good
assessment measures.  And even the development of these within a state
context that would involve more than the few "pilot" states that are
attempting this now in response to the NRS, would be a long way off, if
ever.  In the meantime states have to respond to the pressing
requirements of the NRS *at this time* and most are going to rely on
standardized tests.

This brings us then to Bob Bickerton's point  on the freedom and
flexibility allowed by the WIA.  Perhaps other state directors would like
to weigh in  here on that, though I wonder how freely they would be
willing to speak.  In any event, given the complexity of the
issue -- creating relevant standards that capture significant aspects of
what students achieve through participation in adult literacy/ESOL and
GED classes -- in ways that are meaningful and can be applied to
instruction as well as used for reporting purposes -- does the WIA
legislation facilitate that process or retard it?  In theory it could do
both, yet as Tom points out, the practical impact is that in the vast
majority of states, it is only going to reinforce the pervasive tendency
already to rely on standardized tests as the "primary" measure, premised,
obviously, on the quantitative metaphor.  State directors, I would love
to be proven wrong on this.

I recently purchased a book from Peppercorn Press that addresses the
issue of assessment. ( Peppercorn Press is an excellent clearinghouse of
student-generated and Freirian inspired resources located at 693 Snow
Camp, NC  27359  tel (336) 574-1634).  Give them a call if you'd like to
obtain a catalogue).  The book, called Language and Communication,
identifies the following principles for assessment:

1.  Assessment should reflect what has actually been taught
2.  Assessment should serve instruction rather than drive it
3.  Assessment should fit in with the approach to teaching and reflect
    its value system
4.  Assessment should yield reliable and valid results
5.  Assessment should give learners a sense of their own progress
6.  Assessment should not be culturally or linguistically biased
    Assessment should allow for comparisons of learner progress within and
    between programs
7.  Assessment should be integrated within instruction (formative) and be
    incorporated at the end of learning cycles (summative)

These are obviously big challenges and no one's saying development of
assessment standards based on these principles is easy.  My concern is
given the current climate based on the quantitative and reductionist
mandates of the NRS/WIA, these principles aren't even on the radar
screen.  They may be "nice," but don't count in the "real world" of
policy, power, and funding.

If there is a way out of this morass I don't know, but I'd like to
briefly sketch out a possible world.  First, Mr. Pugsley, any way out of
this morass would require at least a one year MORATORIUM on the imminent
implementation of the NRS.  Very difficult, to be sure, since the NRS
train has well left the station.  But sir, an imminent train wreck is on
the horizons.  I say, *Stop!*, *Halt!*, *Screech!*, *Watch Out!*, *Put on
the Brakes!*

There is simply no way, given current pressures and limitations of
resources, that the current ABE system can step up to the plate as both
Bob and Regie are suggesting, to create the kind of complex,
multi-dimension assessment system needed to come close to capturing the
actual learning that is taking place in classrooms and tutoring sessions
across this great land.  While in principle, the NRS allows for freedom
and flexibility (that Tom says the states already had without the NRS),
the reality is that it will only reinforce the current emphasis on
quantification, standardization and a very limited legitimate view of
adult literacy education.  Mr. Pugsley, that is the reality, sir.

Is there another model?  Here's one that would need much amplification.

At the end of each funding cycle the state office would have to issue a
narrative report on the programs that are funded by federal money.  The
report would be based on the narratives that the programs provided  that
then would be summarized, synthesized and analyzed by the state offices
of adult education. This would be an ongoing progress report.  A broad
range of information would go into the report at both the local and state
level and there would be no need of a "one size fits all" approach."  May
1000 flowers bloom.  Obviously, certain criteria would be needed, though
the plurality of the system in a given state would also be respected, 
(Why should a town run ABE program be judged on the same criteria as a
community-based literacy program?).

This scenario would include quantitative information, but more as support
(secondary measures) that would amplify or help explain the primary
story told through narrative. In this scenario, sampling, rather than
focusing on quantitative information on each student would be more
pervasive.  Given the narrative focus, there would be an emphasis on
development such as:

a)  This is where we've been
b)  This is what we attempted to achieve during this funding cycle
c)  This is what we actually accomplished, including unanticipated
    breakthroughs and a whole host of projects and initiatives that flowed
    out of our work
d)  These are the problems that we still have--some of which we have the
    capacity to improve on, some of which we don't given current resources
e)  This is where we'd like to go for the next funding cycle

This report could be developed by a representative team from both the
field and the state office with consultative support from a research
institute.  The information would flow back to the programs as well as
upward to the federal government and would be a format to stimulate
discussion, analysis, and program development as well as "data" for
national accounting.

I could go on here, but you get the point.

Improbably you say.  Perhaps so given the current official mentality
grounded in the assumptions of standardization, quantification, and
"objectivity," behind all of which is a quest  for control--control of
the system, of the information, and of the lives of the students to be
channeled within certain realms of behavior--get a job, get off welfare,
stay out of jail, vote, read to your kids.  As long as adult literacy is
viewed as a subset of current social policy linked to the maintenance of
the global economy, the more narrow view of the field as implied by the
WIA/NRS will prevail.  A broader and humane view would move in a
direction perhaps in a manner like I am suggesting or in some similar
manner that captures the qualitative dimensions of what our field is
about.

To assume that my suggestion is "subjective" as opposed to the
"objectivity" provided by a quantitative, measurable, and standardized
format that drives the assumptions of the NRS is to assume that numbers
accurately depict reality rather than interpret it.  I do not make that
assumption."

Nancy Hansen, from Sioux Falls, South Dakota, replying to Tom Sticht:

"Tom:

In regards to: "If funding was based on the accountability reflected in
the praise adult learners have for their teachers and programs, the Adult
Education and Literacy System (AELS) in the United States would not be in
the obscenely under funded position it is presently in."

Then why in the world are we not identifying tools that will show what
those "praises" are??  It's obvious to *me* that the NRS is *not* going to
do that -- unless *you* can document "the praise" aligning with the
questions in the NRS.

And Yes!  It *is* obscene that literacy efforts are so "obscenely under
funded" presently.  And it will get even more *obscene* when the tasks are
impossible for under funded/under staffed literacy programs to provide
these unrealistic accountability documents using unreasonable
documentation tools!"


Replying to Nancy, Art LaChance, from Ellijay, Georgia,  said:

"Could it be that IF we did begin to document learners' praise then we
would also be documenting the negative side and the finger of
responsibility may begin to point backwards to the K-12 system that is
failing so miserably.

How better to escape that than to overwhelm the AELS with insurmountable 
and unreasonable documentation requirements that sidestep these issues?"


Catherine King, replying to Art LaChance:

"Colleagues:

Art says:  "Could it be that IF we did begin to document 
learners' praise then we would also be documenting the 
negative side and the finger of responsibility may begin 
to point backwards to the K-12 system that is failing so 
miserably."

Art's note also points to the naming question.  It seems to
me, though there is some legitimacy to understanding our
field as changing into a more comprehensive notion, it
remains that much of what we do ***is*** remedial.  In our
culture, we do intend to prepare people from very early on
to read, write, express themselves, perform basic math, 
to be healthy, psychologically sound, and to be somewhat 
socialized and educated in the broad sense, especially now
as we become more "global," so that our "pursuit of 
happiness" is not shackled by our own horizonal limitations.  

Do we not deal with adults all of the time who have not met 
these intentions or been prepared in this way?   If this is 
true, aren't we being rather sophistic in our efforts to 
get away from this "negative image"?

Also, did anyone see the PBS program about teaching
religious studies in our secular schools, and the
relationship of religion and government mandates and
statues?  This program is the only thing I have seen on
it, but instead of focusing on what schools can't do,
the swing apparently has been going on for some time to
focus methodically on what the schools **can** do, and
provide curriculum accordingly--ending up with teaching
about other religions besides Christianity, and about
including the integral place religion has played in our
historical development.  

If this is true, it plays into our prior discussion on
EFF and the broader definition of adult education as
inclusive of the spiritual questions that all people
live in?"

====================================================