[NLA] support for advocacy

[Bickerton, Robert P]

 Tom, et al,

There are few adult educators across our fifty states who pay as much
attention as I do to your (Tom Sticht's) analysis of issues that confront
our field.  Rarely do I find myself coming to a different conclusion.  Such
is the case, however, with at least a couple of points you raised in your
3/5/02 posting.  Following are my thoughts about a couple of the reports you
cite and some different conclusions about their place in advocacy.

The controversy about the 80% cut point used by the NALS has been with us
for some time.  If I recall, you were one of a number of researchers who
cautioned USDOE and the field to use the more widely accepted 60% cut point
used by the NAEP (National Assessment of Educational Progress -- 4th, 8th
and 12th? grade tests) and some other similar instruments.  Kolstad's
arguement that it should have been set at 50% pushes this arguement even
further from the cut points that were actually used.  Both you and Kolstad
are better trained and prepared than I to engage these issues -- however,
I'm sure you won't be surprised that I'm willing to jump into the fray
regardless.  I understand and have followed the arguements about false
positives, but will submit once again an arguement I made almost a decade
ago as the first round of this furor raged (by the way, there's no
comparable "furor" or "rage" this time around -- it's more like a tempest in
a teapot):  cut scores are NOT arrived at solely as a matter of resolving
technical measurement issues -- they are also driven by the context(s) of
the claims that one wishes to make about the results of an assessment AND
these include no small measure of philosophical and political judgements.
In a simplified view of these issues, it's easy to understand how the cut
scores for "proficient" vs. "mastery" would be different.  With large scale
literacy assessments, cut scores are influenced by whether we want to know
if someone's knowledge, skills and abilities are "adequate" for some purpose
or, to the hackneyed phrase/"standard," "world class."  My point is, there
are many who could and have argued that the 80% cut point didn't reflect
false positives, it reflected what we should/need to be expecting as a level
of performance in today's increasingly demanding environments -- at the
workplace, at home, making judgements about public policy issues in our
communities (e.g., environmental), et al.  By the way, I'm not supporting
the 80% cut point, I'm simply pointing out that it can be argued from many
points of view.  

Now to what feels like firmer ground for me.  
>From a public policy perspective, I am more interested in arriving at a
consensus among the adult education/literacy, research, measurement and
other interested/invested communities about what the cut point for the new
NAAL should be than I am in any particular percentage.  The most destructive
dynamic surrounding NALS data was not whether the "actual" (estimated) level
of need for our services was 90 million, 60 million, or even half that
number.  It was that in making the case -- in good faith -- that the current
service delivery system is inadequately supported in terms of BOTH the
adequacy of quality and quantity of services, that this controversy about
the cut points would be inserted and momentum for a response would be
deflated.  My experience is that the thoughtful critique of really good
people was coopted by those antagonistic to our field and people we seek to
serve.  I am hopeful that we can do a better job of "singing with one voice"
the next time around -- we'll get a hell of a lot further to the extent that
we can and do.

Now on to your reference to what happened in Massachusetts.  I can
understand how you and others may have come to the conclusions summarized in
your post, but it's simply just not correct.  The fact is that the MassInc
Report has been extraordinarily well received across virtually all policy
and elected leaders as well as business and labor leaders in our state.  The
fact is that even in the depths of the budget crisis in Massachusetts, a
substantial majority of state representatives and senators were solid
supporters of ABE/literacy.  Yes, this reality has been the result of many
years of skillful and tenacious advocacy -- but the MassInc Report has
demonstrably added momentum.  So, if all of this is true, how did we end up
confronting a cut of almost 50% and the dismantling of what we have been
building over the past decade?  The last minute cut to ABE and a few other
programs came from at most, one or two legislative leaders who made a
political calculation that there was more to gain by shifting this funding
to another more popular program than there was to lose from the
invisible/unempowered/non-constituencies" served by those programs.  Several
legislators who support ABE/literacy in our state warned them that this was
a miscalculation, however, the vote on the ENTIRE state budget was scheduled
for less than 24 hours from the time these cuts were announced.  No
amendments were allowed -- the budget, already 5 months late, had to be
voted up or down.  Still, the vote was delayed for several hours based on
resistance by ABE/literacy's supporters and went forward only after
assurances were received that it could (and would) be fixed.  Yes, an
extraordinarily powerful advocacy movement then ensured the turn-around --
to the point that ABE/literacy received one of the smallest cuts of all
state programs -- but the MassInc Report was frequently cited and used by a
whole host of constituencies along the way.  In fact, an extraordinary press
conference in which our state's top business and labor leaders stood side by
side and stated that while they didn't agree on many things, they were
united in their position that this cut was dumber than dirt (my poetic
license added) and had to be reversed.  

Now, the fact that the analysis used by the MassInc report was conducted and
signed off on by Harvard (NCSALL) and Northeastern (Center for Labor Market
Studies) Universities has a lot to do with its acceptance.  As I alluded to
earlier, I've been making the best case I can using the best (often only)
data available to me for decades, but whether the data I cite is on firmer
ground or not is irrelevant, I'm seen as a self-interested adult educator
and a bureaucrat to boot!  The bottom line -- what gains momentum will
almost always have more to do with perception and good marketing than with
technical merit -- although a good dose of the latter can be very helpful.  

I am very committed to finding common ground in how the NAAL can and should
be constructed, analyzed and used.  I hope we can get far more of us on the
same page this time around.

take care -- and hope we bump into each other sometime soon,
bob bickerton, MA director of adult ed

-----Original Message-----
From: Thomas Sticht
To: nla at lists.literacytent.org
Cc: tsticht at aznet.net
Sent: 3/5/02 2:16 PM
Subject: [NLA] support for advocacy

What a difference a couple of year makes. Here are a couple of recent
items that have made me scratch my head in puzzlement. I wonder if they 
affect other NLA list members in a similar manner. Tom Sticht

Item 1: Determining the scale of need for adult education and literacy
development. 

In 2001. In a research note in June of last year I noted that in January
2001, a final technical report on the National Adult Literacy Survey was
published by NCES entitled: Technical Report and Data File User's Manual
for the 1992 National Adult Literacy Survey, NCES 2001-457, January
2001. I reported that Chapter 14, written by Andrew Kolstad, the
original project director for the NALS at NCES,  systematically
undermined the entire test and questioned its construct validity, that
is, the question of just what it is that the test measures, its
standards validity, that is,  the validity of the 80 percent probability
standard that was used to assign people to the five literacy levels, and
the use validity, that is, the validity of the NALS for accurately
identifying adults at risk for poor literacy skills. In a July 17, 2001
article in the Washington Post Kolstad confirmed his thinking about the
standards validity position and argued that instead of the 80 percent
response probability, a much higher level of validity in making
inferences about adult's literacy skills was made if a response
probability of .50 was used. In short, one of the reasons why adults who
were classified as poorly literate and placed in levels 1 or 2 of the
NALS when they thought they were fairly literate was that they were
likely classified incorrectly by the use of the 80 percent RP standard. 

In 2002. Fast forward to this year. In a recent posting Debby D'Amico
points to a new research report from the National Center for Adult
Learning and Literacy, called Building a Level Playing Field, that uses
what according to Kolstad are the old,  invalid NALS data, as a large
part of the basis used to establish the scale of need for adult basic
skills education in the U. S. I didn't find anything in the report that
countered Kolstad's data questioning the validity of the NALS from the
final technical report. This suggests to me that using the NCSALL report
as a source for establishing the scale of need for advocacy could be
misleading. The authors go on to say that the new National Assessment of
Adult Literacy (NAAL) will use much the same methodology that was used
in the NALS so that ten year trends can be determined. I wonder if this
means that the NAAL will overcome the issues of validity raised by
Kolstad regarding the NALS? I also wonder if anyone cares about data on
the scale of need or how accurate it is. I seem to recall that in
Massachusetts despite the considerable publicity in the media after the
MassInc report on New Skills for a New Economy came out, using
essentially the same methodology for determining the scale of need for
adult education and literacy development in Massachusetts as the recent
report on Leveling the Playing Field, the Massachusetts legislature cut
the adult education budget by 40 percent or so anyhow. Skillful advocacy
got most of the money back but I don't know how much this depended on
the scale of need data from the MassInc report. 

Item 2. The push to raise the education bar from high school to
post-secondary education. 

In 2000. In a post on the NLA list I cited the Thursday Notes for March
16,2000, an electronic newsletter attributed as From the Desk of Ron
Pugsley, then Director of the U. S. Department of Education's Division
of Adult Education and Literacy which said QUOTE "Jobs requiring an
associate's degree or higher account for... just 16% of actual job
openings, ... Meanwhile, nearly one third of 25-29 year-olds attain
bachelor's or associate's degrees and will compete for those jobs."
UNQUOTE At the time I mentioned that this suggests a "surplus" of
post-secondary educated adults over the jobs requiring this much
education.

In 2001. In a research note last year I mentioned a new report by
Carnevale and Desrochers (2001) called Help Wanted...Credentials Required
which presented data showing that  from 1973 to 1998 the percentage of
prime-age workers (both male and female) with some college/AA degree
more than doubled,  from 12 percent to 27 percent.  But from 1979 to
1998 earnings (in constant 1996 dollars) of prime-age male workers
(30-59 years old) declined for those who failed to finish high school,
those with a high school diploma, AND those with some college or an AA
degree. These data lead to questions as to the likely outcomes of
educating more and more people to higher and higher levels of education.
Was it possible that as more and more people acquire higher education
credentials and higher literacy skills wages will fall because employers
have a larger pool of better qualified workers from which to recruit? 
Is that what happened from 1979 to 1998 as the percentages of men with
some college or AA degrees increased, and wages for these men decreased
(in constant 1996 dollars)? 

In 2002. Fast forward to this year. In the report called Building a
Level Playing Field  the authors argue that the new economy calls for
educating more and more adults up to post-secondary levels. A similar
call is made in the report entitled Adult Literacy & the American Dream
by Forrest Chisman from the Council for Advancement of Adult Literacy
(CAAL) (this report was recently reviewed by George Demetrion on the NLA
list). Neither report addresses the issues raised by Pugsley's 2000
newsletter about the supply of adults with post-secondary and bachelor's
degrees far exceeding the availability of jobs requiring this much
education, or the work of Carnevale and Desrochers suggesting that if
more and more adults are educated to post-secondary levels, wages are
likely to drop for these adults (at least for men; women with
post-secondary education did not show the same drop in wages as for men
in the Carnevale & Desrochers report, but women earned less than men in
all categories of education). 

In short, these recent calls for raising the education standard for
adults from a high school diploma to some post-secondary education would
seem to set up a condition that once again points to the possibility of
driving up educational achievement such that the supply of adults with
post-secondary education exceeds the demand to the point where wages for
adults with post-secondary education are driven down even more than in
the report by Carnevale and Desrochers.

All these various research studies seem to set up conflicting and
sometimes contradictory messages about the scale of need for adult
education and literacy development in the United States. What is an
advocate to do? Can anything be done to provide advocates with accurate,
valid, consistent, and convincing information about  how many adults in
the U. S. need adult education and literacy development, and how many
want it? Does anyone care?
_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org
_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org