[NLA] Research in adult literacy education

[Debbie Yoho]

Thank you, Nancy, for your real-world, painstaking summary of this thread. 
In regard to policy, I wish someone could sort out the quagmire of
balancing local control of our programs with fiats from on-high.  To me,
this is the Great Theme of what has been called the Troubled
Crusade--America's educational system-- at all levels.  Every classroom
teacher from kindergarten to higher ed struggles with this problem.  It
wasn't alway so--education was once solely in the hands of the grass-roots
practitioner, but justice was served unevenly, as inequities abounded. 
Despite the relatively recent involvement (interference?) by state and
national government to rectify the situation, it seems the more things
change, the more they stay the same. 

We have come full circle.  
   
Deborah W. Yoho
Co-moderator, NIFL-Health Listserv
President, SC Adult Literacy Educators
Executive Director, Greater Columbia Literacy Council
2728 Devine Street,  Columbia, SC  29205
803-765-2555   Fax  803-779-8417   dwyoho at earthlink.net


> [Original Message]
> From: <Nashansen at aol.com>
> To: <nla at lists.literacytent.org>
> Date: 11/17/2002 10:21:36 PM
> Subject: Re: [NLA] Research in adult literacy education
>
> Dear Andrea W, Sheryl G and Debbie Y,
>
> I'm going to attempt to reply to six emails posted on the NLA - by Andrea
on=
> =20
> 11/15/2002 at 11:32 am CDT and later in the day by Sheryl and Deb.  So
littl=
> e=20
> time, so much to say - THIS BECAME LONG. =20
>
> And yes, David Rosen.  After this post, I intend to "return to the NLA
list=20
> purpose."  My next project is to type a letter to fax tomorrow to my
two=20
> Senators.  Can't seem to "get away" from this thing called Work.
>
> First of all, I would like to quote another NLA subscriber (who wrote
the=20
> following response earlier last week) in response to a post by
statistician=20
> Tom Sticht:
>
> <<  =E2=80=A6 A.  Interactions between researchers and practitioners:
> I feel that both can learn from each other.  Unfortunately we do not do=20
> enough of this.  In my experience, I have noticed that neither group
wants t=
> o=20
> really listen to each other.  I place the onus of the burden on
researchers=20
> to change this.  We need to listen to practitioners, respect their=20
> experience, their views, their knowledge.  When that day arrives, I
think=20
> that we will be able to move the field forward with more efficiency.     
=20
> Daphne Greenberg " >>
>
> Daphne also wrote other important points.  She brought home for me how=20
> different our plateaus of knowledge and feelings are as various writers
of=20
> NLA posts.  (Those who want to read the complete version should look in
the=20
> archives for "Response to Tom and others" dated 13 Nov 2002 11:00:48.) =20
>
> Thank you very much, Daphne, for making this and the other points in
your=20
> post.  I couldn't agree with you more.  Listening and respect WILL help
to=20
> create a more coordinated and effective field.
>
> So with the above in mind, here are my responses:
>
> Andrea wrote:
> << How was the decision made to begin the NRS reporting system? >>
>
> My response:
> Our state DoL (under which a new AELS director was hired) brought in a
North=
> =20
> Dakota trainer to give NRS system training to South Dakota Adult
Beginning=20
> Education administrators two to three years ago.  I accidentally heard
about=
> =20
> this seminar (due to my partnership with our adult learning center
director)=
> =20
> and invited myself.  The state had already made the decision to
implement=20
> NRS, even though not one literacy director was asked for input.  We were
tol=
> d=20
> by the AELS director the fed's are requiring it. Like It or Lump It.
>
> << Now, I think you said that administrators did not actually use the
system=
> ,=20
> and that resulted in a drop of numbers reported, and hence less funding. 
Is=
> =20
> this correct? >>
>
> Correct -- there are minimal S.D. literacy council administrators using
the=20
> NRS.  Reports are submitted to the state through each ABE program, if
we=20
> apply for funding.  The minimum requirement is TABE testing in these
GED-pre=
> p=20
> programs.  I don't agree to THAT requirement either.
>
> << If what I am saying is not correct, then how is it that your local=20
> literacy=20
> program (is this you?) was stripped of federal funding? >>
>
> Our local program is not receiving the trickle-on-down federal funding
for=20
> FY02-03 unless TABE testing and data for the NRS are included.  We are=20
> ineligible to apply for funding, we were told.  Fine with me.  If they
are=20
> interested enough in my numbers, they will come up with a compromisal=20
> solution.  Besides, there is less red-tape seeking grants anyway.  And=20
> besides that?  Foundations are more generous than the state ever was.
>
> A bit of history:  Prior to the switch to the DoL and the implementation
of=20
> the NRS, about 25 S.D. literacy programs were  accountable to the SD DoE.
=20
> The standards for our submitted reports were the same reported annually
to=20
> our national organization (then called Laubach).  So it's not that we
haven'=
> t=20
> tracked and reported the number of people we serve and what it is that
were=20
> the outcomes.  Also, we've been required to do outcome reporting for
our=20
> United Way funding for three years now.  Neither of these reports are
good=20
> enough for the fed's.
>
> It's more the fact that our program meets other adults' needs than to go
fro=
> m=20
> one grade level or one level book to the other.  The achievements of
life=20
> skills weren't satisfactory to the new AELS.  How could they be?  They
didn'=
> t=20
> fit the NRS.  And they sure can't be measured by TABE testing!
>
> << How is the use of the NRS related to PRIOR inaccuracy in literacy=20
> statistics=20
> as reported to policy makers?  How do you know the figures were
inaccurate?=20=
> =20
> How were the figures calculated? >>
>
> My response:
> I'm going to refer back to Daphne's comments:  I am a practitioner
wanting t=
> o=20
> have my opinions heard and accepted at face value for my knowledge of
what i=
> s=20
> happening in Sioux Falls, SD.  I am not a number-cruncher, nor do I wish
to=20
> be, and will not comment on "prior inaccuracies" when that point was not
my=20
> focus, as far as I was concerned, when I responded to Cecil Smith.=20
>
> How do I know "the figures were inaccurate"?  My comment is about my=20
> immediate area.  I know the numbers of S.D. literacy students who are=20
> currently receiving services are inaccurate because *I* reported my
"numbers=
> =20
> served" each year to the DoE.  For FY01 - FY03 years the state does not
have=
> =20
> total Sioux Falls area "numbers served" nor, since it was a sore point
with=20
> the AELS director, likely does the state have the majority of literacy=20
> learners' numbers from the other 20-something councils in the state. =20
>
> Figures calculated: # of learners served during the July 1st through
June=20
> 31st reporting period.
>
> << Now about documents quoted on the NLA.  Which are you talking about
here?=
> =20
> >>
>
> My response:
> So, by reading the above, do YOU feel the true picture of literacy need
and=20
> service numbers in S. D. has been accurately portrayed in either federal
or=20
> statisticians' reporting taking into account the above scenario?  I
don't=20
> know how it COULD be.  AND I do NOT think that ours is the only state
that i=
> s=20
> "short" literacy student numbers, which apparently causes a need to
question=
> ,=20
> as did T. Sticht not that long ago:   "Where did the adult learners GO?"
>
> The "documents" I referred to were called "Papers" and have been
published=20
> on-line by several different statisticians/NLA subscribers. When I read
them=
> ,=20
> I didn't find the documents included "where this statistical data came
from"=
> =20
> as part of the paper.  It has been said that a good researcher puts that
in=20
> their final report.  I believe the point, but that's not true of several
of=20
> the papers that *I* read on the NLA.  Maybe I missed that documentation.
>
> Andrea continued:
> <<  =E2=80=A6. Now to educational=20
> content, where I think you also have some feelings. >>
>
> << I think your concern here is that the ongoing program of research=20
> supported=20
> by your organization will be pushed aside by new federal requirements=20
> dictating new modes of teaching.  Is this right?  >>
>
> << I am not sure exactly how this would happen, and if there would be=20
> considerable bargaining around teaching issues.  I should think that an=20
> ongoing program of research, supported by the report of outside
evaluators,=20
> which Marsha Tait spoke about in a spring post would increase the
credibilit=
> y=20
> of your teaching methods and outcomes.  Could be wrong, though. >>
>
> << Overall, you feel that volunteer literacy programs have lost funding=20
> support=20
> and now may be forced to change teaching methodology? >>
>
>
> My response:
> Not just FEEL, but also KNOW the literacy programs have lost funding
support=
> !=20
>  And in at least one other state as well.  Yes, I believe federally
required=
> =20
> methodology using what the feds deem "more effective" could occur, taking
th=
> e=20
> word of well-meaning researchers writing papers with data gathered from
a=20
> sampling.  If it isn't true, why is the latest NLA focus on "finding the
way=
> =20
> adults learn to read" so important a thread on this policy listserv? =20
>
> I never negated program evaluation - provided knowledgeable evaluators do
th=
> e=20
> evals.  But please don't send the same people who designed the likes of=20
> assessment tools like TABE to evaluate literacy programs, and
subsequently=20
> tell practitioners what works best for literacy students when
practitioners=20
> at least APPEAR to know more about What Works!
>
> If the evaluation were done within the structure of the ProLiteracy
Worldwid=
> e=20
> environment, which Marsha Tait and Peter Waite represent, I could
support=20
> that.  I would also favor the exploration of supplemental program=20
> development.   Just don't tell me, federal policy-makers, "Sorry you
can't=20
> get that set of books anymore because researchers have deemed the
techniques=
> =20
> ineffective." - especially when I *know* better!
>
> As a whole, the program administrators I've had contact with are
satisfied=20
> with the various techniques they use to teach adult learners.  Not all,
mind=
> =20
> you, but many.  I do realize that the volunteer-based programs have to
use=20
> the materials they can afford to purchase within tight budgets.  If one=20
> particular multi-sensory approach doesn't work, we try another, but
within=20
> the program we found to be effective.  So should a research project
include=20=
> a=20
> supposition that the money will BE there to purchase any new tools and=20
> provide free training to all personnel, maybe THEN it would be an option.
=20
> Not until then for many of us.
>
> Being told that I *have* to buy specific materials/technologies and pay
for=20
> staff training to utilize Their Program with our volunteers is not
acceptabl=
> e=20
> to me personally and has the potential of happening.  I feel our local=20
> council already has an effective, volunteer-friendly program for 1-to-1=20
> instruction. =20
>
> We've been asked by some locally if our program should consider a
different=20
> delivery system because 1-to-1 would be too expensive, should we have to
pay=
> =20
> staff.  I won't agree to THAT either. Input from learners says 1-to-1 is
the=
> =20
> best part of our program.  Our current tutor base of volunteers is=20
> comfortable with 1-to-1 assignments rather than teaching a group. The
fed's=20
> have already proven they "don't put their money where their mouth is"=20
> regarding financing such change-orders. Personally I'm not looking for a
new=
> =20
> way to invent the wheel, thank you very much.
>
> If the policy-makers REEEEEAALLY want to help practitioners, they would=20
> research and design a more effective assessment tool to be used with
literac=
> y=20
> level students who don't want to pursue a GED - a tool that can be used
by=20
> local program administrators to determine level of skills and their
learners=
> '=20
> needs, yet not mean an expensive psychological evaluation.  Those of us
with=
> =20
> minimal budgets canNOT cover such a costly individual evaluative expense
wit=
> h=20
> every registrant.  =20
>
>
> Later, on 11/15/02, Sheryl Gowen, Associate Professor of Research,=20
> Measurement and Statistics, Department of Educational Policy Studies at=20
> Georgia State University added:
>
> << (Andrea) ask(ed) several important questions and make(s) one
essential=20
> observation:  Literacy programs need an "ongoing program of research,=20
> supported by the report of outside evaluators" in order to secure and=20
> maintain funding levels.  Like it or not, the need for systematic
program=20
> evaluation is essential in the current economic and political climate. 
It i=
> s=20
> possible to find evaluators that are pro-active and supportive of the=20
> program's goals and objectives.  I would encourage program directors to
seek=
> =20
> those evaluators out. >>
>
> My response is:
> Like I said - I see it coming.  Expert evaluators come in with their
New=20
> Ideas for "meeting goals and objectives" whether they match the local=20
> program's goals and objectives and programming or NOT.  Assessment
experts=20
> aren't realistic about what it IS the adult literacy learner is learning.
=20
> Incorrect assumptions might occur - literacy programs have nothing of
value,=
> =20
> show no growth, do not show grade level increases  - should they come in
and=
> =20
> evaluate using the current available evaluation tools.
>
> Andrea wrote back at 2:00 pm Friday:
> << =E2=80=A6. an excellent point and a very good strategy.  I should
think t=
> hat=20
> programs could band together to do this--ongoing evaluation with
outside=20
> evluators.  This might result in more funding, and potentially it would
open=
> =20
> up legitimation of a variety of teaching techniques.  I like the idea. >>
>
> Hmm-mm, maybe banding together would work fine in a heavily populated
area=20
> like Boston and Cambridge, but there is little potential for that where
mile=
> s=20
> of prairie stretch between "metropolitan" areas.  In the Midwest the=20
> landscape looks considerably different than it does where you live. =20
>
> In S.D. banding together two metropolitan areas would mean bringing
together=
> =20
> two cities that are 400 miles apart - if banding toether the two
largest=20
> cities in the state should be the goal.  If you count the smaller
communitie=
> s=20
> between, the closest one - where MAYbe literacy students are being served
-=20
> is 1 hour's drive to the West.  Yep!  Going to just jump in the car
every=20
> evaluation period and drive back and forth for evaluation.=20
>
> Debbie Yoho, Executive Director, Greater Columbia Literacy Council,
responde=
> d=20
> at 2:46 saying:
>
> << From Sheryl:  "It is possible to find evaluators that are pro-active
and
> supportive of the program's goals and objectives. I would encourage
program
> directors to seek those evaluators out." >>
>
> << Really?  Now this is interesting.  I'm wondering if those NLA
commentator=
> s
> who believe strongly in "rigorous" research would insist on evaluators who
> specifically know NOTHING about a given program's goals in order to be
> objective?  This raises the question of WHO should be minding the store.
>>
>
> My comment:
> When I read Sheryl's comment, Debbie?  My very first reaction, before I
foun=
> d=20
> your follow-up email post, was "So people don't think local program
personne=
> l=20
> is capable of evaluating their own programs objectively." -- an "I'm
the=20
> expert" stance, as far as I was concerned.  Nobody is listening again!?!
=20
> There are some really fine people "minding the store" in the grassroots=20
> programs who already DO evaluate the effectiveness of their programs
with=20
> individual learners and volunteers plus staff.  We, like you, are
required t=
> o=20
> do annual evaluation - ours is done in order to pursue United Way
allocation=
> s=20
> for the upcoming year.  I agree with the post, Deb, that states
learners=20
> should be a part of the evaluation process.  We do that here.=20
>
> Response from Andrea came that night at 9:43:08 PM CST:
> << This is a day for teasing out word meanings.>>
>
> My response:
> Therein is the problem, Andrea.  What one person gives as an opinion is=20
> heard/read differently than it is by another from a different level of=20
> understanding of literacy services, program opportunities and
individual=20
> needs.
>
> You wrote that you "didn't take Sheryl's post to mean that evaluators
should=
> =20
> know nothing about a program's goals, objectives or methods."  The
double=20
> jeopardy being played at the crap table is that evaluators WOULD "know
plent=
> y=20
> about methods and goals and (have) wide experience" - be "rigorous".  In
my=20
> view, this would HAVE to be part of the criteria to gain accurate=20
> information.   I am not altogether sure that I'm that big a gambler to
take=20=
> a=20
> chance on an evaluator "from the outside"! =20
>
> Andrea, I enjoyed your story about a friend with a diagnosed LD.  You
said,=20
> and I quote, "Evaluators would have to know about language and about
learnin=
> g=20
> disabilities and diagnosis."  And you know something?  When I read your=20
> story, it occurred to me that the expert evaluators better also have to
know=
> =20
> about the differences between adults and teenagers and how their life
skills=
> =20
> and personal needs come into play in the educational environment. =20
>
> Andrea, you expressed a strong message about serving our learners in=20
> "multiple rigorous ways" to provide, what you called, "becoming
literate". =20
> Debbie (or any other lurking practitioner here) tell me if I'm wrong, but
I=20
> feel that literacy programs are using multiple instructional techniques
(I=20
> also have a problem with those techniques being "rigorous") that allow us
to=
> =20
> be flexible and effective in providing not just reading and writing
skills,=20
> but increasing learner self-confidence and gaining a measure of
independence=
> =20
> -- *in addition to "becoming literate".=20
>
> I would poor pity the literacy program that wasted an adult's precious
time=20
> using the wrong method.  (In reference to Andrea's comment about "Given
how=20
> little time adult learners have to spend in class, I think it would be
great=
> =20
> to use the time as effectively as possible".)  You realize, I hope, that
our=
> =20
> New Readers are volunteers, too?  They LEAVE if the program is wasting
their=
> =20
> time!!  I can give testimony to that because when I first began as this=20
> program's director, I did -- see them leave, I mean.
>
> In my local program, "upfront goal(s)" are a part of the New Reader's=20
> registration and assessment appointment with me.   If they can't name
what i=
> t=20
> is they want to learn, for what purpose and what commitment they are
willing=
> =20
> to make to it, they aren't ready for a tutor assignment.  Each adult is=20
> experiencing "life's challenges" at the point they finally make the=20
> appointment.   They have pondered these points because I screen them on
the=20
> phone when they call.  They know ahead of time what we will discuss.
>
> Friday afternoon I met with a 35-year, retired and "knowledgeable
teacher"=20
> regarding the phonetic program she wrote and used with her fourth grade=20
> students before retirement.  I tried to recruit her, but she said:  "You
are=
> =20
> dong good work here, Nancy.  I wouldn't have the patience to work with
adult=
> s=20
> like you are."   She was seeing the "differences" between teaching adults
an=
> d=20
> the kids she'd had in her 4th grade classes.  She turned me down.  So=20
> "knowledgeable teacher" may not be the only criteria expected -
'empathetic'=
> =20
> volunteers should also be a part of the critical factor for success in
areas=
> =20
> where 1-to-1 programming is offered.
>
> Regarding the comment about:  Literacy programs should have "constant=20
> teaching upgrades, evaluators who are also coaches, and funding" -- Hate
to=20
> "burst your bubble", but in the "real world", where the political
climate=20
> exists as referred to by Sheryl, there is not enough money at the
federal=20
> level to carry your dream-plan to fruition nationwide.  Won't happen.
>
> Thanks for taking the time to write.  I'm like Daphne - swamped - yet I=20
> believe in Doing Right by the adult learners whom I serve.  Sorry about
the=20
> length of this response.
>
> Respectfully submitted,
> Nancy Hansen
> Executive Director
> Sioux Falls Area Literacy Council
> Sioux Falls, SD
> <sfallsliteracy at yahoo.com>
> and=20
> <Nashansen at aol.com>
> _______________________________________________
> NLA mailing list: NLA at lists.literacytent.org
> http://lists.literacytent.org/mailman/listinfo/nla
> LiteracyTent: web hosting, news, community and goodies for literacy
> http://literacytent.org



_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org