[NLA] Research in adult literacy education

[Nashansen@aol.com]

Dear Andrea W, Sheryl G and Debbie Y,

I'm going to attempt to reply to six emails posted on the NLA - by Andrea on 
11/15/2002 at 11:32 am CDT and later in the day by Sheryl and Deb.  So little 
time, so much to say - THIS BECAME LONG.  

And yes, David Rosen.  After this post, I intend to "return to the NLA list 
purpose."  My next project is to type a letter to fax tomorrow to my two 
Senators.  Can't seem to "get away" from this thing called Work.

First of all, I would like to quote another NLA subscriber (who wrote the 
following response earlier last week) in response to a post by statistician 
Tom Sticht:

<<  … A.  Interactions between researchers and practitioners:
I feel that both can learn from each other.  Unfortunately we do not do 
enough of this.  In my experience, I have noticed that neither group wants to 
really listen to each other.  I place the onus of the burden on researchers 
to change this.  We need to listen to practitioners, respect their 
experience, their views, their knowledge.  When that day arrives, I think 
that we will be able to move the field forward with more efficiency.        
Daphne Greenberg " >>

Daphne also wrote other important points.  She brought home for me how 
different our plateaus of knowledge and feelings are as various writers of 
NLA posts.  (Those who want to read the complete version should look in the 
archives for "Response to Tom and others" dated 13 Nov 2002 11:00:48.)  

Thank you very much, Daphne, for making this and the other points in your 
post.  I couldn't agree with you more.  Listening and respect WILL help to 
create a more coordinated and effective field.

So with the above in mind, here are my responses:

Andrea wrote:
<< How was the decision made to begin the NRS reporting system? >>

My response:
Our state DoL (under which a new AELS director was hired) brought in a North 
Dakota trainer to give NRS system training to South Dakota Adult Beginning 
Education administrators two to three years ago.  I accidentally heard about 
this seminar (due to my partnership with our adult learning center director) 
and invited myself.  The state had already made the decision to implement 
NRS, even though not one literacy director was asked for input.  We were told 
by the AELS director the fed's are requiring it. Like It or Lump It.

<< Now, I think you said that administrators did not actually use the system, 
and that resulted in a drop of numbers reported, and hence less funding.  Is 
this correct? >>

Correct -- there are minimal S.D. literacy council administrators using the 
NRS.  Reports are submitted to the state through each ABE program, if we 
apply for funding.  The minimum requirement is TABE testing in these GED-prep 
programs.  I don't agree to THAT requirement either.

<< If what I am saying is not correct, then how is it that your local 
literacy 
program (is this you?) was stripped of federal funding? >>

Our local program is not receiving the trickle-on-down federal funding for 
FY02-03 unless TABE testing and data for the NRS are included.  We are 
ineligible to apply for funding, we were told.  Fine with me.  If they are 
interested enough in my numbers, they will come up with a compromisal 
solution.  Besides, there is less red-tape seeking grants anyway.  And 
besides that?  Foundations are more generous than the state ever was.

A bit of history:  Prior to the switch to the DoL and the implementation of 
the NRS, about 25 S.D. literacy programs were  accountable to the SD DoE.  
The standards for our submitted reports were the same reported annually to 
our national organization (then called Laubach).  So it's not that we haven't 
tracked and reported the number of people we serve and what it is that were 
the outcomes.  Also, we've been required to do outcome reporting for our 
United Way funding for three years now.  Neither of these reports are good 
enough for the fed's.

It's more the fact that our program meets other adults' needs than to go from 
one grade level or one level book to the other.  The achievements of life 
skills weren't satisfactory to the new AELS.  How could they be?  They didn't 
fit the NRS.  And they sure can't be measured by TABE testing!

<< How is the use of the NRS related to PRIOR inaccuracy in literacy 
statistics 
as reported to policy makers?  How do you know the figures were inaccurate?  
How were the figures calculated? >>

My response:
I'm going to refer back to Daphne's comments:  I am a practitioner wanting to 
have my opinions heard and accepted at face value for my knowledge of what is 
happening in Sioux Falls, SD.  I am not a number-cruncher, nor do I wish to 
be, and will not comment on "prior inaccuracies" when that point was not my 
focus, as far as I was concerned, when I responded to Cecil Smith. 

How do I know "the figures were inaccurate"?  My comment is about my 
immediate area.  I know the numbers of S.D. literacy students who are 
currently receiving services are inaccurate because *I* reported my "numbers 
served" each year to the DoE.  For FY01 - FY03 years the state does not have 
total Sioux Falls area "numbers served" nor, since it was a sore point with 
the AELS director, likely does the state have the majority of literacy 
learners' numbers from the other 20-something councils in the state.  

Figures calculated: # of learners served during the July 1st through June 
31st reporting period.

<< Now about documents quoted on the NLA.  Which are you talking about here? 
>>

My response:
So, by reading the above, do YOU feel the true picture of literacy need and 
service numbers in S. D. has been accurately portrayed in either federal or 
statisticians' reporting taking into account the above scenario?  I don't 
know how it COULD be.  AND I do NOT think that ours is the only state that is 
"short" literacy student numbers, which apparently causes a need to question, 
as did T. Sticht not that long ago:   "Where did the adult learners GO?"

The "documents" I referred to were called "Papers" and have been published 
on-line by several different statisticians/NLA subscribers. When I read them, 
I didn't find the documents included "where this statistical data came from" 
as part of the paper.  It has been said that a good researcher puts that in 
their final report.  I believe the point, but that's not true of several of 
the papers that *I* read on the NLA.  Maybe I missed that documentation.

Andrea continued:
<<  …. Now to educational 
content, where I think you also have some feelings. >>

<< I think your concern here is that the ongoing program of research 
supported 
by your organization will be pushed aside by new federal requirements 
dictating new modes of teaching.  Is this right?  >>

<< I am not sure exactly how this would happen, and if there would be 
considerable bargaining around teaching issues.  I should think that an 
ongoing program of research, supported by the report of outside evaluators, 
which Marsha Tait spoke about in a spring post would increase the credibility 
of your teaching methods and outcomes.  Could be wrong, though. >>

<< Overall, you feel that volunteer literacy programs have lost funding 
support 
and now may be forced to change teaching methodology? >>


My response:
Not just FEEL, but also KNOW the literacy programs have lost funding support! 
 And in at least one other state as well.  Yes, I believe federally required 
methodology using what the feds deem "more effective" could occur, taking the 
word of well-meaning researchers writing papers with data gathered from a 
sampling.  If it isn't true, why is the latest NLA focus on "finding the way 
adults learn to read" so important a thread on this policy listserv?  

I never negated program evaluation - provided knowledgeable evaluators do the 
evals.  But please don't send the same people who designed the likes of 
assessment tools like TABE to evaluate literacy programs, and subsequently 
tell practitioners what works best for literacy students when practitioners 
at least APPEAR to know more about What Works!

If the evaluation were done within the structure of the ProLiteracy Worldwide 
environment, which Marsha Tait and Peter Waite represent, I could support 
that.  I would also favor the exploration of supplemental program 
development.   Just don't tell me, federal policy-makers, "Sorry you can't 
get that set of books anymore because researchers have deemed the techniques 
ineffective." - especially when I *know* better!

As a whole, the program administrators I've had contact with are satisfied 
with the various techniques they use to teach adult learners.  Not all, mind 
you, but many.  I do realize that the volunteer-based programs have to use 
the materials they can afford to purchase within tight budgets.  If one 
particular multi-sensory approach doesn't work, we try another, but within 
the program we found to be effective.  So should a research project include a 
supposition that the money will BE there to purchase any new tools and 
provide free training to all personnel, maybe THEN it would be an option.  
Not until then for many of us.

Being told that I *have* to buy specific materials/technologies and pay for 
staff training to utilize Their Program with our volunteers is not acceptable 
to me personally and has the potential of happening.  I feel our local 
council already has an effective, volunteer-friendly program for 1-to-1 
instruction.  

We've been asked by some locally if our program should consider a different 
delivery system because 1-to-1 would be too expensive, should we have to pay 
staff.  I won't agree to THAT either. Input from learners says 1-to-1 is the 
best part of our program.  Our current tutor base of volunteers is 
comfortable with 1-to-1 assignments rather than teaching a group. The fed's 
have already proven they "don't put their money where their mouth is" 
regarding financing such change-orders. Personally I'm not looking for a new 
way to invent the wheel, thank you very much.

If the policy-makers REEEEEAALLY want to help practitioners, they would 
research and design a more effective assessment tool to be used with literacy 
level students who don't want to pursue a GED - a tool that can be used by 
local program administrators to determine level of skills and their learners' 
needs, yet not mean an expensive psychological evaluation.  Those of us with 
minimal budgets canNOT cover such a costly individual evaluative expense with 
every registrant.   


Later, on 11/15/02, Sheryl Gowen, Associate Professor of Research, 
Measurement and Statistics, Department of Educational Policy Studies at 
Georgia State University added:

<< (Andrea) ask(ed) several important questions and make(s) one essential 
observation:  Literacy programs need an "ongoing program of research, 
supported by the report of outside evaluators" in order to secure and 
maintain funding levels.  Like it or not, the need for systematic program 
evaluation is essential in the current economic and political climate.  It is 
possible to find evaluators that are pro-active and supportive of the 
program's goals and objectives.  I would encourage program directors to seek 
those evaluators out. >>

My response is:
Like I said - I see it coming.  Expert evaluators come in with their New 
Ideas for "meeting goals and objectives" whether they match the local 
program's goals and objectives and programming or NOT.  Assessment experts 
aren't realistic about what it IS the adult literacy learner is learning.  
Incorrect assumptions might occur - literacy programs have nothing of value, 
show no growth, do not show grade level increases  - should they come in and 
evaluate using the current available evaluation tools.

Andrea wrote back at 2:00 pm Friday:
<< …. an excellent point and a very good strategy.  I should think that 
programs could band together to do this--ongoing evaluation with outside 
evluators.  This might result in more funding, and potentially it would open 
up legitimation of a variety of teaching techniques.  I like the idea. >>

Hmm-mm, maybe banding together would work fine in a heavily populated area 
like Boston and Cambridge, but there is little potential for that where miles 
of prairie stretch between "metropolitan" areas.  In the Midwest the 
landscape looks considerably different than it does where you live.  

In S.D. banding together two metropolitan areas would mean bringing together 
two cities that are 400 miles apart - if banding toether the two largest 
cities in the state should be the goal.  If you count the smaller communities 
between, the closest one - where MAYbe literacy students are being served - 
is 1 hour's drive to the West.  Yep!  Going to just jump in the car every 
evaluation period and drive back and forth for evaluation. 

Debbie Yoho, Executive Director, Greater Columbia Literacy Council, responded 
at 2:46 saying:

<< From Sheryl:  "It is possible to find evaluators that are pro-active and
supportive of the program's goals and objectives. I would encourage program
directors to seek those evaluators out." >>

<< Really?  Now this is interesting.  I'm wondering if those NLA commentators
who believe strongly in "rigorous" research would insist on evaluators who
specifically know NOTHING about a given program's goals in order to be
objective?  This raises the question of WHO should be minding the store. >>

My comment:
When I read Sheryl's comment, Debbie?  My very first reaction, before I found 
your follow-up email post, was "So people don't think local program personnel 
is capable of evaluating their own programs objectively." -- an "I'm the 
expert" stance, as far as I was concerned.  Nobody is listening again!?!  
There are some really fine people "minding the store" in the grassroots 
programs who already DO evaluate the effectiveness of their programs with 
individual learners and volunteers plus staff.  We, like you, are required to 
do annual evaluation - ours is done in order to pursue United Way allocations 
for the upcoming year.  I agree with the post, Deb, that states learners 
should be a part of the evaluation process.  We do that here. 

Response from Andrea came that night at 9:43:08 PM CST:
<< This is a day for teasing out word meanings.>>

My response:
Therein is the problem, Andrea.  What one person gives as an opinion is 
heard/read differently than it is by another from a different level of 
understanding of literacy services, program opportunities and individual 
needs.

You wrote that you "didn't take Sheryl's post to mean that evaluators should 
know nothing about a program's goals, objectives or methods."  The double 
jeopardy being played at the crap table is that evaluators WOULD "know plenty 
about methods and goals and (have) wide experience" - be "rigorous".  In my 
view, this would HAVE to be part of the criteria to gain accurate 
information.   I am not altogether sure that I'm that big a gambler to take a 
chance on an evaluator "from the outside"!  

Andrea, I enjoyed your story about a friend with a diagnosed LD.  You said, 
and I quote, "Evaluators would have to know about language and about learning 
disabilities and diagnosis."  And you know something?  When I read your 
story, it occurred to me that the expert evaluators better also have to know 
about the differences between adults and teenagers and how their life skills 
and personal needs come into play in the educational environment.  

Andrea, you expressed a strong message about serving our learners in 
"multiple rigorous ways" to provide, what you called, "becoming literate".  
Debbie (or any other lurking practitioner here) tell me if I'm wrong, but I 
feel that literacy programs are using multiple instructional techniques (I 
also have a problem with those techniques being "rigorous") that allow us to 
be flexible and effective in providing not just reading and writing skills, 
but increasing learner self-confidence and gaining a measure of independence 
-- *in addition to "becoming literate". 

I would poor pity the literacy program that wasted an adult's precious time 
using the wrong method.  (In reference to Andrea's comment about "Given how 
little time adult learners have to spend in class, I think it would be great 
to use the time as effectively as possible".)  You realize, I hope, that our 
New Readers are volunteers, too?  They LEAVE if the program is wasting their 
time!!  I can give testimony to that because when I first began as this 
program's director, I did -- see them leave, I mean.

In my local program, "upfront goal(s)" are a part of the New Reader's 
registration and assessment appointment with me.   If they can't name what it 
is they want to learn, for what purpose and what commitment they are willing 
to make to it, they aren't ready for a tutor assignment.  Each adult is 
experiencing "life's challenges" at the point they finally make the 
appointment.   They have pondered these points because I screen them on the 
phone when they call.  They know ahead of time what we will discuss.

Friday afternoon I met with a 35-year, retired and "knowledgeable teacher" 
regarding the phonetic program she wrote and used with her fourth grade 
students before retirement.  I tried to recruit her, but she said:  "You are 
dong good work here, Nancy.  I wouldn't have the patience to work with adults 
like you are."   She was seeing the "differences" between teaching adults and 
the kids she'd had in her 4th grade classes.  She turned me down.  So 
"knowledgeable teacher" may not be the only criteria expected - 'empathetic' 
volunteers should also be a part of the critical factor for success in areas 
where 1-to-1 programming is offered.

Regarding the comment about:  Literacy programs should have "constant 
teaching upgrades, evaluators who are also coaches, and funding" -- Hate to 
"burst your bubble", but in the "real world", where the political climate 
exists as referred to by Sheryl, there is not enough money at the federal 
level to carry your dream-plan to fruition nationwide.  Won't happen.

Thanks for taking the time to write.  I'm like Daphne - swamped - yet I 
believe in Doing Right by the adult learners whom I serve.  Sorry about the 
length of this response.

Respectfully submitted,
Nancy Hansen
Executive Director
Sioux Falls Area Literacy Council
Sioux Falls, SD
<sfallsliteracy at yahoo.com>
and 
<Nashansen at aol.com>
_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org