[NLA] "breaking news"

[Dominique Brillanceau]

DAvid: i don't think that Tom  would mind that I'd quote from his work which is online anyway.  If you see a problem with it, I'd be happy to ask him first.

Greetings!

It seems that it is time for Tom Sticht's contribution on negative gains in ABE/ESL programs ( mentioned on this list a few years ago.)

The following reference can be found on-line at
www.nald.ca under Full Text Documents searched by S for his last name. 


Sticht, T. (1999, November) Testing and Accountability in Adult Literacy
Education. El Cajon, CA: Applied Behavioral & Cognitive Sciences, Inc. 


Chapter 4 discusses standardized testing and includes a brief discussion
about negative gain.
An excerpt:
 "Negative Gain.
 In ABE or ESL programs it is not unusual to find that 10-20 percent of
learners score poorer on the post-test than they do
  on the pre-test. Therefore, when the post-test score is subtracted
from the pre-test score to calculate the gain score, the
  gain is a negative number (Taggart, 1985; Caylor & Sticht, 1974).

  It is possible (though not very probable, perhaps) that negative gain
may occur because learners on the pre-test do not
  work at any given item too long, because they think they cannot
perform the test task, and so they simply guess at all the
  items. On the post-test they spend more time on each item because they
have new competence and think they should not
  guess but try to actually comprehend and perform each item. This could
lead to more accurate, but fewer test items being
  completed at the post-test, and hence a negative gain score. 
enerally, however, negative gain reflects guessing or other regression
effects. In this case, guessing on the pre-test is
  better than guessing on the post-test and this leads to negative gain.
This can be reduced by using tests that require
  constructed responses, or that offer many alternatives for multiple
choice tests. The latter reduces the effects of guessing.
  In one study where tests with very low probability for guessing were
introduced, negative gain was reduced from 30
  percent to 6 percent (Sticht, 1975).

  For those programs in which tests with higher potential for negative
gain exists, and this includes all multiple choice
  tests, frequency distributions showing numbers and percentages of
learners making various amounts of negative and zero
  gain should be included. This permits evaluators to gauge the amount
of regression occuring in the program. Simply
  showing average pre-and post-test scores that includes the zero and
negative gains obscures this valuable information and
  produces inaccurate indications of lower improvement in the program
than actually occurs." end of quote

Would we get/give/ report a more accurate picture of our students if we used standardized assessment through performance tasks? Or through holistic scoring  of an essay (as it has been done for  the GED) ? Probably. Would we get a complete picture of what our students are capable of? Probably not, but it might be help.
How about one self reporting line whereby the student would bubble either:
learned a lot.
learned some
didn't learn
unlearned some

 That would give them the voice they ought to have.

Dominique Brillanceau
PCC/ PSU
Portland, Oregon

----- Original Message ----- 
From: <AWilder106 at aol.com>
To: <nla at lists.literacytent.org>
Sent: Tuesday, June 11, 2002 5:21 AM
Subject: Re: [NLA] "breaking news"


> Re NRS:
> 
> Now I'm really confused!  
> 
> Does the discrepancy between numbers exist between state and national level 
> figures?  Does the discrepancy exist between numbers last year and numbers 
> this year?
> 
> Nancy Hansen is suggesting that low level learners head for the hills when 
> faced with a test, is this born out by other AELS administrators?
> 
> In other words:
> 
> 1)  Is there a real drop in numbers?
> 2)  Is there a problem in the paperwork requirement which results in low 
> numbers actually being reported?
> 3)  Have ABE students' behavior changed, leading to swift program exit?
> 
> Andrea
> _______________________________________________
> NLA mailing list: NLA at lists.literacytent.org
> http://lists.literacytent.org/mailman/listinfo/nla
> LiteracyTent: web hosting, news, community and goodies for literacy
> http://literacytent.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.literacytent.org/pipermail/nla/attachments/20020613/02e62d10/attachment.htm