[AAACE-NLA] Unlearning Literacy in ABE
ttweeton at comcast.net
ttweeton at comcast.net
Sun Nov 7 14:09:43 EST 2004
Oh my gosh, negative gain scores!! This very thing has happened to me and I thought I hadn't done my job! I was really perplexed. I didn't know that this can occur. Frankly I was very disheartened when this happened to two of my GED students after several weeks of working on the same subject! My opinion for this is that they are afraid of tests so therefore those results . Is there an answer to this?
Tom , you also said
"At the present time I do
not see how the U. S. Congress, or anyone else for that matter, can read
these data and use them to determine anything about teaching and learning
in adult education and literacy programs funded by the U. S. Education
Department’s State Grants program. I also wonder about the legality of
continuing to use such blatantly invalid methods for holding individual
teachers or programs accountable for teaching."
I love this conclusion of course. You aren't looking for a lobbying job in Broward are you by any chance ? I am SURE we could find an opening!
Tanya Tweeton
ESOL and GED Programs
Fort Lauderdale, Florida
-------------- Original message --------------
> Research Note November 6, 2004
>
> Unlearning Literacy in Adult Basic Education Programs
>
> Tom Sticht
> International Consultant in Adult Education
>
> Thirty years ago a colleague and I prepared a paper entitled "The Problem
> of Negative Gain Scores in the Evaluation of Reading Programs."(1) Today,
> as then, the issue of zero or negative gain in adult literacy education is
> hardly ever discussed, though it is still occurring For example, a year
> 2000 report from MassInc. (2) reported some 4 percent negative gain scores
> and 40 percent no change in scores in the Adult Basic Education programs
> they examined. A year later a report from the United Kingdom's Basic
> Skills Agency (3) reported over 30 percent negative gain scores with no
> discussion of how zero and negative gain scores were treated.
>
> Generally, data from pre and post test scores are interpreted as
> indicating the extent to which learning is taking place in a program. For
> instance, if average reading test scores for students increase, say from a
> grade level of 4.8 to 5.8, this is interpreted to mean that students
> gained one grade level in their reading skills. However, by the same
> logic, if some students do better on the pre test than they do on the post
> test, say they score at the 4.8 grade level on the pre test but only 4.2
> on the post test, this would indicate an "unlearning" of 6 months in
> reading skills. From the point of view of interpretive validity, if we
> interpret positive gain to mean learning has occured, shouldn't we
> interpret zero gain to mean no learning and negative gain to mean
> unlearning has occured?
>
> This raises fundamental questions about the validity of using tests for
> which zero or negative gain scores are not infrequent for assessing the
> extent to which programs are promoting learning by students. Can the same
> tests measure learning, no learning, and unlearning? How are zero and
> unlearning to be interpreted for program accountability? Can students who
> have spent time in programs and learned nothing and those who have had
> their literacy unlearned claim they have been subjected to mal-literacy
> practice and sue for damages?
>
> National Negative Gain Reports
>
> A different, though related problem of interpreting zero or negative gain
> can be found in the federal accountability system for adult education and
> literacy programs. Today, to assess learning in the federally funded State
> Grants program, the National Reporting System (NRS) obtains data from the
> States and U.S. Territories to determine the percentage of students in the
> State Grants program that increase their literacy proficiency enough to
> progress from one of the NRS six levels of proficiency to a higher level.
> These data are derived by the States and Territories from pre and post
> scores on nationally normed and standardized tests, such as the TABE,
> ABLE, CASAS and others. However, in reporting these percentages of
> students progressing from one learning level to another, the data on those
> making zero or negative test score gain are not revealed, only the
> percentage of program enrollees moving upward from one level of
> proficiency to another is reported by the NRS.
>
> Each year now the NRS prepares a report for the U.S. Congress that
> reports, among other things, the percentage of adult learners moving
> upward from one level of basic skills proficiency to a higher level. The
> report for Program year 2001-2002 presents tables showing the percent of
> enrolled adults who acquired the level of basic skills needed to complete
> at least one education level (minimum Grade Level Equivalent—2 years) in
> Program Year 2000-2001, which is called the Baseline Year, and then these
> same kinds of data are presented for Program Year 2001-2002. This allows
> one to determine if the percent of adults progressing from one level up to
> another has increased from the Baseline Year (2000-201) to the current
> year (PY 01-02).
>
> Averaged over the 50 States, District of Columbia, and Puerto Rico, the
> data indicate that in the Baseline Year of 2000-2001, 36% of program
> enrollees moved up from one level to a higher level as measured by a
> particular State’s testing system. In PY2001-2002 this percentage
> increased by one percentage point to 37%, suggesting improvement in the
> federally funded State Grants programs overall, but still short of the U.
> S. Education Department’s performance goal of having 40 percent of adults
> show increases in their learning by moving from one level to a higher
> level. Interestingly, no indication of how the federal performance goal of
> 40 percent making improvements was determined is given in the report.
>
> More to the point here, though, are the individual data for the 50 States
> and Territories. These data are given below for Adult Basic and Secondary
> Education. Similar data are available for English as a Second Language
> learning but those data are not discussed here.
>
> The following table shows the percentage of adults enrolled in adult basic
> and secondary education programs who acquired the basic skills needed to
> complete the level of instruction in which they were initially enrolled
> and move up to a higher level. The first column shows the State or
> Territory, the second column shows the Baseline year of PY00-01 and the
> third column shows the current PY01-02 year being reported to Congress.
> The table shows that 20 States made negative gains in percent of adults
> making level improvements from PY00-01 to PY01-02, 3 States made zero
> gain, and 29 made positive gains (note that there are 52 total reports,
> but reference will be just to States to avoid redundancy in referring to
> Territories).
>
> Now the question is, what do these data mean? Do they mean that in 20
> States the ability of programs to teach basic skills to adults declined
> and in three States their teaching ability stayed the same from one year
> to the next? Do they mean that in 29 States the programs got better at
> teaching basic skills to adults? Does it mean that students in PY 01-02
> got more difficult to teach in those States where improvements declined
> from the baseline, or that teachers were less apt? Why do States have such
> large differences in baseline and current program years in the percentage
> of adults making some movement up the NRS levels?
>
> Along with the issues of zero and negative gain using individual
> standardized, nationally normed tests, the problem of interpreting data
> such as that reported by the NRS raises serious questions regarding the
> validity of our national accountability system for federally supported
> programs across the nation and its Territories. At the present time I do
> not see how the U. S. Congress, or anyone else for that matter, can read
> these data and use them to determine anything about teaching and learning
> in adult education and literacy programs funded by the U. S. Education
> Department’s State Grants program. I also wonder about the legality of
> continuing to use such blatantly invalid methods for holding individual
> teachers or programs accountable for teaching.
>
> PY PY
> 00- 01-
> State 01 02
>
> Alabama 26 25 down
> Alaska 55 44 down
> DC 55 37 down
> Georgia 29 28 down
> Hawaii 39 36 down
> Idaho 55 44 down
> Illinois 30 29 down
> Indiana 38 37 down
> Kentucky 58 57 down
> Maryland 55 46 down
> Minnesota 24 22 down
> Montana 53 35 down
> N. Hampshire 55 40 down
> N. Mexico 42 29 down
> New York 35 31 down
> N. Dakota 82 69 down
> Ohio 59 55 down
> Puerto Rico 75 53 down
> Utah 43 42 down
> W. Virginia 55 54 down
> 20 down
>
> Florida 36 36 same
> Missouri 31 31 same
> N. Carolina 36 36 same
> 3 same
>
> Arizona 34 39 up
> Arkansas 36 44 up
> California 25 29 up
> Colorado 48 50 up
> Connecticutt 28 39 up
> Delaware 27 35 up
> Iowa 27 39 up
> Kansas 53 76 up
> Louisiana 38 45 up
> Maine 38 43 up
> Massachusetts 22 23 up
> Michigan 26 29 up
> Mississippi 42 43 up
> Nebraska 30 36 up
> Nevada 31 46 up
> New Jersey 27 32 up
> Oklahoma 29 42 up
> Oregon 43 47 up
> Pennsylvania 29 33 up
> Rhode Island 55 82 up
> South Carolina 20 36 up
> South Dakota 34 42 up
> Tennessee 38 40 up
> Texas 25 29 up
> Vermont 10 11 up
> Virginia 30 40 up
> Washington 33 41 up
> Wisconsin 57 80 up
> Wyoming 48 53 up
> 29 up
>
>
>
> 1. Caylor, J . & Sticht, T. (1974, April). The Problem of Negative Gain
> Scores in the Evaluation of Reading Programs.
> Chicago, IL: paper presented at the meeting of the American
> Educational Research Association.
>
> 2. Comings, J., Sum, A. & Uvin, J. (with others) (2000,
> December).New Skills for a New Economy: Adult Education's Key Role
> in Sustaining Economic Growth and Expanding Opportunity, Boston,
> MA: MassInc.
>
> 3. Brooks, G. et al (2001, January). Progress in Adult Literacy: Do
> Learners Learn? London: Basic Skills Agency.
>
>
>
> Thomas G. Sticht
> International Consultant in Adult Education
>
> 2062 Valley View Blvd.
> El Cajon, CA 92019-2059
> Tel/fax: (619) 444-9133
> Email: tsticht at aznet.net
>
>
>
>
>
> _______________________________________________
> AAACE-NLA mailing list: AAACE-NLA at lists.literacytent.org
> http://lists.literacytent.org/mailman/listinfo/aaace-nla
> LiteracyTent: web hosting, news, community and goodies for literacy
> http://literacytent.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.literacytent.org/pipermail/aaace-nla/attachments/20041107/364d2a9f/attachment.html>
More information about the AAACE-NLA
mailing list