[NLA] Support for advocacy

[Thomas Sticht]

According to John Coming’s recent post, Andrew Kolstad’s criticisms of
the NALS are addressed on page 7 of the report Building a Level Playing
Field.  In my earlier post, I noted that Kolstad questioned the
construct validity (what it is the NALS three scales actually measure),
the standards validity  (using the 80 percent probability standard which
produces four times the false negative errors (saying an adult cannot do
some tasks when in fact he can),  over the number of false positive
errors  (saying that an adult can do a task when in fact he cannot), and
the use validity, that is, the validity of the NALS for accurately
identifying adults at risk for poor literacy skills. 

I took John’s advise and re-read the report again to find out how the
report rebutted Kolstad’s criticisms. I went to page seven as he advised
and found the following statements:

Paragraph A: Quote: "Like most standardized tests and national surveys,
NALS has received criticism. Some of the critics suggest that the way in
which the NALS data were analyzed placed too many people in Levels 1 and
2. The basis for this argument is that some people in NALS Level 2, for
example, were able to accomplish some but not all of the NALS Level 3
tasks. The other argument is that NALS used tasks (balancing a
checkbook, for example) that might be unfamiliar to some people (in the
preceding example, those who do not have checking accounts). People who
have never had a checking account might have been able to answer a
question that used the same math had the task been something they do all
the time, possibly a similar task that takes place in their work."
Unquote

Paragraph A mentions some of the criticisms of NALS, but does not go
into the three problems of validity that Kolstad found with the NALS.
Also, note that the paragraph does not rebut either of the two lines of
criticism against the NALS that it cites. That is left to the next
paragraph. It goes like this:

Paragraph B: Quote: "These criticisms do not undermine the findings
presented here because this report is focused on the demands of the new
economy. The kinds of tasks used in the NALS are similar to those
demanded in the 21st century workplace. In addition, the new economy
needs workers who are able to apply these skills with accuracy and speed
and in diverse settings." Unquote

No evidence is provided for these assertions and as far as I can tell
that is the end of how the report addresses Kolstad’s concerns about the
validity of the NALS. 

I know that teaching critical thinking is one of the goals of adult
educators to prepare adult literacy learners for the "new economy." So
I’ll leave it as an exercise for the reader to judge whether Paragraph B
rebuts the specific criticisms identified in Paragraph A, or any of the
other concerns that Kolstad has identified. 

Fortunately, other than the NALS data, and the need to read around a few
more non sequiturs, the report has what I think are some very
interesting and useful data and citations of research that can be
profitably used by advocates. I join John in hoping that others will
read the report. 

Tom Sticht
_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org