[NLA] Consequential validity

Fri Apr 5 16:44:19 EST 2002

consequential validity seams to work like this?

They gave a test and got results.  Now what to do with
the results (consequences) of the test.  Did their
interpretation of the test results lead to "bias,
fairness, and distributive justice" perhaps in
situations such as hiring or promotion.  Perhaps there
were unintended consequences of the test?

So this seams to say that a test may be valid because
it measures what it intends to measure.  It may be
reliable as well getting the same results over and
over.  But actions taken be those who make decisions
based on test results somehow might be wrong based on
an unintended fault in the test itself?  Or is it that
the decision maker is merely chucking the results and
making wrong decisions anyway.  If its the latter, why
discuss it.

tz

--- Regie Stites <regie.stites at sri.com> wrote:
> The late Samuel Messick developed the concept of
> consequential validity as one of
> six aspects of a unified notion validity (for all
> educational and psychological
> measurement).  He defined the consequential aspect
> of validity as an appraisal of
> "the value implications of score interpretation as a
> basis for action as well as
> the actual and potential consequences of test use,
> especially in regard to sources
> of invalidity related to issues of bias, fairness,
> and distributive justice" (see
> Messick's essay on "Validity" in the 1989 (3rd)
> edition of Educational Measurement,
> R. L. Linn (Editor), New York: Macmillan).
> 
> Including consequences of test interpretation and
> use as explicit considerations in
> the validation of educational tests and assessment
> is still a relatively new idea
> in psychometrics, and something that needs more
> attention from test developers and
> from test users.
> 
> The latest edition (1999) of the "Standards for
> Educational and Psychological
> Testing" (published jointly by the American
> Educational Research Association,
> American Pscyhological Association, and the National
> Council on Measurement in
> Education) incorporates consequential validity in
> test validation.  But, in the
> section introducing standards for validity, they
> make a distinction between
> "intended and unintended consequences of test use
> ... that can inform validity
> decisions"  on the one hand, and "evidence that may
> inform decisions about social
> policy but falls outside the realm of validity" on
> the other (p. 16).   In other
> words, not every problem of differential
> consequences from test use (in making
> decisions about employment or about academic
> promotion or selection, for example)
> is a problem with the validity of the test.  Some
> problems with the consequences of
> test use are strictly matters of social policy --
> bad decisions made by people
> interpreting test results (or research results), not
> flaws in the design of the
> tests (or research methods).
> 
> Regie Stites, educational researcher
> 
> Elsa Auerbach wrote:
> 
> > Can anyone explain the concept of consequential
> validity?   I think it's
> > relevant to Freire and other social change
> pedagogies and research.  Elsa
> > Auerbach
> >
> > PDRNRI at aol.com wrote:
> >
> > > A few short observations from a quick read of
> Catherine's meaningful letter:
> > >
> > > 1.  I would like to hear from other researchers
> on this list on how the
> > > research field is grappling with the govt.'s
> emphatic prioritizing of a
> > > rather one-dimensional approach to research.  It
> seems to me that there are a
> > > great many researchers in the field today who
> seek to approach learning and
> > > development as something "dialogic,
> self-reflective, political, ethical and
> > > creative" (Catherine's terms).  How are you
> responding to this?
> > >
> > >  2.  Reflecting on Catherine's important,
> ongoing reminder that we view our
> > > society as a "maturing democracy" - it occurs to
> me that for most of our
> > > history (certainly the 20th century) the
> predominant question challenging the
> > > democratic process in the US. has been "who gets
> to participate?" .  The
> > > dilemma of modern adult education, it seems to
> me, is partly connected to a
> > > shift (or the lack thereof) in the popular
> discourse from  "who gets to
> > > participate?" to "how do we prepare ourselves to
> participate?"  I suspect
> > > that while many of the folks on this list -- who
> have thought hard about
> > > democratic participation for so long -- see this
> question clearly and have
> > > developed very carefully considered answers. 
> But I'm less sure that the
> > > popular thinking on this question has advanced a
> great deal -- partly because
> > > issues around "who gets to participate" are not
> fully resolved, partly
> > > because actual preparation for participation is
> more threatening to those who
> > > exert the most power, and perhaps partly because
> many of those who have had a
> > > long-standing right to participate have been
> effectively dissuaded from doing
> > > so, having bought into the idea that
> particaption ends, rather than begins,
> > > at the ballot box.  In that case, the issue of
> preparing for more effective
> > > participation would simply not be on the radar
> among those who can apply
> > > political pressure through popular opinion.
> > >
> > > 3.  More good books, for the same purposes, I
> think, as Catherine's:
> > > Making Spaces (Sissel and Sheared, eds.)  In
> Defense of the Lifeworld
> > > (Welton, ed.) -- and for teacher development
> purposes, Becoming a Critically
> > > Reflective Teacher (Brookfield)
> > >
> > > David Hayes
> > > _______________________________________________
> > > NLA mailing list: NLA at lists.literacytent.org
> > >
> http://lists.literacytent.org/mailman/listinfo/nla
> > > LiteracyTent: web hosting, news, community and
> goodies for literacy
> > > http://literacytent.org
> >
> > _______________________________________________
> > NLA mailing list: NLA at lists.literacytent.org
> > http://lists.literacytent.org/mailman/listinfo/nla
> > LiteracyTent: web hosting, news, community and
> goodies for literacy
> > http://literacytent.org
> > begin:vcard 
> n:Stites;Regie
> tel;cell:(510) 325-3968
> tel;fax:(650) 859-3375
> tel;home:(510) 339-8905
> tel;work:(650) 859-3768
> x-mozilla-html:FALSE
> url:http://www.sri.com/policy/cep/literacy
> org:SRI International;Center for Education Policy
> adr:;;333 Ravenswood Avenue;Menlo Park;CA;94025;
> version:2.1
> email;internet:regie.stites at sri.com
> title:Program Manager
> fn:Regie Stites, Ph.D.
> end:vcard
> 

=====
Read all about truespel at truespel.com.  
Convert text to truespel USA accent by copy/pasting it at: http://www.foreignword.com/dictionary/truespel/transpel.htm

__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/
_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org