[NLA] research: long and technical

[Eileen Eckert]

>From Nancy's post on Practitioner-based research: "However, I raise several 
questions and ask whether the answers will mean The Listeners and Readers of 
such a research paper might not accept the data as 'legitimate' or 'valid' 
(as referred to by both George and Eileen Eckert)"

Nancy, if all you want to do is find a way to convince policymakers of 
something, get a lobbyist and bribe them legally--that's how everyone else 
gets what they want, right? Sorry to sound so grouchy, but I'm not 
advocating for an all-or-nothing type deal on the research issue, and I'm 
not an ivory-tower, anti-learner academic. I don't think it's unreasonable 
or unfriendly to learners to ask anyone to provide some evidence to support 
their opinions. You, for example, didn't accept Sandy's belief that learners 
are okay with the initial testing; you (appropriately) wanted some 
description of the context, some evidence to support a statement that didn't 
match your own experience. And she provided it, because she's done some 
research and had some data to back up her opinion.

So, if you want to improve your program based on data, and in the process 
get some evidence to use when talking with funders, policymakers, and 
others, consider doing some research. If you want to improve your program 
based on data that matter to you and your learners, consider doing 
naturalistic research. I've mentioned trustworthiness before, the criteria 
for evaluating naturalistic research, AND I've said I think it's more useful 
and relevant to practitioners than the positivist standards of validity and 
reliability. Andrea says generalizability is the key to research; I say 
generalizability is the key to positivist research. There's a difference. 
Another apology, for lecturing this time (hopefully, the grouchy tone 
diminishes and I won't have two things to apologize for at once).

Positivist (or conventional, or scientific) research is based on the idea 
that there is one objective reality and if we can just put enough pieces of 
the puzzle together with our research, it will all make one big picture that 
looks the same to everybody. Hence, the criteria of validity and 
reliability.

Naturalistic research is based on the idea that much (or all) of reality is 
constructed, individually and socially. Not to say there is no "there" out 
there, just that it operates differently on each of us, and we each 
understand it differently. So the task of the naturalistic researcher is not 
to describe, explain, or manipulate some reality that exists apart from 
people's minds, but rather to understand and communicate the realities 
people create.

So in naturalistic research, we wouldn't try to hold everything else 
constant in order to, for example, isolate the variables that contribute to 
beginning students' attrition. Instead we'd try to record and describe as 
much as possible of the complexity of their experience, and communicate it 
in such a way that others could learn from it too.

Okay, from here on I'm getting even more technical, no main ideas, only 
details, so here's where to stop reading unless you are really into this 
discussion!

To do "good" naturalistic research, researchers need to build 
trustworthiness by establishing credibility, transferability, dependability, 
and confirmability. I'm mainly using and quoting from a book called "Doing 
Naturalistic Inquiry" by Erlandson, Harris, Skipper, and Allen and 
"Designing Qualitative Research" by Marshall and Rossman.

Credibility: "the degree to which [the] findings are the product of the 
focus of [the] inquiry and not the biases of the researcher"(p.34). This is 
communicated through an audit trail, keeping track of the processes used in 
the research.

Transferability: the researcher provides "thick description" (i.e., lots of 
detail) about her own research setting or subjects (though not detail that 
would lead to individual identification) so others can make a judgment about 
whether findings are applicable in their setting or with their 
population(s).

Dependability: "the researcher attempts to account for changing conditions 
in the phenomenon chosen for study as well as changes in the design created 
by increasingly refined understanding of the setting" (From Marshall and 
Rossman's "Desigining Qualitative Research" p. 145). This contrasts with the 
idea of reliability, which "assume an unchanging universe where inquiry 
could, quite logically, be replicated"(p.145)

Confirmability: the researcher attempts to present findings that match those 
of others reviewing the data.

Data collection methods that help build trustworthy research: prolonged 
engagement, persistent observation, interviewing, document analysis, audio- 
or videotaping, subject narratives, and many more! Ways to build 
trustworthiness during data collection and analysis include: triangulation 
of data, member checks (checking with the subjects of the research that your 
understanding matches theirs), peer debriefing (reality checks with 
colleagues), keeping a journal, looking for negative instances of the themes 
you see emerging, constantly comparing data and themes. Of course, nobody 
can do everything, but in the case of naturalistic research it's not all or 
nothing; everything builds trustworthiness.



_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com

_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org