[NLA] research questions and methods

[Eileen Eckert]

Contributions to the discussion of research have highlighted differences in 
philosophy and methods, and discussed the matter of which are priveleged and 
which may be marginalized in the current political climate. Maybe it would 
be useful to look at research in terms of purposes or questions to be 
answered as well. As I think about it, the categories of exploratory and 
confirmatory research serve to make clear for me the value of a range of 
research methods. I'm going to take a stab at describing two approaches.

Exploratory research, aka inquiry-guided or grounded theory research, starts 
with a question. The researcher does not make predictions about the answer, 
so the data collection and data analysis methods need to be sensitive to all 
the possibilities, and the criteria for evaluating the quality of the 
research are based on the degree to which the researcher makes the case that 
s/he has considered and weighed the possibilities and substantiated her/his 
findings.

Confirmatory research starts with a hypothesis to be tested. Data collection 
and analysis methods need to control for confounding influences and isolate 
the phenomenon being studied. The criteria for evaluating the research are 
based on the degree to which the researcher has set up an experiment that 
tests the hypothesis and controlled for factors that might influence the 
test results.

Here's an example:
Exploratory research into development of reading proficiency might start 
with the question, "What activities and strategies facilitate the 
development of reading skill?" The researcher could choose a sample of low 
level readers, either purposely choosing a homogeneous sample knowing that 
the findings may not be transferable beyond the population sampled, or 
deliberately seeking a heterogeneous group. Data collection methods might 
include doing talk-aloud protocols, interviews and observations, and 
measurement of reading skills at some point(s). Data analysis could include 
a search for common themes and/or single cases of "best practices" among 
those who demonstrated skill gains. It could include creating groups of high 
skill and low skill readers at the end of the data collection period and 
doing a comparison of activities and strategies used by members of the two 
groups. The quality of the research would be judged by how well the 
researcher makes the case that s/he has considered and explained the various 
possibilities and how well s/he substantiates her/his conclusions about what 
activities and strategies facilitate the development of reading skill.

A confirmatory approach to reading research would start with previous 
research and test a hypothesis, usually the null hypothesis, such as "There 
will be no significant difference in reading level among those instructed 
using a 'balanced' approach versus those instructed using an approach 
stressing phonemic awareness. The sample would have to be chosen to control 
for factors not being tested, so the researcher would want groups (a control 
and two different treatment groups, or just two treatment groups) that are 
similar. Amount of instructional time, time spent reading outside of 
instructional time, and other possibly confounding factors would be 
controlled for as much as possible, and the quality of the research would be 
judged on how well the researcher controlled for "threats to validity" and 
how well the test measured differences in the groups in terms of the 
outcome, reading skill, being studied.

When there is not much known about something, or when the case could be made 
that prior research has not addressed important aspects of a phenomenon, 
then exploratory research is valuable. When there is a research base that 
has produced a testable hypothesis, then confirmatory research is valuable. 
Each has its place, and though the criteria for judging them is different, 
there are criteria for evaluating both. One is not "more rigorous" than the 
other.

Of course, that still leaves the question: Even if both kinds of research 
are valued and funded, how do the findings get applied? We seem to rely on a 
trickle down theory: academics do and publish the research, policy makers 
use the findings to make new rules, administrators intepret the rules to 
make new procedures, professional developers interpret the research to 
teachers, teachers use it with students. Lots of opportunities for loss of 
meaning in the journey!





_________________________________________________________________
Surf the Web without missing calls! Get MSN Broadband. 
http://resourcecenter.msn.com/access/plans/freeactivation.asp

_______________________________________________
NLA mailing list: NLA at lists.literacytent.org
http://lists.literacytent.org/mailman/listinfo/nla
LiteracyTent: web hosting, news, community and goodies for literacy
http://literacytent.org