Background
- Teachers & Research - Types
of Research - Metiri Rubrics
Rubrics
The review process that
serves as the foundation of the Metiri TSW (Technology Solutions
That Work) database involves a team of educators, researchers,
and Metiri partners.
Listed below are the questions
that our review team explores when reviewing individual research
studies. These questions are aligned both with Campbell and Cooks
seminal work on “Threats to Validity” as well as with
the “Study Design and Implementation Assessment Device”
from the Department of Education’s “What Works Clearinghouse.”
The specific rubrics that are used by our researchers are considered
intellectual property and are not for general release. The less
specific version of these rubrics provided below will give educators
a deep understanding of the questions we ask and provide a resource
to vendors who wish to interpret the review of their product.
Criteria for Review
of Each Research Study
The criteria and associated rubrics are designed to consider individual
research studies about an intervention and make a judgment about
the merits of researchers’ findings and conclusions about
that intervention. A rubric has been developed to judge the relative
merit of the study with respect to the following questions.
1. What was the nature of the research?
Experimental, Quasi-experimental, Correlational?
2. Confounding variables have been
controlled for.
3. Was the intervention
implemented adequately? Was the degree of implementation monitored
or measured?
4. Were the tools used for measuring
treatments and outcomes valid, reliable & available for inspection?
5. Was the sample size adequate?
6. Can the findings be generalized
to U.S. student populations for whom the intervention is intended?
Any quality that differentiates what happened in the study to
conditions that are likely to exist in real, U.S. schools is suspect.
7. The findings have been replicated in other studies.
8. The findings make sense given what you know about current thinking
in teaching and learning – either in general, or in this
specific content area.
9. The research is conducted by an independent party who does
not have a stake in the success or failure of the intervention.
10. The research has been published in a peer-reviewed journal
or otherwise submitted to peer scrutiny.
11. The statistics used are appropriate and robust.
