How Are Standards for Quality Research Determined?
Authors have asserted that standards for quality research should be premised upon the principles of scientific inquiry (i.e., empirical observations using systematic designs), the theoretical underpinnings and philosophy of science (both positivist and post-positivist), and a consensus of a community of scholars (Shavelson & Towne, 2002; Singleton, Straits, & Straits, 1993).
While space limitations prevent a description of these premises, the role of consensus can be discussed in brief. Consensus among a community of scholars is one of the most respected means of quality assessment. Strategies for reaching consensus include position statements, conferences, the peer review process, and systematic review. For example, RAND Europe (Wooding & Grant, 2003) organized and convened a conference of multidisciplinary scholars (e.g., physical sciences, natural sciences, humanities, and the arts) to reach consensus on standards for quality research. According to Odom et al. (2005), divisions within the American Psychological Association (APA) have established criteria on group experimental design, single subject design, and qualitative data gathering methods for research on school psychology and clinical psychology. The consensus approach has been used to evaluate and critique federally sponsored research.
As part of the Government Performance and Reporting Act (GPRA) initiative, a Committee On Science, Engineering, and Public Policy (COSEPUP) was organized to help determine evaluation criteria for government-sponsored research. COSEPUP has stated that "the people best qualified to evaluate basic or applied research are those with the knowledge and expertise to understand its quality and, in the case of applied research, its connection to public and agency goals" (COSEPUP, 1999).
References
COSEPUP. Evaluating federal research programs: Research and the Government Performance and Results Act. Committee on Science, Engineering and Public Policy. Washington, DC: National Academy Press.
Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71(2), 137—148.
Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: National Research Council, National Academy Press.
Singleton, R. A., Straits, B. C., & Straits, M. M. (1993). Approaches to social research. New York: Oxford University Press.
Wooding, S., & Grant, J. (2003). Assessing research: The researchers' view. Cambridge, England: RAND Europe.
Adapted from Focus: Technical Brief Number 9, What Are the Standards for Quality Research?