Standardized Research Reporting
Another form of consensus is standardized reporting of research. In published research, quality assessment is often poor because essential information is frequently absent regarding samples, statistics, randomization, analysis, or interventions. For example, Garcia-Berthou and Alcaraz (2004) conclude that the reporting of test statistics and degrees of freedom, two items needed to calculate P-values, is often absent from published articles in medical research. Moher, Schulz, and Altman (2001) suggest that "inadequate reporting borders on unethical practice when biased results receive false credibility."
To facilitate quality review, several groups of scholars, particularly among public health and medical researchers, have recommended standardized research reporting frameworks to help ensure that essential research information needed to assess quality is included in journal articles. Often described as "checklists," these standards for reporting are more comprehensive than the basic IMRAD (Introduction, Methods, Results, and Discussion or Conclusion) framework for general scientific reporting. Checklists vary by methodology used and specific research designs. There are several standardized formats for general and specific research designs, including the following:
- CHERRIES (Checklist for Reporting Results of Internet E-Surveys): a 30-item checklist for reporting Web-based surveys, published in the Journal of Medical Internet Research (JMIR) (Gunther, 2004). Available at http://www.jmir.org/2004/3/e34/
- CONSORT (Consolidated Standards for Reporting Trials): a 22-item checklist for reporting simple two group, parallel, randomized controlled trials (Moher, Schulz, & Altman, 2001). Available at
http://www.consort-statement.org/Downloads/Checklist.doc [ MS Word format ]
or http://www.consort-statement.org/Downloads/checklist.pdf [ PDF format ]. - MOOSE (Meta-Analysis Of Observational Studies in Epidemiology): a 35-item checklist for reporting observational studies (Stroup et al., 2000). Available at http://www.greenjournal.org/misc/moose.pdf
or http://jama.ama-assn.org/cgi/content/full/283/15/2008. - QUOROM (Quality of Reporting of Meta-Analyses): a 17-item checklist for reporting systematic reviews (Moher et al., 1999). Available at http://www.consort-statement.org/QUOROM.pdf. [ QUOROM is only available in .pdf format.]
- Reporting of Observational Longitudinal Research: a 33-item checklist patterned after CONSORT. The checklist criteria focus on threats to the internal and external validity of observational longitudinal studies. Additional items concern recruitment, data collection, biases, data analysis, descriptive issues and generalizability (Tooth et al., 2005). Available at http://aje.oxfordjournals.org/cgi/content/abstract/161/3/280
- STARD (Standards for Reporting of Diagnostic Accuracy): a 25-item checklist for diagnostic test accuracy (STARD, 2001). Available at http://www.clinchem.org/cgi/content/full/49/1/7/.
- STROBE (Strengthening the Reporting of Observational Studies in Epidemiology): Checklist for case-control, cohort, and cross-sectional studies (STROBE Group, 2005). A draft of this checklist is available in .pdf and text format at http://www.strobe-statement.org
- TREND (Transparent Reporting of Evaluations with Nonrandomized Designs): a 22-item checklist for nonrandomized designs (Des Jarlais et al., 2004). Available at: http://www.ajph.org/cgi/content/full/94/3/361/ [ HTML page ]
or http://www.trend-statement.org/asp/documents/statements/AJPH_Mar2004_Trendstatement.pdf. [ PDF format ] - Qualitative Checklist(s): Several checklists and evaluative guides have been developed for empirical research using qualitative methods or designs. While not an exhaustive list, representative checklists for qualitative research include:
- Criteria for evaluating qualitative studies (Bromely, et al., 2002). Available at http://www.liv.ac.uk/lstm/learning_teaching/masters/masters_docs/criteria_eval_qual_studies.pdf [ PDF format ]
There are also standardized reporting instruments for specific subspecialties ranging from acupuncture (STRICTA: Standards for Reporting Interventions in Controlled Trials of Acupuncture) to acute ischemic stroke (Higashida, 2003; MacPherson et al., 2002). These reporting frameworks include key appraisal points for assessing quality that are specific to the research design and are intended to facilitate the review of research studies (Des Jarlais, Lyles, & Crepaz, 2004; Lohr, 2004). While checklists are not evaluation instruments, their use has been associated with improved reporting (Moher, Jones, & Lepage, 2001). While this discussion focuses on checklists for research that report quantitative data, the literature also indicates guides for authors using research that reports qualitative data (Greenhalgh & Taylor, 1997; Patton, 2003; Ragin et al., July 2003; Rowan & Huston, 1997). Some authors have criticized the concept of checklists for research designed to generate qualitative data as being overly prescriptive (Barbour, 2001).
Summary
As quality research is a precursor to statements about evidence, consensus standards on quality research and consistent reporting are needed. Consensus standards also are needed to facilitate the knowledge translation (KT) process, as research quality and evidence must be assessed and deemed sufficient prior to dissemination and knowledge utilization initiatives (CIHR, 2004; Davis et al., 2003). In the fields of disability and rehabilitation research, there is a healthy debate regarding the specific criteria for quality research, and the specific checklists to be used to standardize reporting. As this debate continues, there are many ideas in the public domain regarding standards for quality research and strategies for standardized reporting that can be used to help guide the ongoing discussion and decision-making process.
References
Barbour, R. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog?British Medical Journal, 322, 1115-1117.
Bromely, H., Dockery, G., Fenton, C., Nhlema, B., Smith, H., Tolhurst, R., and Theobald, S. (2002).Criteria for evaluating qualitative studies. Qualitative Research and Health Working Group, International Health Research Group, Liverpool School of Tropical Medicine, Liverpool, UK.
CIHR. (2004).Knowledge translation strategy 2004-2009: Innovation in Action. Ottawa, ON: Canadian Institutes of Health Research.
Davis, D., Evans, M., Jadad, A., Perrier, L., Rath, D., Ryan, D., et al. (2003). The case for knowledge translation: shortening the journey from evidence to effect. British Medical Journal, 327(7405), 33-35.
Des Jarlais,D. C., Lyles, C., & Crepaz, N. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health, 94(3), 361-366.
Garcia-Berthou, E., & Alcaraz, C. (2004). Incongruence between test statistics and P values in medical papers. BMC Medical Research Methodology, 4(13).
Greenhalgh,T., & Taylor, R. (1997). How to read a paper: Papers that go beyond numbers (qualitative research). British Medical Journal, 315, 740-743.
Guther, E. (2004). Improving the quality of web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). Journal of Medical Internet Research, 6(3) e34. Retrieved on September 20, 2006, from http://www.jmir.org/2004/3/e34/
Higashida, R. T., Furlan, A.J., Roberts, H., Tomsick, T., Connors, B., Barr, J., et al. (2003). Trial design and reporting standards for intra-arterial cerebral thrombolysis for acute ischemic stroke. Stroke, 34(8), 109-137.
Lohr, K. N. (2004). Rating the strength of scientific evidence: Relevance for quality improvement programs.International Journal for Quality in Health Care, 16(1), 9-18.
MacPherson, H., White, A., Cummings, M., Jobst, K. A., Rose, K., & Niemtzow, R. C. (2002). Standards for Reporting Interventions in Controlled Trials of Acupuncture: The STRICTA recommendations. Journal of Alternative and Complementary Medicine, 8(1), 85.
Moher, D., Cook,D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomized controlled trials: The QUORUM statement. The Lancet, 354(1896-1900).
Moher, D., Schulz, K. F., & Altman, D. G. (2001). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials.The Lancet, 357, 1191-1194.
Patton, M. Q. (2003).Qualitative evaluation checklist. Retrieved March 19, 2005, from http://www.wmich.edu/evalctr/checklists/
Ragin, C.C., Nagel, J., & White, P. (July 2003). Workshop on scientific foundations of qualitative research. National Science Foundation, Arlington, VA.
Rowan, M., & Huston, P. (1997). Qualitative research articles: Information for authors and peer reviewers. Canadian Medical Association Journal, 157(10), 1442-1446.
STARD Group. (2001).The STARD initiative - Towards complete and accurate reporting of studies on diagnostic accuracy. Retrieved February 18, 2004, from http://www.consort-statement.org/Initiatives/newstard.htm
STROBE Group (Sept 2005). STROBE statement:
Strengthening the reporting of observational studies in epidemiology [Version 3]. Retrieved on February 12, 2007, from http://www.strobe-statement.org/
Stroup, D. F., Berlin, J. A.,Morton, S. C., Olkin, I., Williamson, G. D., & Rennie, D. (2000). Metaanalysis of observational studies in epidemiology: A proposal for reporting. Journal of the American Medical Association, 283, 2008-2012.
Tooth, L., Ware, R., Bain, C., Purdie, D. M. & Dobson, A. (2005). Quality of reporting of observational longitudinal research. American Journal of Epidemiology, 161(3), 280-288.
Adapted from Focus: Technical Brief Number 9, What Are the Standards for Quality Research?