Knowledge Translation: Introduction to Models, Strategies, and Measures

Effectiveness of Knowledge Translation Strategies

Many strategies related to knowledge translation were reported in the literature. The majority of these strategies focused on the dissemination and implementation of existing knowledge. Only a few studies reported strategies involving knowledge creation. Those studies, however, mainly described the strategies that were used and their successes, but not the measurable outcomes of such strategies.

The evidence on the effectiveness of knowledge dissemination and implementation strategies came largely from studies on physicians, with a much smaller portion coming from other allied health professions, such as nursing, physician assistants, and other staff. In addition, the knowledge targeted for implementation in these studies was not all research-based. However, much can be learned about what might influence changes in providers' practice behavior. Ultimately, the translation of research knowledge into practice implies a need for change in providers' behavior to enable adoption and use of the new research-based knowledge.

Overall Effectiveness of Implementation Strategies

A considerable number of studies are available in the area of implementation strategies' effectiveness, and several systematic reviews of those studies have been conducted. Also available are the overviews of systematic reviews.

Bero et al. (1998) conducted an overview of 18 systematic reviews, published between 1989 and 1995, to investigate the effectiveness of dissemination and implementation strategies of research findings. The systematic reviews were obtained from searches in MEDLINE records from 1966 to June 1995 (the Cochrane Library was also searched, but no relevant reviews were located). To be included, the reviews had to focus on interventions to improve professional performance, report explicit selection criteria, and measure changes in performance or outcome. The researchers found the following consistent themes from their review:

• Most of the reviews reported modest improvements in performance after interventions.
• Passive dissemination of information was generally ineffective in altering practices, no matter how important the issue or how valid the assessment methods.
• Multifaceted interventions, a combination of methods including two or more interventions, seemed to be more effective than single interventions.

Bero et al. (1998) also categorized three tiers of specific interventions based on their level of effectiveness in promoting behavioral change among health professionals: (1) consistently effective interventions, (2) interventions of variable effectiveness, (3) and interventions that have little or no effect. These levels of effectiveness are elaborated below:

• Consistently effective interventions included educational outreach visits for prescribing in North America; manual or computerized reminders; multifaceted interventions such as two or more approaches of audit and feedback, reminders, local census processing, or marketing; and interactive educational meetings.
• Interventions of variable effectiveness included audit and feedback, the use of local opinion leaders, local consensus processes, and patient-mediated interventions.
• Interventions that have little or no effect included educational materials and didactic educational meetings.

The authors concluded that the use of specific strategies to implement research-based recommendations seems necessary to promote practice changes and that more intensive efforts are generally more successful. The authors also indicated a need to conduct studies to evaluate two or more interventions in a specific setting to clarify the circumstances likely to modify the effectiveness of such interventions.

In an updated version of the Bero et al. overview (1998), Grimshaw et al. (2001) conducted an overview of 41 systematic reviews published between 1989 and 1998 to examine the effectiveness of intervention strategies in changing providers' behavior to improve quality of care. Similar inclusion criteria were used, although the searches for systematic reviews were expanded to include the Health Star database and feedback from the Cochrane Effective Practice and Organization of Care group's Listserv of potentially relevant published reviews that might have been omitted. The systematic reviews included in this overview were found to be widely dispersed across 27 different medical journals and covered a wide range of interventions.

Similar to the findings from the original overview (Bero et al., 1998), Grimshaw et al. (2001) found that, in general, passive approaches are ineffective and unlikely to result in behavior change (although they could be useful in raising awareness). Also, as previously reported, multifaceted interventions that targeted several barriers to change are more likely to be effective than single interventions. Specifically, the interventions of variable effectiveness included audit and feedback and use of local opinion leaders, whereas the interventions that were generally effective included educational outreach and reminders. However, most of the interventions were effective under some circumstances, and none were effective under all circumstances.

The researchers noted considerable overlap across the reviews in terms of original studies included, and they found that one reason for such overlap could be the lack of agreement within the research community on the theoretical or empirical framework for classifying interventions aimed at facilitating professional behavior change. In addition, although the findings indicated that multifaceted interventions are more likely to be successful, it is difficult to disentangle which components of such interventions led to success. Therefore, Grimshaw et al. (2001) suggested that more research is needed to facilitate a further understanding about the components contributing to the effectiveness of the multifaceted interventions.

Grol (2001) reported on experiences with more than 10 years of development and dissemination of clinical guidelines for family medicine in the Netherlands, arguably one of the most comprehensive programs for evidence-based guidelines development and implementation in the world. A stepwise approach was used for guidelines development and dissemination. First, a relevant topic was selected by an independent advisory board of practitioners. Second, researchers and practitioners worked together to develop guidelines that were later tested for feasibility and acceptability through a written survey and external reviewers who were specialized on the topic, and the guidelines were then revised. Next, the guidelines were presented to an independent scientific board for official approval. After approval, the final version of the guidelines was published in a scientific journal for family physicians. In addition to the formal publication, the dissemination activities included developing special educational programs and packages for each set of the guidelines and sending them to regional and local coordinators for continuing medical education and quality improvement. Support materials, such as summaries of the guidelines for receptionists, leaflets, letters for patients, and consensus agreements with specialists' societies, were also developed and disseminated.

Several aspects of these experiences were empirically investigated, including knowledge and acceptance of the guidelines, the use of the guidelines, barriers to their implementation, and intervention effects. Grol (2001) concluded that the development of clinical guidelines is feasible and accepted by the target group when such development is owned and operated by the profession itself. Second, a comprehensive strategy to disseminate the guidelines through various channels appears to be very important. Third, a program to implement a guideline should be well designed, well prepared, and preferably pilot-tested for use.

Grimshaw et al. (2004) conducted a comprehensive systematic review of 235 original studies to evaluate the effectiveness and efficiency of guideline dissemination and implementation strategies. Literature searches were conducted in MEDLINE (1966–1998), Health Star (1975–1988), Cochrane Controlled Trial Register (4th ed., 1998), EMBASE (1980–1998), SIGLE (1980–1988), and the specialized register of the Cochrane Effective Practice and Organization of Care group. The studies included in this review employed randomized controlled trials, controlled clinical trials, controlled before-and-after studies, and interrupted time series methodologies. They also included medically qualified health-care professionals, used guideline dissemination and implementation strategies, and used objective measures of provider behavior and/or patient outcome.

The researchers found that the majority of interventions produced modest to moderate improvements in care, with a considerable variation both within and across interventions. Commonly evaluated single interventions were reminders, dissemination of educational materials, and audit and feedback. For multifaceted interventions, no relationship was found between the number of components in the interventions and the effects of such interventions. Fewer than one third of the 309 comparisons within the included studies reported economic data.

Grimshaw et al. (2004) concluded that there is an imperfect evidence base to support decisions about which guideline dissemination and implementation strategies are likely to be efficient under different circumstances. In addition, there is also a need to develop and validate a coherent theoretical framework of health-professional behaviors, organizational behaviors, and behavior change to better inform appropriate choices for dissemination and implementation strategies and to better estimate the efficiency of such strategies.

Effectiveness of Specific Implementation Strategies

Audit and Feedback

To further investigate the features or contextual factors that determine the effectiveness of audit and feedback, Jamtvedt, Young, Kristoffersen, O'Brien, and Oxman (2006) conducted a systematic review to investigate the effects of audit and feedback on both professional practice and health outcomes using 118 randomized controlled trials (RCTs) published between 1977 and 2004. The original studies were located through the Cochrane Effective Practice and Organization of Care register for 2004, MEDLINE from January 1966 to April 2000, and the Research and Development Resource Base in Continuing Medical Education. The RCTs in this review included participants who were health-care professionals responsible for patient care; used audit and feedback as one of the interventions; and objectively measured provider performance in a health-care setting or measured health outcomes. Audit and feedback was defined as any summary of clinical performance of health care over a specified period of time, with information given in a written, electronic, or verbal format.

The findings showed that the effects of audit and feedback on compliance with the desired practice varied from negative effect to a very large positive effect, and the differences in effects were not found to relate to study quality. Some possible explanations offered by the authors for the differences included the level of baseline compliance of the targeted behavior, the intensity of audit and feedback, and the level of motivation of health professionals to change the targeted behavior (not assessed due to lack of sufficient data from original studies). The authors concluded that audit and feedback could be effective in improving professional practice, although the degree of effectiveness is likely to be greater in cases where the baseline adherence to recommended practice is low and the delivery intensity of audit and feedback is high. However, the results do not support mandatory use of audit and feedback as an intervention to change practice.

Recently, Hysong, Best, and Pugh (2006) used a purposeful sample of six Veterans Affairs Medical Centers to further investigate the impact of feedback's characteristics on its effectiveness, comparing facilities with high adherence to six clinical practice guidelines to those with low adherence. Data were obtained through individual and group interviews of all levels of staff at each facility, including facility leadership, middle management and support management, and outpatient clinic personnel. The researchers found four discernible characteristics that distinguished the high- and low-performing facilities: (1) the timeliness with which the providers received feedback, (2) the degree to which providers received feedback individually, (3) the punitive or nonpunitive nature of the feedback, and (4) the customization of feedback in ways that made it meaningful to individual providers. Based on these findings, the authors proposed a model of actionable feedback in which feedback should be provided in a timely fashion; feedback should be about the clinician's individual performance rather than aggregated performance at a clinic or facility level; feedback should be nonpunitive; and the individual should be engaged as an active participant in the process rather than be merely a passive recipient of information.

Tailored Interventions

Shaw et al. (2005) conducted a systematic review of 15 randomized controlled trials, published between 1983 and 2002, to assess the effectiveness of planning and delivery interventions tailored to address specific and prospectively identified barriers to changing professional practice and health outcomes. The pool of studies was retrieved from the Cochrane Collaboration Effective Practice and Organization of Care group's specialized register and pending files until the end of December 2002. Studies in this review were RCTs that included participants who were health-care professionals responsible for patient care; included at least one group that received an intervention tailored to address explicitly and prospectively identified barriers to change; either involved a comparison group that did not receive a tailored intervention or compared a tailored intervention addressing individual barriers with an intervention addressing both individual and social- or organizational-level barriers; and included objectively measured professional performance (excluding self-report). A tailored intervention was defined as an intervention chosen to overcome barriers identified before the design and delivery of the intervention. A meta-analysis method was used to synthesize subsets of the studies when possible.

The authors concluded that the effectiveness of tailored interventions is not conclusive and more rigorous trials are needed before the effectiveness of such an approach can be established. In addition, because of the small number of studies limited to a small number of organizational settings, the authors stated that it was not possible to determine whether interventions targeting both individual and organizational barriers are more effective than those targeting only individual barriers. Based on the review, the authors concluded that decisions about whether the tailored intervention approach is likely to be effective for specific problems should be made based on the knowledge of the problem in each setting and other practical considerations.

Organizational Structures

Foxcroft and Cole (2000) conducted a systematic review to determine the extent to which change in organizational structures would promote the implementation of high-quality research evidence in nursing. Literature searches were performed in the following databases: the Cochrane library, MEDLINE, EMBASE, CINAHL, SIGLE, HEALTHLINE, the National Research Register, the Nuffield Database of Health Outcomes, and the NIH Database up to August 2002 (the review was updated in 2003). Hand searches were performed in the periodicals Journal of Advanced Nursing, Applied Nursing Research, and Journal of Nursing Administration (to 1999). Reference lists of the articles obtained were also checked. In addition, experts in the field and relevant Internet groups were contacted for any other potential studies. To be included in this review, each study had to employ randomized controlled trial, controlled clinical trial, or interrupted-time-series research designs; include health-care organizations (where nurses practiced) as participants and measure and report nursing practice outcomes separately; evaluate an entire or identified component of organizational infrastructure aimed at promoting effective nursing interventions; and use objective measures of evidence-based practice.

Foxcroft and Cole (2000) found that no studies were sufficiently rigorous to be included in the formal systematic review, neither at that time nor when they updated the review in 2003. Therefore, they reported on seven studies that did not meet the design quality required but had evaluated organizational infrastructural interventions to establish the evidence baseline in this area. All of these studies were retrospective case reports. The researchers found that although positive results were reported, there was no offering of any substantive evidence to support the studies' conclusions. Therefore, no clear implication for practice can yet be drawn based on the findings of those studies.

Interactive Strategies

Vingillis et al. (2003) reported the use of knowledge translation strategies that integrated knowledge generation with knowledge diffusion and utilization. During the project's 3-year period, the researchers developed the initial research proposal in response to frustration expressed by local professionals. The researchers also built into the original research proposal a series of three colloquia of potential knowledge users, and they established an open-door policy so that interested parties could request meetings with the team. Vingillis et al. used a partnership culture model in which researchers and potential knowledge users develop a partnership of trust, respect, ownership, and common ground as the fundamental first step to successful knowledge dissemination and utilization. The strategies used by this group included viewing research as the means and not as the end, linking the university and research services to the community, using a participatory research approach, embracing transdisciplinary research and interactions, and using connectors to assist potential knowledge users in identifying knowledge needs and to assist researchers in translating knowledge to users. However, the effectiveness of these strategies in promoting knowledge translation was not reported.

Evidence in Rehabilitation

The evidence to support or dispute the effectiveness of knowledge translation strategies in rehabilitation is very limited and available in the form of original studies only (as opposed to systematic reviews or overviews seen in medicine). Thomas, Cullum, McColl, Rousseau, and Steen (1999), in their searches of primary studies to conduct a systematic review to evaluate the effectiveness of practice guidelines in professions allied to medicine, could not locate such studies in rehabilitation. At present, the studies of knowledge translation strategies' effectiveness in rehabilitation continues to be scarce.

Wolfe, Stojcevic, Rudd, Warburton, and Beech (1997) investigated the uptake of stroke guidelines in a district of southern England using the 18-month audit process. The guidelines were developed through an interdisciplinary collaboration and included rehabilitation standards of services for physiotherapy, occupational therapy, and speech therapy. The guidelines were made available for participating hospitals, although no other specific strategies appeared to have been used to facilitate the adoption of the guidelines. The researchers found that, over the period of 18 months, only modest changes in practice occurred according to the guidelines, and the overall level of rehabilitation service remained low despite the significant increase in occupational therapy services within 72 hours of acute stroke episode. The researchers concluded that despite local development and feedback, adherence to guidelines was relatively poor, there was little improvement over time, and a considerable amount of unmet need continued to exist.

Heinemann et al. (2003) evaluated the effectiveness of a lecture-based educational program about post-stroke rehabilitation guidelines. The knowledge and referral practice questionnaire was used to assess changes in knowledge and practice of physicians and allied health professionals who provided services in acute care settings. The findings indicated no significant increase in knowledge or referrals at 6 months following the educational program. The researchers concluded that simply providing information about the guidelines did not result in an increase in knowledge and referrals.

Fritz, Delitto, and Erhard (2003) conducted a randomized clinical trial comparing the effectiveness of providing therapy based on a classification system matching patients to specific interventions with providing therapy based on the Agency for Health Research and Quality guidelines. At 4 weeks after treatment, the researchers found that the patients who received classification-based therapy showed greater improvement on the general health measure (SF-36) on the physical component score, greater satisfaction, and increased likelihood of returning to full-duty work status. At 1 year, however, there were no statistical differences between the groups in all outcomes.

Pennington et al. (2005) evaluated the effectiveness of two training strategies to promote the use of research evidence in speech and language therapy management of post-stroke dysphagia. The first strategy was the training on critical appraisal of published research papers and practice guidelines, and the second strategy was the same training plus additional training on management of change in clinical practice. The researchers found that, overall, there was no significant change in the adherence to clinical practice guidelines within 6 months of intervention. No other outcomes were measured.

Bekkering et al. (2005), using a cluster-randomized controlled trial, compared an active strategy for implementation of the Dutch guidelines for patients with low back pain with the standard passive method of dissemination of the same guidelines in 113 primary-care physical therapists who treated a total of 500 patients. The passive method group received the guidelines by mail, and the active method group received an additional active training strategy consisting of two sessions of education, group discussion, role-playing, feedback, and reminders. The outcomes were measured through self-reported patients' questionnaires on physical functioning, pain, sick leave, coping, and beliefs. The researchers found that physical functioning and pain in both groups improved substantially in the first 12 weeks but there were no significant differences between the two groups. At a follow-up at 12 months, there continued to be no difference between the two strategies. The researchers concluded that there was no additional benefit to applying active strategy to implement the physical therapy guidelines for patients with low back pain.

Because of the limited evidence, no conclusion or even pattern can be drawn from these studies at this time. The knowledge translation inquiry in rehabilitation appears to be in the beginning stage, and further development in this area is clearly needed. Although some information is available from other health-care fields, such as medicine and nursing, it is not certain how applicable those findings would be for rehabilitation, because of the conceivable difference in practice focus, culture, methods, and contexts.

Next Section: Measures of Knowledge Use

Previous Section: Knowledge Translation Models

Table of Contents