SEDL Logo

Southwest Educational Development Laboratory



Limitations

No research is without limitations; this survey study is no exception. The most apparent limitation in the phase 2 portion of the survey is the low response rate for some stakeholder groups. However, in survey research, no single response rate is considered a standard (Fink, 1995; Fowler, 1993). For some surveys, a response rate of 90 percent is desired; in others, 65 percent is deemed adequate. Mail surveys typically have lower response rates than other types of surveys and because nonresponse may introduce error, researchers should take steps designed to promote responses. Some of these steps include personally contacting potential respondents and asking them to participate, sending a reminder to nonrespondents, assuring respondents of confidentiality, and making the survey short and easy to complete. For this study, all these measures were adopted.

More critical than response rate is the degree to which nonrespondents are similar to respondents. For example, Legislators/Aides had the lowest response rate, and therefore the largest percentage of nonrespondents of the stakeholder groups sampled. This raises the question: Are nonrespondents similar to respondents? If the answer is no, then conclusions may be biased. Unfortunately, this question is difficult to answer. Therefore, caution should be taken with drawing conclusions from the results for the stakeholder groups with the lowest response rates (i.e., RESNA, CEPD, and LEG).

Another limitation is the variable sample sizes as well as response rates within groups. Ideally, the sample would be chosen randomly, stratifying on stakeholder group and with every group having the same response rate. If this were the case, a mean percentage could be calculated for the 'All Stakeholders' category and each stakeholder group would be equally represented. Unfortunately, several of the stakeholder groups were difficult to define (e.g., MEDIA), and others were difficult to personally contact (e.g., LEG). Others were so large or so varied that it would be impossible to identify an inclusive list for the entire group (e.g., CEPD represented different ways local communities choose to address the concerns of people with disabilities, and are not comparable across the country).

In addition to the limitations in regard to sample size and response rate, is the survey questionnaire itself. The questions were few and the content of a limited nature. NCDDR used this survey to gather baseline information and it was important for consumers and stakeholders to respond. Therefore, the decision was made to limit the number and content of the questions, with the intent to follow this survey with another in-depth survey.

Future Research

Phase 1 of this study identified the sources and formats used and preferred by consumers to get information. Phase 2 gives the NCDDR an indication of the ways stakeholder groups get information and share information with their consumers. While comparing the consumer results with stakeholder groups' results showed some alignment, stakeholders do not necessarily use the formats most used or preferred by consumers.

The results of this stakeholder survey will serve as a framework for phase 3 of this survey research, which will focus on NIDRR grantees and their dissemination practices. The survey will be modified to reflect the following:

  • Question six will be reworded to more accurately reflect the sources and formats from which consumers get information;
  • Several open-ended questions will be added at the end of the survey to allow NIDRR grantees an opportunity to provide additional information; and
  • If necessary, nonrespondents will be contacted and asked to complete the survey by telephone.

It bears repeating that while disability research is important for many stakeholders, it is also important for consumers with disabilities and their families. Phase 1 results indicated that consumers think disability research is useful, but have difficulty accessing it. Interestingly, phase 2 results showed that stakeholder groups also value disability research and generally have more ways to find such information. Phase 3 survey activities will gather information from NIDRR grantees to learn more about how disability research information is disseminated. The extent of integration and/or comparability among the three major survey groups of consumers, stakeholders, and NIDRR grantees will be the subject of further analyses.

References

Americans With Disabilities Act of 1990, 42 U.S.C. § 12101 et seq.

The Complete Directory for People with Disabilities 1996-97 (5th ed.). (1996). Lakeville, CT: Grey House Publishing.

Congressional Yellow Book. (1997). Washington: Leadership Directories, Inc.

Fink, A. (1995). How to sample in surveys. Thousand Oaks, CA: SAGE.

Fowler, F. J. (1993). Survey research methods (2nd ed.). Thousand Oaks, CA: SAGE.

Independent Living Research Utilization (ILRU). (1996, January.) Directory of independent living centers and related organizations, Vol. 18. (Available from ILRU, The Institute for Rehabilitation Research, 2323 S. Shepherd Blvd., Suite 1000, Houston, TX 77019).

NCDDR. (1996). A review of the literature on dissemination and knowledge utilization. Austin: Author.

NCDDR. (1997). How do consumers get information they can use? The Research Exchange, 2(4), 1-8.

State Yellow Book. (1997). Washington: Leadership Directories, Inc.

Winston, C. (1995). America's telability media (1st ed.). (Available from the National Telability Media Center, PO Box 1488, Columbia, MO 65205).


Top

Copyright ©2004 Southwest Educational Development Laboratory

About SEDL | Contact SEDL | Terms of Use