 |
 |
|
||||||||
|
|
|
|
|
|
|
||
|
|
|
||||||
|
|
| Home |
Volume 3: No. 1, January 2006
Suggested citation for this article: Mukhtar Q, Mehta P, Brody ER, Camponeschi J, Friedrichs M, Kemple AM, et al. Development of the Diabetes Indicators and Data Sources Internet Tool (DIDIT). Prev Chronic Dis [serial online] 2006 Jan [date cited].
Available from: URL: http://www.cdc.gov/pcd/issues/2006/
jan/05_0109.htm.
Developing a Web-based tool that involves the input, buy-in, and collaboration of multiple stakeholders and contractors is a complex process. Several elements facilitated the development of the Web-based Diabetes Indicators and Data Sources Internet Tool (DIDIT). The DIDIT is designed to enhance the ability of staff within the state-based Diabetes Prevention and Control Programs (DPCPs) and the Centers for Disease Control and Prevention (CDC) to perform diabetes surveillance. It contains information on 38 diabetes indicators (measures of health or factors associated with health) and 12 national- and state-level data sources. Developing the DIDIT required one contractor to conduct research on content for diabetes indicators and data sources and another contractor to develop the Web-based application to house and manage the information. During 3 years, a work group composed of representatives from the DPCPs and the Division of Diabetes Translation (DDT) at the CDC guided the development process by 1) gathering information on and communicating the needs of users and their vision for the DIDIT, 2) reviewing and approving content, and 3) providing input into the design and system functions. Strong leadership and vision of the project lead, clear communication and collaboration among all team members, and a commitment from the management of the DDT were essential elements in developing and implementing the DIDIT. Expertise in diabetes surveillance and software development, enthusiasm, and dedication were also instrumental in developing the DIDIT.
The Diabetes Indicators and Data Sources Internet Tool (DIDIT) is a Web-based resource designed to strengthen the capacity of the staff and partners of state-based Diabetes Prevention and Control Programs (DPCPs) and staff of the Centers for Disease Control and Prevention (CDC) to conduct diabetes surveillance and program evaluation. The tool contains detailed information on 38 diabetes indicators (measures of health or factors associated with health) and their associated data sources (e.g., Behavioral Risk Factor Surveillance System [BRFSS]). The content, design, and function of the DIDIT have been described elsewhere (1). This article describes the process of developing the DIDIT, beginning with the conceptual phase and proceeding through the content and systems phases (Table 1). In so doing, we provide an example for other agencies and organizations as well as other entities within the CDC that are interested in developing a similar tool.
The DIDIT was developed in response to a request from the DPCPs for technical assistance in surveillance and program evaluation (2). In August 2001, a work group comprised of representatives from the CDC’s Division of Diabetes Translation (DDT) and eight DPCPs (representing Alabama, Minnesota, Montana, New Mexico, New York, Oregon, Utah, and Wisconsin) was convened to develop a tool that would provide comprehensive information about 1) diabetes indicators (e.g., foot examinations, diabetes-related hospitalizations) and 2) their associated data sources (e.g., BRFSS, Medicare) in one centralized place. The work group generated a list of 55 indicators (later reduced to 38) and developed an outline describing the desired contents and format of a tool called the diabetes indicator tool.
In October 2001, an overview and vision of the DIDIT was presented to six focus groups at the annual meeting of DPCP directors. Three key themes emerged:
Input from these initial focus groups and subsequent feedback from the DPCPs served as the basis for developing the concept, content, and Web application for the DIDIT.
Content development took place in two stages. During the first stage, the work group selected 10 of the originally identified 55 indicators to develop a prototype. The 10 indicators were as follows: 1) diabetes prevalence, 2) annual hemoglobin A1c test, 3) annual influenza vaccination, 4) pneumococcal vaccination, 5) level of diabetes education, 6) diabetes-related hospitalizations, 7) prevalence of end-stage renal disease, 8) hospitalization for lower extremity amputations, 9) physical inactivity, and 10) overweight. Because members of the work group lived in different states, discussions were conducted through a series of telephone conferences and two in-person meetings.
During the second stage, the work group selected an additional 28 indicators from the original 55 through a two-round modified Delphi process. Indicators were ranked in priority according to the following four criteria:
The typical reason for excluding an indicator was that no state-level data source could be identified to measure it. A list of indicators that were excluded and the rationale for excluding them can be found on the DIDIT (available from www.cdc.gov/diabetes/statistics/index.htm). All 10 indicators used to develop the prototype as well as all 28 selected during the second stage were retained, with a total of 38 indicators selected for inclusion. At this stage, the selection of fields to describe each indicator (e.g., definitions of indicators) and data source (e.g., its methodology for data collection, data access) was also finalized with input from the DIDIT work group.
After selecting indicators, associated data sources, and their related fields of information, the DIDIT team, including work group members, contractors for both content and Web site development, and DDT leaders, met in January 2003 (Table 1). The major focus of the meeting was to develop a process to ensure that the information included in the DIDIT was accurate and complete. The work group viewed the Web-based prototype with 10 indicators and eight associated data sources. Each work group member was assigned a set of indicators from the group of 38 to review and revise.
During content development, a contractor researched information on selected indicators and their associated data sources, and the work group provided feedback on its accuracy and completeness. Comments and suggestions on content were discussed during monthly conference calls, and the content was approved by all work group members before it was finalized. A second in-person meeting was held to review and refine content and to ensure that the process of reviewing content was efficient and effective. The protocol for developing and refining content resulted in valuable end-user feedback. Development of the Web-based application took place at the same time as development of content.
The parallel development of the application and its content relied on the software development life cycle (Table 2). This process included the following four major phases: 1) planning, 2) analysis, 3) design, and 4) implementation (6). We adopted an iterative and incremental approach, with overlap between analysis, design, and implementation.
The primary purpose of project planning was to articulate the objectives and scope of the DIDIT and ensure the technical feasibility of the system. We noted previously that the purpose and vision for the DIDIT were defined early during the concept development phase. During the project planning phase, the work group articulated the purpose and vision to the technical contractors, and they produced a document that outlined the definition and scope of the project; this document served as a blueprint for system development efforts. Additional activities in this phase included confirming the project’s operational and organizational feasibility, developing a schedule, estimating costs, and allocating resources. The project planning phase culminated in the development of a final project plan, which was reviewed by the project lead and management staff. After the plan was approved, technical development efforts, or systems analysis, for the DIDIT were initiated.
The analysis phase defined in detail what the information system needed to accomplish to provide users with the benefits they desired. Several storyboards were created to display preliminary design options for Web page content and format. The storyboards were uploaded onto a secure Internet site to allow sharing among and feedback from a geographically dispersed user group. As design options were presented, users were quickly able to provide comments. An iterative feedback process allowed further revisions to DIDIT storyboards. System requirements were prioritized as they were identified. The analysis phase culminated with the development of model diagrams, which were used to drive the next phase, system design.
While the analysis phase focused on what the system should do, the design phase focused on how the system should function. Information from the analysis phase was used to design the application, the database, the user interface, and the operating environment. The application and database were designed in parallel with the user interface. The user interface is a critical component because it ensures ease of use. It was designed with stakeholder input to ensure that the final product would reflect stakeholder needs.
During the implementation phase, a demonstration model was built, tested, and released with information on the 10 pilot indicators and associated data sources. The model included core functionalities such as the ability to browse, sort, and search, and it was demonstrated at the first in-person work group meeting in early 2003. Input was solicited on additional features, including the addition of DPCP-specific data sources, system-searching functions, and report formats.
Phase 4a: Merging the content and the Web system
After the demonstration model was developed, the development of the content and the Web system converged. The work group began using the DIDIT to review and refine the content of indicators and data sources. The contractor responsible for researching and developing the content delivered a data set on indicators and data sources to the systems developers for upload into the DIDIT. Once uploaded, the content was reviewed by the work group, and appropriate modifications were made. This process took place iteratively between February and April 2003, with a total of three uploads, until all 38 indicators and 12 data sources had been uploaded, reviewed, and finalized. An unexpected positive outcome from this process was that as the work group reviewed DIDIT content, it also tested and evaluated system functionality and design, leading to several important changes.
|
Before the DIDIT was formally implemented in September 2003, both usability testing and pilot testing were conducted to obtain user feedback about system design and usage to further refine the new tool. Usability testing was conducted in April 2003 at the DDT national conference, with pilot testing in July and August 2003 through conference calls and NetMeeting (Microsoft Corp, Redmond, Wash). NetMeeting allowed participants throughout the nation to view the DIDIT as it was being demonstrated at the CDC in Atlanta, Ga. Pilot testing is a critical and often overlooked component of the software life cycle, and there are important reasons for conducting it (Sidebar). The objective of usability testing and pilot testing was to formally validate that system function, design, and content were consistent with the needs of users as determined during earlier phases. Feedback from these processes was evaluated and used to make further refinements to the system before its final release in September 2003.
Phase 4b: System maintenance and training
Shortly after the release of the DIDIT, the project lead conducted a national training session for DPCPs and CDC staff using NetMeeting. A team of DDT professionals was then assigned the responsibility of providing ongoing user support and training for technical and functional aspects of the DIDIT. The project lead’s responsibilities included providing support on questions and issues related to the content and application of the DIDIT in the context of DPCP programs.
Several factors were critical to successfully developing and implementing the DIDIT. The factors have practical implications for other agencies that want to undertake a similar effort.
The work group members had extensive knowledge and experience in diabetes surveillance and epidemiology, which proved essential in guiding the content and technical contractors during the development process. DIDIT team members were a motivated, dedicated, enthusiastic, and knowledgeable group of DPCP representatives and DDT staff. In addition, the knowledge and skills of the contractor were critical to researching and developing content on indicators and data sources.
The project lead effectively solicited the interest and support of DDT management to ensure that financial and staff resources were available to develop the new tool. To sustain interest and support of management, the project lead presented draft content and DIDIT prototypes at various CDC and national public health meetings throughout the development process (Table 1). These presentations allowed management to realize the high level of interest among prospective users and the potential for the DIDIT as an important tool for diabetes surveillance. Updates were also shared with DDT management on an ongoing basis.
Development of a comprehensive reference tool such as the DIDIT requires a commitment of time and resources. The management of the DDT supported allocation of resources and time needed to create the DIDIT.
The DIDIT project lead had a clear vision of the type of tool that would fulfill the surveillance needs of the DPCP and the DDT. A strength of the project lead was her ability to communicate the vision of the DIDIT to the project team and stakeholders throughout the development process.
Development of the DIDIT involved input from stakeholders across the country. Clear and ongoing communication among stakeholders was essential to the development process. During the first in-person work group meeting, we learned that face-to-face interactions were highly appreciated by work group members and that these interactions helped build rapport among members. In-person meetings were arranged at national conferences to avoid issues of travel approval and costs. Timelines and other defined plans facilitated collaboration. A contractor who was skillful at organizing materials, facilitating meetings, motivating work group members, and responding to the needs of work group members was also essential. Because the work group volunteered its time to create the DIDIT, efforts were made to minimize the burden placed on its members. Minimizing this burden helped to maintain a core group of members who have actively participated for more than 3 years.
Development of both content and Web application took place incrementally and iteratively. Members of the work group reviewed the content in phases, allowing the content contractor to apply feedback to subsequent phases. Similarly, because an incremental process was conducted that involved analysis, design, and implementation at the same time, the contractors were able to make a demonstration model of the DIDIT during the early phases of development, which facilitated refinements to its content and design. Working with an actual tool triggered ideas among users for additional functions and alternative designs that may have been overlooked at the prototyping stage. A model also allowed us to obtain user input on database-driven features such as system searches.
The ability to assess the status of the public’s health in a timely, consistent, and accurate manner satisfies the first two of the 10 essential public health services as defined by the Institute of Medicine: 1) “monitor health status to identify community health problems” and 2) “diagnose and investigate health problems and health hazards in the community” (7).
The DIDIT represents an innovative approach to enhancing the capacity of state and federal agencies to perform public health surveillance. As one user has described, “The DIDIT offers a one-stop shop that is available 24 hours a day.” It empowers users by providing them easy access to information that has been reviewed by DIDIT work group members for accuracy and content. In addition to providing a road map for development, this article highlights components that were critical to the successful development of the DIDIT. These components synergistically influenced the development process. Having adequate time, expertise, and commitment of resources, for example, would not have been sufficient for success without the clear communication and rapport among the project team members or buy-in and involvement of all stakeholders. Because these critical factors enhance one another, it is difficult to prioritize them. Other entities that wish to undertake a similar effort of systems development can use these requirements as guiding principles and customize them for their own needs and circumstances.
A major benefit of sharing these elements is to prevent other agencies from having to “reinvent the wheel” when they can draw directly on the experiences of the DIDIT team. While the technology is available to develop information technology solutions for addressing public health problems, it is vital to have effective processes and methods in place to successfully identify the needs of users and harness and customize appropriate technology to meet those needs.
We thank the following individuals at the DDT: Stephanie Benjamin, Surveillance Coordinator; Rich Gerber, Deputy Branch Chief; and Ed Gregg, Epidemiologist. In addition, we thank Tom Chapel, the CDC, and all DPCPs that participated in DIDIT usability and pilot testing.
Corresponding Author: Qaiser Mukhtar, PhD, Division of Diabetes Translation, Centers for Disease Control and Prevention, 4770 Buford Hwy, Mail Stop K-10, Atlanta, GA 30341. Telephone: 770-488-5505. E-mail: qam2@cdc.gov.
Author Affiliations: Prachi Mehta, DrPH, Northrop Grumman Mission Systems, Atlanta, Ga; Erica R. Brody, MPH, RTI International, Research Triangle Park, NC; Jenny Camponeschi, MS, Wisconsin Department of Health and Family Services, Division of Public Health Diabetes Prevention and Control Program, Madison, Wis; Michael Friedrichs, MS, Brenda Ralls, PhD, Diabetes Prevention and Control Program, Utah Bureau of Health Promotion, Utah Department of Health, Salt Lake City, Utah; Angela M. Kemple, MS, Oregon Department of Human Services, Diabetes Prevention and Control Program, Portland, Ore.
|
|
|
|
|
|
|
|
The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors’ affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.
Privacy Policy | Accessibility This page last reviewed March 30, 2012
|
|
