CDC Evaluation Plan: Executive Summary

Introduction to the NBCCEDP

Evidence shows that deaths from both breast and cervical cancers can be reduced by increasing the use of screening services—mammography and Pap tests—among women. However, screening is underutilized among women who are under- or uninsured, have no regular source of healthcare, or have recently immigrated to the U.S.¹ The Centers for Disease Control and Prevention’s (CDC) recent funding opportunity announcement (FOA), Cancer Prevention and Control Programs for State, Territorial, and Tribal Organizations (DP17-1701), supports implementation of a comprehensive, coordinated approach to inform policy, systems, and environmental change strategies to prevent and control cancer. The five-year FOA (DP17-1701, 2017–2022) comprises three distinct national programs: (1) the National Breast and Cervical Cancer Early Detection Program (NBCCEDP); (2) the National Comprehensive Cancer Control Program; and (3) the National Program of Cancer Registries. This document details a plan for evaluation and performance monitoring of the NBCCEDP.

Seventy grantees, including 50 states, the District of Columbia, 13 tribes/tribal organizations, and 6 U.S. territories, received CDC funding for the NBCCEDP ranging from $137,500 to $7,721,730 in program year one. The NBCCEDP grantees are charged with implementing a program to provide breast and cervical cancer screening services to eligible women, and implement evidence-based interventions (EBIs) to reduce barriers to cancer screening within health systems and their clinics. Women eligible to receive program-funded screening and diagnostic services are: (1) at or below 250% of the federal poverty level; (2) aged 40 to 64 years for breast cancer services, or aged 21 to 64 years for cervical cancer services; and (3) under- or uninsured. Under DP17-1701, the NBCCEDP expands its previous scope to support grantees in partnering with health systems and their clinics (that have low screening rates and serve disadvantaged populations) to implement EBIs to increase breast and cervical cancer screening rates.

Evaluation of the NBCCEDP

CDC will evaluate the NBCCEDP in order to track progress and measure outputs and outcomes. The three main purposes of evaluation and performance monitoring of the NBCCEDP are to: (1) improve grantee programs; (2) strengthen CDC’s accountability to the public and Congress, as well as grantees’ accountability to CDC; and (3) inform future program planning and policy-making. CDC’s program evaluation team developed this evaluation plan for the NBCCEDP. The evaluation plan is based on CDC’s Framework for Program Evaluation (Figure 1) and is guided by the NBCCEDP logic model [PDF-53KB]. In addition to program management requirements, the logic model provides a visual representation of the seven strategies that grantees are expected to implement:

• Program collaboration.
• External partnerships.
• Cancer data and surveillance.
• Environmental approaches for sustainable cancer control.
• Community-clinical linkages.
• Health systems change and provider-focused activities.
• Program monitoring and evaluation.

We expect the implementation of these strategies to contribute to multiple short-, intermediate-, and long-term outcomes, including the use of multiple EBIs to support high-quality screening and increased breast and cervical screening among priority populations. A variety of internal stakeholders (e.g., U.S. Congress, CDC Division- and Branch-level leadership, CDC program consultants) and external stakeholders (e.g., NBCCEDP grantees and national partner organizations) will be engaged during evaluation planning, implementation, and the dissemination of findings. CDC will also prioritize increasing grantees’ capacity to participate in national evaluation of the NBCCEDP, as well as to plan and implement evaluations of their individual programs.

Design

The CDC evaluation team will conduct a multi-component, mixed methods evaluation of the NBCCEDP to include data collection at the grantee, health system clinic, and patient levels. The primary outcomes of interest include: (1) the number of women served and their screening outcomes, and (2) changes in clinic-level breast and cervical cancer screening rates. The CDC evaluation team developed a comprehensive list of evaluation questions that address the following general focus areas:

• Patient screening, including screening quality and results.
• Clinic-level screening rate changes.
• Patient demographics and eligibility criteria.
• Characteristics of clinics where EBIs are implemented.
• EBI implementation and sustainability.
• Program reach.
• Partnerships to support screening.
• Community outreach and patient navigation.
• Support of healthy lifestyle behaviors.
• Grantee infrastructure.
• Grantees’ monitoring and evaluation.

Methods

CDC will conduct annual data collection combined with periodic special studies, conduct analysis, and generate routine reports to monitor incremental program progress (Figure 2). Annual data sources include primary data collected systematically at the patient, clinic, and grantee levels, including the minimum data elements, baseline and annual clinic data, and an annual grantee survey. Secondary data sources include routine financial reports (i.e. grantee program budgets, Federal Financial Reports) and grantee program reports (i.e. evaluation plans, implementation plans). CDC will utilize a variety of quantitative and qualitative analytic methods to summarize and interpret these data. Periodic special studies may include case studies and cost analyses.

Use of Findings and Dissemination

The use of routine and periodic evaluation findings will vary by stakeholders. We anticipate that federal stakeholders (e.g., U.S. Congress, CDC leadership) would be most interested in the program’s reach to priority populations, patient screening results, changes in clinic-based breast and cervical screening rates, and cost-related analyses of program strategies. CDC program consultants would utilize findings to inform the technical assistance they provide to grantees. Grantees will use findings to inform program planning and improve implementation practices to maximize program outcomes. Other external stakeholders (e.g., national partners, the general public) would be interested in better understanding program reach at the local level and screening results across specific priority populations.

Reference

¹Centers for Disease Control and Prevention. Cancer screening—United States, 2010. MMWR 2012;61(3):41–45.