|
|
Chapter 5: Justify Conclusions
The next step in program evaluation is to prepare the data for the
intended use(s) of the evaluation. Whether your evaluation is conducted to
show program effectiveness, help improve the program, or demonstrate
accountability, you will need to analyze and interpret your findings.
Data analysis is the process of organizing and classifying the
information you have collected, tabulating it, analyzing it, comparing the
results with other appropriate information, and presenting the results in an
easily understandable manner. There are five steps in data analysis:
- Enter the data into a database and check for errors. If you are using
a surveillance system such as BRFSS or PRAMS, the data have already been
checked, entered, and tabulated by those conducting the survey. If you are
collecting data with your own instrument, you will need 1) to select the
computer program you will use to enter and analyze the data, and 2) to
determine who will enter, check, tabulate, and analyze the data.
- Tabulate the data. The data need to be tabulated to provide
information (such as a number or percentage) for each indicator. Some
basic calculations include determining
- The number of participants.
- The number of participants achieving the desired outcome.
- The percentage of participants achieving the desired outcome.
- Analyze and stratify your data by various demographic variables of
interest, such as participants' race, sex, age, income level, or
geographic location.
- Make comparisons. Use statistical tests to show differences between
comparison and intervention groups, between geographic areas, or between
the pre-intervention and post-intervention status of the target
population.
- Present your data in a clear and understandable form. To interpret
your findings and make your recommendations, you must ensure that your
results are easy to understand and clearly presented. Data can be
presented in tables, bar charts, pie charts, line graphs, and maps.
Return to top of page
|
Tips to
remember when interpreting your findings
[Footnote 29] |
|
Interpret evaluation results with the goals of
your tobacco control program in mind |
|
Keep your audience in mind when preparing the
report. What do they need and want to know? |
|
Consider the limitations of the evaluation:
Possible biases.
Validity of results.
Reliability of results.
Generalizability of results.
|
|
Are there alternative explanations for your
results? |
|
How do your results compare with those of
similar programs? |
|
Have the different data collection methods used
to measure your progress shown similar results? |
|
Are your results consistent with theories
supported by previous research? |
|
Are your results similar to what you expected?
If not, why do you think they may be different? |
|
|
|
After analyzing your findings, the next step is to examine your results
and determine what they actually say about the program. The purpose of the
evaluation, the social and political context of your program, and the needs
of the stakeholders are all issues to be considered in relation to your
results.
Return to top of page
To measure your progress within the national context of tobacco
prevention and control, you need to compare your data with national data and
with the data of other states. Healthy People 2010 objectives provide
a starting point for performance measurement from a national perspective.
However, a clear set of standards for assessing a tobacco prevention and
control program's success in attaining short-term and intermediate outcomes
has not been developed. Therefore, it is important to develop a set of
standards against which you will measure your progress. Possible standards
include
- Needs of participants.
- Community values, expectations, and norms.
- Program mission and objectives.
- Program protocols and procedures.
- Changes in selected indicators over time.
- Performance by similar programs.
- Performance by a control or comparison group.
- Resource efficiency.
- Mandates, policies, regulations, and laws.
- Judgments of participants, experts, and funders.
- Institutional goals.
- Social equity.
- Human rights.
Checklist for
justifying your conclusions |
|
Analyze data using appropriate techniques. |
|
Check data for errors. |
|
Consider issues of context when interpreting data. |
|
Describe plausible mechanisms or pathways toward change. |
|
Assess results against available literature. |
|
Compare different methods for consistent findings. |
|
Consider alternative explanations. |
|
Compare evaluation results with those of similar programs. |
|
Use existing standards (e.g., Healthy People 2010
objectives) as a starting point for comparisons. |
|
Compare program outcomes with those of previous years. |
|
Compare actual with intended outcomes. |
|
Document potential biases. |
|
Examine the limitations of the evaluation. |
|
Return to top of page
- CDC Evaluation Working Group
www.cdc.gov/eval
|
Links to non-Federal organizations are provided solely as a service to our users. Links do not constitute an endorsement of any organization by CDC or the Federal Government, and none should be inferred.
CDC is not responsible for the content of the individual organization Web pages found at
non-Federal
links. |
|
Return to top of page |
|