Chapter 5: Justify Conclusions

The next step in program evaluation is to prepare the data for the intended use(s) of the evaluation. Whether your evaluation is conducted to show program effectiveness, help improve the program, or demonstrate accountability, you will need to analyze and interpret your findings.

Analyzing the findings

Data analysis is the process of organizing and classifying the information you have collected, tabulating it, analyzing it, comparing the results with other appropriate information, and presenting the results in an easily understandable manner. There are five steps in data analysis:

Enter the data into a database and check for errors. If you are using a surveillance system such as BRFSS or PRAMS, the data have already been checked, entered, and tabulated by those conducting the survey. If you are collecting data with your own instrument, you will need 1) to select the computer program you will use to enter and analyze the data, and 2) to determine who will enter, check, tabulate, and analyze the data.

Tabulate the data. The data need to be tabulated to provide information (such as a number or percentage) for each indicator. Some basic calculations include determining—

The number of participants.
The number of participants achieving the desired outcome.
The percentage of participants achieving the desired outcome.

Analyze and stratify your data by various demographic variables of interest, such as participants' race, sex, age, income level, or geographic location.

Make comparisons. Use statistical tests to show differences between comparison and intervention groups, between geographic areas, or between the pre-intervention and post-intervention status of the target population.

Present your data in a clear and understandable form. To interpret your findings and make your recommendations, you must ensure that your results are easy to understand and clearly presented. Data can be presented in tables, bar charts, pie charts, line graphs, and maps.

Tips to remember when interpreting your findings [Footnote 29]
•	Interpret evaluation results with the goals of your tobacco control program in mind
•	Keep your audience in mind when preparing the report. What do they need and want to know?
•	Consider the limitations of the evaluation: – Possible biases. – Validity of results. – Reliability of results. – Generalizability of results.
•	Are there alternative explanations for your results?
•	How do your results compare with those of similar programs?
•	Have the different data collection methods used to measure your progress shown similar results?
•	Are your results consistent with theories supported by previous research?
•	Are your results similar to what you expected? If not, why do you think they may be different?

Interpreting the findings

After analyzing your findings, the next step is to examine your results and determine what they actually say about the program. The purpose of the evaluation, the social and political context of your program, and the needs of the stakeholders are all issues to be considered in relation to your results.

Sample benchmarks for performance

To measure your progress within the national context of tobacco prevention and control, you need to compare your data with national data and with the data of other states. Healthy People 2010 objectives provide a starting point for performance measurement from a national perspective. However, a clear set of standards for assessing a tobacco prevention and control program's success in attaining short-term and intermediate outcomes has not been developed. Therefore, it is important to develop a set of standards against which you will measure your progress. Possible standards include—

Needs of participants.

Community values, expectations, and norms.

Program mission and objectives.

Program protocols and procedures.

Changes in selected indicators over time.

Performance by similar programs.

Performance by a control or comparison group.

Resource efficiency.

Mandates, policies, regulations, and laws.

Judgments of participants, experts, and funders.

Institutional goals.

Social equity.

Human rights.

Checklist for justifying your conclusions
	Analyze data using appropriate techniques.
	Check data for errors.
	Consider issues of context when interpreting data.
	Describe plausible mechanisms or pathways toward change.
	Assess results against available literature.
	Compare different methods for consistent findings.
	Consider alternative explanations.
	Compare evaluation results with those of similar programs.
	Use existing standards (e.g., Healthy People 2010 objectives) as a starting point for comparisons.
	Compare program outcomes with those of previous years.
	Compare actual with intended outcomes.
	Document potential biases.
	Examine the limitations of the evaluation.

CDC Evaluation Working Group
www.cdc.gov/eval

Links to non-Federal organizations are provided solely as a service to our users. Links do not constitute an endorsement of any organization by CDC or the Federal Government, and none should be inferred. CDC is not responsible for the content of the individual organization Web pages found at non-Federal links.

Analyzing the findings

Interpreting the findings

Sample benchmarks for performance

Resources