National Evaluation of Welfare-to-Work Strategies

How Effective Are Different Welfare-to-Work Approaches?
Five-Year Adult and Child Impacts for Eleven Programs:

Chapter 13:
Costs and Benefits

[ Main Page of Report | Contents of Report ]

Contents

  1. Key Findings
  2. Issues in the Cost Analysis
  3. Major Components of the Cost Analysis
  4. Expenditures for In-Program Employment-Related Services
    1. Welfare Department Expenditures
    2. Non-Welfare Agency Expenditures
  5. Expenditures for Out-of-Program Employment-Related Services
    1. Welfare Department Expenditures
    2. Non-Welfare Agency Expenditures
  6. Gross Costs
    1. Gross Costs for Program Group Members
    2. Gross Costs for Control Group Members
  7. Net Costs
    1. Results by Educational Attainment Subgroups
    2. LFA-HCD Program Differences
  8. Analytical Approach for the Benefit-Cost Analysis
    1. Accounting Methods
    2. Analytical Perspectives
    3. Limitations of the Analysis
  9. Effects for Sample Members
    1. Earnings and Fringe Benefits
    2. Personal Taxes
    3. Transfer Payments and Benefits
  10. Net Gains and Losses by Accounting Perspective
    1. Welfare Sample Members
    2. Government Budgets
  11. Sensitivity of the Results
  12. Conclusions

Endnotes

Earlier chapters described the implementation of the NEWWS programs and the effects on sample members and their families. In sum, NEWWS programs increased participation in employment-related activities and produced some positive effects for program group members, including higher earnings and less reliance on welfare. Overall, the NEWWS programs had little impact — positive or negative — on the well-being of the children of program group members. This chapter presents an analysis of the costs of providing the array of services that produced these effects and uses the results of the cost analysis to examine the net financial benefits and costs of the NEWWS programs from two perspectives: program group members and government budgets. The benefit-cost analysis includes key financial effects discussed in earlier chapters, such as effects on earnings, cash assistance payments, and Food Stamp payments, and expands the scope to consider effects such as fringe benefits from employment, taxes, and Medicaid coverage.

This chapter extends the analysis of impacts on use of program services (Chapter 3) and impacts on earnings, welfare and Food Stamps, and combined income (Chapters 4-6). It examines the costs and benefits of 10 of the 11 NEWWS programs, those for which five years of follow-up data were available from administrative records. (Owing to data limitations five-year costs and benefits were not estimated for Oklahoma City.)

The chapter presents details on the analysis of the costs of running the NEWWS programs, discusses the financial benefits of NEWWS, and compares the benefits and costs from the two perspectives mentioned above.

[Go To Contents]

I. Key Findings

The costs presented in this chapter consist of all costs associated with providing employment services and related support services to sample members. The gross cost per program group member consists of costs paid by the welfare department and non-welfare agencies while sample members were enrolled in NEWWS programs, as well as for employment and support services after they exited the programs and left the welfare rolls.

The net cost per program group member is the gross cost per program group member minus what would have been spent in the absence of the NEWWS program — the gross cost per control group member. As discussed in Chapter 3, a sizable number of controls participated on their own in community-provided employment-related activities, usually vocational training or post-secondary education.

This is not surprising, given the high rates of participation in higher-cost education activities, such as post-secondary education and vocational training. The average cost of the NEWWS programs was comparable to other high-cost programs: the Alameda and Los Angeles Greater Avenues for Independence (GAIN) programs operated in the late 1980s and early 1990s.

This comparison is based on the three sites where head-to-head LFA-HCD tests were conducted: Atlanta, Grand Rapids, and Riverside. Because the Riverside HCD sample includes only those who did not have a high school diploma or GED when they entered the evaluation, the LFA-HCD comparison in this site is limited to this subgroup.

In all programs, gross costs per program group member were higher for the subgroup that entered the evaluation with a high school diploma or GED. As expected, they were more likely to participate in post-secondary education or vocational training activities, whereas those lacking these credentials were more likely to participate in basic education activities, which typically cost less. The differences in net costs between the subgroups were less pronounced, but still higher for those with a high school diploma or GED.

From the perspective of the welfare recipients subject to the programs, the benefit-cost findings show that:

From the perspective of government budgets, the findings show that:

[Go to Contents]

II. Issues in the Cost Analysis

The primary purpose of the cost analysis is to estimate the cost of providing NEWWS services, over and above the cost that would have been incurred in the absence of the program — that is, to estimate the average net cost per program group member. The net cost is the difference between the gross cost per program group member and the gross cost per control group member, where the gross costs reflect the cost of all services that sample members used in the NEWWS programs and of the education and training services that they used outside the programs, when they were no longer receiving welfare benefits. In other words, the cost for the control group is the benchmark used to determine the additional costs incurred as a result of the NEWWS programs.

This report updates cost figures based on two years of follow-up data presented in earlier reports.(1) (As noted above, data limitations did not allow inclusion of Oklahoma City in the five-year analysis.) Table 13.1 shows that, at two years, education-focused programs were more costly to operate than employment-focused programs. The average net cost was $1,846 (in 1999 dollars) for the employment-focused programs and $2,523 for the education-focused programs. Similarly, the net costs were 1.5 to 2.6 times greater for the HCD programs than for the LFA programs. The earlier reports presented only program costs because a full cost-benefit accounting with only two years of data was considered premature, in that the total return on program investments would be evident only after several years.

Table 13.1
Estimated Gross and Net Costs Within a Two-Year Follow-Up Period,
by Program and Service Component (in 1999 dollars)

Program and Component

 Total Gross Cost per Program Group Member ($) Total Gross Cost per Control Group Member ($) Total Net Cost per Program Group Member ($)

Atlanta Labor Force Attachment
Orientation and appraisal 72 n/a 72
Formal assessment n/a n/a n/a
Job search 816 8 807
Basic education 507 58 449
Post-secondary education 289 315 -26
Vocational training 751 443 309
Work experience 174 20 152
Subtotal (operating) 2,609 843 1,765
Child care 883 291 591
Other support services 193 17 176
Total 3,685 1,152 2,533

Atlanta Human Capital Development
Orientation and appraisal 72 n/a 72
Formal assessment n/a n/a n/a
Job search 194 8 185
Basic education 1,263 58 1,205
Post-secondary education 300 315 -14
Vocational training 1,792 443 1,350
Work experience 126 20 105
Subtotal (operating) 3,746 843 2,903
Child care 806 291 515
Other support services 413 17 397
Total 4,966 1,152 3,814

Grand Rapids Labor Force Attachment
Orientation and appraisal 18 n/a 18
Formal assessment 9 n/a 9
Job search 823 78 745
Basic education 819 793 27
Post-secondary education 1,944 1,715 229
Vocational training 730 841 -111
Work experience 122 12 110
Subtotal (operating) 4,465 3,438 1,026
Child care 408 230 177
Other support services 30 n/a 30
Total 4,903 3,669 1,233

Grand Rapids Human Capital Development
Orientation and appraisal 18 n/a 18
Formal assessment 312 n/a 312
Job search 289 78 210
Basic education 2,064 793 1,272
Post-secondary education 2,123 1,715 408
Vocational training 1,205 841 364
Work experience 213 12 201
Subtotal (operating) 6,224 3,438 2,786
Child care 603 230 372
Other support services 37 n/a 37
Total 6,865 3,669 3,195

Riverside Labor Force Attachment
Orientation and appraisal 111 n/a 111
Formal assessment 6 n/a 6
Job search 877 45 832
Basic education 172 106 67
Post-secondary education 707 421 286
Vocational training 237 243 -6
Work experience 56 65 -10
Subtotal (operating) 2,164 878 1,286
Child care 98 33 66
Other support services 54 n/a 54
Total 2,316 911 1,405

Riverside Human Capital Development(without a high school diploma or GED)
Orientation and appraisal 107 n/a 107
Formal assessment 13 n/a 13
Job search 729 43 685
Basic education 2,340 181 2,158
Post-secondary education 186 150 36
Vocational training 236 258 -22
Work experience 63 30 33
Subtotal (operating) 3,674 662 3,012
Child care 183 16 166
Other support services 82 n/a 82
Total 3,938 678 3,260

Columbus Integrated
Orientation and appraisal 17 n/a 17
Formal assessment n/a n/a n/a
Job search 79 7 72
Basic education 516 89 427
Post-secondary education 1,163 270 892
Vocational training 693 226 467
Work experience 82 7 76
Subtotal (operating) 2,550 599 1,952
Child care 569 346 221
Other support services 221 11 211
Total 3,340 956 2,384

Columbus Traditional
Orientation and appraisal 9 n/a 9
Formal assessment n/a n/a n/a
Job search 100 7 93
Basic education 632 89 543
Post-secondary education 1,046 270 775
Vocational training 313 226 87
Work experience 63 7 57
Subtotal (operating) 2,163 599 1,564
Child care 552 346 205
Other support services 148 11 138
Total 2,863 956 1,907

Detroit
Job search 265 47 218
Education and training a 3,541 1,890 1,651
Work experience 78 10 68
Subtotal (operating) 3,884 1,947 1,938
Child care 407 337 71
Other support services 61 17 45
Total 4,354 2,300 2,053

Oklahoma City
Orientation and appraisal n/a n/a n/a
Formal assessment n/a n/a n/a
Job search 127 46 80
Basic education 553 276 277
Post-secondary education 814 738 77
Vocational training 1,048 625 423
Work experience 34 14 20
Subtotal (operating) 2,576 1,698 878
Child care 593 506 87
Other support services 87 n/a 87
Total 3,255 2,204 1,051

Portland
Orientation and appraisal 149 n/a 149
Formal assessment n/a n/a n/a
Job search 468 47 422
Basic education 543 337 206
Vocational training and post-secondary education 1,358 1,159 199
Work experience 247 37 210
Subtotal (operating) 2,766 1,580 1,186
Child care 1,582 629 953
Other support services 79 6 73
Total 4,427 2,215 2,212
SOURCES:  MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton County Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia Department of Technical and Adult Education, the Board of Regents University System of Georgia; Grand Rapids - the Michigan Department of Social Services, the Michigan Department of Education Office of Extended Learning Services, the Grand Rapids Community College, the Wyoming Community Education Center; Riverside - the California Department of Social Services, the California Department of Education, the Chancellor’s Office of California Community Colleges; Columbus - the Franklin County Department of Human Services, the Ohio Department of Education, the Office of Vocational and Adult Education, the Ohio Board of Regents, the National Center for Education Statistics; Detroit - the Michigan Family Independence Agency, the Michigan Department of Education, the Michigan Jobs Commission; Portland - the Oregon Department of Human Resources, Adult and Family Services Division, the Oregon Office of Community College Services; in all sites - information collected on tuition charged at proprietary schools attended by sample members, from MDRC-collected JOBS case file data, the MDRC Two-Year Client Survey, and the MDRC Five-Year Client Survey. MDRC child care and other support service calculations from Fulton County, Kent County, Riverside County, Ohio Department of Human Services, Wayne County, Washington County, and Multnomah County(District 2) payment data, other support service data from country records.

NOTES:  Rounding may cause slight discrepancies in calculating sums and differences.
N/a = not applicable.
a Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of "education and training." Furthermore, in this site orientation and assessment costs are spread across these three activities.

As described in the earlier reports, costs per sample member are the product of unit costs and behavioral variables. The unit cost of an activity is an estimate of the average cost of serving one person in a specified activity for a specified unit of time (one month or one hour, for example). In general, unit costs were calculated by dividing expenditures for an activity (or service) during a steady-state period by the total number of participant-months in that activity during the same period. The number of participant-months was obtained by counting the number of participants in an activity in each month of the steady-state period and summing across the months.(2) Once the unit cost of an activity was determined, it was multiplied by the average number of months that sample members spent in the activity, called the behavioral variable, to determine the average cost incurred per program group member or control group member during the follow-up period.

The costs presented here were calculated using the same unit costs calculated at the two-year point. However, the behavioral variables used in this analysis cover the five-year period following each sample member's entry into the study. Behavioral variables for years 3 to 5 of follow-up were estimated using Two-Year and Five-Year Client Survey data, as well as administrative records data. The Five-Year Client Survey was not administered in Columbus and Detroit. Therefore, in Columbus, behavioral variables were estimated using participation trends from the two-year and five-year follow-up points in the education-focused sites where longer-term data were available. Because Detroit shifted program focus from education to employment mid-follow-up, it was considered inappropriate to estimate participation for this program using data from the other education-focused programs. Three years of participation data were available from the management information system maintained by the Detroit Work First program. Participation in years 4 and 5 of the follow-up period was estimated based on patterns of participation and welfare receipt in the earlier period.

As discussed in Chapter 2, in several sites service embargoes for control group members were lifted during the follow-up period. As a result, in those sites some control group members were required to participate in employment-related activities in follow-up year 4 or 5. Welfare receipt data were used to determine whether participation should be counted as in-program or out-of-program for both program group members and control group members for whom embargoes were lifted. In addition to the cost of providing the activity, in-program participation incurs an additional cost for case management.

As noted above, costs are estimated for the five-year period following sample members' entrance into the study. Later in the chapter, to assess whether the programs in the NEWWS Evaluation were cost-effective from the perspective of the government's budget, this five-year net cost is compared with the value of any budgetary savings during the same period (for example, from lower welfare or Food Stamp payments) and of any tax revenue increases associated with the additional earnings of program group members.

The costs presented here include the costs of program services as well as the costs of employment-related services that sample members used outside the programs when they were not receiving welfare. The off-welfare costs are important because they represent an additional investment of resources that could have differentially affected program and control group members' future earnings and welfare receipt (effects that are accounted for in the benefit-cost analysis).

All sample members, not just those who participated in mandatory welfare-to-work program services, were included in calculating the net costs because the requirement to participate may have affected some recipients' behavior: Some people may have chosen to avoid the participation mandate by finding a job on their own or by leaving the welfare rolls. In addition, sample members who did not participate in welfare-to-work program services may have taken part in education and training services on their own, and these costs need to be taken into account as well.(3)

Owing to the fact that findings might be expected to differ according to sample members' educational background, all results are presented for the full samples for each program and then separately for those with and without a high school diploma or GED at the time they entered the evaluation. As noted in earlier chapters, all HCDs in Riverside had no high school diploma or GED. Therefore, the Riverside LFA-HCD comparisons include only those without a high school diploma or GED.

[Go To Contents]

III. Major Components of the Cost Analysis

The costs presented in this chapter consist of all costs associated with providing employment services and related support services to sample members. Figure 13.1 illustrates the cost components. For each group of sample members, costs were calculated for employment-related services that sample members participated in when they were receiving welfare (in-program) and services that they participated in when they were not receiving welfare (out-of-program). The employment-related services are divided into those that were paid for by the welfare department, either directly or indirectly, and those that were paid for by other agencies in the community; this information may be useful to administrators and planners who want to understand the nature of the government's investment.

Figure 13.1
Major Components of Gross and Net Costs for Employment-Related Services

Mmmajor Components of Gross and Net Costs for Employment-Related Services

The remainder of the cost portion of the chapter follows the organization of Figure 13.1, beginning with in-program expenditures paid by the welfare departments and ending with the net cost per program group member, which is the sum of the program costs less the costs of services used by control group members.

[Go To Contents]

IV. Expenditures for In-Program Employment-Related Services

Across the programs, costs varied widely for employment-related services for sample members when they were receiving welfare — from less than $4,000 in the Riverside LFA and Detroit programs to more than $7,000 in the Grand Rapids and Atlanta HCD programs. The welfare department paid some of these costs, with the remainder picked up by various other agencies in the community. The proportion that was paid for out of welfare department funds varied across programs, from 27 percent in the Grand Rapids LFA program to 63 percent in the Portland program. This section examines these expenditures in more detail. (See Figure 13.1, Boxes 1 and 2.)

A. Welfare Department Expenditures

Welfare department costs consisted of program operating costs and the costs of support services that sample members received to enable their participation in employment and employment-related activities.

1. Operating costs. The welfare departments paid for the day-to-day operation of the NEWWS programs, including expenditures for employment-related case management services, overhead, program orientation, and other activities. These expenditures cover services provided directly by welfare department staff as well as services provided by other agencies under contract to the welfare departments. Welfare department staff directly provided case management (following up on recipients who failed to attend scheduled appointments, providing employability planning, and referring and monitoring individuals assigned to activities). In some sites, assessments, job search assistance, and job development services were provided by welfare department staff, but in others these activities were provided by contracted outside agencies. Costs incurred by the welfare department to accommodate NEWWS research requirements and requests were excluded from the analysis.

Table 13.2 shows the unit costs (in 1999 dollars) — that is, estimates of the average cost of providing specified services to one person for a month — used in the cost analysis. Differences in unit measures make it difficult to compare all unit costs across all programs, but it is clear from those with common units that costs varied widely across programs. These differences reflect a number of factors: differences in local markets, the types of employment-related services that participants selected or were steered toward, and the decisions that administrators made in implementing their programs. For example, some programs referred sample members to community providers for basic education services, but did not provide any additional funding to these generally publicly funded programs. However, the welfare departments in Riverside, Detroit, and Portland paid for services provided to sample members through formal contracts with education providers. As a result, welfare department unit costs for basic education (column 1) are higher in these three sites than in the others.(4) Non-welfare agency unit costs for basic education (column 3) were much higher for the Portland program than for the other programs, because community colleges were the largest provider of basic education services in Portland, whereas public adult education systems were the main providers of basic education in the other sites.

Table 13.2
R Estimated Unit Costs for Employment-Related Activities (in 1999 dollars)
Program and Activity Program Group Control Group
Welfare Department Unit Cost Non-Welfare Agency Unit Cost Non-Welfare Agency Unit Cost
Average per Month of Participation ($) Average per Session ($) Average per Hour ($) Average per Month of Participation ($) Average per Hour ($)

Atlanta Labor Force Attachment
Orientation and appraisal n/a 72 n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job search 416 n/a n/a 61 n/a
Basic education 115 n/a 3.18 n/a 3.20
Post-secondary education 73 n/a 9.03 n/a 8.44
Vocational training 153 n/a 7.31 n/a 6.88
Work experience 167 n/a n/a 185 n/a

Atlanta Human Capital Development
Orientation and appraisal n/a 72 n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job search 463 n/a n/a 60.80 n/a
Basic education 99 n/a 3.09 n/a 3.20
Post-secondary education 94 n/a 8.07 n/a 8.44
Vocational training 140 n/a 7.25 n/a 6.88
Work experience 203 n/a n/a 185 n/a

Grand Rapids Labor Force Attachment
Orientation and appraisal n/a 18 n/a n/a n/a
Formal assessment 395 n/a n/a n/a n/a
Job search a 259 n/a n/a n/a n/a
Basic education 132 n/a 6.37 n/a 6.38
Post-secondary education 97 n/a 9.31 n/a 9.37
Vocational training 110 n/a 7.82 n/a 7.97
Work experience 241 n/a n/a 241 n/a

Grand Rapids Human Capital Development
Orientation and appraisal n/a 18 n/a n/a n/a
Formal assessment 395 n/a n/a n/a n/a
Job searcha 259 n/a n/a n/a n/a
Basic education 132 n/a 6.31 n/a 6.38
Post-secondary education 97 n/a 9.15 n/a 9.37
Vocational training 110 n/a 7.23 n/a 7.97
Work experience 241 n/a n/a 241 n/a

Riverside Labor Force Attachment
Orientation and appraisal n/a 88 n/a n/a n/a
Formal assessment n/a 596 n/a n/a n/a
Job search 759 n/a n/a 254 n/a
Basic education 255 n/a 4.45 n/a 4.09
Post-secondary education 122 n/a 6.37 n/a 6.39
Vocational training 122 n/a 5.52 n/a 5.90
Work experience 572 n/a n/a 572 n/a

Riverside Human Capital Development(without a high school diploma or GED)
Orientation and appraisal n/a 88 n/a n/a n/a
Formal assessment n/a 596 n/a n/a n/a
Job search 759 n/a n/a 254 n/a
Basic education 255 n/a 4.05 n/a 4.09
Post-secondary education 122 n/a 6.29 n/a 6.39
Vocational training 122 n/a 5.31 n/a 5.90
Work experience 572 n/a n/a 572 n/a

Columbus Integrated
Orientation and appraisal 19 n/a n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job search 220 n/a n/a 100 n/a
Basic education 86 n/a 3.82 n/a n/a
Post-secondary education 225 n/a 6.16 n/a n/a
Vocational trainingb 225 n/a n/a n/a n/a
Work experience 106 n/a n/a 76 n/a
Columbus Traditional
Orientation and appraisal 14 n/a n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job search 205 n/a n/a 100 n/a
Basic education 54 n/a 3.99 n/a n/a
Post-secondary education 55 n/a 6.16 n/a n/a
Vocational trainingb 55 n/a n/a n/a n/a
Work experience 46 n/a n/a 76 n/a

Detroitc
Orientation and appraisal n/a n/a n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job searchd 752 n/a n/a 597 n/a
Education and traininge 587 n/a 8.44 n/a 8.21
Work experience 557 n/a n/a n/a n/a

Portland
Orientation and appraisal 138 n/a n/a n/a n/a
Formal assessment n/a n/a n/a n/a n/a
Job search 359 n/a n/a 280 n/a
Basic education 257 n/a 11.75 n/a 11.75
Post-secondary education 131 n/a 7.82 n/a 8.01
Vocational training 131 n/a 7.82 n/a 8.01
Work experience 400 n/a n/a 322 n/a
SOURCES:  MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton County Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia Department of Technical and Adult Education, the Board of Regents University System of Georgia; Grand Rapids - the Michigan Department of Social Services, the Michigan Department of Education Office of Extended Learning Services, the Grand Rapids Community College, the Wyoming Community Education Center; Riverside - the California Department of Social Services, the California Department of Education, the Chancellor’s Office of California Community Colleges; Columbus - the Franklin County Department of Human Services, the Ohio Department of Education, the Office of Vocational and Adult Education, the Ohio Board of Regents, the National Center for Education Statistics; Detroit - the Michigan Family Independence Agency, the Michigan Department of Education, the Michigan Jobs Commission; Portland - the Oregon Department of Human Resources, Adult and Family Services Division, the Oregon Office of Community College Services; in all sites - information collected on tuition charged at proprietary schools attended by sample members, from MDRC-collected JOBS case file data, the MDRC Two-Year Client Survey, and the MDRC Five-Year Client Survey.
NOTES:  Welfare department unit costs include the cost of providing activities (e.g., classroom instruction, job search facilitation, space rental, and case management expenditures).
Work experience unit costs for program and control group members receiving services from non-welfare agencies were assumed to be equal to the welfare department JOBS unit cost. Atlanta’s unit cost for controls was the average of the LFA and HCD welfare department unit costs.
N/a = not applicable.
a The estimated unit cost of job search to Grand Rapids non-welfare agencies was $1,025 per participant.
b The estimated unit cost of vocational training for non-welfare agencies in Columbus was $5,032 per participant in the Integrated program and $2,771 per participant in the Traditional program.
c For the Detroit program, welfare department unit costs were calculated by dividing total activity expenditures by the total number of participant-months for the corresponding activity. At the time of the two-year follow-up, Farrell, 2000, presented a range of unit costs, based on two different methods of calculating participant-months. However, both the two- and five-year gross cost estimates were based only on the upper-bound unit cost estimates, and so only "upper-bound" estimates of unit costs are presented on this table. See Farrell, 2000, for a complete discussion of this issue and the full range of two-year cost and participation estimates for this site.
d The estimated unit cost of job search to Detroit non-welfare agencies was $816 per month of participation. In addition, the estimated unit cost to the welfare department for control group participation in job search, education and training, and work experience activities was $271 per month of participation.
e Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of "education and training." Furthermore, in this site orientation and assessment costs are spread across these three activities.

The unit cost multiplied by sample members' average number of months of participation in the activity (the behavioral variable) yields the cost per sample member. These costs are shown in Table 13.3 (column 1). In-program welfare department operating costs ranged from $603 (Columbus Traditional) to $2,753 (Riverside HCD) and were not systematically different between employment- and education-focused programs.

Table 13.3
Estimated Cost per Program Group Member for Employment-Related Services
Within a Five-Year Follow-Up Period,
by Program, Service Component, and Agency (in 1999 dollars)
Activity or Service In-Program Cost Out-of-Program Cost Total Gross Cost per Program Group
Welfare Department Cost ($) Non-Welfare Agency Cost ($) Total Program Cost ($) Welfare Department Cost ($) Non-Welfare Agency Cost ($)

Atlanta Labor Force Attachment
Orientation and appraisal 72 0 72 0 0 72
Formal assessment n/a n/a n/a n/a n/a n/a
Job search 1,081 0 1,081 0 23 1,104
Basic education 210 453 663 0 166 829
Post-secondary education 62 632 694 0 177 871
Vocational training 265 1,166 1,431 0 641 2,072
Work experience 256 n/a 256 0 67 323
Subtotal (operating) 1,946 2,251 4,198 0 1,074 5,271
Child care 1,191 0 1,191 301 0 1,491
Other support services 317 0 317 0 0 317
Total 3,454 2,251 5,705 301 1,074 7,080

Atlanta Human Capital Development
Orientation and appraisal 72 0 72 0 0 72
Formal assessment n/a n/a n/a n/a n/a n/a
Job search 503 0 503 0 28 530
Basic education 430 1,004 1,434 0 161 1,596
Post-secondary education 58 425 483 0 415 898
Vocational training 429 2,185 2,614 0 605 3,219
Work experience 216 0 216 0 45 261
Subtotal (operating) 1,708 3,614 5,323 0 1,253 6,576
Child care 1,148 0 1,148 251 0 1,400
Other support services 610 0 610 0 0 610
Total 3,466 3,614 7,080 251 1,253 8,585

Grand Rapids Labor Force Attachment
Orientation and appraisal 18 n/a 18 0 0 18
Formal assessment 9 n/a 9 0 0 9
Job search 362 908 1,270 0 152 1,422
Basic education 166 719 884 0 484 1,368
Post-secondary education 263 1,710 1,974 0 1,340 3,314
Vocational training 152 1,033 1,186 0 618 1,804
Work experience 130 0 130 0 53 183
Subtotal (operating) 1,100 4,371 5,470 0 2,648 8,118
Child care 451 0 451 312 0 762
Other support services 60 0 60 0 0 60
Total 1,610 4,371 5,981 312 2,648 8,940

Grand Rapids Human Capital Development
Orientation and appraisal 18 0 18 0 0 18
Formal assessment 312 0 312 0 0 312
Job search 206 432 638 0 116 754
Basic education 474 1,672 2,146 0 443 2,589
Post-secondary education 302 2,285 2,586 0 995 3,582
Vocational training 163 900 1,062 0 956 2,018
Work experience 131 0 131 0 139 271
Subtotal (operating) 1,607 5,288 6,895 0 2,649 9,544
Child care 606 0 606 502 0 1,108
Other support services 75 0 75 0 0 75
Total 2,288 5,288 7,576 502 2,649 10,727

Riverside Labor Force Attachment
Orientation and appraisal 111 0 111 0 0 111
Formal assessment 6 0 6 0 0 6
Job search 1,056 0 1,056 0 145 1,202
Basic education 157 345 502 0 226 728
Post-secondary education 218 988 1,205 0 984 2,189
Vocational training 97 392 489 0 586 1,075
Work experience 147 0 147 0 134 281
Subtotal (operating) 1,793 1,725 3,518 0 2,075 5,593
Child care 252 0 252 29 0 281
Other support services 171 0 171 0 0 171
Total 2,217 1,725 3,941 29 2,075 6,045

Riverside Human Capital Development
(without a high school diploma or GED)
Orientation and appraisal 107 0 107 0 0 107
Formal assessment 13 0 13 0 0 13
Job search 1,008 0 1,008 0 169 1,177
Basic education 1,188 1,420 2,608 0 291 2,900
Post-secondary education 89 209 298 0 282 580
Vocational training 179 688 867 0 182 1,049
Work experience 169 0 169 0 92 261
Subtotal (operating) 2,753 2,318 5,071 0 1,015 6,086
Child care 361 0 361 16 0 376
Other support services 170 0 170 0 0 170
Total 3,284 2,318 5,601 16 1,015 6,632

Columbus Integrated
Orientation and appraisal 18 0 18 0 0 18
Formal assessment n/a n/a n/a n/a n/a n/a
Job search 141 0 141 0 15 156
Basic education 108 259 367 0 334 700
Post-secondary education 732 1,241 1,973 0 887 2,860
Vocational training 206 542 749 0 534 1,283
Work experience 85 0 85 0 170 254
Subtotal (operating) 1,290 2,042 3,331 0 1,939 5,271
Child care 571 0 571 558 0 1,129
Other support services 334 0 334 0 0 334
Total 2,195 2,042 4,237 558 1,939 6,734
Columbus Traditional
Orientation and appraisal 9 0 9 0 0 9
Formal assessment n/a n/a n/a n/a n/a n/a
Job search 192 0 192 0 0 192
Basic education 84 494 577 0 218 796
Post-secondary education 194 1,870 2,064 0 556 2,620
Vocational training 59 270 329 0 830 1,158
Work experience 66 0 66 0 127 194
Subtotal (operating) 603 2,634 3,237 0 1,731 4,969
Child care 584 0 584 547 0 1,131
Other support services 224 0 224 0 0 224
Total 1,411 2,634 4,045 547 1,731 6,323

Detroit
Job search 114 446 560 0 5 565
Education and traininga 1,198 1,403 2,602 0 1,879 4,481
Work experience 77 0 77 0 3 80
Subtotal (operating) 1,389 1,849 3,238 0 1,887 5,126
Child care 313 0 313 435 0 748
Other support services 117 0 117 0 0 117
Total 1,819 1,849 3,668 435 1,887 5,990

Portland
Orientation and appraisal 138 11 149 0 0 149
Formal assessment n/a n/a   n/a n/a
Job search 609 14 623 0 131 754
Basic education 361 12 373 0 624 997
Post-secondary education 127 709 836 0 755 1,591
Vocational training 182 928 1,110 0 831 1,941
Work experience 286 6 292 0 122 414
Subtotal (operating) 1,702 1,680 3,382 0 2,464 5,846
Child care 1,043 0 1,043 1,883 0 2,926
Other support services 158 0 158 0 0 159
Total 2,903 1,680 4,583 1,883 2,464 8,930
SOURCES:  MDRC calculations based on fiscal and participation data from the following: Atlanta - the Fulton County Department of Family and Children Services, the Georgia Department of Human Resources, the Georgia Department of Technical and Adult Education, the Board of Regents University System of Georgia; Grand Rapids - the Michigan Department of Social Services, the Michigan Department of Education Office of Extended Learning Services, the Grand Rapids Community College, the Wyoming Community Education Center; Riverside - the California Department of Social Services, the California Department of Education, the Chancellor’s Office of California Community Colleges; Columbus - the Franklin County Department of Human Services, the Ohio Department of Education, the Office of Vocational and Adult Education, the Ohio Board of Regents, the National Center for Education Statistics; Detroit - the Michigan Family Independence Agency, the Michigan Department of Education, the Michigan Jobs Commission; Portland - the Oregon Department of Human Resources, Adult and Family Services Division, the Oregon Office of Community College Services; in all sites - information collected on tuition charged at proprietary schools attended by sample members, from MDRC-collected JOBS case file data, the MDRC Two-Year Client Survey, and the MDRC Five-Year Client Survey. MDRC child care and other support service calculations from Fulton County, Kent County, Riverside County, Ohio Department of Human Services, Wayne County, Washington County, and Multnomah County (District 2) payment data. Other support service data from county records.
NOTES:  Rounding may cause slight discrepancies in calculating sums and differences.
N/a = not applicable.
a Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of "education and training." Furthermore, in this site orientation and assessment costs are spread across these three activities.

2. Support service costs. Programs paid for child care, transportation, and ancillary services (such as uniforms, tools, equipment, and books) to support recipients' participation in employment and employment-related activities. Detailed information on support service payments was available through the end year 2 of follow-up. These data, together with data from the Five-Year Client Survey, were used to estimate payments for the remainder of the follow-up period.

As shown in Table 13.3, the welfare departments spent an average of $252 (Riverside LFA) to more than $1,000 (Atlanta LFA and HCD and Portland) per program group member on in-program child care. The costs of other support services, which included payments to cover transportation costs, class fees, uniforms, and so on, ranged from less than $100 (Grand Rapids LFA and HCD) to $610 (Atlanta HCD).

B. Non-Welfare Agency Expenditures

Table 13.3 (column 2) shows the non-welfare agency cost of providing employment-related services. These costs ranged from $1,680 (Portland) to $5,288 (Grand Rapids HCD). As discussed above, Portland's low non-welfare agency costs reflect the fact that the welfare department paid for the majority of in-program activities (except for vocational training and college). The costs were high in Grand Rapids, especially for the HCD program, because job search services were provided and paid for by the local community education center and because of the extensive use of education and training activities, which also were not paid for by the welfare department.

[Go To Contents]

V. Expenditures for Out-of-Program Employment-Related Services

Program and control group members participated in some employment-related activities when they were not receiving welfare benefits. Although these services were not part of the NEWWS programs, if program and control group members participated in such activities and participated at different rates, the off-welfare services have the potential to differentially increase sample members' earnings and reduce their welfare receipt, and thus are included in the cost estimates. These expenditures are examined in more detail below. (See Figure 13.1, Boxes 4 and 5.)

A. Welfare Department Expenditures

The bulk of the costs incurred when participants were not receiving welfare were paid by non-welfare agencies; welfare agencies provided child care assistance only. Sample members were eligible for one year of transitional child care assistance if they left welfare for work and could also receive income-eligible child care.

As shown in Table 13.3, the average cost of transitional, income-eligible, and other child care ranged from less than $30 per program group member (Riverside LFA and HCD) to $1,883 (Portland). This wide variation in child care payments was seen after two years of follow-up. As explained in the earlier report, Portland's high child care costs may reflect a combination of the following: The program included more families with younger children than most of the other NEWWS programs,(5) the staff believed in the importance of providing child care to everyone who was eligible for it, and integrated case management allowed better administration of child care payments.(6)

B. Non-Welfare Agency Expenditures

When sample members were not receiving welfare benefits, they most commonly participated in basic education, post-secondary education, and vocational training; some sample members participated in other activities (job search, unpaid work experience, and on-the-job training).(7) These costs ranged from about $1,000 (Riverside HCD) to about $2,500 (Grand Rapids LFA and HCD and Portland). The programs in Grand Rapids and Portland had high rates of self-initiated participation, particularly in education and training activities. As will be discussed later, this was also true for control group members in these programs.

[Go to Contents]

VI. Gross Costs

The gross costs were obtained by adding the cost of in-program services to the out-of-program costs. This total investment must be compared with the total gross cost per control group member to determine the government's net investment per program group member and, in the benefit-cost analysis, the net payoff of that investment. As shown in Table 13.3, the gross costs of the NEWWS programs ranged from $5,990 (Detroit) to $10,727 (Grand Rapids HCD) per program group member over the five-year follow-up period. On average, gross costs were slightly higher for education-focused programs than for employment-focused programs. (See Figure 13.1, Boxes 7 and 10.)

A. Gross Costs for Program Group Members

Participation rates and duration of participation, as well as whether participation took place while sample members were on or off welfare, determined both overall costs and the proportion of costs paid for by the welfare department. For example, post-secondary education unit costs were similar in the Atlanta and Grand Rapids HCD programs, but participation in both in-program and out-of-program (that is, self-initiated) activities was much greater in Grand Rapids, resulting in a total gross cost for post-secondary education that was nearly four times as high as that for Atlanta. As a result, the Grand Rapids HCD program incurred the highest in-program costs (column 3) and total costs (column 6). However, the proportion of these costs paid by the welfare department was lower in the Grand Rapids programs than in the other programs (column 1 divided by column 6): These programs had more participation in higher-cost activities such as education and training that was leveraged with fewer dollars by the welfare department.

Overall, higher costs were associated with higher rates of participation in education and training activities. Therefore, employment-focused programs might be expected to have lower total gross costs than education-focused programs. However, the gross costs of the Grand Rapids LFA and Portland programs were among the highest in this evaluation because sample members in these programs participated in education and training activities at high rates, much of it self-initiated. As a result, on average, employment-focused programs were slightly more expensive than education-focused ones.

The average gross cost was higher than costs found in other MDRC evaluations of welfare-to-work programs. On average, total gross costs of the NEWWS programs were similar to those found in other high-cost programs, including the Alameda and Los Angeles GAIN programs operated in the late 1980s and early 1990s, which provided extensive education and training services. The average cost per sample member across all NEWWS programs was $7,599, compared with $7,763 in Alameda and $7,123 in Los Angeles (all in 1999 dollars).(8) The higher costs in NEWWS programs (Atlanta and Grand Rapids HCD, Grand Rapids LFA, and Portland) are likely due to higher participation overall, and particularly in high-cost activities such as post-secondary education and vocational training.

B. Gross Costs for Control Group Members

Control group costs reflect the cost of participation in employment-related services, most of which was self-initiated. However, in the sites where control embargoes were lifted before the end of the five-year follow-up period, a small portion of participation was program-related.(9)

In general, control group costs were high. This was particularly true in Grand Rapids and Portland owing to high rates of self-initiated participation, especially in post-secondary education and vocational training activities. The average control group cost in the GAIN program was $1,638 (in 1999 dollars). The control group costs in Grand Rapids and Portland were about three times higher than the highest control group cost in the GAIN program, which was $2,233 (in 1999 dollars) in San Diego. The control group costs in the other NEWWS programs ranged from 8 percent (Riverside HCD) to 76 percent (Detroit) higher.

[Go To Contents]

VII. Net Costs

The net cost per program group member is the gross cost per program group member minus what would have been spent in the absence of the NEWWS programs, represented by the gross cost per control group member. (See Figure 13.1, Box 11.) As shown in Table 13.4 (column 3), net costs ranged from about $2,000 (Grand Rapids LFA and Detroit) to $5,480 (Atlanta HCD). Depending on the rate of participation, and the types of activities in which control group members enroll on their own, programs with high gross costs can have low net costs. For example, although Grand Rapids HCD had the highest gross costs, owing to high participation rates in education and training activities, similarly high participation rates in these activities among control group members resulted in a net cost that was lower than costs for the other education-focused programs. Likewise, Grand Rapids LFA and Portland had the highest gross costs among the employment-focused programs, but high gross costs for control group members produced low net costs for these two programs. As a result, the average net cost of the NEWWS programs was similar to that of the GAIN program.

Table 13.4
Estimated Gross and Net Costs Within a Five-Year Follow-Up Period,
by Program and Service Component (in 1999 dollars)

Program and Component

 Total Gross Cost per Program Group Member ($) Total Gross Cost per Control Group Member ($) Total Net Cost per Program Group Member ($)

Atlanta Labor Force Attachment
Orientation and appraisal 72 11 61
Formal assessment n/a n/a n/a
Job search 1,104 79 1,025
Basic education 829 166 663
Post-secondary education 871 844 27
Vocational training 2,072 1,392 680
Work experience 323 126 198
Subtotal (operating) 5,271 2,618 2,654
Child care 1,491 459 1,032
Other support services 317 29 288
Total 7,080 3,105 3,974

Atlanta Human Capital Development
Orientation and appraisal 72 11 61
Formal assessment n/a n/a n/a
Job search 530 79 452
Basic education 1,596 166 1,429
Post-secondary education 898 844 54
Vocational training 3,219 1,392 1,827
Work experience 261 126 135
Subtotal (operating) 6,576 2,618 3,958
Child care 1,400 459 940
Other support services 610 29 581
Total 8,585 3,105 5,480

Grand Rapids Labor Force Attachment
Orientation and appraisal 18 1 17
Formal assessment 9 n/a 9
Job search 1,422 329 1,093
Basic education 1,368 1,170 198
Post-secondary education 3,314 2,906 408
Vocational training 1,804 1,931 -127
Work experience 183 54 129
Subtotal (operating) 8,118 6,392 1,726
Child care 762 561 202
Other support services 60 n/a 60
Total 8,940 6,953 1,987

Grand Rapids Human Capital Development
Orientation and appraisal 18 1 17
Formal assessment 312 n/a 312
Job search 754 329 425
Basic education 2,589 1,170 1,419
Post-secondary education 3,582 2,906 675
Vocational training 2,018 1,931 87
Work experience 271 54 216
Subtotal (operating) 9,544 6,392 3,151
Child care 1,108 561 548
Other support services 75 n/a 75
Total 10,727 6,953 3,773

Riverside Labor Force Attachment
Orientation and appraisal 111 n/a 111
Formal assessment 6 n/a 6
Job search 1,202 149 1,053
Basic education 728 230 498
Post-secondary education 2,189 1,301 889
Vocational training 1,075 856 218
Work experience 281 126 155
Subtotal (operating) 5,593 2,661 2,931
Child care 281 63 218
Other support services 171 n/a 171
Total 6,045 2,724 3,320

Riverside Human Capital Development
(without a high school diploma or GED)
Orientation and appraisal 107 n/a 107
Formal assessment 13 n/a 13
Job search 1,177 127 1,050
Basic education 2,900 409 2,491
Post-secondary education 580 606 -26
Vocational training 1,049 820 229
Work experience 261 63 198
Subtotal (operating) 6,086 2,025 4,061
Child care 376 35 342
Other support services 170 n/a 170
Total 6,632 2,059 4,573

Columbus Integrated
Orientation and appraisal 18 n/a 18
Formal assessment n/a n/a n/a
Job search 156 31 125
Basic education 700 180 520
Post-secondary education 2,860 606 2,254
Vocational training 1,283 593 690
Work experience 254 25 229
Subtotal (operating) 5,271 1,434 3,837
Child care 1,129 1,084 45
Other support services 334 33 301
Total 6,734 2,551 4,183

Columbus Traditional
Orientation and appraisal 9 n/a 9
Formal assessment n/a n/a n/a
Job search 192 31 161
Basic education 796 180 616
Post-secondary education 2,620 606 2,014
Vocational training 1,158 593 566
Work experience 194 25 169
Subtotal (operating) 4,969 1,434 3,534
Child care 1,131 1,084 47
Other support services 224 33 190
Total 6,323 2,551 3,772

Detroit
Job search 565 328 237
Education and traininga 4,481 2,830 1,651
Work experience 80 12 67
Subtotal (operating) 5,126 3,170 1,956
Child care 748 734 14
Other support services 117 34 83
Total 5,990 3,937 2,053

Portland
Orientation and appraisal 149 n/a 149
Formal assessment n/a n/a n/a
Job search 754 273 481
Basic education 997 593 404
Post-secondary education 1,591 2,353 -763
Vocational training 1,941 1,049 892
Work experience 414 240 174
Subtotal (operating) 5,846 4,508 1,337
Child care 2,926 1,542 1,384
Other support services 159 15 144
Total 8,930 6,065 2,865
SOURCES: See Table 13.3.
NOTES:  Rounding may cause slight discrepancies in calculating sums and differences.
N/a = not applicable.
a Owing to data limitations, in Detroit it was not possible to separate vocational training, post-secondary education, and basic education costs. Thus, the gross costs of all three activities are included in the cost of "education and training." Furthermore, in this site orientation and assessment costs are spread across these three activities.

Net costs were generally lower for employment-focused programs than for education-focused ones, largely due to the particularly low net costs in Grand Rapids LFA and Portland. As the two-year findings suggested, the Grand Rapids LFA program may have diverted program group members into job search who would have enrolled on their own in education or training programs. The same may have been true of the Portland program.

A. Results by Educational Attainment Subgroups

Table 13.5 shows gross and net costs for those with and without a high school diploma or GED when they entered the evaluation. For the most part, gross and net costs were higher for graduates. This is not surprising, given that those with high school credentials were more likely to participate in higher-cost activities, such as post-secondary education or vocational training, whereas those who lacked these credentials were more likely to participate in typically lower-cost activities, such as basic education.

Table 13.5
Estimated Gross and Net Costs Within a Five-Year Follow-Up Period,
by Program and Education Subgroup (in 1999 dollars)
Program and Component Total Gross Cost per Program Group Member ($) Total Gross Cost per Control Group Member ($) Total Net Cost per Program Group Member ($)
Without a high school diploma or GED
Atlanta Labor Force Attachment
Operating costs 3,761 1,118 2,642
Support services 1,469 370 1,099
Total 5,230 1,489 3,741
Atlanta Human Capital Development
Operating costs 5,646 1,118 4,528
Support services 1,982 370 1,612
Total 7,628 1,489 6,140
Grand Rapids Labor Force Attachment
Operating costs 6,244 5,699 544
Support services 493 445 48
Total 6,736 6,144 592
Grand Rapids Human Capital Development
Operating costs 8,789 5,699 3,090
Support services 1,001 445 556
Total 9,790 6,144 3,645
Riverside Labor Force Attachment
Operating costs 5,306 2,025 3,282
Support services 309 35 274
Total 5,615 2,060 3,555
Riverside Human Capital Development
Operating costs 6,086 2,025 4,061
Support services 546 35 511
Total 6,632 2,060 4,572
Columbus Integrated
Operating costs 4,564 914 3,650
Support services 1,357 624 733
Total 5,921 1,538 4,383
Columbus Traditional
Operating costs 3,500 914 2,585
Support services 1,078 624 454
Total 4,577 1,538 3,039
Detroit
Operating costs 4,560 3,033 1,527
Support services 781 902 -121
Total 5,341 3,935 1,406
Portland
Operating costs 4,690 2,927 1,763
Support services 2,874 1,346 1,528
Total 7,563 4,273 3,290
With a high school diploma or GED
Atlanta Labor Force Attachment
Operating costs 6,259 3,596 2,663
Support services 2,105 559 1,546
Total 8,365 4,155 4,209
Atlanta Human Capital Development
Operating costs 7,207 3,596 3,611
Support services 2,021 559 1,462
Total 9,228 4,155 5,073
Grand Rapids Labor Force Attachment
Operating costs 8,961 7,112 1,849
Support services 958 630 328
Total 9,918 7,741 2,177
Grand Rapids Human Capital Development
Operating costs 10,273 7,112 3,162
Support services 1,440 630 810
Total 11,713 7,741 3,972
Riverside Labor Force Attachment
Operating costs 5,877 3,625 2,252
Support services 581 58 523
Total 6,457 3,682 2,775
Columbus Integrated
Operating costs 5,938 1,790 4,149
Support services 1,524 1,368 157
Total 7,463 3,157 4,305
Columbus Traditional
Operating costs 5,759 1,790 3,969
Support services 1,495 1,368 127
Total 7,253 3,157 4,096
Detroit
Operating costs 5,510 3,292 2,218
Support services 859 589 270
Total 6,370 3,882 2,488
Portland
Operating costs 6,568 5,438 1,131
Support services 3,358 1,540 1,818
Total 9,926 6,978 2,948
SOURCES:  See Table 13.3.
NOTES:  Rounding may cause slight discrepancies in calculating sums and differences.

B. LFA-HCD Program Differences

As noted above, net costs were generally higher for education-focused programs than for employment-focused programs. Similarly, HCD programs had higher net costs than LFA programs. Again, this was not a surprising finding. As discussed in Chapter 1, the HCD programs were designed to make larger upfront investments in building skills, mainly through education and training activities, which tend to be longer in duration and more expensive to operate than job search activities. It was anticipated that the programs would incur higher costs, but with the hope that this greater investment in human capital would produce greater returns (earnings increases and welfare savings) over time.

After two years of follow-up, HCD programs were 1.5 to 2.6 times more expensive than LFA programs. Although the net cost gap at the five-year point narrowed relative to net costs at the two-year point, HCD programs continued to be more expensive than LFA programs — from 1.3 to 1.9 times more expensive.(10)

[Go To Contents]

VIII. Analytical Approach for the Benefit-Cost Analysis

The analytical approach used in this benefit-cost analysis is similar to that used in previous evaluations conducted by MDRC.(11) The general approach is to place dollar values on the program's effects and its use of resources whenever possible, either by directly measuring them or by estimating them.

Program effects on earnings, welfare, and Food Stamp payments were measured directly. Effects on earnings were based on quarterly earnings reported by employers to states' unemployment insurance (UI) systems, and effects on welfare and Food Stamp payments were measured using computerized administrative records (the same data sources were used in the impact analysis presented in Chapters 4 and 5). Effects on Medicaid, fringe benefits, federal income taxes, state income and sales taxes, and the costs of administering transfer programs could not be measured directly, but were estimated or imputed using various data sources (details are provided below).

All of these effects were considered along with the estimated net costs of the NEWWS programs, presented above, to ascertain the net gains and losses to program group members and to government budgets.

A. Accounting Methods

The benefit-cost estimates cover a five-year period starting with the quarter after random assignment (quarter 2) for each sample member. This time frame is similar to that used in most previous MDRC benefit-cost analyses of welfare-to-work programs. Projecting benefits and costs beyond five years would be problematic because it is difficult to predict future behavior. However, given the extent to which impacts on earnings and welfare had decayed in most programs by the end of the five-year period, it seems unlikely that there would be many future net returns to the programs.

The benefit-cost estimates in this analysis are expressed in terms of net present values per program group member. The "net" means that, like impacts, the estimated amounts represent differences between estimates for program and control group members. The estimates are in "present value" terms because the accounting method of discounting was used to express the dollar worth at the end of the first year of follow-up of program effects that occurred later in the follow-up period.(12) As in the cost analysis, all estimated amounts in the benefit-cost analysis are expressed in 1999 dollars, eliminating the effects of inflation on the values.

B. Analytical Perspectives

An important issue in benefit-cost analyses of government programs is determining who bears any benefits or costs of the program. A program effect can generate gains from one perspective while generating losses from another. For example, a decrease in welfare payments is a financial loss from the perspective of the program group but a financial gain from the perspective of the government. This makes it important to consider the perspectives of all the directly affected groups when assessing each main program effect.

This analysis presents the net benefits and costs of the NEWWS programs from the perspective of program participants and government budgets. In the following table, the main financial effects are shown as an expected gain or benefit (+), a loss or cost (-), or neither a benefit nor a cost (0), according to a priori expectations regarding their value. (The tables in the following sections show the actual gains and losses in dollars.)

Expected Main Financial Effects of NEWWS Programs

 Accounting Perspective
Welfare Sample Government Budget
Increased earnings and fringe benefits + 0
Increased tax payments - +
Reduced use of transfer programs - +
NEWWS operating costs 0 -
Increased use of support services 0 -

The welfare sample perspective identifies net gains or losses for program group members, indicating how they fared as a result of the program. As illustrated, it is expected that earnings impacts represent gains for participants, whereas reductions in welfare payments and higher tax payments (resulting from earnings gains) represent losses. The program may be considered a net gain from the standpoint of program group members if the gains from earnings exceed losses from reduced transfer payments and higher taxes. The net cost of providing eligibility and employment-related services to participants has no direct effect on their income.

The government budget perspective identifies net gains and losses incurred by a combination of federal, state, and local government budgets. Net gains to the government budget occur through savings in transfer payments and their related administrative costs and through higher taxes paid by program group members compared with control group members. The government budget comes out ahead if tax increases and savings in transfer payments and administrative costs exceed the net cost of providing NEWWS program services. Program group members' earnings gains do not directly affect the government budget's net gains or losses.

C. Limitations of the Analysis

Some limits on the comprehensiveness of the benefit-cost analysis should be recognized. Some program effects, whose costs and benefits are difficult to quantify or to express in dollars, are not estimated. For example, this analysis does not include estimates of out-of-pocket expenditures (for child care, transportation, clothing, and so on) that sample members incurred when they went to work.(13) In addition, the estimates in this chapter reflect the direct effects of programs and do not consider secondary effects. These secondary effects include the possible displacement of other workers by the increased employment of program group members; these displaced workers may have become unemployed or employed in lower-paying jobs. In addition, the analysis does not consider the sample members' forgone personal and family activities that resulted from increased work or the intrinsic benefits of education that are not reflected in earnings. The analysis does not place a dollar value on family or child well-being or the clear but difficult-to-monetize benefits associated with society's (or sample members') preference for work over welfare.

[Go to Contents]

IX. Effects for Sample Members

This section presents estimates of the financial benefits of the NEWWS programs per program group member during the five-year follow-up period.(14) It presents an account of the main benefit components: earnings and fringe benefits, personal taxes, and transfer payments and benefits.

A. Earnings and Fringe Benefits

As reported in Chapter 4, the NEWWS programs produced increases in employment and earnings for program group members (compared with control group members) over the five-year follow-up period. Table 13.6 shows that the impact on earnings ranged from about $900 (Grand Rapids HCD) to more than $5,000 (Portland) per program group member.

Table 13.6
Estimated Program-Control Group Differences in
Earnings, Fringe Benefits, and Personal Taxes per Sample Member
Within a Five-Year Follow-Up Period (in 1999 dollars)
Component of Analysis Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Riverside HCD ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)
Earnings 2,575 2,089 1,678 906 2,697 1,398 2,086 1,456 1,490 5,171
Fringe benefitsa 384 311 265 143 369 192 330 230 235 708
Total 2,959 2,400 1,943 1,050 3,066 1,590 2,416 1,687 1,726 5,879

Personal taxes
Social Security taxb 174 141 113 61 182 94 141 98 101 349
Federal income tax 9 62 -321 -99 -278 -340 0 -53 59 -258
State income tax 77 73 46 33 5 2 27 15 64 175
State sales and excise tax 25 20 -21 -19 -5 -44 10 6 19 0
Total taxes 285 296 -182 -24 -95 -287 178 67 242 266
Sample size 2,938 2,992 3,012 2,997 6,726 3,135 4,672 4,729 4,459 4,028
SOURCES:  MDRC calculations from state unemployment insurance (UI) earnings records and published data on fringe benefits and tax rates.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.
a These include employer-paid health and life insurance, pension contributions, and workers' compensation. Paid leave is captured directly by the earnings estimate. Employee-paid Social Security and Medicare taxes are included as tax payments.
b Employee portion only.

Fringe benefits — employer-provided health and life insurance, pension contributions, and worker's compensation — were part of sample members' total compensation from working and thus were included in the benefit-cost analysis.(15) They ranged by region from 13.7 percent (the West) to 15.8 percent (the Midwest) of earnings.(16) Increases from earnings and fringe benefits yielded an average increase in total work-related compensation that ranged from $1,050 per program group member in Grand Rapids HCD to $5,879 per program group member in Portland.(17)

B. Personal Taxes

Since all of the programs increased earnings, one might expect them to also increase income taxes, payroll taxes, and sales and excise taxes (there is no sales tax in Oregon). Federal tax payments, along with the Earned Income Tax Credit (EITC),(18) were imputed from the relevant earnings base using tax rates and rules for 1995.(19) As shown in Table 13.6, in fact, about half of the programs decreased income taxes by $53 (Columbus Traditional) to $340 (Riverside HCD) per sample member. In these cases, the expected increase in tax payments was outweighed by an increase in federal EITC payments to sample members. There was an increase in federal income taxes in the remaining sites, ranging from less than $1 (Columbus Integrated) to $62 (Atlanta HCD).

C. Transfer Payments and Benefits

As discussed in Chapter 5, all of the programs in the evaluation decreased welfare and Food Stamp payments to program group members over the follow-up period. As shown in Table 13.7, the impacts on welfare payments ranged from -$571 in the Detroit program to -$3,058 in the Riverside HCD program. During the same period, reductions in Food Stamp payments ranged from $160 in the Atlanta HCD program to over $1,000 in the Riverside HCD and Columbus Integrated programs. In sum, program group members received from about $900 (Detroit and Atlanta) to over $4,000 (Riverside HCD) less in cash payments over the five-year period than their control group counterparts.

Table 13.7
Estimated Program-Control Group Differences in
Transfer Payments and Administrative Costs per Sample Member
Within a Five-Year Follow-Up Period (in 1999 dollars)
Component of Analysis Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Riverside HCD ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)

Transfer payments
Welfare -919 -747 -2,711 -1,860 -2,868 -3,058 -1,575 -1,148 -571 -2,783
Food Stamps -434 -160 -643 -396 -928 -1,040 -1,052 -673 -341 -848
Medicaid -1,159 -727 -1,026 -775 -510 -766 -1,102 -875 -309 -2,598
Total -2,512 -1,633 -4,380 -3,031 -4,306 -4,864 -3,728 -2,696 -1,222 -6,228

Administrative costs
Welfare -119 -97 -434 -298 -258 -275 -173 -126 -91 -918
Food Stamps -67 -25 -99 -61 -143 -160 -162 -104 -53 -131
Medicaid -46 -29 -51 -39 -71 -107 -44 -35 -15 -208
Total -233 -151 -584 -397 -472 -543 -379 -265 -159 -1,257
Sample size 2,938 2,992 3,012 2,997 6,726 3,135 4,672 4,729 4,459 4,028
SOURCES:  MDRC calculations from state welfare and Food Stamp payment records, unemployment insurance (UI) earnings records, and published data on administrative costs.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.

In addition to cash benefits, sample members were also eligible for Medicaid benefits. In low-grant states especially, Medicaid benefits can have a larger cash value than welfare payments. Sample members and their children were categorically eligible for Medicaid while receiving welfare. In addition, those who left welfare because of employment were eligible for Transitional Medicaid for one year.(20) Eligibility for these programs was estimated using welfare administrative data, Transitional Medicaid take-up rates calculated from the Five-Year Client Survey,(21) and published data on Medicaid expenditures.(22)

As noted in Chapter 8, the programs in the NEWWS Evaluation did not result in significant differences in health care coverage between program and control group members at the end of the follow-up period. It is important to note that the earlier chapter looked at differences in health coverage at a single point in time, that is, in the month before the five-year survey. As shown in Figure 5.1, in most programs the smallest differences in rate of welfare receipt were observed at the end of the follow-up period. However, as shown in Table 5.1, the differences in months of welfare receipt (and, therefore, Medicaid coverage) over the entire five-year period were substantial, ranging from 1.6 months (Detroit) to 5.6 months (Portland). Although program group members were more likely than their control group counterparts to receive Transitional Medicaid, this did not make up for the loss in Medicaid benefits due to their shorter stay on welfare. As shown in Table 13.7, average Medicaid losses over the five-year period ranged from $309 (Detroit) to $2,598 (Portland).

The costs of administering transfer programs were calculated as the cost per benefit paid. As a result, the NEWWS programs also decreased transfer program administration costs.(23) These changes, shown in Table 13.7, yielded decreases of between about $150 (Atlanta HCD and Detroit) and about $1,250 in Portland.(24)

[Go To Contents]

X. Net Gains and Losses by Accounting Perspective

A. Welfare Sample Members

Table 13.8 summarizes the main financial effects from the perspective of welfare sample members. Program-control group differences were defined as gains, indicated by positive values, and losses, indicated by negative values. These results were then summed to attain an estimate of the overall net gain or loss of each program.

Table 13.8
From the Welfare Sample Perspective:
Estimated Monetary Gains and Losses per Program Group Member
Within a Five -Year Follow-Up Period (in 1999 dollars)
Component of Analysis Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)

Gains
Earnings 2,575 2,089 1,678 906 2,697 2,086 1,456 1,490 5,171
Fringe benefitsa 384 311 265 143 369 330 230 235 708
Total 2,959 2,400 1,943 1,050 3,066 2,416 1,687 1,726 5,879

Losses
Tax payments -285 -296 182 24 95 -178 -67 -242 -266
Welfare payments -919 -747 -2,711 -1,860 -2,868 -1,575 -1,148 -571 -2,783
Food Stamps -434 -160 -643 -396 -928 -1,052 -673 -341 -848
Medicaid -1,159 -727 -1,026 -775 -510 -1,102 -875 -309 -2,598
Total -2,797 -1,930 -4,197 -3,007 -4,211 -3,907 -2,763 -1,464 -6,494
Net gain or loss(net present value) 162 471 -2,254 -1,957 -1,145 -1,491 -1,076 262 -615
SOURCES:  MDRC calculations from state welfare and Food Stamp payment records, unemployment insurance (UI) earnings records, and published data on fringe benefits, taxes, and administrative costs.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.
a These include employer-paid health and life insurance, pension contributions, and worker's compensation. Paid leave is captured directly by the earnings estimate. Employee-paid Social Security and Medicare taxes are included as tax payments.

The overall financial effect from the welfare sample perspective was estimated by subtracting the value of transfer payment losses from the value of gains in earnings, fringe benefits, and taxes. Clearly, programs where sample members experience large losses in transfer payments are less likely to produce net gains to sample members, because they have greater losses to offset with earned income. The three programs with the smallest welfare reductions (under $1,000 for both Atlanta programs and the Detroit program) produced very small net gains for sample members. However, these gains were very small; sample members effectively broke even. All of the programs with larger welfare reductions (ranging from about $1,148 in the Columbus Traditional program to $2,868 in the Riverside LFA program) produced net losses for sample members.

As noted earlier, this analysis does not take into account losses of time for personal and family activities (leisure) and out-of-pocket expenditures associated with increases in employment. Given that the three programs with gains did no more than break even, even small values attached to these effects would have resulted in losses for all programs.

Table 13.9 shows the net present values from the perspective of welfare sample members who did not have a high school diploma or GED at the time they entered the evaluation. Only two programs (Detroit and Portland) produced gains for these welfare sample members. However, these gains were small (sample members in Detroit effectively broke even). For the subgroup of sample members with a high school diploma or GED, three programs produced gains: both programs in Atlanta and the Detroit program. Again, gains were small. In fact, welfare sample members in the Atlanta LFA and Detroit programs did little more than break even. Generally, those with a high school diploma did better (that is, bigger gains or smaller losses accrued to this subgroup). However, for the Grand Rapids LFA and Portland programs, this trend was reversed. Although welfare reductions in Portland were as large for nongraduates as for graduates, nongraduates experienced smaller losses in Food Stamps and Medicaid, resulting in smaller reductions in total benefits from transfer programs.

Table 13.9
From the Welfare Sample Perspective:
Estimated Monetary Gains and Losses per Program Group Member Within a Five-Year Follow-Up Period,
by High School Diploma or GED Status at Random Assignment (in 1999 dollars)

Component of Analysis

 Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Riverside HCD ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)

Without a high school diploma or GED

Gains
Earnings 1,143 0 3,241 1,143 2,378 1,398 3,066 1,352 1,533 5,067
Fringe benefitsa 170 0 512 181 326 192 484 214 242 694
Total 1,313 0 3,753 1,323 2,704 1,590 3,551 1,565 1,775 5,761

Losses
Tax payments 195 -133 17 -101 452 324 -467 -225 -196 81
Welfare payments -774 -249 -3,218 -2,240 -3,121 -3,058 -2,207 -1,393 -518 -2,628
Food Stamps 14 227 -849 -776 -967 -1,040 -1,489 -809 -444 -273
Medicaid -964 -276 -1,183 -1,019 -533 -767 -1,674 -1,014 -411 -1,858
Total -1,528 -432 -5,233 -4,135 -4,168 -4,541 -5,837 -3,441 -1,568 -4,677

Net gain or loss(net present value)

 -215 -432 -1,479 -2,812 -1,465 -2,951 -2,286 -1,876 207 1,084

With a high school diploma or GED

Gains
Earnings 3,452 3,358 535 763 3,117   1,252 1,467 1,318 5,075
Fringe benefitsa 514 500 85 121 427   198 232 208 695
Total 3,966 3,858 619 883 3,545   1,449 1,699 1,527 5,770

Losses
Tax payments -571 -370 299 87 -292   57 5 -198 -384
Welfare payments -1,014 -1,034 -2,359 -1,572 -2,539   -1,124 -969 -576 -2,939
Food Stamps -732 -378 -513 -159 -878   -759 -587 -238 -1,132
Medicaid -1,290 -964 -906 -598 -481   -713 -840 -211 -3,066
Total -3,608 -2,746 -3,478 -2,242 -4,190   -2,540 -2,391 -1,224 -7,521

Net gain or loss(net present value)

 358 1,112 -2,859 -1,359 -646   -1,090 -693 303 -1,751
SOURCES: See Table 13.8.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.
a These include employer-paid health and life insurance, pension contributions, and worker's compensation. Paid leave is captured directly by the earnings estimate. Employee-paid Social Security and Medicare taxes are included as tax payments.

B. Government Budgets

From the perspective of government budgets, programs are considered "budget neutral" if they can generate savings that equal the expenditures for services. It follows, then, that programs that produce large welfare savings have the best chance of paying for themselves and programs with small welfare savings will have a harder time breaking even. As shown in Table 13.10 the three programs (Grand Rapids LFA, Riverside LFA, and Portland) that resulted in gains to government budgets had the largest welfare savings. The programs with the smallest welfare reductions, ranging from $571 (Detroit) to $1,148 (Columbus Traditional), resulted in net losses. Programs with welfare reductions in the midrange (Columbus Integrated and Grand Rapids HCD) essentially broke even.

Table 13.10
From the Government Budget Perspective:
Estimated Monetary Gains and Losses per Program Group Member
Within a Five-Year Follow-Up Period (in 1999 dollars)

Component of Analysis

 Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)

Gains
Payroll taxes a 348 282 227 122 364 282 197 201 698
Income and sales tax 111 155 -295 -86 -277 37 -31 141 -83
Welfare payments 919 747 2,711 1,860 2,868 1,575 1,148 571 2,783
Food Stamps 434 160 643 396 928 1,052 673 341 848
Medicaid 1,159 727 1,026 775 510 1,102 875 309 2,598
Transfer administration 233 151 584 397 472 379 265 159 1,257
Total 3,204 2,221 4,895 3,465 4,865 4,427 3,126 1,724 8,100
Losses
Net cost of program and nonprogram activities and services -3,974 -5,480 -1,987 -3,773 -3,320 -4,183 -3,772 -2,053 -2,865
Net gain or loss (net present value) -770 -3,259 2,908 -308 1,545 244 -646 -329 5,235
Return to budget per net dollar invested in program and nonprogram activities and services b 0.81 0.41 2.46 0.92 1.47 1.06 0.83 0.84 2.83
SOURCES: See Table 13.8.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.
a Payroll taxes include employer- and employee-paid Social Security and Medicare taxes.
b The return to budget per net dollar invested is computed by dividing total savings and tax increases by the net cost of activities and services.

To provide some basis for interpreting these results and to facilitate cross-program comparisons, Table 13.10 also presents an additional measure of the cost-effectiveness of the NEWWS programs from the government budget perspective. This measure is called the return to budget per net dollar invested, and it is calculated by dividing the gains (taxes and savings in transfer payments and associated administrative costs) by the total net costs of services. Using this metric, government budgets come out ahead if programs produce more than a dollar's worth of additional revenues and savings for each dollar spent on employment-related services to program group members (compared with controls).

Both the Portland and the Grand Rapids LFA programs produced over $2.00 in increased revenue and savings for every additional dollar spent on program group members. The Riverside LFA program also produced a considerable return, $1.47 per dollar invested. As noted above, the Grand Rapids HCD and Columbus Integrated programs essentially caused the government to break even ($0.92 to $1.06, respectively). The Atlanta, Columbus Traditional, and Detroit programs were not as successful, returning considerably less than one dollar for each dollar invested, ranging from $0.41 in the Atlanta HCD program to just over $0.80 in the other programs. The average return across all of the programs was $1.29 (not shown in any table).

On average, these results are more positive than those found in the GAIN evaluation, in which returns to the government budget ranged from $0.17 per dollar invested (Tulare) to $2.84 (Riverside). The average across all counties in that study was $0.76. In the NEWWS Evaluation, the return on investments in Portland ($2.83) and Grand Rapids LFA ($2.46) compare favorably with the Riverside GAIN program and the SWIM program, which returned $2.84 and $2.34, respectively.

From the perspective of government budgets, the results were mostly positive for those without a high school diploma or GED. As shown in Table 13.11, both Atlanta programs produced losses. Gains ranged from $437 in Detroit to $5,564 in Grand Rapids LFA. For those with a high school diploma or GED, only the employment-focused programs paid for themselves (Atlanta LFA broke even). This is consistent with other findings: The welfare savings were larger for employment-focused programs; and those entering education-focused programs with a high school diploma or GED were more likely to participate in higher-cost activities, such as post-secondary education or vocational training, than those entering employment-focused (particularly LFA). However, this is not to say that programs with high rates of participation in higher-cost activities cannot pay for themselves: The mixed-approach Portland program had the highest returns to the government budget.

Table 13.11
From the Government Budget Perspective:
Estimated Monetary Gains and Losses per Program Group Member
Within a Five-Year Follow-Up Period,
by High School Diploma or GED Status at Random Assignment (in 1999 dollars)

Component of Analysis

 Atlanta LFA ($) Atlanta HCD ($) Grand Rapids LFA ($) Grand Rapids HCD ($) Riverside LFA ($) Riverside HCD ($) Columbus Integrated ($) Columbus Traditional ($) Detroit ($) Portland ($)

Without a high school diploma or GED

Gains
Payroll taxesa 154 -0 438 154 321 189 414 182 207 684
Income and sales tax -272 133 -236 24 -613 -418 260 133 92 -423
Welfare payments 774 249 3,218 2,240 3,121 3,058 2,207 1,393 518 2,628
Food Stamps -14 -227 849 776 967 1,040 1,489 809 444 273
Medicaid 964 276 1,183 1,019 533 767 1,674 1,014 411 1,858
Transfer administration 137 9 705 529 504 543 539 318 172 1,058
Total 1,742 440 6,156 4,741 4,833 5,178 6,583 3,850 1,843 6,077

Losses
Net cost of program and nonprogram activities and services -3,741 -6,140 -592 -3,645 -3,555 -4,572 -4,383 -3,039 -1,406 -3,290
Net gain or loss(net present value) -1,999 -5,700 5,564 1,096 1,278 606 2,200 811 437 2,787
Return to budget per net dollar invested in program and nonprogram activities and servicesb 0.47 0.07 10.40 1.30 1.36 1.13 1.50 1.27 1.31 1.85

With a high school diploma or GED

Gains
Payroll taxesa 466 453 72 103 421   169 198 178 685
Income and sales tax 338 143 -335 -138 82   -141 -104 109 42
Welfare payments 1,014 1,034 2,359 1,572 2,539   1,124 969 576 2,939
Food Stamps 732 378 513 159 878   759 587 238 1,132
Medicaid 1,290 964 906 598 481   713 840 211 3,066
Transfer administration 296 231 502 306 431   269 231 139 1,390
Total 4,137 3,204 4,016 2,599 4,832   2,893 2,721 1,452 9,253

Losses
Net cost of program and nonprogram activities and services -4,209 -5,073 -2,177 -3,972 -2,775   -4,305 -4,096 -2,488 -2,948
Net gain or loss (net present value) -72 -1,869 1,839 -1,373 2,057   -1,412 -1,375 -1,036 6,305
Return to budget per net dollar invested in program and nonprogram activities and servicesb 0.98 0.63 1.84 0.65 1.74   0.67 0.66 0.58 3.14
SOURCES: See Table 13.8.
NOTES:  Estimates reflect discounting and adjustment for inflation.
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members.
Rounding may cause slight discrepancies in calculating sums and differences.
Tests of statistical significance were not performed.
a Payroll taxes include employer- and employee-paid Social Security and Medicare taxes.
b The return to budget per net dollar invested is computed by dividing total savings and tax increases by the net cost of activities and services.

[Go To Contents]

XI. Sensitivity of the Results

As is common in cost-benefit analyses, assumptions were made in producing the estimates in this analysis. Owing to data limitations, several major assumptions had to be made about the levels of participation in follow-up years 3 to 5, as well as the proportion of participation that was considered in-program or out-of-program. The Five-Year Client Survey asked about participation at any point during the follow-up period, as well as specifically about participation in the last year of the follow-up period. This information was put together with data from the Two-Year Client Survey, which provided more detailed information about participation during the two-year period following program entry, to classify the likelihood of having participated during follow-up years 3 and 4. For the cost-benefit analysis, sample members were considered to have participated during years 3 and 4 if there was a good chance that they could have participated during this period.

In addition, the available data could not provide details on whether participation took place as part of the program or as a self-initiated activity. Therefore, in the cost-benefit analysis, it was assumed that the proportion of in-program participation increased with the number of months of welfare receipt during the period in question. This was the best approximation of the participation patterns seen in the detailed two-year data used in earlier reports.

To measure the effect of these two assumptions, a sensitivity test was conducted (see the following table). In this analysis, the alternative assumptions were that only those with the highest likelihood of having participated in years 3 and 4 did so, and all participation was evenly divided between "in-program" and "out-of-program" for those who received welfare for a portion of the period.(25) The resulting gross and net costs, as well as the net present value from the perspective of government budgets, are presented below. (The welfare perspective is unaffected by program costs.)

Site and Program

 Gross Cost Net Cost ($) From the Government Budget Perspective
Program Group ($) Control Group ($) Net Gain or Loss (NPV) ($) Return to Budget per Net Dollar Invested in Program and Nonprogram Activities and Services ($)
Atlanta LFA 6,432 3,006 3,426 -222 0.94
Atlanta HCD 7,843 3,006 4,837 -2,616 0.46
Grand Rapids LFA 8,311 6,513 1,798 3,097 2.72
Grand Rapids HCD 9,993 6,513 3,480 -15 1.00
Riverside LFA 5,280 2,538 2,742 2,123 1.77
Columbus Integrated 6,158 2,433 3,724 703 1.19
Columbus Traditional 5,782 2,433 3,348 -222 0.93
Detroit 5,478 3,751 1,727 -3 1.00
Portland 8,456 5,884 2,572 5,528 3.15

Using these alternative assumptions, gross costs were lower by 5 percent (Portland) to 13 percent (Riverside LFA) for program group members and by 3 percent (Atlanta LFA and HCD and Portland) to 7 percent (Riverside LFA) for control group members. The alternative assumptions resulted in reductions in net costs ranging from 8 percent (Grand Rapids HCD) to 17 percent (Riverside LFA).

As these results show, the point estimates of gross and net costs are indeed sensitive to the assumptions made. However, using these lower cost estimates, the same conclusions were drawn with regard to the ranking of the various sites overall and between program approaches in the NEWWS Evaluation. In addition, the comparisons with the GAIN programs also remain largely unaffected. In the benefit-cost analysis, the lower cost estimates result in very few changes to the bottom line for government budgets: Although there were changes in the magnitude of gains and losses, none of the estimates changed direction. In the sites where there were small losses to government budgets using the original assumptions, the losses are smaller yet, and these programs could be considered to have reached the "break-even" point.

[Go To Contents]

XII. Conclusions

The costs of services were relatively high everywhere. This was true for controls as well. Gross costs for employment-focused programs were similar to those for education-focused programs. However, net costs were lower for employment-focused programs. Similarly, LFA programs continued to have lower costs than HCD programs.

The NEWWS programs mostly left sample members worse off. Where there were gains to the welfare sample, they were very small. In half of the programs, the average earnings gains over the five-year period were smaller than reductions in welfare and Food Stamps. In Atlanta, which had the lowest welfare benefit levels, program group members had less to lose. As a result, both programs in this site produced earnings gains that were greater than the loss in combined welfare, Food Stamp, and Medicaid benefits. Detroit was the only other program to produce earnings gains greater than its reductions in transfer benefits. However, it bears repeating that the gains were very small and could easily have become losses had leisure and out-of-pocket work expenditures been included.

Except for Portland and Grand Rapids LFA, sample members without a high school diploma or GED at random assignment fared worse than those with these credentials regardless of the program approach. However, even among those who entered the evaluation with a high school diploma or GED, programs mostly left sample members worse off.

Although earnings increases were too small to cover benefit losses for program group members, in most programs the savings to government budgets were not large enough to cover their investments. The three programs that produced gains to government budgets were all employment-focused and had among the lowest net costs. Similar results were seen for sample members who entered the programs with a high school diploma or GED. That is, within this subgroup, the four employment-focused programs broke even or produced gains to government budgets, and all of the education-focused programs produced losses. For the subgroup without these credentials, all programs, except for the two in Atlanta, produced gains to government budgets. In general, the greater investment of education-focused programs did not pay off for government budgets or for program group members.

For the full samples, no program produced gains to both the welfare sample and government budgets. This was true for the educational attainment subgroups as well, except for nongraduates in Portland: Small gains ($1,084 over five years) accrued to nongraduates in that program, and the government budget experienced a moderate gain of $2,787. (Detroit produced a small gain to the government budget but broke even from the perspective of the welfare sample.)

[Go To Contents]

Endnotes

1. See Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001.

2. See the two-year reports mentioned in footnote 1 for more detailed information on the calculation of unit costs.

3. As noted in Chapter 2, unlike those in other programs in this analysis, Columbus sample members were randomly assigned to the two programs prior to an orientation. These differences in sample composition are reflected in the cost and benefit outcomes.

4. A unit cost for basic education in Detroit is not available. Table 13.2 presents an "education and training" unit cost for Detroit, which combines vocational training, post-secondary education, and basic education. (See footnote "e" on Table 13.2.)

5. The Portland, Grand Rapids, and Detroit programs required welfare recipients with children as young as age 1 to participate in the NEWWS program, whereas the requirement extended only to recipients with children as young as age 3 in the other programs. The Grand Rapids and Detroit programs did not have particularly high child care costs.

6. A comparison of child care costs for the Columbus Integrated and Traditional programs does not support the idea that the case management approach alone was responsible for this result.

7. The Two-Year and Five-Year Client Survey data, along with welfare payment records, were used to estimate participation in employment-related activities that sample members took part in when they were not receiving welfare.

8. In Riccio, Friedlander, and Freedman, 1994, the GAIN costs were presented in 1993 dollars and were as follows: Alameda, $6,977; and Los Angeles, $6,402.

9. A portion of job search participation that occurred after the embargoes were lifted was considered program-related. The proportion of participation that was considered in-program was estimated based on the number of months that sample members received welfare in the post-embargo period. All participation in education and training activities was considered to be self-initiated.

10. The Riverside comparison includes only HCD and LFA program group members without a high school diploma or GED at random assignment.

11. Many of the techniques were originally developed for the benefit-cost analysis conducted as part of the MDRC Demonstration of State Work/Welfare Initiatives (for additional information, see Long and Knox, 1985). In this report the description of the analytical approach was adapted from previous MDRC reports (Riccio, Friedlander, and Freedman, 1994; Kemple, Friedlander, and Fellerath, 1995; Miller et al., 2000; and Bloom et al., 2000a).

12. In programs such as these, particularly education-focused programs, many costs are incurred early, when welfare receipt is heaviest; however, many benefits, such as earnings gains, continue to be realized in later years. Simply comparing the nominal dollar value of program costs with benefits over multiple years would be problematic because a dollar's value is greater in the present than in the future: A dollar available today, to either sample members or the government, can be invested and may produce income over time, making it worth more than a dollar available in the future. In order to make a fair comparison between benefits and costs over multiple years, it is essential to determine their value at a common point in time — for example, the present. In benefit-cost analyses, this is often accomplished by discounting, a method for reducing the value of benefits and costs accrued in later years relative to benefits and costs accrued in early years. In this analysis, the end of the first year following random assignment was used as the comparison point for the investment period. In other words, gains that were accrued after that point were discounted to reflect their value at the end of year 1. In calculating these discounted values, it was assumed that a dollar invested at the end of year 1 would earn a real rate of return of 5 percent annually (this assumption has been used in other MDRC benefit-cost analyses).

13. Some work-related expenditures were paid by the welfare agencies and are reflected as a cost to government budgets in the support services line.

14. The presentation of benefit-cost results in this report was adapted from previous MDRC reports (Miller et al., 2000; Bloom et al., 2000a).

15. Data limitations did not allow for an accounting of out-of-pocket costs paid by sample members for health care coverage.

16. These rates were based on published information on employers' compensation costs for 1995 from the U.S. Department of Labor, Bureau of Labor Statistics.

17. This analysis does not account for out-of-pocket child care costs. With the exception of the Portland program, no out-of-pocket cost differences were found between program and control group members in the programs for which Five-Year Client Survey data were available.

18. The federal Earned Income Tax Credit is a credit against federal income taxes for taxpayers with annual earnings below a certain level. For 1995, taxpayers with earnings up to $26,673 were eligible for the EITC. Not all eligible taxpayers receive the EITC. As has been the practice in earlier benefit-cost analyses performed by MDRC, the EITC "take-up" rate was estimated at 80 percent.

19. Income from earnings was used in calculating federal and state income taxes. Income from earnings and welfare benefits was used in calculating sales and excise taxes. Sales and excise tax rates were based on information from state tax boards.

20. This analysis does not include an accounting of public health care coverage for children through programs for low-income families, such as the Child Health Insurance Program.

21. Take-up rates for programs without Five-Year Client Survey data were estimated from those with these data.

22. Average statewide Medicaid costs per eligible month were calculated using 1995 data from the Health Care Financing Administration Web site (www.hcfa.gov).

23. The costs of administering welfare, Food Stamp, and Medicaid benefits were estimated using statewide administrative cost data.

24. One of the hypotheses of the case management experiment in Columbus was that integrated case management would allow more efficient delivery of services. As discussed in Scrivener and Walter, 2001, integrated case managers reported seeing such benefits. This analysis assumes that the costs of administering benefits were the same for both programs. Because the five-year administration net costs were relatively low ($379 for the Integrated program and $265 for the Traditional program), even if the Integrated program had realized some savings in this area, the overall cost of the program would still have been slightly higher than the cost for the Traditional program. That is, the combined net operating and transfer administration costs for the Traditional program were $4,037, about $150 less than the net operating cost of the Integrated program.

25. Both scenarios counted all participation as "in-program" for those who received welfare for the full period and all participation as "out-of-program" for those who did not receive any welfare during the entire period.

Where to?

Top of Page | Contents

Home Pages:
National Evaluation of Welfare-to-Work Strategies (NEWWS)
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)