[ Main Page of Report | Contents of Report ]
Chapters 4-6 revealed the overall effects on economic outcomes of the NEWWS programs studied. With regard to earnings, for example, employment-focused programs had larger immediate effects than education-focused programs, Portland had large and persistent effects, and education-focused programs in Detroit and Oklahoma City had relatively small effects. This chapter investigates whether some groups were affected more or less than others. In particular, program effects on average earnings, welfare benefits, and income are compared for long-term and short-term welfare recipients; those who had worked in the year prior to random assignment and those who had not; by race and ethnicity; and for groups defined by whether they faced multiple barriers to work as long-term welfare recipients, high school dropouts, or long-term unemployed.
Knowing how welfare-to-work services affect various subgroups can help in deciding where to target new resources or where to develop new services. For example, recipients who are more disadvantaged and who are likely to have the most difficulty finding a job will be particularly at risk of losing income if they lose eligibility for benefits under TANF. Programs that showed positive effects for these recipients may serve as good models under time-limited welfare.
Results for more job-ready recipients are also of interest. Programs may only have assisted these individuals to secure jobs more quickly than they would have otherwise, but have had little long-term effect. If that was the case, targeting them might have been an inefficient use of resources. On the other hand, the programs might have helped job-ready recipients find higher-quality jobs, which could very well have had substantial positive effects on their long-term earnings. The results for job-ready subgroups can inform this debate, which is likely to become more heated as states continue to try different strategies under TANF.
[Go to Contents]
[Go to Contents]
Subgroups were organized around three barriers to employment: high school education (high school graduates compared with nongraduates), recent work experience (those who had worked in the year prior to random assignment compared with those who had not), and welfare history (those who had ever been on welfare at least two years prior to random assignment compared with those who had not). Results by high school credential were presented in Chapters 4-6. This chapter presents results for the other barriers. In addition, the three individual barriers to employment were used to define three mutually exclusive subgroups based on relative disadvantage. The "most disadvantaged" were sample members who did not have a high school diploma or GED prior to random assignment, did not work in the year prior to random assignment, and were on welfare for two years or more prior to random assignment. "Moderately disadvantaged" sample members faced only one or two of the three barriers, while the "least disadvantaged" faced none. Finally, results are presented in this chapter for racial and ethnic groups.
Subgroups were identified using information collected just before individuals were randomly assigned. Because these groups were defined by pre-existing characteristics observed at study enrollment, control and program group members in the subgroup should be comparable at the time of random assignment, and any systematic differences that emerge between the two groups can be reliably attributed to the programs being studied.
The chapter presents results for three outcomes: (1) earnings, (2) cash assistance, and (3) combined income from earnings, cash assistance, Food Stamps, and the federal Earned Income Credit (EITC) net of payroll taxes. The three outcomes represent three different perspectives. Many policymakers want to encourage welfare recipients to work; for them, the "best" program may be the one that increases earnings the most. Other policymakers may be primarily interested in reducing spending on welfare; for them the best program may be the one that reduces cash assistance the most. Welfare recipients and policymakers concerned about child and family poverty may care most about their total income; for them, the best program may be the one that increases income the most.
For each outcome, the chapter focuses on cumulative dollar amounts over a five-year follow-up period. Although use of program services by control group members might have reduced the effects of some programs in years 4 and 5, five years of follow-up are used for two reasons. First, the program effects over five years are generally similar to their effects over three years. Second, an earlier report presented a detailed analysis of subgroup impacts over three years.(1)
In analyzing subgroups, several types of comparisons are made, with each comparison answering a different question. The first question is whether there is evidence that the welfare-to-work programs taken as a whole — without regard to the approach they used or where they operated — tended to affect a particular subgroup. For example, with welfare time limits, administrators probably want to make sure that long-term welfare recipients are able to find work and leave welfare; if the programs were not particularly effective at benefiting long-term recipients, policymakers might want to target them for more resources or devise different and better services.
A second question is whether the programs tended to have larger effects for one subgroup than another. Again, the answer to this question can help policymakers think about how to use their precious resources or whether new services should be developed. Suppose that long-term recipients were generally affected by the programs being studied. Suppose, however, that they were affected less than short-term recipients. This might suggest that more effort or different services should be considered for long-term recipients to increase the effectiveness of welfare-to-work services.
A third question is whether one program approach benefited a subgroup more than another approach or whether it benefited one subgroup more than another subgroup. Because there are fewer programs of each type, however, statements about the effects of particular program models might be more speculative. For example, Portland is the only NEWWS program that was employment-focused with varied first activities. Although its impacts on earnings were by far the largest — and this chapter shows that the effects were also the largest for most subgroups — it cannot be determined whether this is a consequence of Portland's approach, the way sample members were chosen, the Portland economy (or other local factors), or unobserved differences between welfare recipients in Portland and in the other sites.
Since the number of people in a subgroup is, obviously, less than the number in the full sample, it is consequently more difficult to confidently say that an individual program had an effect for a subgroup than it is for the full sample, and it is more difficult to say whether the estimated effects are bigger for one group than another because of the program rather than by chance. However, the pattern of impacts across programs can provide statistical evidence that the programs taken as a whole had a particular effect, even if no individual program had a statistically significant effect. For example, suppose the question is whether welfare-to-work programs have a larger impact on earnings for long-term welfare recipients or for short-term welfare recipients. If 9 or more programs had a larger impact for long-term recipients than for short-term recipients, say, the hypothesis that the impacts are the same for the two groups can be rejected at the 10 percent significance level — even if no single program had a statistically significant different effect for long-term recipients than for short-term recipients. Likewise, if 10 or more programs have an impact in the same direction, we can reject the hypothesis of no difference at the 5 percent significance level, and if all 11 programs have an impact in the same direction, we can reject the hypothesis of no difference at the 1 percent significance level.(2)
The most rigorous means of examining the effects of program models is to compare the effects of the three LFA programs with their HCD counterparts. The chapter consequently devotes a section to this comparison. With only three programs of each type, however, it can be difficult to draw firm conclusions about the relative benefits of the two approaches by subgroup. As mentioned above, it is harder to find statistically significant effects for a subgroup than for the full sample and unlikely that the impacts between two subgroups will be statistically significantly different in any specific site. Moreover, three sites are too few to use only the pattern of results to draw conclusions about the relative effectiveness of the two approaches unless the differences in impacts between the two approaches are large. If, for example, the LFA and HCD approaches are equally effective, then the chance that all three LFA programs would have larger impacts than all three HCD programs would be 12.5 percent, or greater than the usual threshold for drawing conclusions based on statistical significance. However, it is extremely unlikely that all three LFA programs would increase earnings significantly more than all three HCD programs simply by chance, and statistically significant differences in all three sites would be enough to draw solid conclusions based on the statistical evidence.
[Go to Contents]
The Family Support Act (FSA) of 1988, the provisions of which helped shape the programs studied in this report, was designed to help individuals who were most likely to become long-term welfare recipients. The FSA required states to target this group for welfare-to-work resources and to offer services that were thought to provide them the greatest benefit. An important question about the NEWWS programs, therefore, is whether the programs had positive effects for long-term recipients.
Table 7.1 shows impacts on earnings; cash assistance payments; and income from earnings, cash assistance, Food Stamps, and the federal EITC net of payroll taxes, all measured over the five years following random assignment. Impacts are shown for each program for two subgroups: long-term welfare recipients (those who had been on welfare for two years or more prior to random assignment) and short-term welfare recipients and welfare applicants (those who had been on welfare for less than two years prior to random assignment). To allow comparisons between the two Riverside programs to be made, results for the Riverside LFA program are presented both for the entire sample and for sample members considered in need of basic education (the only group assigned to the Riverside HCD program).
Long-term recipients. Table 7.1 shows that the NEWWS programs did increase earnings for long-term recipients. In all of the programs, long-term recipient program group members had higher earnings over three years than long-term recipient control group members. Impacts were highest in Portland, at nearly $6,000 over five years, with most programs having impacts between $1,000 and $3,700 per person. In all cases but three, moreover, the differences were statistically significant.
Site and Program |
SampleSize | Average Total Earnings in Years1 to 5 ($) | Average Welfare Payments in Years 1 to 5 ($) | Combined Income in Years 1 to 5 ($) |
|---|---|---|---|---|
On welfare for two years or more |
||||
| Atlanta Labor Force Attachment | 2,063 | 2,522*** | -1,137*** | 1,002 |
| Atlanta Human Capital Development | 2,100 | 2,059** | -856*** | 1,080 |
| Grand Rapids Labor Force Attachment | 1,791 | 1,221 | -2,931*** | -2,206** |
| Grand Rapids Human Capital Development | 1,775 | 205 | -2,121*** | -2,341** |
| Riverside Labor Force Attachment | 3,510 | 3,657*** | -3,427*** | -568 |
| Lacked high school diploma or basic skills | 1,831 | 3,016*** | -3,302*** | -901 |
| Riverside Human Capital Development | 1,841 | 2,582*** | -3,018*** | -1,177 |
| Columbus Integrated | 3,392 | 2,523*** | -1,686*** | -176 |
| Columbus Traditional | 3,415 | 1,424* | -1,154*** | -129 |
| Detroit | 3,313 | 1,710* | -860*** | 370 |
| Oklahoma City | 2,076 | 699 | n/a | n/a |
| Portland | 2,443 | 5,859*** | -2,905*** | 2,415 |
On welfare for less than two years |
||||
| Atlanta Labor Force Attachment | 840 | 2,262 | -227 | 1,623 |
| Atlanta Human Capital Development | 847 | 1,540 | -419 | 850 |
| Grand Rapids Labor Force Attachment | 1,219 | 2,125 | -2,002*** | -280 |
| Grand Rapids Human Capital Development | 1,215 | 1,603 | -1,241*** | 169 |
| Riverside Labor Force Attachment | 3,101 | 1,506 | -1,979*** | -1,111 |
| Lacked high school diploma or basic skills | 1,248 | 1,550 | -2,565*** | -1,624 |
| Riverside Human Capital Development | 1,238 | -415 | -2,855*** | -4,066*** |
| Columbus Integrated | 806 | -843 | -1,468*** | -3,362 |
| Columbus Traditional | 793 | 1,291 | -1,018*** | -917 |
| Detroit | 1,015 | 1,298 | -16 | 1,300 |
| Oklahoma City | 2,683 | -236 | n/a | n/a |
| Portland | 1,494 | 4,410* | -2,638*** | 1,538 |
| SOURCES: MDRC calculations from unemployment insurance
(UI) earnings records and AFDC records. NOTES: Impacts on earnings were significantly different across subgroups in Riverside HCD. Impacts on AFDC were significantly different across subgroups in Atlanta LFA and Grand Rapids LFA. N/a = not applicable. |
||||
In all 10 programs for which welfare benefits could be measured, long-term recipients in the program group received less in cash assistance than long-term recipients in the control group. In part, this is a natural consequence of going to work. In all sites, welfare recipients' cash benefits were reduced somewhat when their earnings increased. In a number of programs, however, impacts on cash assistance were greater than impacts on earnings. In addition, as discussed in Chapter 5, sanctions would have resulted in welfare savings over and above the program effects on earnings, particularly in Grand Rapids and Columbus Integrated. Alternatively, some people may have left welfare because they were unable or unwilling to comply with program requirements, and others may not have returned to welfare when they lost their jobs.
Because the programs generally resulted in higher earnings but less cash assistance for long-term recipients than would have occurred otherwise, they generally had relatively small effects on income from earnings, cash assistance, Food Stamps, and projected EITC payments net of payroll taxes. For programs outside Grand Rapids, the impacts on income were small enough that they could not reliably be attributed to the programs. Moreover, the estimated impact was negative in six programs and positive in four, providing a further indication that these programs did not systematically affect income. Nevertheless, the results in Grand Rapids may be considered troubling. The reductions in income that were seen overall (in Chapter 6) appear to be concentrated among long-term recipients.
Short-term recipients. Table 7.1 indicates that the programs also generally had effects for short-term welfare recipients, although not as consistently as for long-term recipients. In all cases, short-term recipient program group members had lower cash assistance payments than their control group counterparts (though the difference was statistically significant in only 7 of the 10 programs). In only 8 of the 11 programs, however, did they have higher earnings — and the earnings impact was statistically significant for short-term recipients only in Portland. Particularly troubling are the impacts for the Riverside HCD short-term recipients whose earnings were virtually unchanged, but whose cash assistance was reduced by more than $2,800 over five years. As a result of the program, short-term recipient program group members received more than $4,000 less in income from earnings, cash assistance, and Food Stamps than short-term recipient control group members. The Riverside HCD program was not alone: The Columbus Integrated program reduced income by more than $3,300 for program group members.
Comparing short-term and long-term recipients. Should long-term recipients, short-term recipients, or both be targeted for new services? One means of addressing the question is to look at the relative effects of the programs on the two groups. Even though many of the programs were effective for both groups, long-term recipients were generally helped more than short-term recipients. In 9 of the 11 programs (the only exception was in Grand Rapids), the impact on earnings for long-term recipients was greater than the impact on earnings for short-term recipients. This suggests that the approach of the programs studied in this report was effective at increasing the earnings of long-term recipients.
If a goal of welfare reform is to increase income, however, then these programs were generally equally ineffective for long-term recipients and short-term recipients. If they are not already doing so, states should consider further supplementing the earnings of welfare recipients or recent welfare recipients through enhanced earnings disregards, state Earned Income Tax Credits, or other means, to make it more likely that the programs that encourage them to work also help them to obtain greater financial resources.(3)
Another means of asking whether one group or another should be targeted for future services is to look at their outcomes. If the earnings levels of long-term recipients remained low despite the fact that the programs were generally effective for this group it might suggest the need for more or different services to further ameliorate their barriers to work. Across the 11 programs there is generally a large gap in the earnings levels of the two groups: earnings of long-term recipients were generally about 60 to 75 percent of earnings for short-term recipients (result not shown in Table 7.1). For example, in the Atlanta LFA program, earnings for long-term recipients in the program groups were about $16,500 over five years compared with nearly $28,000 for short-term recipients.
Welfare-to-work programs are likely to have small effects if they offer services only to people whose barriers to work are so serious that they cannot take advantage of the services. People without much work experience could represent such a group; the fact that they have not worked much may indicate the presence of barriers that are keeping them from working (if not the possibility that they would rather raise their children than work). Alternatively, welfare-to-work services could have small effects if control group members are so likely to work on their own (that is, without the benefit of program services) that services are unable to improve their outcomes. People who have worked recently clearly have the ability to find work and may represent such a group.
Recent work history does predict future earnings well and may help identify groups that could benefit from welfare-to-work services and groups that have less need to benefit. Across the 11 programs people who had worked in the year prior to random assignment earned about twice as much as people who had not.(4) The most extreme differences occurred in the Riverside HCD program, where control group members who had worked in the year prior to random assignment earned about $19,000 on average in the five years after random assignment, and control group members who had not worked earned less than $7,000. (In Portland, however, those who had worked recently earned only about 50 percent more than those who had not. This could reflect Portland's very strong economy, the removal of the most disadvantaged participants from the study through up-front screening, or some other factor.)
Table 7.2 addresses whether these differences in ability to earn without employment services translated into differential impacts for the two groups. The table offers evidence — but far from overwhelming evidence — that those who have not worked recently do benefit more from welfare-to-work services, at least in terms of increasing their earnings.
Site and Program |
Sample Size | Average Total Earnings in Years 1 to 5 ($) | Average Welfare Payments in Years 1 to 5 ($) | Combined Income in Years 1 to 5 ($) |
|---|---|---|---|---|
Did not work in year prior to random assignment |
||||
| Atlanta Labor Force Attachment | 1,869 | 3,763*** | -928*** | 2,323*** |
| Atlanta Human Capital Development | 1,937 | 3,110*** | -780*** | 2,020** |
| Grand Rapids Labor Force Attachment | 1,527 | 1,630 | -3,103*** | -2,157** |
| Grand Rapids Human Capital Development | 1,489 | 543 | -2,112*** | -1,896* |
| Riverside Labor Force Attachment | 4,010 | 3,024*** | -3,027*** | -751 |
| Lacked high school diploma or basic skills | 2,074 | 1,986*** | -3,286*** | -1,978** |
| Riverside Human Capital Development | 2,065 | 1,863*** | -2,969*** | -1,950** |
| Columbus Integrated | 2,143 | 2,925*** | -1,506*** | 531 |
| Columbus Traditional | 2,160 | 2,641*** | -1,225*** | 938 |
| Detroit | 2,978 | 904 | -400 | 260 |
| Oklahoma City | 3,910 | 245 | n/a | n/a |
| Portland | 2,317 | 6,276*** | -3,620*** | 2,035 |
Worked in year prior to random assignment |
||||
| Atlanta Labor Force Attachment | 1,069 | 370 | -782** | -691 |
| Atlanta Human Capital Development | 1,055 | 238 | -666** | -402 |
| Grand Rapids Labor Force Attachment | 1,485 | 1,622 | -2,007*** | -586 |
| Grand Rapids Human Capital Development | 1,508 | 1,240 | -1,478*** | -565 |
| Riverside Labor Force Attachment | 2,716 | 1,788 | -2,22*** | -1,073 |
| Lacked high school diploma or basic skills | 1,051 | 2,958* | -2,320*** | 202 |
| Riverside Human Capital Development | 1,070 | 403 | -2,723*** | -2,962* |
| Columbus Integrated | 2,529 | 1,374 | -1,533*** | -1,227 |
| Columbus Traditional | 2,569 | 486 | -987*** | -1,160 |
| Detroit | 1,481 | 2,592* | -907** | 1,155 |
| Oklahoma City | 4,767 | 13 | n/a | n/a |
| Portland | 1,711 | 3,522 | -1,620*** | 1,408 |
| SOURCES: MDRC calculations from unemployment insurance
(UI) earnings records and AFDC records. NOTES: Impacts on earnings were significantly different across subgroups in Atlanta LFA. Impacts on AFDC were significantly different across subgroups in Grand Rapids LFA and Portland. Impacts on total income were significantly different across subgroups in Atlanta LFA. N/a = not applicable. |
||||
Those without recent work experience. The programs generally led to higher earnings among those without recent work experience, which implies that the services helped those who had not worked recently find jobs (although there are many in this group who did not work much). In all 11 programs, earnings were higher for program group members who had not worked recently than for their control group counterparts.
Once again, however, positive impacts on earnings did not generally translate into positive impacts on income. The most positive findings were in Atlanta, where the programs resulted in income gains of more than $2,000 compared with what would have occurred otherwise. This might reflect Atlanta's fill-the-gap budgeting, which allowed working welfare recipients to keep more of their welfare checks than they kept in other sites, or the relatively low welfare benefit levels in Georgia.
Those with recent work experience. In all 11 programs, earnings were higher for program group members who had worked recently than for their control group counterparts. However, the difference was statistically significant only in Detroit, and the impacts were typically not very big. Only in Detroit and Portland did the impact on earnings exceed $2,000. Despite the relatively small gains in earnings, the programs universally led to reduced welfare benefits for this group. In fact, the modest earnings gains combined with systematic reductions in welfare benefits led to the most systematic reductions in income for any subgroup: In 8 of the 10 programs for which income could be measured, program group members with recent work experi-ence ended up with lower income than their control group counterparts, though only in the Riverside HCD program was this difference large enough to be statistically significant.
Comparing those with and without recent work experience. The differences between those who had worked and those who had not are somewhat greater than the differences between long-term and short-term recipients. In the two Atlanta programs, for example, earnings impacts were virtually the same for long-term and short-term recipients, but were about $3,000 to $3,500 greater for those without recent work experience than for those with recent work experience. In Portland, the impact on earnings for the group without recent work experience was nearly twice as large as the impact for those with recent work experience. In 9 of the 11 programs, in fact, impacts on earnings were larger for those who had not worked recently than for those who had. All of this suggests that people who have not worked in a while benefit the most (in terms of increasing their earnings and self-sufficiency, but not in terms of increasing their income) from these types of welfare-to-work programs.
In summarizing results from several welfare-to-work programs from the 1980s, Friedlander found that earnings gains were concentrated neither among groups of long-term recipients, who are expected to have a hard time finding work and leaving welfare, nor among groups such as new welfare applicants, who are most likely to work without assistance from a welfare-to-work program. Instead, Friedlander found the largest earnings gains among a middle group of welfare applicants who had spent some but not a great deal of prior time on welfare. In contrast, welfare savings came primarily from long-term recipients, especially those without a high school diploma or with little recent work experience.(5)
Do the NEWWS sites provide evidence that their approach changed these patterns by making the impacts for long-term recipients closer to those for short-term recipients? Results already presented in this chapter differ from Friedlander's, in that impacts on earnings were generally larger for the more disadvantaged group than for the less disadvantaged group. Table 7.3 shows impacts for three groups defined by three barriers to work. (As noted above, the most disadvantaged did not have a high school diploma or GED prior to random assignment, did not work in the year prior to random assignment, and were on welfare for two or more years prior to random assignment. The moderately disadvantaged faced only one or two of the three barriers, while the least disadvantaged faced none.)
Site and Program |
Sample Size | Average Total Earningsin Years1 to 5 ($) | Average Welfare Payments in Years1 to 5 ($) | Combined Income in Years 1 to 5 ($) |
|---|---|---|---|---|
Most disadvantaged |
||||
| Atlanta Labor Force Attachment | 698 | 1,946* | -766** | 1,285 |
| Atlanta Human Capital Development | 734 | -290 | -243 | -452 |
| Grand Rapids Labor Force Attachment | 458 | 3,994*** | -4,028*** | -828 |
| Grand Rapids Human Capital Development | 453 | 1,566 | -2,886 | -2,136 |
| Riverside Labor Force Attachment | 1,362 | 2,938*** | -3,592*** | -1,395 |
| Riverside Human Capital Development | 1,362 | 3,279*** | -2,973*** | -445 |
| Columbus Integrated | 911 | 2,206* | -2,047*** | -1,055 |
| Columbus Traditional | 901 | 302 | -1,198*** | -1,318 |
| Detroit | 1,119 | 1,187 | -1,219** | -827 |
| Oklahoma City | 429 | 740 | n/a | n/a |
| Portland | 617 | 3,162 | -2,862** | 491 |
Moderately disadvantaged |
||||
| Atlanta Labor Force Attachment | 1,887 | 3,406*** | -1,020*** | 1,772* |
| Atlanta Human Capital Development | 1,911 | 2,617** | -904*** | 1,489 |
| Grand Rapids Labor Force Attachment | 2,123 | 270 | -2,396*** | -2,510** |
| Grand Rapids Human Capital Development | 2,078 | 934 | -1,749*** | -1,187 |
| Riverside Labor Force Attachment | 4,298 | 3,035*** | -2,547*** | -204 |
| Riverside Human Capital Development | 3,049 | -1,307 | -2,185*** | -3,902*** |
| Columbus Integrated | 3,155 | 2,663** | -1,432*** | 235 |
| Columbus Traditional | 3,236 | 1,790* | -1,060*** | 253 |
| Detroit | 3,018 | 2,173** | -271 | 1,740* |
| Oklahoma City | 6,170 | 255 | n/a | n/a |
| Portland | 2,803 | 5,960*** | -3,318*** | 1,935 |
Least disadvantaged |
||||
| Atlanta Labor Force Attachment | 353 | -1,121 | -119 | -1,347 |
| Atlanta Human Capital Development | 347 | 709 | -514 | 161 |
| Grand Rapids Labor Force Attachment | 431 | 1,998 | -1,145* | 665 |
| Grand Rapids Human Capital Development | 466 | 1,869 | -408 | 1,664 |
| Riverside Labor Force Attachment | 1,066 | 516 | -1,871*** | -2,162 |
| Columbus Integrated | 606 | -3,413 | -991*** | -4,916* |
| Columbus Traditional | 592 | -1,951 | -1,184*** | -4,376 |
| Detroit | 322 | -894 | -1,201 | -2,483 |
| Oklahoma City | 2,078 | -448 | n/a | n/a |
| Portland | 608 | 2,430 | -159 | 1,883 |
| SOURCES: MDRC calculations from unemployment
insurance (UI) earnings records and AFDC records. NOTES: Impacts on earnings were significantly different across subgroups in Grand Rapids LFA, Riverside LFA, Riverside HCD, and Columbus Integrated. Impacts on AFDC benefits were significantly different across subgroups in Grand Rapids LFA and HCD and Portland. Impacts on total income were significantly different across subgroups in Riverside HCD, Columbus Traditional, Columbus Integrated, and Detroit. N/a = not applicable. |
||||
This method of defining the most and the least disadvantaged does a good job of finding groups that would fare well and poorly on their own, in terms of their earnings levels. In the five years after random assignment, average earnings for the most disadvantaged control group members ranged from less than $4,000 in Oklahoma City to about $11,500 in Detroit (not shown in Table 7.3). In contrast, earnings levels for the least disadvantaged control group members were at least three times higher in all sites, ranging from about $20,000 in Oklahoma City to nearly $45,000 in Columbus (and more than $40,000 in Detroit).
Examining subgroups by level of disadvantage may also help find more precisely defined groups who are helped most and least by the programs. This may help policymakers and welfare administrators decide whether the more disadvantaged or the less disadvantaged sample members benefit more from employment-focused or education-focused activities. It may also help them determine whether new services are needed for the more disadvantaged or the less disadvantaged sample members. In this sense, results by level of disadvantage are in the spirit of recent work on "profiling."(6)
Impacts for the most disadvantaged. These welfare-to-work programs generally increased earnings for the most disadvantaged, but their effects for this group were not as strong as for either long-term welfare recipients or those who had not worked in the year prior to random assignment. In 10 of the 11 programs, the most disadvantaged program group members earned more than their control group counterparts. Because there were relatively few people in this group, the impacts on earnings were statistically significant in only five of the programs. While several programs were moderately successful, increasing earnings by about $3,000 or more, just as many were not very effective, with impacts on earnings close to zero.
Impacts on welfare benefits were more systematic, with significant reductions in 9 of the 10 programs for which they could be measured. This same pattern was seen for the other subgroups discussed above: more systematic changes in welfare payments than in earnings levels. As a result, the most disadvantaged program group members had lower income than their control group counterparts in 8 of the 10 programs (though the impact was not statistically significant in any of the programs).
These results are somewhat different than the results for the two barriers described above, where earnings impacts were quite strong for those who had not worked in the year prior to random assignment and for long-term welfare recipients. This could imply that it is particularly difficult to assist people with three barriers to work rather than possibly just one. On the other hand, it could indicate an important drawback to this method of counting barriers to work: Not all barriers are equal. In particular, Michalopoulos and Schwartz indicate that earnings gains were generally larger for people with a high school credential than for those who lacked one.(7) Thus, two of the barriers appear to be related to greater effects of the welfare-to-work programs, while a third barrier appears to be related to smaller effects of the programs. In such circumstances, understanding the effects of multiple barriers requires more sophisticated methods than the categorization shown in Table 7.3
Comparing impacts by level of disadvantage. If Friedlander's 1988 findings on pre-FSA programs hold for programs in the NEWWS Evaluation, then Table 7.3should indicate that impacts are larger for the moderately disadvantaged than for the most disadvantaged. It does not. The impacts of the NEWWS programs on earnings were generally more broadly distributed than the impacts of the programs studied by Friedlander. Seven of the 11 programs had significant impacts on earnings for the moderately disadvantaged group, but 5 programs also had significant impacts on earnings for the most disadvantaged group. In addition, in both Grand Rapids programs and the Riverside HCD program earnings impacts were much larger for the most disadvantaged than for the moderately disadvantaged, while in the two Atlanta programs and the Portland program the opposite was true.
In one way, however, results in Table 7.3 are similar to results from the pre-FSA programs studied by Friedlander: The programs had little effect on earnings for the least disadvantaged. Impacts on cumulative earnings were not statistically significant for any of the programs. Moreover, just as many programs left the least disadvantaged program group members with higher earnings than their control group counterparts as left them with lower earnings. Despite the modest effects on earnings, each of the programs reduced cash assistance amounts — four of them significantly so — although the programs did not have a consistently negative effect on income.
The final subgroups discussed in this chapter are defined by race and ethnicity. Results are presented for three groups: white, African-American, and Hispanic.(8) If minority sample members are faring much worse under these programs than white sample members, it might be a signal to policymakers and administrators that something is preventing minority sample members from fully participating in or benefiting from the programs. It may mean that the services offered were not enough to overcome additional barriers to employment often faced by minority group members, such as living far from available jobs, having language barriers, and being discriminated against by employers.
Results in Table 7.4 provide little reason to be concerned. In general, impacts on earnings were larger for African-American and Hispanic sample members than for white sample members. In seven of the nine programs where comparisons between white and African-American sample members could be made (Atlanta had too few white sample members to make reliable comparisons), impacts on earnings were larger for African-American sample members. For example, the Grand Rapids LFA program increased earnings by nearly $2,400 over five years for African-American sample members, but had virtually no effect on earnings for white sample members. In all programs except Portland and Detroit, in fact, the average impact on earnings for African-American sample members exceeded the average impact for white sample members by more than $1,100 over five years.
Site and Program |
Sample Size | Average Total Earnings in Years 1 to 5 ($) | Average Welfare Payments in Years1 to 5 ($) | Combined Income in Years1 to 5 ($) |
|---|---|---|---|---|
White, non-Hispanic |
||||
| Atlanta Labor Force Attachment | n/a | n/a | n/a | n/a |
| Atlanta Human Capital Development | n/a | n/a | n/a | n/a |
| Grand Rapids Labor Force Attachment | 1,470 | -275 | -2,302*** | -3,168*** |
| Grand Rapids Human Capital Development | 1,515 | -394 | -1,569*** | -2,467** |
| Riverside Labor Force Attachment | 3,464 | 1,565* | -2,350*** | -1,426 |
| Lacked high school diploma or basic skills | 1,245 | 2,158* | -2,818*** | -1,148 |
| Riverside Human Capital Development | 1,208 | 623 | -2,482*** | -2,430* |
| Columbus Integrated | 2,161 | 1,762 | -1,489*** | -734 |
| Columbus Traditional | 2,204 | 73 | -829*** | -1,104 |
| Detroit | 481 | 2,794 | -1,129 | 819 |
| Oklahoma City | 5,109 | -241 | n/a | n/a |
| Portland | 2,754 | 6,343*** | -2,971*** | 2,730* |
Black, non-Hispanic |
||||
| Atlanta Labor Force Attachment | 2,791 | 2,385*** | -831*** | 1,180 |
| Atlanta Human Capital Development | 2,838 | 1,828** | -718*** | 998 |
| Grand Rapids Labor Force Attachment | 1,214 | 2,367** | -2,462*** | -162 |
| Grand Rapids Human Capital Development | 1,158 | 999 | -1,472*** | -445 |
| Riverside Labor Force Attachment | 1,121 | 3,775** | -2,248*** | 934 |
| Lacked high school diploma or basic skills | 501 | 1,743 | -2,580** | -1,233 |
| Riverside Human Capital Development | 510 | 1,576 | -2,578** | -1,614 |
| Columbus Integrated | 2,414 | 2,528** | -1,528*** | 8 |
| Columbus Traditional | 2,431 | 2,395** | -1,265*** | 390 |
| Detroit | 3,836 | 1,375 | -557** | 500 |
| Oklahoma City | 2,484 | 1,238* | n/a | n/a |
| Portland | 798 | -278 | -1,740* | -1,754 |
Hispanic |
||||
| Atlanta Labor Force Attachment | n/a | n/a | n/a | n/a |
| Atlanta Human Capital Development | n/a | n/a | n/a | n/a |
| Grand Rapids Labor Force Attachment | 244 | 4,973* | -4,211*** | -464 |
| Grand Rapids Human Capital Development | 249 | 6,270** | -4,002*** | 999 |
| Riverside Labor Force Attachment | 1,858 | 4,357*** | -3,732*** | -420 |
| Lacked high school diploma or basic skills | 1,210 | 3,639*** | -3,669*** | -981 |
| Riverside Human Capital Development | 1,240 | 3,018*** | -3,536*** | -1,682 |
| Columbus Integrated | n/a | n/a | n/a | n/a |
| Columbus Traditional | n/a | n/a | n/a | n/a |
| Detroit | n/a | n/a | n/a | n/a |
| Oklahoma City | 392 | -546 | n/a | n/a |
| Portland | n/a | n/a | n/a | n/a |
| SOURCES: MDRC calculations from unemployment insurance
(UI) earnings records and AFDC records. NOTES: Impacts on earnings were significantly different across subgroups in Grand Rapids LFA and HCD, Riverside LFA and HCD, and Portland. Impacts on AFDC benefits were significantly different across subgroups in Grand Rapids LFA and HCD. N/a = not applicable. |
||||
Despite these differences in the impacts on earnings, the impacts on welfare benefits were fairly similar for African-American and white sample members. In the Grand Rapids LFA program, for example, the impact on welfare benefits was about $2,300 for white sample members and about $2,500 for African-American sample members. As a result, both programs in Riverside and Grand Rapids significantly reduced income for white sample members but not for African-American sample members.
In the five programs with enough Hispanic sample members to allow impacts to be reliably measured, program effects on earnings were generally larger for Hispanic sample members than for white or African-American sample members, although comparisons can be made across the three groups for only the four programs in Grand Rapids and Riverside because most of the sites had too few Hispanic sample members to provide reliable estimates of program impacts. The large earnings gains for Hispanic sample members were accompanied by large welfare reductions.
One exception to the positive results for African-American sample members was in Portland, where earnings increased by more than $6,000 for white sample members, but were little changed for African-American sample members. This result should be interpreted with a great deal of caution, however. The Portland sample used in this report contains only 101 African-American control group members, and the estimated effects for African-American sample members are consequently quite imprecise. In examining impacts through three years, Michalopoulos and Schwartz were able to use the entire Portland control group and found that the program increased earnings for African-American sample members by about $2,000 over three years.(9)
The reasons for the large discrepancy in the Portland effects are unclear. One possible explanation is that African-American control group members had higher rates of job search, employment, and earnings than white control group members, which made it relatively harder to generate employment and earnings gains for African-Americans. According to the Five-Year Client Survey, about 47 percent of African-American control group members reported receiving some sort of job search assistance after entering the study compared with about 32 percent of white control group members. Perhaps as a result of their greater efforts to look for work, African-American control group members worked more often than white control group members (9.2 quarters on average compared with 7.4 quarters) and they earned considerably more than white control group members (about $24,000 over five years compared with less than $20,000, shown in Appendix Table F.4). However, it is unclear why African-American control group members had such high rates of employment.
Perhaps because of the high rates of participation in job search by African-American control group members in Portland, the program effect on job search participation was somewhat lower for African-Americans than for whites (a 22 percentage point increase for African-Americans compared with a 30 percentage point increase for whites in the five years following random assignment). The smaller effect on job search assistance was offset somewhat by larger effects on any education and training for African-American sample members (a 24 percentage point increase for African-Americans compared with an 11 percentage point increase for whites). In addition, the program affected different types of education and training activities for the two groups, with the program's effect on vocational training concentrated among African-American sample members, its effect on basic education concentrated among white sample members, and its effect on post-secondary education about the same for white and African-American sample members. It is important to note that the program increased both job search assistance and education and training for both white and African-American sample members, so these results do not imply that lack of job search assistance or undue reliance on education were the cause of the low earnings gains for African-Americans.
[Go To Contents]
In comparing the LFA and HCD programs in Atlanta, Grand Rapids, and Riverside, it is difficult to draw strong conclusions about the relative effectiveness of the job-search-first and education-first approaches over a five-year period. Across the 10 subgroups discussed in this chapter, only three impacts on earnings were numerically larger in an HCD program, of 27 comparisons that were made. This suggests that the LFA programs were somewhat more effective than the HCD programs at increasing earnings.
However, differences between the impacts of the two approaches were generally fairly small and generally not statistically significant. Moreover, for no subgroup was the evidence compelling. The largest differences between the two approaches were for the most disadvantaged sample members. In Atlanta, the earnings impact was more than $2,000 greater for the LFA program than for the HCD program ($1,946 compared with -$290; see Table 7.3). In Grand Rapids, the difference was nearly $2,500 ($3,994 compared with $1,566). However, in Riverside, the impact was slightly greater for the HCD program than for the LFA program.
Overall, the results suggest that these welfare-to-work programs, both education-focused and employment-focused, increased earnings and decreased welfare receipt for a wide range of subgroups. In particular, they suggest that the approach of the Family Support Act of 1988 was reasonably successful: The welfare-to-work programs did generally result in higher earnings for long-term welfare recipients.
On the other hand, for the least disadvantaged (those sample members who have a high school diploma, have recent work experience, and have little prior welfare history) none of the programs increased earnings significantly, and the programs as a whole were just as likely to increase earnings as to reduce them. This may reflect, again, the fact that the programs were forced by the provisions of the Family Support Act to concentrate resources on the most disadvantaged. Alternatively, it could be interpreted as evidence that concentrating services on the least disadvantaged is an inefficient use of resources. Perhaps more advanced training, training that builds on skills already in place, is needed for the least disadvantaged.
Impacts by subgroup bolster evidence from the full sample regarding the most successful programs and program approaches. LFA programs tended to have larger effects on earnings and welfare benefits for most subgroups, although differences were typically fairly small. The Portland program was the most effective for the broadest range of subgroups. Of the seven subgroups compared (not counting those by race and ethnicity), Portland had the largest effect on earnings for six of the subgroups. This may reflect the program's unique use of both job search and education as initial program activities, but it might also reflect the program's use of job development, its experience running job search programs, its willingness to exempt welfare recipients who were perceived to be the hardest-to-employ from welfare-to-work services, or the interaction of these features with the city's robust economy during this period.
1. Michalopoulos and Schwartz, 2001.
2. The number of programs that produced a larger effect for one group than for another follows a binomial distribution. Significance levels were determined using this distribution, under the hypothesis that the programs had equally large effects for all subgroups. For example, if welfare-to-work programs do not affect the earnings of long-term welfare recipients, then the chance that all 11 programs would have had higher earnings for long-term recipient program group members than for long-term recipient control group members is 0.1 percent. Likewise, the chance that 10 or more of the programs would have had higher earnings for long-term recipient program group members than for long-term recipient control group members is 1.17 percent, and the chance that 9 or more of the programs would have had higher earnings for long-term recipient program group members than for long-term recipient control group members is 6.54 percent.
3. Michalopoulos and Berlin, 2001.
4. Michalopoulos and Schwartz, 2001.
5. Friedlander, 1988.
6. For example, Eberts, 1997; and Rangarajan, Schochet, and Chu, 1998.
7. Michalopoulos and Schwartz, 2001.
8. A few sample members were not part of any of these three groups. Because only Oklahoma City and Riverside had more than a handful of these people, results for them are not shown in Table 7.4.
9. Michalopoulos and Schwartz, 2001.
Home Pages:
National Evaluation of Welfare-to-Work Strategies
(NEWWS)
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)