[ Main Page of Report | Contents of Report ]
For the past 30 years, federal and state policymakers have been looking for new and better ways to increase the employment of welfare recipients. Beginning in the late 1960s, in response to dissatisfaction with the Aid to Families with Dependent Children (AFDC) program, one of the nation's principal safety nets for poor families, Congress began to reshape it, creating a program to encourage welfare recipients to find jobs. In 1988 the Family Support Act (FSA) established a system of mutual obligation within the AFDC benefit entitlement structure: Government was to provide education, employment, and support services to AFDC recipients, who were, in turn, required to participate in the Job Opportunities and Basic Skills Training (JOBS) program. The most recent federal reform effort, the 1996 Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA), replaced AFDC with a flexible, state-directed block grant program, Temporary Assistance for Needy Families (TANF); limited most families to five years of lifetime federal TANF assistance (with some states setting shorter welfare time limits); and created financial incentives for states to run mandatory, work-focused welfare-to-work programs. PRWORA gives states more flexibility than they had before in designing their programs (which has encouraged, for example, some states to implement generous financial work incentives or tough financial sanctions for noncooperation) and more responsibility for moving the nation's poor into the labor market.
Mandatory, work-focused welfare-to-work programs are thus not a new idea. The challenge for welfare policymakers and state and local program administrators, however, is to determine how to design and implement such programs so as to best achieve the goal of fostering adults' long-term economic self-sufficiency without unintentionally jeopardizing the well-being of their children. This report provides guidance on the topic by analyzing the long-term effectiveness of 11 programs in seven sites that aimed to move substantial numbers of people from welfare to work. These programs were begun under FSA but operated under TANF in the later years of the evaluation. (The results in this report pertain to the time period between 1991 and 1999.) Overall, the programs shared TANF's primary goal of moving people from welfare to work. Further, they reflect a range of approaches, implementation features, and environments: Some were strongly employment-focused while others emphasized basic education; they varied in how broadly the program participation mandate was applied to the welfare caseload and how strictly it was enforced, in the amount of child care support provided for program participation or employment, and in methods of case management; and the programs served different welfare populations and operated in a variety of labor markets. Finally, all 11 programs were studied using a strong random assignment design, resulting in reliable information about their relative effectiveness.
This report analyzes the comparative long-term effectiveness of two very different types of programs: employment-focused programs and education-focused programs. Results from the 11 programs studied provide evidence on the relative effectiveness of these two approaches in two ways: First, an innovative research design implemented in three sites is used to directly compare versions of these two approaches within each site, resulting in unusually reliable information about their relative effectiveness. Second, cross-site comparisons of all 11 programs studied are used to determine if more examples of the two approaches, operating in different labor markets and for different populations, uphold or refine the findings from the direct comparisons in three sites. In addition, the cross-site comparisons are used to ascertain if any versions of these approaches emerge as particularly successful. Results are presented for two key subgroups of welfare recipients for whom different program approaches might be expected to work differently: those with a high school diploma or GED and those without these credentials. Covering a five-year follow-up period and using a wealth of data pertaining to single parents (mostly mothers) as well as their children, the report thus addresses the following critical question: What works best and for whom?
The five-year results presented in this report were produced as part of the National Evaluation of Welfare-to-Work Strategies (NEWWS). This evaluation is being conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services (HHS), with support from the U.S. Department of Education. Child Trends, as a subcontractor, is conducting the Child Outcomes Study (COS), the part of the evaluation that examines effects on young children. The NEWWS Evaluation includes programs in seven sites across the country: Atlanta, Georgia (Fulton County); Grand Rapids, Michigan (Kent County); Riverside, California (Riverside County); Columbus, Ohio (Franklin County); Detroit, Michigan (Wayne County); Oklahoma City, Oklahoma (Oklahoma, Cleveland, and Pottawatomie counties); and Portland, Oregon (Multnomah and Washington counties).(1) In these seven locations, more than 55,000 people were randomly assigned to research groups as part of the study.
This chapter begins by presenting a framework for understanding the five-year results and then describes the research questions and design of the overall NEWWS Evaluation. Next, the environments in which the 11 programs studied were operated are discussed, along with the most critical program features. This is followed by an examination of the welfare-to-work program treatments experienced by individuals in the various evaluation research groups over time, given the long follow-up examined in the report. Finally, two-year NEWWS Evaluation results are recapped, and the contents of Chapters 2-13 are briefly described.
[Go to Contents]
The FSA gave program administrators a great deal of flexibility in designing the 11 programs studied in the National Evaluation of Welfare-to-Work Strategies. (Box 1.1 provides a comparison of the key features of FSA and TANF.) That flexibility, combined with local economic, political, and funding environments, resulted in 11 programs that vary on several dimensions. This report focuses primarily on one of those dimensions: the self-sufficiency approach used.
FSA/AFDC (1988-1996)
PRWORA/TANF (1996-2002)
|
While the overarching goal of programs in the past 30 years — to foster the self-sufficiency of welfare recipients through increased employment and decreased welfare receipt — has not changed, there has been disagreement on how best to move individuals from welfare to work. One strategy emphasizes quick employment, reflecting the belief that individuals can best build their employability and improve their skills, eventually achieving self-sufficiency, through actual work, even if their initial jobs are minimum wage and without fringe benefits. The other strategy emphasizes initial investments in short-term education and, in some cases, training, reflecting the view that these investments will eventually enable individuals to obtain higher-wage, longer-lasting jobs with health insurance coverage. Most programs have blended the two strategies and emphasized elements of both. Past research has shown that a program's location on the continuum between these two strategies and the mix of services it provides to enrollees can have an effect on the patterns and magnitude of program impacts measured in the short and long term.(2)
The programs in this report have been categorized by their approach: either employment-focused or education-focused. Three sites in the evaluation (Atlanta, Grand Rapids, and Riverside) simultaneously implemented a Labor Force Attachment (LFA) program and a Human Capital Development (HCD) program, versions of employment-focused and education-focused programs that magnified the differences between the two types of approaches. The six programs in these three sites provide the best test of the relative effectiveness of the two approaches.(3) Another site, Columbus, was also asked to implement two different programs in a head-to-head test. One program used an "integrated case management" staffing structure, in which one worker assumes responsibility for both eligibility and employment and training for her clients. The other program used a "traditional case management" staffing structure, in which separate workers handle the eligibility and employment and training duties. These programs, called the Columbus Integrated and Traditional programs, both used an education-focused approach.(4) Program administrators in the other three sites chose which self-sufficiency approach to implement based on their own goals. Of the 11 programs studied, four programs (Atlanta LFA, Grand Rapids LFA, Riverside LFA, and Portland) were employment-focused; the remaining seven were education-focused.
In the three LFA versions of the employment-focused program, almost all enrollees were first assigned to job search. In Portland, the other employment-focused program, many, but not all, individuals were assigned to job search as a first activity. Some individuals, usually those who were determined to have more barriers to work than other members of the caseload, were first assigned to education or training activities. In the three HCD education-focused programs, as well as in the four other education-focused programs, almost all individuals were first assigned to either education or occupational skills training activities.
This report also focuses on another program implementation dimension: mandatoriness. Past research suggests that the degree to which a program enforces a participation mandate for the welfare caseload is a determinant of whether a program can have an effect.(5) High or low enforcement of the mandate is a product of three factors: how wide a cross section of the welfare caseload is enrolled in a program; how closely a program monitors individuals' participation; and how swiftly and consistently a program imposes financial sanctions (that is, reductions in monthly welfare grants) on those who do not participate. Nine of the programs were high enforcement programs; Detroit and Oklahoma City were not, mostly because of limited program and staff resources.
Table 1.1 categorizes the 11 programs according to their self-sufficiency approach and level of enforcement of the participation mandate. Section IV of this chapter discusses in greater detail these dimensions of the programs, as well as others that may have affected program impacts. It is important to keep in mind, however, that these program categorizations and descriptions apply to these particular programs as they would have been experienced by evaluation sample members in their first three years of follow-up. In the years corresponding to sample members' fourth and fifth years of follow-up, some of the programs changed their approach (in response to TANF). Section V of this chapter describes these changes.
| Employment-focused approach | Education-focused approach | ||
|---|---|---|---|
Job search first |
Varied first activity | Education or training first | |
High enforcement |
High enforcement | High enforcement | Low enforcement |
| Atlanta LFA Grand Rapids LFA Riverside LFA |
Portland | Atlanta HCD Grand Rapids HCD Riverside HCD Columbus Integrated Columbus Traditional |
Detroit Oklahoma City |
NOTES: LFA = Labor Force Attachment program. HCD = Human Capital Development program. |
|||
[Go to Contents]
Within the categorization scheme described above, the report analyzes program effects for single-parent welfare recipients and their children, focusing on results for the five years after individuals entered the programs. Data included in this report are from administrative records (unemployment insurance, state and county welfare payments, and Food Stamp payments) and from client surveys and parent and child assessments conducted over the five years after individuals entered the study. The report specifically addresses the following questions:
The NEWWS Evaluation uses an unusually rigorous research design, a random assignment experiment, to estimate program effects. In each site individuals who were required to participate in the program were assigned, by chance, to either a program group, which had access to employment and training services and whose members were required to participate in the program or risk a reduction in their monthly welfare grant, or a control group, whose members were not subject to a participation mandate and received no services through the program, but could seek out such services from the community.(6) This random assignment design ensures that there are no systematic differences between the background characteristics of people in the program and control groups within each site when they enter the study. Thus, any subsequent differences in outcomes between the groups can be attributed with confidence to the programs. These differences, called impacts, are the primary focus of this report.
Although this design assures that the impact estimates of each program are extremely reliable, there are some limitations. Local conditions, including labor markets, prevailing wages, welfare grant levels, political environments, program funding levels, and staff administration, can all have an effect on the magnitude of impact estimates. For this reason, comparisons of impacts across the 11 employment- and education-focused programs in this report are only suggestive of the relative effectiveness of either approach in the short term.(7) The most definitive judgments on the relative effectiveness of the two approaches come from the results in the three sites in this evaluation that tested versions of the two approaches side by side.
Finally, it should be kept in mind that in some sites it was not possible to bar all control group members from receiving welfare-to-work program services for the entire five-year follow-up period examined in this report. While no control group members were exposed to the services provided and mandates imposed by sites' welfare-to-work programs for the first three years of the follow-up period, their status differed by site in the fourth and fifth years, as detailed in Section VI of this chapter. As a result, in several sites impacts measured for years 4 and 5 of follow-up may underestimate the effects that would have been found had the control embargo been in effect for a full five years in all sites. Given this uncertainly, some of the analyses throughout the report separate impacts for years 1 to 3 from those for years 4 and 5.
[Go to Contents]
When planning this evaluation, HHS and MDRC sought to include sites that would demonstrate operation in a diverse range of conditions, though they would not represent all welfare-to-work programs in the country. As shown in Table 1.2, sites varied along several dimensions, such as geographic location, labor market, and welfare grant level.(8) More striking, however, are the changes in the economy, and the concomitant changes in welfare policy and caseloads, that all sites experienced from 1991, when the first group of individuals was randomly assigned as part of the NEWWS Evaluation, to 1999, the end of the five-year follow-up period for the last group of individuals randomly assigned. (Random assignment took place over a roughly two-year time period in each site, falling somewhere between mid 1991 and the end of 1994, depending on the site.) Part of the richness of the evaluation results thus stems from the examination of program effects over a period of time that reflected unprecedented economic growth and a sea change in welfare policies.(9)
Characteristic |
Atlanta | Grand Rapids | Riverside | Columbus | Detroit | Oklahoma City | Portland |
|---|---|---|---|---|---|---|---|
| Population, 1990 | 648,779 | 500,631 | 1,170,413 | 961,437 | 2,111,687 | 832,624 | 895,441 |
| Population, 1997 | 722,540 | 539,425 | 1,447,791 | 1,017,274 | 2,127,087 | 889,411 | 1,015,954 |
| Population growth, 1990-1997 (%) | 11.4 | 7.7 | 23.7 | 5.8 | 0.7 | 6.8 | 13.5 |
| Employment growth, 1991-1996 (%) | 14.8 | 15.9 | 11.9 | 7.7 | 5.4 | 9.1 | 15.1 |
| 1991-1999 | 25.1 | 28.9 | 28.2 | 11.8 | 12.8 | 16.5 | 17.8 |
| Unemployment rate (%) | |||||||
| 1991 | 5.3 | 7.8 | 9.8 | 3.8 | 10.5 | 6.0 a | 5.4 b |
| 1992 | 7.4 | 7.5 | 11.6 | 4.6 | 10.5 | 5.5 a | 7.3 b |
| 1993 | 6.4 | 5.3 | 11.9 | 4.5 | 8.3 | 5.5 a | 6.6 b |
| 1994 | 5.8 | 4.2 | 10.5 | 3.7 | 6.7 | 5.0 a | 4.9 b |
| 1995 | 5.4 | 3.8 | 9.6 | 2.9 | 6.0 | 4.0 a | 4.1 b |
| 1996 | 4.9 | 4.0 | 8.2 | 2.9 | 5.5 | 3.6 a | 5.2 b |
| 1997 | 4.6 | 3.2 | 7.5 | 2.7 | 4.7 | 3.5 a | 5.0 b |
| 1998 | 4.1 | 2.9 | 6.6 | 2.5 | 4.3 | 3.9 a | 4.9 b |
| 1999 | 3.8 | 3.1 | 5.5 | 2.5 | 4.2 | 2.7 a | 5.2 b |
| AFDC/TANF caseload c | |||||||
| 1991 | 18,507 | 7,660 | 23,325 | 23,192 | 87,992 | 12,305 | 11,234 |
| 1992 | 21,801 | 7,389 | 25,581 | 24,135 | 88,584 | 13,392 | 11,817 |
| 1993 | 23,113 | 7,508 | 27,775 | 24,739 | 89,083 | 14,259 | 11,961 |
| 1994 | 23,121 | 7,137 | 32,044 | 24,807 | 88,337 | 14,257 | 11,981 |
| 1995 | 22,043 | 7,052 | 24,650 | 23,240 | 88,614 | 13,959 | 11,231 |
| 1996 | 19,620 | 5,836 | 25,076 | 19,474 | 74,051 | 12,488 | 10,097 |
| 1997 | 15,754 | 5,362 | 23,519 | 17,363 | 70,052 | 10,239 | 6,721 |
| 1998 | 12,007 | 4,551 | 19,278 | 13,341 | 59,060 | 8,273 | 5,306 |
| 1999 | 8,907 | 3,303 | 15,091 | 11,082 | 43,278 | 7,280 | 5,229 |
| Welfare grant level for a family of 3 ($) | |||||||
| 1993 | 280 | 474 | 624 | 341 | 459 | 324 | 460 |
| 1998 | 280 | 459 | 611 | 362 | 459 | 292 | 503 |
| Food Stamp benefit level for a family of 3 ($) d | |||||||
| 1993 | 292 | 252 | 202 | 292 | 252 | 292 | 287 |
| 1998 | 329 | 314 | 321 | 329 | 314 | 329 | 310 |
| Income disregard policies | Standard; fill-the-gap |
Standard e | Extended; fill-the-gap |
Standard | Standard e | Standard | Standard |
Maximum that a family of 3 could earn and receive AFDC, January 1993 ($) |
|||||||
| In months 1-4 of employment | 756 | 831 | 1,175 | 632 | 809 | 606 | 810 |
| In months 5-12 of employment | 544 | 594 | 823 | 461 | 579 | 444 | 580 |
| After 12 months of employment | 514 | 564 | 793 | 431 | 549 | 414 | 550 |
Maximum that a family of 3 could earn and receive TANF, December 1998 ($) |
|||||||
| In months 1-4 of employment | 756 | 774 | 1,447 | 974 | 774 | 704 | 616 |
| In months 5-12 of employment | 544 | 774 | 1,447 | 974 | 774 | 704 | 616 |
| After 12 months of employment | 514 | 774 | 1,447 | 974 | 774 | 704 | 616 |
| SOURCES: U.S. House of Representatives, 2000;
Gaquin and Littman, 1999; Hall and Gaquin, 1997; Hamilton and Brock, 1994;
Hamilton et al., 1997; Scrivener et al., 1998; State Policy Documentation
Project; U.S. Department of Labor, Bureau of Labor Statistics; Center on
Social Welfare Policy and Law, 1994; Center for Law and Social Policy, 1995;
site contacts. NOTES: Data are for counties: Atlanta (Fulton County), Georgia; Grand Rapids (Kent County), Michigan; Riverside (Riverside County), California; Columbus (Franklin County), Ohio; Detroit (Wayne County), Michigan; Oklahoma City (Oklahoma, Cleveland, and Pottowatomie Counties), Oklahoma; Portland (Multnomah and Washington Counties), Oregon. a Data are for Oklahoma County. The unemployment rates for Cleveland County are: 1991, 4.4%; 1992, 3.5%; 1993, 3.5%; 1994, 3.5%; 1995, 2.9%; 1996, 2.6%; 1997, 2.6%; 1998, 3.2%; 1999, 2.1%. The unemployment rates for Pottowatomie County are: 1991, 7.6%; 1992, 5.9%; 1993, 5.8%; 1994, 5.7%; 1995, 4.5%; 1996, 4.8%; 1997, 4.8%; 1998, 5.4%; 1999, 3.7%. b Data are for Multnomah County. The unemployment rates for Washington County are: 1991, 4.5%; 1992, 6.1%; 1993, 5.3%; 1994, 3.7%; 1995, 3.2%; 1996, 3.9%; 1997, 3.8%; 1998, 3.9%; 1999, 4.1%. c Annual average monthly caseloads, as reported by the state or county. In Atlanta and Portland averages are for calendar years; in all other sites averages are for state fiscal years. d Assumes the receipt of the maximum AFDC/TANF payment. e Although Michigan implemented nonstandard earned income disregards during the evaluation period through the To Strengthen Michigan Families initiative, all sample members in the NEWWS Evaluation were excluded from them. |
|||||||
To be included in the NEWWS Evaluation, sites needed large enough welfare caseloads to meet the sample size requirements of the research design. Accordingly, all seven sites include urban areas. Detroit, with a population that was slightly over 2 million in both 1990 and 1997, is the largest urban area studied in the evaluation. Riverside, with a population of over 1 million in 1990, experienced the most growth during this time period, adding almost 24 percent to its population by 1997. Population growth in the other sites during this seven-year period ranged from 6 to 14 percent.(10)
As population grew, so did labor markets. In three sites employment expanded significantly between 1991 and 1999: the employed labor force grew by 29 percent in Grand Rapids, 28 percent in Riverside, and 25 percent in Atlanta. The other four sites experienced 12 to 18 percent gains.
Rising employment, particularly in localities with rising population, does not necessarily indicate declining unemployment rates. Unemployment rates in all seven sites, however, decreased over this period. Following national trends, unemployment rates peaked in 1992 and, in general, were lowest in 1999. Early in the evaluation period unemployment rates in Detroit and Riverside topped 10 percent. Although rates in both localities steadily declined, Riverside rates remained at 8 percent in 1996, significantly higher than the national average. By 1999, unemployment rates in all evaluation sites were below 6 percent. Throughout the evaluation period the Columbus labor market was notably robust; its unemployment rate never exceeded 5 percent, even during the high point of the national recession, and dropped to less than 3 percent in 1999.
Because individuals in the program and control groups within each site were subject to the same labor market, the quality of the economy by itself should not necessarily affect impact estimates; program and control groups shared the same advantages of a tight labor market or disadvantages of a slack one. However, different economic environments can influence the type of people receiving welfare and thus required to participate in welfare-to-work programs. For example, in a good labor market individuals with more serious barriers to work are likely to be left on the welfare rolls.
The size of welfare (AFDC or TANF) caseloads varied with the size of site populations, ranging in 1991, the beginning of the evaluation, from about 7,500 in Grand Rapids to almost 90,000 in Detroit. In general, welfare caseloads grew in the early part of the evaluation period, peaked in 1993 or 1994, and declined to their 1991 levels or below by 1996, shrinking further to 1999. In almost all evaluation sites, welfare caseloads in 1999 were no more than half the size they were in 1991; decreases in Riverside and Oklahoma City were somewhat smaller, with caseloads reduced by 35 and 41 percent, respectively.
There was considerable variation in welfare grant levels among the sites. In 1993 monthly maximum cash payments for a family of three ranged from $280 in Atlanta to $624 in Riverside. Welfare grant levels in 1998 were similar to 1993 levels, with slight reductions in the maximum amount payable in three sites, a slight increase in two sites, and identical levels in two sites. Food Stamp payments, for which means standards are federally set, varied less across the sites, from $202 in Riverside to $292 in Atlanta, Columbus, and Oklahoma City in 1993, with slightly higher benefit levels in all sites in 1998.(11) To some extent, low welfare grants are offset by higher Food Stamp payments, but this does not change the overall rankings of sites on benefit levels.
All states were required to disregard (that is, not count) some earned income when calculating a family's welfare grant and, over time, more generous earnings disregards were put into place in some of the evaluation sites. At the beginning of the NEWWS Evaluation, five sites applied standard earnings disregard rules. Under these, for the first four months of employment $120 of earnings and an additional one-third of the remainder were disregarded. This $120 disregard included both a $30 flat disregard and a $90 disregard for work expenses. In months 5-12 of employment, the additional one-third disregard was eliminated, leaving the total disregard at $120. After the first year of employment only the $90 work expenses disregard was allowed. In addition, individuals were allowed to disregard child care expenses up to $175 per child aged 2 or over and $200 per child under age 2.(12) Atlanta and Riverside applied nonstandard disregard rules that permitted employed recipients to keep more of their welfare check. Throughout the evaluation period Atlanta employed "fill-the-gap" budgeting. Under fill-the-gap, working welfare recipients can earn up to the state-determined "standard of need" before losing all welfare benefits. For example, in 1993 the standard of need for a family of three was $424 (per month). A parent with two children could earn up to $756 in each of the first four months of employment and still remain on AFDC, $544 in months 5-12, and $514 per month thereafter.
Throughout the course of the NEWWS Evaluation five-year follow-up period, states implemented further earnings disregard policies. In California, for example, the state received a waiver at the end of 1993 to eliminate the time limit on the standard earnings disregard applied to the calculation of welfare benefits and also instituted a version of fill-the-gap. By 1998, relative to 1993, all but one of the evaluation sites had implemented policies that allowed welfare recipients to keep more of their earnings, affecting the likelihood that a sample member could work while remaining on welfare. Increases were greatest in Riverside where, in 1998, a family of three could earn as much as $1,447 and still receive TANF. In Columbus and Oklahoma City, welfare recipients in 1998 similarly could earn almost double what they could in 1993 and remain eligible for welfare.
Differences in welfare grants, earnings disregard standards, and the use of fill-the-gap budgeting may explain some variation in program impacts, even though these grant levels and policies applied to both program and control group members in each site. Impacts on welfare payments in low-grant states are likely to be somewhat lower than those in high-grant states, other things being equal, because there are fewer welfare dollars to reduce. In addition, in low-grant states even low-paying jobs may be more attractive than welfare, providing a greater incentive to work. At the same time, in states that have higher grant levels, or generous earnings disregards, it may be easier for individuals to combine work and welfare in a way that will increase total household income and raise the family standard of living, particularly after factoring in the Earned Income Tax Credit (EITC).(13)
[Go to Contents]
This report presents the long-term impacts, or effects, that the 11 NEWWS programs had on outcomes such as employment, earnings, welfare receipt, and child and family well-being. This section provides a context for interpreting the results in the chapters that follow by showing the range of programs on key implementation dimensions. As will become apparent, there is no typical "package" of welfare-to-work program features. As examples, the most work-focused programs are not necessarily the toughest, and those that use integrated case management do not necessarily monitor their enrollees' progress more effectively than others. Given this information, it is important to interpret each program's impacts as a result of its entire "bundle" of services and features. It is likely that a combination of features rather than one specific feature is associated with successful outcomes and specific impacts.
Nonetheless, this section focuses on the two implementation dimensions used to categorize the 11 NEWWS programs: the self-sufficiency approach used and the level of participation mandate enforcement. These particular dimensions are discussed at length here because they clearly demonstrate the division between the programs and provide a general framework for thinking about program results. In addition, this section explores other implementation features that provide an important context for interpreting the impact results in later chapters.
It should be kept in mind that sample members were subject to the welfare-to-work programs being studied only as long as they were receiving welfare. Welfare recipients frequently cycle on and off the welfare rolls. In the first two years of follow-up in these programs, for example, sample members received welfare and were required to participate in the programs for an average of 9 to 17 months, depending on the site. On average, sample members were either participating in a program activity, employed while subject to the program, or sanctioned for program nonparticipation between 22 and 68 percent of these months, depending on the site.(14) (Those not in one of these statuses in any given month may have been waiting for case managers to refer them to an activity or sanction them, waiting for support services to be arranged or for activities to begin, temporarily deferred from participation for reasons such as illness or caring for an ill relative, or temporarily "lost track" of by program staff.) While such figures on program "dosage" might seem surprising, it is important realize that the goal of welfare-to-work programs is to enable individuals to leave welfare and/or get a job. As a result, one would hope that sample members had not been participating in program activities during every month in the follow-up period, since this would mean that they had never left welfare and/or found employment during the period.
As noted earlier, the implementation features discussed here primarily relate to how sample members would have experienced the programs in the first three years of follow-up in the evaluation. Section V describes changes in program focus in the last years of the five-year follow-up period analyzed in the report. Table 1.3 summarizes for all programs the implementation features discussed in this section.
| Program | Degree of Emphasis on Self-Sufficiency Approach | Degree of Enforcement of the Participation Mandate | Type of Child Care Support Provided for Participation | Extent of Partnership Between Eligibility and Self-Sufficiency Staff | ||||
|---|---|---|---|---|---|---|---|---|
| Employment | Education | Enrollment | Monitoring | Sanctioning | Message | Availability | ||
| Atlanta LFA | High | Moderate | Broad- delayed |
Moderate | High | Encouraged use; licensed care only | No shortage | Limited |
| Atlanta HCD | Low | High | Broad- delayed |
Moderate | High | Encouraged use; licensed care only | No shortage | Limited |
Grand Rapids LFA |
High | Low | Broad | High | Very high | Suggested use; choice of provider | No shortage | Limited |
Grand Rapids HCD |
Low | High | Broad | High | Very high | Suggested use;choice of provider | No shortage | Limited |
Riverside LFA |
High | Low | Broad | High | High | Encouraged low-cost, informal care | Occasional shortage | Limited |
Riverside HCD |
Moderate | High | Broad | High | High | Encouraged low-cost, informal care | Occasional shortage | Limited |
Columbus Integrated |
Low | High | Broad | Moderate | Very high | Suggested use; choice of provider | No shortage | Strong |
Columbus Traditional |
Low | High | Broad | Low | Very high | Suggested use; choice of provider | No shortage | Limited |
Detroit |
Low | High | Selective | Low | Low | Organizational emphasis on providing assistance; choice of licensed or approved provider | No shortage | Very limited |
Oklahoma City |
Low | High | Selective | Low | Low | Organizational emphasis on providing assistance; licensed care only | No shortage | Limited |
Portland |
High | Moderate | Moderately selective | High | Moderate | Emphasis on necessity of arrangements; choice of provider | No shortage | Strong |
As discussed earlier, welfare-to-work program strategies usually emphasize either quick employment or an initial investment in education or training. The 11 programs in the NEWWS Evaluation blended elements of both strategies to varying degrees.
The kinds of messages that case managers send about education and work, the emphasis that they place on different program activities, and the activities in which program enrollees actually participate help to determine whether a client is more likely to get a job shortly after she enters the program or after she has tried to build her skills. The following program descriptions incorporate both the directions that case managers gave and the activities in which enrollees were most likely to participate.(15) Box 1.2 gives a brief description of the services offered by the programs in this evaluation.
Four of the programs are categorized as employment-focused and seven as education-focused. In the descriptions below, programs within each of the two categories are listed in rough rank order, from those that are most purely education- or employment-focused to those that blend the two approaches.
In general, the welfare-to-work programs studied in this evaluation made
available to their enrollees the following services and classes:
*For a more detailed description of service components in the 11 programs, see Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001. |
The Oklahoma City program encouraged long-term education and training activities instead of active job search almost universally. Case managers communicated to clients the importance of education, even in job clubs, as a way to increase skills for later entry into the labor market.
The Atlanta HCD, Grand Rapids HCD, and Columbus Integrated and Traditional programs emphasized increasing skills through formal education and training before entry into the labor market. Because of the generally low educational attainment of participants in these programs, basic education was a common first activity, though Grand Rapids also encouraged participation in vocational training programs. Clients in these programs were given considerable latitude in choosing the kind of education activity they wanted to pursue.
The Detroit program underwent a substantial shift in focus over the study period. Initially, the program emphasized long-term education and training assignments before clients engaged in work search. About midway through the study period clients were referred to a program that required job search first.
The Riverside HCD program, which enrolled only individuals without a high school diploma or GED, generally assigned clients to basic education as a first activity. Short stays in these classes and active job search once a literacy benchmark was reached were stressed by case managers throughout clients' participation. Job developers assisted HCD clients in job club.
Case managers and program staff in the Riverside, Grand Rapids, and Atlanta LFA programs emphasized that employment was the goal of program participation and that job search should be the first activity for participants. Clients were given very little choice in their first program assignment. In Riverside participants were encouraged to take even part-time and low-paying jobs as a first step to self-sufficiency and were assisted by full-time job developers who searched for job leads and followed up on job placements.(16) While Grand Rapids staff stressed to clients the importance of finding work, they believed that it might be justifiable for clients to turn down temporary or part-time jobs. Those who wished to enroll in education programs were encouraged to do so — in addition to, not instead of, working. Atlanta case managers indicated the availability of education and training services as a second step after initial job search. Many Atlanta enrollees did, in fact, participate in education or training if they completed job search without finding a job.
While Portland program staff emphasized that employment was the goal of program participation, not all enrollees were assigned to job search first. For individuals who first enrolled in education or training activities, usually those who were thought by case managers to be the more disadvantaged members of the caseload, program staff communicated that improving employability was the goal of their assignment. Portland also employed full-time job developers to work with participants once they began actively looking for a job though, unlike other developers in work-focused programs in this evaluation, they encouraged participants to seek "good" jobs, that is, higher-paying jobs with benefits.
In addition to the messages about work and education that case managers send to clients and the relative mix of services that a program provides, the degree to which a program enforces a participation mandate has also been shown to affect program impacts. The three elements of enforcement include the broadness with which a program enrolls from its caseload, how well it monitors participants' progress, and how strictly the participation requirements are enforced. In other words, a high or low ranking indicates the likelihood that a client would be told to participate, the likelihood that her case manager would know if she had not been participating, and how swiftly or surely she would be sanctioned for not participating. Nine of the 11 NEWWS programs were "high enforcement" programs; that is, they were rated the equivalent of "high" on at least two of the three elements. Two programs, Detroit and Oklahoma City, were rated the equivalent of "low" on all three elements of enforcement. (See Table 1.3.)
The rest of this section describes how each program was rated with respect to the enforcement of the participation mandate. Within each element of enforcement sites are listed in a rough rank order, from high to low. A number of factors can contribute to a program's overall ranking on an element; a site may be high on one but low on others, but these factors have not been formally weighted.
How likely was it that an individual would have been required to participate in the welfare-to-work programs?(17)
Single parents with children aged 3 or over were required to participate in all programs studied in this evaluation, with some programs requiring participation of women with children as young as age 1. Individuals who had health barriers or were taking care of a household member who was ill or incapacitated, were pregnant, or were already working 30 hours per week could be exempted from this mandate.(18)
At a number of points, administrator and case manager discretion, combined with funding and resource constraints, could affect a welfare applicant's or recipient's chances of enrolling in a welfare-to-work program. First, five of the programs required women with children as young as age 1 to enroll. Since over 40 percent of the welfare cases nationwide in 1994 included a child under age 3,(19) expanding the mandate to this group significantly increased the proportion of the caseload that could be served by the program. Second, case managers might not tell all of those who meet the demographic criteria to enroll. Third, individuals might not show up for the program orientation because they do not wish to participate or they become exempt or leave welfare in the period between referral and orientation date, especially if the period is long.(20) Finally, even recipients who attend an orientation could be deferred from future activities at case managers' discretion.(21)
The Grand Rapids LFA and HCD, Columbus Integrated and Traditional, and Riverside LFA and HCD programs enrolled broadly, including virtually their entire mandatory caseload.(22) Both of the Grand Rapids programs included parents with children as young as age 1 in their participation mandate.
The Atlanta LFA and HCD programs aimed to enroll their entire mandatory caseloads; however, budget limitations created a waiting list, sometimes as long as six months, before those who had been referred to the program could actually enroll. During a waiting period welfare recipients with the fewest barriers to work leave the rolls on their own; thus, the clients who actually enroll may be slightly more disadvantaged than they would be if there were no waiting list.(23) Indeed, the Atlanta sample includes more long-term recipients than samples in most other sites. Because the Atlanta programs did refer virtually all members of their mandatory population to the program, and enrolled all those who were left after the delay, their enrollment is termed "broad-delayed" in Table 1.3.
The Portland program extended its mandate to parents of very young children (as young as age 1), but selectively enrolled from its mandatory population. Some individuals determined "hard-to-serve," that is, less employable, either would not be referred for enrollment in the program or, after attending a program orientation, would not be assigned to further activities. For these reasons Portland can be considered moderately selective.
The Detroit and Oklahoma City programs also extended their mandate to women with very young children, but were more selective than other programs. Like Atlanta, Detroit had a waiting list for "slots" in the program. Guided by the principle that the program would rather spend scarce resources on those who wished to participate than on cajoling those who might never participate, staff tended to give priority to "mandatory" clients who volunteered for the program. In addition, case managers spent a large proportion of their time authorizing child care and support service payments, leaving little time to focus on individuals who were not eager to enroll. Oklahoma City referred all those eligible to its program; however, since the program was also limited by resources and rising caseloads, much of the responsibility for enrolling in program activities fell on the client. Case managers assisted clients in finding appropriate services, but the self-directed enrollment allowed resistant individuals to avoid the mandate. As discussed in Chapter 2, this evaluation examines only the experiences of individuals applying for welfare ("applicants") in the Oklahoma City program; the treatment of those already receiving welfare ("recipients") may have been different from the situation described here.
How often or how quickly would an enrollee be contacted by her case manager if she was not participating?
Once clients begin participating, they may drop out of activities or attend irregularly because they have a new job, have new problems with child care or transportation, or no longer want to participate. Close monitoring can help case managers maintain and increase participation among their caseload, facilitate the authorization of transitional benefits for individuals who leave welfare for work, or speed case closures for individuals who become ineligible. In order to monitor participation closely, case managers must learn about attendance problems from activity providers, determine the reasons for them, contact clients about their options or the consequences of nonparticipation, and then inform the income maintenance branch of a case's outcome. How closely an individual will be monitored depends on the level of information that case managers get from the activity instructors and providers and on the time that case managers have to devote to this task.
The Riverside LFA and HCD, Grand Rapids LFA and HCD, and Portland programs all intensively monitored their participants' progress. Overall, more case managers in these sites indicated receiving a lot more information about attendance from providers than those in most other sites. In addition, case managers reported that it took them between one and two weeks to both hear about attendance problems from providers and contact clients about their attendance, the shortest in the range of time among the programs.
The Atlanta LFA and HCD and Columbus Integrated programs engaged in moderate monitoring of their clients. Information sharing between providers and case managers was not as regular in these programs as in the intensive-monitoring programs, and it took between two and a half and three and a half weeks to get information from providers. These programs did, however, contact clients in less than two weeks once they learned of attendance problems.
The Oklahoma City, Detroit, and Columbus Traditional programs engaged in less intensive monitoring of their clients than the other programs. Regular protocols for obtaining attendance information from providers were not in place for at least two of the programs. It took a little longer, on average, for all three programs to get information from providers than it did for the moderate-monitoring programs. Moreover, it took between two and three weeks for case managers to contact clients about their attendance problems, on average one week longer than for the moderate-monitoring programs.
How much would an individual be encouraged, or coerced through financial sanctions, to participate in a program activity if she did not want to?
The great majority of welfare recipients who are required to participate in welfare-to-work programs believe, prior to hearing details about the program, that they will have trouble participating, citing barriers such as a lack of child care or transportation or having a health or emotional problem.(24) All the programs in this evaluation provided monetary assistance to help participants (both program and control group members) with child care and transportation, but they also relied on case managers to work with clients to remove participation barriers or to coerce participation through the imposition of a financial sanction.(25) Most of the programs were strongly committed to enforcing the participation mandate for their welfare caseload, though the degree to which clients were more likely to be cajoled or coerced differed. Individuals in Detroit and Oklahoma City were not as likely to be cajoled into participating if they did not want to, though this was largely the consequence of limited program funding and staffing.
The Grand Rapids LFA and HCD and Columbus Integrated and Traditional programs were very highly committed to the enforcement of clients' participation obligation. Case managers sent strong messages about the consequences of nonparticipation and, in instances of noncompliance, imposed financial sanctions swiftly on a large percentage of their caseloads.
While other programs informed clients of the necessity of program participation, they gave them more chances to comply than Grand Rapids or Columbus. Atlanta LFA and HCD program case managers were somewhat less comfortable with enforcing participation requirements through financial sanctions, though they did so on a regular basis. More clients were sanctioned in the Atlanta HCD program than in its LFA program, though the messages that case managers sent about requirements were not different.
Riverside LFA and HCD program staff tended to view sanctions as one tool to get clients to attend activities and initially emphasized to clients the importance of personal responsibility. Riverside staff did not delay requests for or impositions of sanctions; the process, however, took longer than it did in most other programs because of extensive state-mandated due process procedures. The Portland program staff also emphasized ways to solve problems related to nonparticipation rather than reductions in clients' grants. Staff in Riverside and, to a greater extent, in Portland were more willing to defer individuals from participation requirements than staff in either Columbus or Grand Rapids. Staff in Portland did, however, ultimately sanction noncompliant individuals.
The mandatory participation requirements in the Detroit and Oklahoma City programs were communicated less intensively to clients. As already mentioned, staff in these two sites focused on those who wanted to participate. Resource constraints kept staff from following up on nonparticipation, and staff tended to delay imposing sanctions.
How much support in the form of child care assistance could an individual expect for her participation in a program or subsequent employment?
For many welfare recipients with young children the major obstacle to working or attending an education or job training program is child care. All 11 programs studied in the evaluation provided this assistance to participants in the program (and to control group members who enrolled in activities on their own in the community) as well as transitional child care (TCC) for those who left welfare for work. However, the relative emphasis that the programs placed on making this assistance available and the messages that case managers sent to clients about the type of care they should choose varied by site (not by research group within site).
Participation-related child care. In the Atlanta LFA and HCD, Oklahoma, Portland, and Detroit programs, child care assistance was emphasized either by site staff or by the welfare department's organizational structure. In both Atlanta programs case managers actively promoted the availability of child care reimbursement as a benefit of program participation and even used it as an inducement for noncompliant clients to participate. In Oklahoma statewide emphasis on access to child care made assistance to clients readily available while they were in the program and after they left welfare for work. Oklahoma had no set caps on the amount of child care assistance that clients could receive. Atlanta and Oklahoma reimbursed only for care given by licensed providers.
Portland program caseworkers told clients that not having child care arrangements was not an acceptable reason for not participating in program activities. Staff often encouraged clients to have backup arrangements in case their regular provider fell through. Although case managers did not push specific types or locations of providers, they did emphasize the necessity for clients to make arrangements and assisted clients who were unable to make arrangements on their own.
Detroit program case managers reported that they spent much of their time on child care payment authorizations and that the priority placed on making child care payments took time away from employment and training counseling. Detroit staff would make referrals to licensed providers in the area on request, but the choice of provider (including choosing licensed child care or unlicensed care approved by the welfare department) was left to the client.
Both the Grand Rapids and Columbus programs would reimburse expenses from child care in licensed as well as unlicensed care, but expected clients to make their own arrangements. Referrals to licensed providers in the area could be made for clients at their request.
In all sites except Riverside, case managers said that child care providers were not difficult to come by. In Riverside, case managers noted that some area providers did not like working with the program or its participants because they did not approve of the reimbursement rates or procedures. These case managers encouraged clients to use low-cost, more informal arrangements, both to contain program costs and because they believed that clients would be more able to afford such arrangements after program or other government supports expired. Clients and case managers often clashed about the providers they wished to use, especially if clients chose more expensive care.
Transitional child care. In the Detroit, Portland, Columbus, and Oklahoma City programs, authorization for TCC payments did not appear to be difficult. In Portland, Columbus Integrated, and Oklahoma the use of integrated case managers, who are more likely to know both the welfare and employment information needed to determine if a client is eligible for TCC, may have made authorization in these sites easier for both program and control group members.
In the Atlanta, Grand Rapids, and Riverside LFA and HCD programs, few clients who began working received TCC; case managers in all three sites cited a lack of information about clients' welfare status when authorizing child care payments. Thus, TCC authorization in these sites would have been infrequent for both program and control group members.
How likely was a welfare recipient to get a unified self-sufficiency message from the welfare department?
The eligibility-compliance culture of the welfare system (more common prior to FSA), in which contact between a client and an agency is focused solely on determining eligibility for staying on welfare, has been harshly criticized. Implementing a mandatory welfare-to-work program was one way that welfare offices hoped to change from an eligibility-compliance culture to a self-sufficiency culture, which would structure interactions and expectations around leaving welfare for work and preparation and supports for it. Yet this task is formidable; it requires the income maintenance and employment services staffs of the welfare offices to work together to send a unified message of the self-sufficiency goal to the client. If the sole responsibility for delivering the self-sufficiency message is remanded to the employment and training program, programs can be interpreted by clients and workers as requirements for continued receipt of assistance, or another element of compliance, instead of an overhaul of the philosophy of the welfare department. Implementing an integrated case management approach, in which one worker is responsible for both the eligibility determination and employment services functions, is one way that has been suggested to achieve a more unified culture.(26) Three of the programs in the NEWWS Evaluation used integrated case management, but these and the other eight programs met with different levels of success in coordinating the messages between their eligibility and employment preparation staffs and in refocusing the welfare department's interactions with clients on the road toward self-sufficiency.
As part of a specially formulated research experiment, the Columbus Integrated program used integrated case management. Staff had sufficient resources and small enough caseloads that they were able to perform both their income maintenance and self-sufficiency roles. Thus, an individual's case manager could both monitor her progress in becoming self-sufficient and verify her credentials for staying on welfare. As of the two-year follow-up point, this program had the largest effect on changing clients' minds about whether they agreed that the welfare office tried hard to get recipients employed or enrolled in school.
The Portland program was marked by a strong partnership between welfare-to-work staff (eligibility workers and integrated case managers) and case management staff contracted by the Portland welfare department. The division of labor was flexible between contractor staff and welfare department integrated case managers, with responsibility for case management services such as reassignment to activities and attendance monitoring, as well as a mission of promoting self-sufficiency, shared by both. Moreover, eligibility workers in Portland were among the most knowledgeable about the program and spent more time discussing the program with recipients than those in most other programs. These results suggest that together eligibility workers, integrated case managers, and contractor staff were able to send a unified self-sufficiency message to welfare recipients.
The Oklahoma City program also used integrated case management. However, limited resources and large caseloads led case managers to put little overall emphasis on the employment services function of their position; in fact, their performance evaluation benchmarks were primarily related to the accuracy of their eligibility duties. Like Portland, Oklahoma City supplemented its integrated case managers with some caseworkers who focused on employment-related services. However, owing to staffing constraints, not all clients received this added case management. The result was a program with little overall emphasis on self-sufficiency.
The Atlanta, Grand Rapids, Riverside, and Columbus Traditional programs all used a separated, or "traditional," case management structure, in which a client had two different case managers, one who specialized in determining eligibility and processing payments and another who focused on her participation and progress in a welfare-to-work program. Although the different staffs did not report any major problems in their working relationship, they mentioned that there was a lack of partnership between the two. Income maintenance workers knew little about the programs and most often discussed with clients the penalties for nonparticipation in the program, not the services it provided, suggesting that participation was cast as a compliance requirement and not a route to self-sufficiency.
The separation between the two staffs of the welfare department in the Detroit program was even more pronounced. Income maintenance workers knew little about the program and had almost no contact with clients regarding their participation; the welfare-to-work program case managers in Detroit handled some income-related functions related to program participation, such as child care payments, that income maintenance workers were responsible for in the other traditional sites. Staff mentioned that this separation was intentional, so that the welfare-to-work case managers would be able to communicate consistent messages and information. In short, the priorities of the two staffs were so dissimilar that an individual was likely to experience very different cultures during her contact with the department.
[Go to Contents]
The previous section described the 11 programs as they existed during the first three years of the follow-up period. In the last two years of the five-year follow-up period, however, the focus of many of the programs changed. Most commonly, some of the education-focused programs became more employment-focused, a change driven in part by the 1996 welfare reform law.(27) This shift was not incompatible with what might normally happen in an education-focused program. It would have occurred several years after most program group members started their participation in education or training activities. Most education-focused welfare-to-work programs involve periodic job search, particularly once individuals have improved their skills or achieved a credential. Therefore, it is unlikely that the shift to an employment-focused program would have greatly compromised the nature of the treatment studied in these originally education-focused programs. In addition, the changes in program emphases would have affected only those program group members still receiving welfare in the last two years of the five-year follow-up period. As discussed in the next section, which covers changes in control group members' treatment over time, this proportion varied by site. The following paragraphs highlight the most important program changes.
In Atlanta, the site (and state) implemented a strongly job search-focused "work first" program in December 1996. For LFA sample members, the new program continued their employment-focused treatment. For HCD sample members, the new program represented a change in program message and emphasized activities. At the same time, Georgia began the count of months toward a welfare time limit (which would have affected sample members in all three of the Atlanta research groups).
In Grand Rapids, a Work First program was implemented during the five-year follow-up period as well, in October 1994, but referral to this program was delayed for most LFA and HCD sample members. For individuals assigned to the LFA and HCD research groups in roughly the first half of the Grand Rapids random assignment period, referrals to the Work First program could be made three years after random assignment if these sample members were still receiving welfare. Reviews of Work First program databases, however, indicated that as of May 1996, which would have been about the fourth year of the five-year NEWWS follow-up period for most of these sample members, less than 10 percent of this group had, in fact, been referred to the new program. Individuals assigned to the LFA and HCD research groups in roughly the second half of the Grand Rapids random assignment period continued to be part of the original LFA and HCD programs throughout the five-year follow-up period. Grand Rapids also took part in another statewide initiative, Project Zero, which sought to drastically reduce the number of nonworking adults on the welfare caseload, but implementation of this program did not occur until almost all NEWWS sample members were beyond the end of their five-year follow-up period.
In Riverside, the LFA program focus and components generally carried over into years 4 and 5 of the five-year follow-up period. The HCD program, however, had more of an employment focus in years 4 and 5. Once HCD sample members completed their education activities in the last two years, it is likely that they would have been assigned, if still on welfare, to job search. If they did not find a job through job search, it is unlikely that they would have been assigned to more basic education; rather, they probably would have been assigned to do more job search or to attend group self-esteem-building sessions. Assignments to vocational training remained rare for both LFA and HCD sample members in the last two years of follow-up, as it had in the first three years. (The CalWorks program in California was implemented at a point that would have been, for most Riverside sample members, after the end of the five-year follow-up period examined in this report; its provisions are thus largely not reflected in the behavior of sample members examined here.)
In Columbus, several program changes took place in October 1997. First, all sample members in both the Integrated and Traditional program groups began to receive integrated case management, reflecting a county-wide shift to this type of arrangement. Second, the program became much more employment-focused than education-focused. Third, Ohio began the count of months toward a welfare time limit (which would have affected sample members in all three Columbus research groups).
Reflecting the Michigan changes outlined above for the Grand Rapids site, the Detroit program was transformed in October 1994 to the strongly employment-focused Work First program, operated by an agency other than the welfare department. In this site, unlike Grand Rapids, it was not possible to implement procedures whereby program group members could continue to be eligible for their "original" program. As a result, sample members still on welfare in late 1994 would have experienced an abrupt change in program type and focus, from an almost voluntary program emphasizing education to a strongly mandatory one emphasizing employment.(28)
The Oklahoma City program also became more employment-focused in years 4 and 5 of the five-year follow-up period. In addition, in October 1996 Oklahoma began the count of months toward a welfare time limit (which would have affected both program and control group members in the site).
In Portland, program group members would have continued to receive an employment-focused treatment in the last two years of the five-year follow-up period. Several other program changes, however, occurred in these years. First, over the course of the five years, the program made more use of job search and less use of education. In the last year of follow-up, for example, program staff were urged to reduce the number of assignments to GED classes and, when made, to limit the assignments to only three to six months. In 1999, GED and other education programs were viewed more as activities that could help working individuals retain and advance in jobs than as activities for initial work preparation. Along with this, life skills activities were still offered, but were compressed to two weeks, from the five to six weeks observed in the first three years of the follow-up. Second, in the last two years of follow-up, staff made more short-term training assignments (of three to six months) and worksite placements (a type of "supportive" work experience). Finally, toward the end of the follow-up period, Portland started to move away from a type of integrated case management and toward traditional case management.
[Go to Contents]
Part of the reliability of the findings from a random assignment social experiment rests on the assumption that control group members are not exposed to the specific program being evaluated. If control group members are exposed to the program, measured impacts could represent an underestimate of the true effects of the program. Rarely, however, is it possible to maintain this lack of exposure for a long period of time. In this regard, the NEWWS Evaluation was no exception.(29)
Agreements reached with the NEWWS Evaluation sites at the beginning of the study specified that all control group members were to be kept out of the welfare-to-work programs being studied in each site for at least three years from their random assignment date. (As noted earlier, control group members were free to enroll in other employment-related activities offered in their communities during this period and, as will be discussed below and in Chapter 3 of this report, many controls did, in fact, enroll in such activities.) Midway through the evaluation, program operators in Riverside and MDRC agreed to extend the embargo on providing welfare-to-work program services to control group members to five years. Program operators in Portland and Grand Rapids agreed to a similar extension, but only for a subsample of control group members. In Portland, about one-quarter of the control group was randomly selected (from every month of the random assignment period) to remain ineligible for welfare-to-work program services until the end of year 5; in Grand Rapids, the embargo on welfare-to-work program services was extended for all control group members who were randomly assigned during 1993, the last year of sample intake in the site. However, Atlanta, Columbus, and Oklahoma City could not continue barring control group members from welfare-to-work program services longer than three years because of the implementation of welfare time limits in their states; if NEWWS control group members could not be exempted from welfare time limits, then both HHS and MDRC felt it necessary to allow control group members to be eligible for some type of welfare-to-work program services once their time limit clock "starting ticking." Finally, in Detroit, it was not feasible to continue the control services embargo, after a different agency took over administration of the site's welfare-to-work program and entry points into the new program were so numerous and widespread that screening to identify NEWWS control group members would have been close to impossible.
Most probably program impacts on employment and earnings and other outcomes in the last two years of follow-up in a few of the five sites above would have been somewhat larger had some control group members not been exposed to welfare-to-work programs. For several reasons, however, lifting the control group embargo on services prior to the end of the five-year follow-up period in these five sites most likely had only a small effect on measured program impacts. As discussed in detail in Chapter 2, most control group members were not eligible to receive program services when the control group embargo ended, often because they had already left welfare. From one-quarter to one-half of control group members in these five sites were receiving welfare when their embargo on program services was lifted. In addition, after the embargo was lifted, some control group members were not contacted about enrolling in the program until after the end of the follow-up period, and others were assigned to a program orientation but did not show up. Moreover, the likely effects of ending the control group embargo were estimated for the sites where the embargo was lifted by calculating impacts for a subsample of control group members who were precluded from program services for four to five years. It was found that the patterns of impacts in years 4 and 5 resembled those for all sample members in those sites.
It should be emphasized that the control group situations described above do not affect the assessments in this report of the relative merits of the Labor Force Attachment and Human Capital Development approaches in welfare-to-work programs. As described in Chapter 2, the three-group random assignment designs in the three sites in which these two types of programs were simultaneously operated permit a direct comparison of these two approaches, that is, a comparison that does not need to take into account the services received by and the behavior of control group members.
[Go to Contents]
Several reports have documented the range of effects achieved by all 11 NEWWS programs within a two-year follow-up period.(30) This report extends and expands on these two-year findings, lengthening the follow-up period to five years and including additional outcome measures. As a basis for understanding the long-term findings, this section recaps the two-year findings, noting which welfare-to-work program strategies were more or less successful in helping welfare recipients achieve self-sufficiency in the relatively short run.
All programs, regardless of their approach, increased participation in activities designed to promote employment during the two-year follow-up period. As expected, employment-focused programs increased participation primarily in job search activities, whereas education-focused programs increased levels primarily in basic education and vocational skills training classes. Very different patterns of participation impacts were found for individuals who entered the study with a high school diploma or GED certificate and for those who did not have these credentials. In most education-focused programs participation impacts were concentrated among those without a high school diploma or GED and resulted primarily from large increases in attendance in basic education; only small increases in attendance in post-secondary education or vocational training were found for the education-focused programs, and they were generally among only high school graduates or GED holders. In contrast, large impacts on participation in job search were achieved for both groups in the employment-focused programs.
Some education-focused programs, as well as the Portland program, were able to produce relatively large two-year impacts (about 10 percentage points) on GED attainment among sample members who did not have a high school diploma or GED certificate at study entry. Of the seven education-focused programs, the Grand Rapids HCD, Riverside HCD, and Columbus Traditional programs had this effect. The Portland program, in addition to boosting GED receipt, increased the rate at which those without education credentials obtained a trade license or certificate by 12 percentage points. For sample members with a high school diploma or GED certificate at study entry, only three programs (Atlanta LFA and HCD and Grand Rapids HCD) increased receipt of a trade license or certificate.
As expected, employment-focused programs produced larger gains in employment and earnings over the two-year follow-up period than education-focused programs. A majority of control group members in all sites except Riverside, the site with the most difficult labor market, found jobs on their own at some point within two years of random assignment and, as a group (including zeroes for nonearners), had average earnings during the second year of follow-up ranging from $2,127 (Oklahoma City) to $3,978 (Columbus). The Portland program attained the largest earnings increase of all programs, with members averaging more than $900 per year in earnings above control group members. Equally important, employment and earnings gains in Portland grew larger over time and reached their highest levels at the end of year 2. The other employment-focused programs produced moderate earnings increases, ranging from $400 to $650 per year, that grew smaller toward the end of year 2.
Several of the education-focused programs began to show moderate impacts in year 2. By the end of year 2 all but two of the education-focused programs had attained increases in employment and earnings that equaled or exceeded the gains achieved by all employment-focused programs except the Portland program. The two exceptions to this pattern, the Riverside HCD and Oklahoma City programs, did not raise employment or earnings levels in year 2.
All programs reduced two-year welfare dependency to some degree. Control group members in all but one site remained on welfare for an average of 16 to 20 months during the two-year follow-up period and received payments averaging between $3,624 (Oklahoma City) and $10,302 (Riverside HCD) during this period. Seven of the 11 programs, a mixture of employment- and education-focused approaches, decreased cumulative welfare expenditures by more than 10 percent, a historically large effect; welfare reductions in the other four programs were smaller. The Portland program produced a large decrease in welfare receipt that persisted at a high level throughout the follow-up period, showing a 12 percentage point decrease in welfare receipt during the last quarter of the two-year period; all other programs had reduced welfare receipt at this point by 3 to 7 percentage points. However, at least 40 percent of sample members in the programs were still relying to some extent on welfare at the end of two years.
Most programs increased sample members' reliance on earnings rather than welfare, but family net incomes were largely unchanged. As a result, within a two-year follow-up period the programs lifted few families above the poverty line. Impact estimates of reductions in welfare, Food Stamps, and other benefits generally matched or exceeded impact estimates of earnings gains. Including estimates of income from the Earned Income Tax Credit (EITC) produced little change in this finding for all programs except Portland's, which attained the largest and most consistent gain in total combined income ($238, or $425 including the EITC estimate, for year 2 of the follow-up) and also produced a small increase in the proportion with combined incomes above the poverty level (4 percentage points, or 7 percentage points including the EITC estimate, in year 2).
Although no programs had pervasive negative effects on sample members in the first two years of follow-up, some individuals were adversely affected. In year 2 of follow-up six programs (some employment-focused and some education-focused) produced small increases in the proportion of sample members with combined income from welfare, Food Stamps, and earnings equivalent to less than 50 percent of poverty levels. In addition, several programs (representing both types of approaches) increased the rate at which individuals left welfare without a job (although other persons in their household might have been working). Finally, some programs that increased employment also decreased family health insurance coverage (as reported by parents) and increased out-of-pocket child care expenditures.
The programs did not have widespread, large, or consistent two-year effects on the children of sample members, but positive and negative effects occurred in some programs. No programs in the evaluation provided direct services (with the exception of child care assistance) to children. Program-produced changes in the lives of sample members (virtually all mothers) may, nevertheless, influence the well-being of children. There is evidence that some of the programs affected the likelihood of at least one child in a family having behavioral, educational, or health and safety problems. There was not, however, a consistent pattern of benefit or harm to children. In addition, employment- and education-focused programs did not appear to affect children differently; there was no consistent evidence that one particular approach affected children more or less or was more likely to help or harm children.
Several employment- and education-focused programs attained at least moderate two-year employment and earnings gains for the "most disadvantaged" sample members. Five programs (Portland, Grand Rapids LFA and HCD, and Riverside LFA and HCD) increased employment and earnings for individuals who at study entry did not have a high school diploma or GED, had not worked in the prior year, and had been on welfare cumulatively for two years or more. These five programs and two others (Detroit and Columbus Integrated) also reduced the amount of time that the most disadvantaged individuals spent on welfare during the two-year follow-up period. Notably, for individuals who entered the study without a high school diploma or GED, the two-year employment and earnings impacts of the education-focused programs did not exceed those of the employment-focused programs.(31)
High enforcement programs did not produce the largest two-year impacts, but low enforcement programs resulted in only small effects. High enforcement programs, notably those in Grand Rapids and Columbus, did not necessarily produce the largest impacts. However, the two low enforcement programs — Oklahoma City and, in its early stages, Detroit — yielded only small impacts. It thus appears that a minimum level of enforcement by program staff is required to produce at least moderate earnings and welfare impacts, presumably because this extra "push" is needed in order to engage in program activities those who normally would not participate on their own initiative.
While many programs achieved positive two-year effects on employment, earnings, and reduced use of welfare, the Portland program stood out as unusually successful. The Portland program substantially increased employment and earnings, helped people to get good jobs, lowered welfare receipt, and achieved these outcomes for a cross section of sample members. The results are probably due to a combination of factors. While its employment message was strong, the program offered high-quality education and training services as well as job search, enforced a participation mandate, and had strong job development and placement services. In addition, contextual factors may have contributed to the program's success. In particular, it worked with a less disadvantaged welfare caseload (relative to the other studied programs) and operated within a good labor market with a relatively high state minimum wage.
[Go to Contents]
This report builds and expands on the two-year findings. Chapter 2 and 3 give important background information about the NEWWS Evaluation and its participants and their activities. Chapter 2 describes the random assignment research design used to test the effectiveness of the programs, the definition and characteristics of the various samples included in this report, and the types and sources of data used. Chapter 3 describes the five-year effects of the programs on increasing participation in work-related activities. The chapter also documents whether programs increased the percentage of recipients who earned GEDs or other education credentials after random assignment. Importantly, the chapter expands the discussion in this chapter on the extent to which both control and program group members received welfare-to-work program services in the last two years of the five-year follow-up period.
Chapters 4, 5, 6, and 7 examine the five-year economic impacts of the programs, for all sample members as well as selected subgroups. These longer-term impacts could reflect an extension of the effects of services and experiences in the first two years of follow-up, the effect of new services and experiences in follow-up years 3 to 5, or a combination of both. Chapter 4 discusses the impacts of the programs on sample members' employment, earnings, job stability, and job quality. The chapter investigates whether employment- or education-focused programs fared better and what caused increases in average earnings: putting to work welfare recipients who would not have found jobs on their own, improving job quality for those who would have been employed anyway, or both. Chapter 5 presents impacts on welfare and Food Stamp receipt and payments, determining whether the programs achieved welfare savings and whether they did so by increasing the speed or frequency of welfare exits or by decreasing average grants for those on public assistance. Chapter 6 looks at earnings gains and welfare reductions from the perspective of sample members and presents impacts on individuals' combined income from earnings and benefits, level of self-sufficiency, and prospects for longer-term economic security. Chapter 7 determines the effects of alternative program strategies for different subgroups of welfare recipients. It explores the degree to which programs helped groups of the welfare population likely to have different capacities to find work on their own: those who had limited education credentials, those who were more disadvantaged (without recent work experience and who had been on welfare for two or more years), and those who were less disadvantaged.
Chapters 8, 9, and 10 are concerned with several noneconomic family outcomes. Chapter 8 examines the ways that programs affected sample members' health care coverage, for adults as well as children in the families studied. Chapter 9 discusses the effects of the programs on individuals' household and personal circumstances, examining impacts on marital status, the structure and composition of families and households, fertility, and barriers to work from and abuse by intimate partners and others. Chapter 10 looks at the effects of these welfare-to-work programs on sample members' work-related child care arrangements and on children's daily activities.
Chapters 11 and 12 examine the effects of the programs on children. Chapter 11 presents impacts on children of all ages in four of the sites using limited measures of child well-being. Chapter 12 presents impacts on a subset of children in three sites who were preschool-age at the start of the NEWWS Evaluation and generally between ages 8 and 10 at the five-year follow-up point; these analyses use in-depth child well-being measures, constructed from information supplied by parents, elementary school teachers, and the children.
Finally, Chapter 13 presents a benefit-cost analysis for each program studied, weighing benefits and costs from the perspectives of government and the sample members themselves.
[Go to Contents]
1. The programs and individuals studied in this evaluation are drawn from the entire county (or counties) mentioned in parentheses after the city name; for ease of reference, in this report the sites will be referred to by the name of their corresponding urban area.
2. See Friedlander and Burtless, 1995; Bloom, 1997; and Riccio, Friedlander, and Freedman, 1994.
3. It is important to note that the studies of the programs in the education-focused category yield information about the effects of increasing welfare recipients' participation in basic education programs (including Adult Basic Education, GED preparation, and English as a Second Language classes) and, to a much lesser extent, in vocational skills training programs, but not in college. On their own, many welfare recipients enroll in various types of education or training classes and reap benefits from them; the education-focused programs in the evaluation, however, sought to increase participation in education or training activities beyond what would normally occur. As will be discussed in Chapter 3, most of the programs did indeed increase such participation, but the increases in enrollments were in basic education courses and, to some degree, in vocational training courses, but generally not in college-level ones.
4. See Scrivener and Walter, 2001, for a full discussion of the results of the direct test of Integrated versus Traditional case management in the Columbus site.
5. See Freedman et al., 2000a, p. ES-6; Hamilton and Scrivener, 1999, pp. 24-31; Bloom, 1997, p. 51; Kemple, Friedlander, and Fellerath, 1995, pp. ES-2 and ES-3; and Friedlander et al., 1987, pp. vii-x.
6. Control group members were eligible for child care assistance similar to that offered to program group members if they were participating in nonprogram activities in which they had enrolled on their own.
7. While comparisons of impacts across programs or sites are not as reliable as the impacts for each program or site in the evaluation, they are a much more accurate determination of which types of programs are high and low performers than simple comparisons of statistics, such as welfare caseload reductions, across localities or states.
8. For a description of the site selection process, see Hamilton and Brock, 1994, Appendix A.
9. The participation of single mothers in the labor market also increased dramatically during this time period. Rates of employment for single mothers with any children under age 18 increased from 57 percent in 1992 to 71 percent in 1999 and for those with a child under age 3 increased from 35 to 56 percent over this same time period (U.S. House of Representatives, 2000, pp. 1412-1427).
10. Data presented in this chapter are for the entire county (or counties) from which each site draws its sample members.
11. These amounts assume the receipt of the maximum welfare payment.
12. Greenberg, 1992.
13. The federal EITC is a credit against federal income taxes for taxpayers with annual earnings below a certain level.
14. See Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001.
15. See also Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001. In addition, Bos et al., 2001, provides a detailed description of how adult education programs were implemented in the Atlanta, Grand Rapids, and Riverside HCD programs.
16. Given Riverside's high grant level and earnings disregard policies, individuals could remain on welfare until they had a full-time wage of $8.35 per hour.
17. See also Hamilton and Brock, 1994, pp. 51-55, for a more detailed description of the sites' enrollment practices.
18. Exceptions for health reasons were typically few. In Columbus, for example, where detailed data on exemptions are available, the majority of exemptions were due to the youngest child being under age 3; less than 9 percent of the exempt single parents had a long-term illness or incapacitation and less than 3 percent were caring for an ill or incapacitated family member (including a child). See Hamilton, 1995.
19. U.S. House of Representatives, 1996, Table 8-32; percentage of all AFDC households with a child under age 3.
20. See Hamilton and Brock, 1994, and Knab et al., 2001, for a discussion of the length of time between referral to and enrollment in welfare-to-work programs and reasons for orientation nonattendance.
21. As described in more detail in Chapter 2, in most of the sites, random assignment to research groups occurred once individuals attended program orientations. Only in Columbus and Oklahoma City did random assignment occur earlier.
22. In Riverside, individuals working 30 hours per week or more were not enrolled in the program. In addition, program enrollees who were employed 15 to 29 hours per week were not assigned to additional, concurrent program activities.
23. See Bane and Ellwood, 1983; Pavetti, 1992; Gueron and Pauly, 1991; and Hamilton and Brock, 1994.
24. See Hamilton and Brock, 1994.
25. The financial sanctions in effect during the NEWWS Evaluation were not the "full-family" sanctions currently being implemented in numerous states. During the period under study, sanctions affected only the adult member of the welfare case; for a three-person family in 1993, for example, a sanction would reduce the welfare grant by approximately 20 percent, depending on the site. In addition, the financial penalty continued until the sanctioned individual complied with the program participation mandate, with a minimum sanction length of three months for the second "offense" and a minimum length of six months for the third offense (with no minimum length for the first offense). As a result, some sanctioned individuals endured a penalty for a short amount of time while some experienced a penalty for much longer. For example, almost half of those sanctioned in the two Grand Rapids programs were in this status for at least 12 months of a 24-month follow-up period. See Hamilton et al., 1997; Scrivener et al., 1998; Farrell, 2000; Storto et al., 2000; and Scrivener and Walter, 2001, for more details on sanction practices in each of the NEWWS Evaluation sites.
26. See Scrivener and Walter, 2001, pp. 3-5; and Bane and Ellwood, 1994, p. 127.
27. Three evaluation sites also implemented welfare time limits during the later part of the five-year follow-up period. These would have applied to both program and control group members. This means that the count of months toward their welfare limit would have started; no sample members actually would have reached their welfare time limit during the follow-up for this evaluation.
28. See Farrell, 2000, for details on how this change came about.
29. The authors know of only one large-scale social program evaluation — the GAIN Evaluation, a study of California's late 1980s and early 1990s welfare-to-work program — that, by design, kept control group members from exposure to the specific program being evaluated for a follow-up period as long as five years.
30. See Freedman et al., 2000a; McGroder et al., 2000; and Hamilton, 2000.
31. For an extension of these results into the third year of follow-up, see Michalopoulos and Schwartz, 2001.
Main Page of Report
Contents of Report
Home Pages:
National Evaluation of Welfare-to-Work Strategies
(NEWWS)
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health and Human Services
(HHS)