Denied Claims Accuracy Pilot Project:
Follow-Up Report
August 2000
Prepared for:
Division of Performance Review
Office of Workforce Security
Employment and Training Administration
U.S. Department of Labor
200 Constitution Avenue, N.W. Room S-4231
Washington, D.C. 20210
Submitted by:
Stephen Woodbury
W.E. Upjohn Institute, 300 South Westnedge Avenue, Kalamazoo, MI 49007
(616/343-5541, woodbury@we.upjohninst.org) and
Department of Economics, Michigan State University, East Lansing, MI 48824
(517/355-4587, woodbur2@msu.edu)
and
Wayne Vroman
The Urban Institute, 2100 M Street, NW, Washington, DC 20037
(202/261-5573, wvroman@urban.ui.org)
This report was prepared under Contract No. M-5538-5-00-97-30. The authors are grateful to Burman Skrable and Andrew Spisak of the Office of Performance Review for extensive discussion and comments, and to Leland Teal, Layne Waters, Bob Branham, and Doug Potter of the South Carolina Department of Employment Security for their cooperation and efforts in making available the supplemental data used in chapter 5.
Table of Contents
Executive Summary
List of Tables
Introduction
1. QPI versus DCA: The Value-Added of DCA
1.1. "Undetectable" Errors Discovered by DCA Investigations
1.2. Point of Detection of Erroneous Denials and the QPI
1.3. What If a Claimant Changes His or Her Story for the DCA Investigation?
1.4. Implications of QPI and DCA for States' Procedures
1.5. Summary and Conclusions
2. Can the QPI Review Mimic the Findings of DCA Investigations?
2.1. Lowering the QPI Pass/Fail Threshold
2.2. Eliminating Apparently Misleading Elements from the QPI
2.3. Discriminant Analysis of the QPI and Denied Claims Accuracy
Appendix to Chapter 2: Nonlinear Transformation of Least Squares Coefficients to Obtain Weights of the Discriminant Function
3. Erroneous Denials Compared with Overpayments
3.1. Erroneous Denials and Payment Errors
3.2. Why the Differences between Overpayment and Denial Error Rates?
3.3. Claimant Characteristics and Denial Probabilities
4. Benefits Lost Due to Erroneous Denials
4.1. Issues in Estimating Lost Benefits
4.2. Three Approaches to Estimating Lost Benefits<br>
4.3. Description of Penalties
4.4. National Estimates
5. Regression-Based Estimates of Benefits Received by Erroneously Denied Claimants
5.1. Description of the Data and Samples
5.2. Modeling Benefits Received and Benefit Duration
5.3. Expected Benefits and Benefit Duration of Erroneously Denied Claimants
5.4. Implications for Benefits Lost Due to Erroneous Denials
5.5. Summary and Conclusion
References
This follow-up report addresses several questions and issues surrounding the accuracy of denied claims for Unemployment Insurance (UI) that either were not addressed, or were addressed only in a preliminary way, in the Denied Claims Accuracy Pilot Project Final Report (Woodbury and Vroman 1999). Chapter 1 explores further the relationship between the findings of the Denied Claims Accuracy (DCA) intensive field audit and the scoring of denied cases under the Quality Performance Indicator (QPI) review. In each of the five pilot states, approximately 100 Separation and Nonseparation denials were subjected to both the DCA intensive field audit and the QPI review. These parallel reviews provide a rich source of information for examining the correlation between the DCA findings and QPI scoring.
Chapter 1 reaches three main conclusions. First, the great majority of separation and nonseparation errors result from agency error or some kind. Second, erroneous separation and nonseparation denials that passed QPI were more likely to involve an issue that the agency could not detect, indicating that the DCA tends to pick up errors that the QPI misses. That is, the QPI tends to miss a significant subset of problems in the separation and nonseparation determinations processes. In particular, the findings suggest that that the QPI cannot detect errors that require new information, especially the type of information that would be obtained from interviews with claimants or third parties. Third, in addition to giving an upward-biased picture of the extent to which the determinations process is flawed (as discussed in the May 1999 Denied Claims Accuracy Pilot Project Final Report), the QPI understates the extent to which incorrect action is a problem and overstates the extent to which inadequate information is a problem. That is, although the outcomes of the QPI review and the DCA investigation point to the same main problems with the determination process, an administrator using the QPI alone would have difficulty correctly allocating resources to improving the quality of decisions made by adjudicators relative to improving the information on which decisions are based.
Chapter 2 attempts to modify the QPI so as to mimic the results obtained by DCA investigations. The question addressed is whether the relatively inexpensive QPI review could substitute for the relatively expensive DCA audit. Several methods of modifying and adjusting the QPI are examined, including discriminant analysis. We conclude that, although the QPI could be modified so as to reduce the number and proportion of proper denials that fail, it cannot be modified so as to reduce the number or proportion of erroneous denials that pass. Under the best circumstances QPI is capable of classifying at most 41 percent of erroneous nonmonetary denials as failing.
Chapter 3 examines two related questions. First, we compare the denial error rates found by the DCA pilot project with the overpayment error rates found under the BAM program in the same five pilot states during the same time period as the DCA. The main finding is that total overpayment rates are sharply lower than erroneous denial rates. For example, in the five states taken together, the total overpayment rate on monetary determinations was 0.6 percent, compared with a monetary denial error rate of 16 percent. That is, the monetary denial error rate was nearly 27 times the total overpayment rate on monetary determinations. Also, total overpayment errors result mainly from lack of information rather than from human error. This contrasts with the situation for erroneous denials, where errors of judgment appear to play a larger role.
Second, chapter 3 examines whether the characteristics of claimants are related to the probabilities of erroneous denial or total overpayment. The results suggest four main conclusions: (a) Claimants whose earnings history puts them near the threshold of benefit eligibility are more likely to receive an erroneous monetary denial than are other claimants. (b) Adjudicators may incorrectly use a relatively weak earnings history as an indicator that a claimant does not meet the separation or nonseparation conditions for eligibility. (c) Claimants who are at the maximum potential duration, but whose WBA is below the maximum, are more likely than others to receive a total overpayment. (d) Individual characteristics such as race and gender may play a role in erroneous denials and total overpayments, although the evidence is not strong on this point.
Chapter 4 examines the benefits lost to claimants due to erroneous denials. As discussed in the May 1999 final report, a variety of problems arise in estimating the benefit losses due to erroneous denials. Four of these are reviewed and discussed in section 4.1: self-correction of initial administrative errors, the interconnectedness of error corrections, estimating the cost per case, and aggregation issues. Section 4.2 discusses three methods of estimating the dollar costs of denied claims: (a) a key week approach, (b) a benefit year approach, which is used in this report, and (c) a hybrid approach. Section 4.3 describes the penalties associated with each of the three types of denials. Finally section 4.4 presents estimates of the benefits lost by claimants due to erroneous denials. The estimates suggest that, overall, about $625 million in benefits were erroneously denied during fiscal year 1998, amounting to just over 3 percent of total regular UI benefit payments. Of this total, about $240 million were erroneously denied due to incorrect monetary determinations, about $230 million were erroneously denied due to incorrect separation determinations, and about $150 million were erroneously denied due to incorrect nonseparation determinations.
Chapter 5 outlines and implements a regression strategy for estimating the benefits that erroneously denied UI claimants would have received during the full benefit year, had they been correctly determined eligible. The main practical barrier to such an approach is lack of data on the full benefit-year experience of a sample of eligible claimants. This was overcome with the cooperation of the South Carolina Quality Control Division, which provided data on the full benefit-year payments made to its BAM sample during the period of the DCA Pilot Project. The supplemental data from South Carolina allows estimation of models of benefits received that serve as the basis for imputing the benefits (and weeks of benefits) that would have been received by erroneously denied claimants if they had not been denied.
Imputations based on the estimated models (see section 5.3) suggest that the benefits lost by erroneously denied claimants (as a percentage of the benefits received by a typical correctly determined claimant) amount to just under 80 percent for erroneous monetary denials and about 55 percent for erroneous separation denials. (The weeks of benefits lost as a percentage of the weeks of benefits received by a typical correctly determined claimant amount to 91 percent for erroneous monetary denials and 67 to 68 percent for erroneous separation denials.) These findings imply that the total lost benefits due to erroneous denials amount to about $565 million in fiscal year 1998, or about 3.1 percent of total regular UI benefit payments (section 5.4). Of this total, about $220 million were erroneously denied due to incorrect monetary determinations, about $190 million were erroneously denied due to incorrect separation determinations, and about $155 million were erroneously denied due to incorrect nonseparation determinations. These estimates are only slightly less than the estimate of lost benefits developed in chapter 4.
List of Tables
Table 1-1: Prior agency action on erroneous denials, by state and type of denial
Table 1-2: Prior agency action on erroneous denials, by type of denial and whether erroneous denials passed QPI
Table 1-3: Error detection point on erroneous denials, by type of denial and whether erroneous denials passed QPI
Table 1-4: Prior agency action on erroneous denials by detection point, erroneous separation and nonseparation denials
Table 1-5: Percentages of erroneous denials that failed each element of the Quality Performance Indicator review, by type of denial
Table 1-6: Percentages of "undetectable" erroneous denials that failed each element of the Quality Performance Indicator review, by type of denial
Table 2-1: Crosstabulations of separation denial accuracy by alternative QPI pass/fail scores
Table 2-2: Crosstabulations of nonseparation denial accuracy by alternative QPI pass/fail scores
Table 2-3: Components of QPI as a predictor of denied claims accuracy
Table 2-4: Crosstabulations of separation denial accuracy by QPI-based discriminant function pass/fail scores
Table 2-5: Crosstabulations of nonseparation denial accuracy by QPI-based discriminant function pass/fail scores
Table 3-1: Error rates on denied claims (DCA pilot) and paid claims (BAM) in DCA pilot states, September 1997 through August 1998
Table 3-2: Prior agency action on claims "totally overpaid" (BAM) in DCA pilot states, September 1997 through August 1998
Table 3-3: Prior agency action on erroneous denials (DCA pilot) and claims "totally overpaid" (BAM) in all five DCA pilot states, September 1997 through August 1998
Table 3-4: Relationships between claimant characteristics and the unconditional probability of (a) erroneous monetary denial, (b) erroneous separation denial, (c) erroneous nonseparation denial, and (d) total overpayment: regression analysis
Table 4-1: Denial error rates, unadjusted and adjusted for "self-corrections," by pilot project and issue
Table 4-2: Framework for viewing the duration of erroneous denials
Table 4-3: Penalties for separation issues
Table 4-4: Determinations and denials for the United States, and imputations of erroneous denials for the United States based on DCA pilot project, 1998
Table 4-5: Estimates of benefits lost due to erroneous denials, United States, fiscal year 1998
Table 4-6: Mean nonseparation penalty periods, imputations for the five pilot states and the United States, 1998
Table 5-1: Characteristics of UI claimants correctly paid and erroneously denied for monetary, separation, and nonseparation reasons, South Carolina, 1997-98
Table 5-2: Models of benefits paid and number of weeks paid during the benefit year, various groups of UI claimants, South Carolina, 1997-98
Table 5-3: Observed and expected benefits paid and number of weeks paid during the benefit year, various groups of UI claimants, South Carolina, 1997-98
Table 5-4: Imputed benefits (and weeks of benefits) lost by erroneously denied claimants as a percentage of benefits received by correctly determined claimants
Table 5-5: Estimates of benefits lost due to erroneous denials, United States, fiscal year 1998
This follow-up report addresses several questions and issues surrounding the accuracy of denied claims for Unemployment Insurance (UI) that either were not addressed, or were addressed only in a preliminary way, in the Denied Claims Accuracy Pilot Project Final Report (May 1999). The Office of Performance Review requested the contractor to pursue further research on these questions, and this report describes the findings of that follow-up work.
The first two chapters of this follow-up explore further the relationship between the findings of the Denied Claims Accuracy (DCA) intensive field audit and the scoring of denied cases under the Quality Performance Indicator (QPI) review. In each of the five pilot states, approximately 100 Separation and Nonseparation denials were subjected to both the DCA intensive field audit and the QPI review. These parallel reviews provide a rich source of information for examining the correlation between the DCA findings and QPI scoring. Chapter 1 offers a descriptive discussion of the valueadded of the DCA investigation and concludes that the DCA provides new information without which a substantial number of separation and nonseparation denial errors could not be detected or diagnosed. That is, a desk audit like the QPI review misses such errors because it relies on information on file with the agency. Chapter 2 attempts to modify the QPI so as to mimic the results obtained by DCA investigations. The question addressed here is whether the relatively inexpensive QPI review could substitute for the relatively expensive DCA audit. (The conclusion is that it could not.)
Chapter 3 examines two related questions. Sections 3.1 and 3.2 compare the denial error rates found by the DCA pilot project with the overpayment error rates found under the BAM program in the same five pilot states during the same time period as the DCA. The main purpose is to examine differences between denial and overpayment error rates and to understand what factors account for the differences. Section 3.3 examines whether the characteristics of claimants are related to the probabilities of erroneous denial or total overpayment. The DCA Pilot data are pooled with BAM data in order to estimate four models: one each for the probability of erroneous monetary, separation, and nonseparation denial, and one for the probability of overpayment.
Chapters 4 and 5 examine the benefits lost by claimants due to erroneous denials. As discussed in the May 1999 final report, a variety of problems arise in estimating the dollar impact of erroneous denials. Chapters 4 and 5 of this follow-up report addresses the various problems and offers some estimates that are based on plausible assumptions.
A main goal of the DCA Pilot Project was to compare the results of a comprehensive field investigation (the Denied Claims Accuracy audit) with the Quality Performance Indicator (QPI) assessment of nonmonetary determinations. Section 4.6 of the Denied Claims Accuracy Pilot Project Final Report (May 1999) developed some basic comparisons of the DCA findings and the QPI assessment, and this chapter pursues the topic further. As will be seen, the DCA codings of prior agency action and point of detection of erroneous denials provide a way of diagnosing each erroneous denial and relating the circumstances surrounding each back to the QPI review.
Section 1.1 uses DCA data on prior agency action to address the following questions: What percentage of erroneous denials did the DCA investigation determine the agency did not detect or cannot detect? What does the DCA investigation show about the effectiveness of the QPI in detecting problems in the separation and nonseparation determinations processes?
Section 1.2 uses DCA data on the point of error detection in the DCA investigation to examine further the causes of denials that were found improper by the DCA investigation. In particular, we compare the error detection point in erroneous denials that passed the QPI with the error detection point in erroneous denials that failed the QPI, with an eye to understanding differences between DCA and QPI in the types of error that each can detect. One goal is to understand better why DCA and QPI give such different results.
Section 1.3 is a brief digression on whether claimants who give a different story to the DCA investigator than they gave in the original fact-finding could pose a problem for the DCA method.
Section 1.4 attempts to pull together what has been learned about the implications of the QPI and the DCA for modifications that the states could make, either to improve error detection or to prevent denial errors in the first place. The section discusses what the QPI review and the DCA determination each can reveal about errors in separation and nonseparation denials.
Chapter 1 has two related goals. The first is to understand better the relationship between the DCA findings and the QPI review. Given that the QPI is a process-oriented review, whereas the DCA field audit is an outcome-oriented determination, how do these two reviews supplement and complement one another? The second goal of chapter 1 is to evaluate whether the DCA could lead states to take actions other than those the QPI review would indicate. The DCA is more expensive than the QPI. What is the value-added of the DCA? Is there a reasonable expectation that state agencies would act on the additional information offered by the DCA?
1.1. "Undetectable" Errors Discovered by DCA Investigations
An important question about including separation and nonseparation determinations in the DCA program is whether the DCA investigation will yield additional information that contributes to the improvement of each state's program. In this section, we examine (a) the state agency actions that occurred before the DCA investigation and (b) the point in the DCA investigation at which an error was discovered. Cross-tabulating these two variables provides insights into the value of the DCA investigation. Two questions in particular are addressed: What is the percentage of erroneous denials that DCA determined the agency did not detect or cannot detect (and which the QPI review would not detect)? What does the QPI review show about state agency efforts to obtain information on such cases?
1.1.1. Prior Agency Action on Erroneous Denials. For each erroneous denial, DCA investigators classified the state agency action that occurred before the DCA investigation into one of six categories:
The first category (could not detect issue) is especially important because it indicates that the agency could not uncover the error issue even though it followed its normal procedures. The error was discovered only through the DCA field investigation. It follows that one or more aspects of the agency's existing procedures should be reviewed with an eye to making changes that would result in more complete fact finding.
Similarly, the second category (was already resolving issue) suggests that the agency needs to review its procedures to see whether improvements could be made that would speed collection of information needed to make a fully informed decision. This category also alerts the agency to incomplete or inaccurate initial decisions that require additional work by the agency for a correct determination.
In contrast, the last three categories indicate that the agency had enough information to identify and resolve the error issue but either failed to do so, took incorrect action, or did not follow its own procedures. Knowing this is clearly useful to managers and administrators in identifying aspects of program operations that require correction.
Table 1-1 shows the actions that were being taken (or had already been taken) by each state agency for each type of denial at the time of the DCA investigation. Although the focus is on separation and nonseparation denials, Table 1-1 also includes monetary denials so that comparisons can be drawn across all three type of denials. (The figures on erroneous monetary denials will be referred to again in chapter 3.)
The first panel of Table 1-1 shows that prior agency actions on monetary denials vary significantly among the five pilot states. In three states (Nebraska, South Carolina, and West Virginia), the error-causing issue could not have been detected through normal procedures for over half of the erroneous monetary denials. Also, in New Jersey, South Carolina, and Wisconsin, from 30 to 50 percent of the erroneous monetary denials were in the process of being corrected by the agency. Finally, in 15 to 55 percent of erroneous monetary denials, the agency (a) identified the issue but took incorrect action, (b) had adequate documentation to identify the issue but did not do so, or (c) did not follow official procedures. In sum, the first panel of Table 1-1 suggests that, although existing agency procedures would have resolved about 30 percent of the erroneous monetary denials, nearly 40 percent of the errors could not have been detected under existing procedures, and the remaining 30 percent of monetary denial errors result from incorrect action or failure to identify the issue or to follow procedures.
The second panel of Table 1-1 shows the actions taken by states on erroneous separation denials. There are three main findings. First, the error-causing issue could not have been detected through normal procedures for about 20 percent of the erroneous separation denials overall. Second, only 8 percent (7 out of 86) of erroneous separation denials were in the process of being corrected by the agency at the time of the DCA investigation. Overall, erroneous separation denials appear less likely to be corrected by agency actions than erroneous monetary denials. Third, in all five pilot states, the agency took incorrect action, did not identify the issue, or did not follow procedures in 60 percent of more of the erroneous separation denials.
Similarly, the findings on erroneous nonseparation denials show that nearly 70 percent of the erroneous nonseparation denials involved the agency taking an incorrect action, not identifying an issue, or not following official procedures. Only 22 percent of the erroneous nonseparation denials were undetectable. As with erroneous separation denials, relatively few erroneous nonseparation denials were in the process of being corrected by the agency (under 9 percent).
In sum, the findings in Table 1-1 suggest that the great majority of erroneous separation and nonseparation denials result from agency error of some kind. This is in sharp contrast to monetary denial errors, of which only 30 percent result from agency error.
1.1.2. Prior Agency Action and the QPI Review. By means of the prior agency action code, the DCA investigation provides program managers with information that can be used to improve the system of determining UI eligibility. Can the QPI review provide the same or similar information?
Table 1-2 shows, for the five pilot states combined, the prior agency action taken by state agencies in separation and nonseparation denials, and for each prior action shows the percentage of improper denials that passed the QPI review. The purpose of the table is to suggest the extent to which the QPI review could identify the error issues revealed by the DCA investigation, and whether the QPI's potential effectiveness varies by prior agency action.
As already noted, for the majority of erroneous separation and nonseparation denials, state agencies took incorrect action, did not identify an issue, or did not follow agency procedures (see the "total" column in Table 1-2). However, the mix of prior agency action differs between denial errors that passed the QPI review and those that failed the QPI review. The erroneous denials that passed QPI were more likely to involve an issue that the agency could not detect. For example, whereas 11 percent of the separation denial errors that failed QPI involved an "undetectable" error, about 35 percent of the separation denial errors that passed QPI involved an "undetectable" error. Similarly, about 14 percent of the nonseparation errors that failed QPI involved an "undetectable" error, whereas 32 percent of the nonseparation errors that passed QPI involved an "undetectable" error. This is clear evidence that the DCA investigation tends to "catch" errors that the QPI cannot detect. Stated differently, the DCA investigation has the clear potential to uncover and add information to the process of improving the system's performance.
1.2. Point of Detection of Erroneous Denials and the QPI
The DCA investigation also gives information on the method by which each case error was discovered the error detection point. As can be seen in Table 1-3, four detection points are coded in the Data Collection Instrument: verification of wages or separation, the claimant interview, through a third party, and UI records. Table 1-3 suggests that UI records were the most common way of detecting erroneous separation and nonseparation denials: 45 percent of separation errors were detected through UI records, and nearly 63 percent of nonseparation errors were detected though UI records. The claimant interview also played a significant role in detecting both separation and nonseparation errors, and verification of wages and separation played a significant role in detecting separation errors.
Table 1-3 also breaks down the error detection point by whether the case passed or failed the QPI review. This breakdown suggests that the error detection point for a case that fails the QPI review is most likely to be either verification of wages/separation or UI records. (The column percentages for these two error detection points exceed the overall row percentages for failing QPI in both the separation and nonseparation denials.) This suggests again the value and importance of the DCA field audit in detecting errors. By its nature, the QPI review tends to pick up errors that can be detected by examining agency records that are on hand although it is still true that the QPI passes over 25 percent of erroneous separation denials that were detected from UI records and over 40 percent of erroneous nonseparation denials that were detected from UI records. (These are cases, in general, where the agency had adequate information but took the wrong action.) The QPI, however, cannot detect errors that require new information from, for example, claimant interviews and third parties.
1.3. What If a Claimant Changes His or Her Story for the DCA Investigation?
A concern that has been raised about the DCA investigation is that a claimant could give a different story to the DCA investigator than was given in the original factfinding. If this were to occur, the argument goes, then the DCA investigator could conclude that the determination was originally incorrect, but this would unfairly impugn the original investigation because it was the claimant's testimony that originally misled the determination.
One way to appraise whether this argument should be a serious concern is to look at data on point of detection jointly with data on prior agency action. This is done in Table 1-4. If a claimant changes his or her story, then the DCA point of detection should be "claimant interview," and the prior agency action should be "not detectable." In Table 1-4, it can be seen that only 6 out of the 86 erroneous separation denials (or 7 percent), and 15 out of the 149 nonseparation denials (or 10 percent), fit this description.
Although these figures suggest that "story-switching" is not a major problem, the number of erroneous denials that fit this description may give an imperfect idea of the extent to which "story-changing" could be a problem. First, the claimant interview could be the detection point in erroneous denials that were coded as "undetectable" even if the claimant did not change his or her story. If so, then the number of cases that fit the above description overstates the number of "story-switchers." On the other hand, a claimant could switch his or her story without the claimant interview being the detection point. But in this latter case, the change in the story would be a moot point because the error was found by some other means.
The conclusion is that the figures given above (7 percent for separation denials and 10 percent for nonseparation denials) are upper-bound estimates of the extent to which story-switching might be responsible for error detection. Based on the evidence, then, it would be difficult to conclude that "story-switching" is a serious problem.
1.4. Implications of QPI and DCA for States' Procedures
Most of the evidence to this point has suggested that the DCA has the potential to offer information that the QPI could not provide. However, the QPI also has the potential to serve as a tool in detecting errors and diagnosing problems with a state UI system. As discussed in greater detail in the next chapter, the QPI scores each case along six lines:
Accordingly, it should be useful to examine denials that DCA found to be in error in light of what the QPI review found, with an eye to understanding what the QPI would suggest to an administrator about problems in eligibility determination.
Table 1-5 displays, for the separation and nonseparation denials that DCA found to be in error, tabulations of the QPI findings on each of the six aspects of a case listed above. The first column shows that the QPI review failed two-thirds of the separation errors for not meeting the provisions of law and policy. (Under the QPI scoring system, a determination that fails law and policy fails the QPI; therefore, the same two-thirds of these denials, all of which were erroneous, failed the QPI see the bottom row.) Also, the QPI review indicated that for over one-third of the separation error cases, claimant information and/or employer information was inadequate or missing, and the written determination was inadequate or wrong.
The second column shows that the QPI review failed over half (54.5 percent) of the nonseparation errors for not meeting the provisions of law and policy. (The same 54.5 percent of these erroneous denials failed the QPI see again the bottom row.) Also, the QPI review indicated that for nearly 30 percent of the nonseparation error cases, claimant information was inadequate or missing, and for nearly 40 percent of the cases, the written determination was inadequate or wrong.
Looking at the QPI results in Table 1-5, one would conclude that there are two main problems with the separation and nonseparation determination processes: incorrect application of the state's law and policy, and inadequate or missing information (from the claimant in both separation and nonseparation denials, and especially from employers in separation denials). In effect, these problems are similar to the main problems identified by the DCA in its investigation: incorrect actions taken by the agency, and inability to detect issues (usually due to incomplete information). Based on such findings, one could perhaps defend the QPI and say that it identifies the same problems as are identified by the DCA.
The difficulty in such an argument is that the QPI offers no benchmark for identifying the actual extent and nature of determination outcomes that are erroneous. As already noted, overall, the QPI gives the impression that there are far more denial errors than actually occur (based on the DCA). Further, the QPI tends to miss denial errors that involve inadequate or incorrect information on file with the agency. This last point can be seen in Table 1-6, which shows the same information as Table 1-5 but for separation and nonseparation denials that were found to be "undetectable" as well as erroneous by the DCA investigation. Of the erroneous separation denials that were undetectable, the QPI review failed only 40 percent (as opposed to failing two-thirds of all erroneous separation denials). Similarly, of the erroneous nonseparation denials that were undetectable, the QPI review failed only 34 percent (as opposed to failing nearly 55 percent of all erroneous nonseparation denials). This suggests again that the QPI review has a difficult time with the undetectable cases the cases for which agency procedures are relatively weak or for which information is inadequate.
It follows that, even though the outcomes of the QPI review and the DCA investigation point to the same main problems with the separation and nonseparation determination processes, the QPI is a rougher gauge of the extent of the problems. First, the QPI gives an upward-biased picture of the extent to which the determinations process is flawed. Second, the QPI overstates the extent to which incorrect action is a problem and understates the extent to which inadequate information is a problem. It follows that an administrator using the QPI alone would have trouble knowing the appropriate quantity of additional resources to devote to a problem, or of knowing when to stop increasing the resources devoted to a problem. Because the DCA is designed with the purpose of estimating the accuracy of denied claims, it seems natural to use the QPI in conjunction with DCA to gain a rounded picture of the accuracy and quality of nonmonetary determinations.
This chapter has focused on the potential value-added of the DCA investigation, relative to the existing QPI review, in helping states administer the UI program. In particular, it has attempted to draw out features of the DCA that would give states an advantage in detecting errors in separation and nonseparation denials. Section 1.1 used DCA data on prior agency action to examine the causes of denial errors and the extent to which the QPI could detect denial errors. That section drew two main conclusions. First, the great majority of separation and nonseparation errors result from agency error of some kind. Second, erroneous separation and nonseparation denials that passed QPI were more likely to involve an issue that the agency could not detect, indicating that the DCA tends to pick up errors that the QPI misses. That is, the QPI tends to miss a significant subset of problems in the separation and nonseparation determinations processes.
Section 1.2 used DCA data on point of error detection to examine further the causes of denial errors. Comparison of the error detection point in erroneous denials that passed the QPI with the error detection point in erroneous denials that failed the QPI confirmed the reasonable suspicion that the QPI cannot detect errors that require new information, especially the type of information that would be obtained from interviews with claimants or third parties.
Section 1.4 attempted to bring together the various analyses of the DCA and QPI and draw out their implications for practice. The main questions addressed are: What can the QPI review suggest about procedures that the states need to modify either to improve error detection or (preferably) to prevent denial errors in the first place? Can the DCA determination reveal additional information that the QPI cannot offer? In particular, are there gaps in the QPI approach that can be filled only with an intensive field audit like the DCA?
Section 1.4 highlighted two significant drawbacks of the QPI. First, the QPI gives an upward-biased picture of the extent to which the determinations process is flawed, as discussed in the Denied Claims Accuracy Pilot Project Final Report (May 1999). Second, the QPI understates the extent to which incorrect action is a problem and overstates the extent to which inadequate information is a problem. Although the outcomes of the QPI review and the DCA investigation point to the same main problems with the separation and nonseparation determination processes, an administrator using the QPI alone would have difficulty correctly allocating resources to improving the quality of decisions made by adjudicators relative to improving the information on which decisions are based. It seems natural to use the QPI and the DCA together to gain a rounded picture of the accuracy and quality of nonmonetary determinations.
Can the QPI Review Mimic the Findings of DCA Investigations?Chapter 2
2.1. Lowering the QPI Pass/Fail Threshold
The overall QPI score is calculated as the sum of scores on six components: adequacy of claimant information, adequacy of employer information, adequacy of other information, provision of opportunity for rebuttal, whether the determination meets the provision of state law and policy, and adequacy of the written determination. Accordingly, the overall QPI score (denoted simply as QPI) is calculated as: QPI = clmtinfo + empinfo + othinfo + rebutprv + lawpol + writdet, where:
clmtinfo = the case's score on the adequacy of information obtained from the claimant (0 if not available or missing, 5 if inadequate, and 10 if adequate or not applicable);
empinfo = the case's score on the adequacy of information obtained from the employer (0 if not available or missing, 5 if inadequate, and 10 if adequate or not applicable);
othinfo = the case's score on the adequacy of information obtained from others (0 if not available or missing, 5 if inadequate, and 10 if adequate or not applicable);
rebutprv = the case's score on whether the adjudicator provided the opportunity for rebuttal to the applicable parties (0 if opportunity not provided, 10 if opportunity provided or not applicable);
lawpol = the case's score on whether the non monetary determination met the provisions of state law and/or policy (0 if denial determination met provisions of state law and policy, 30 if determination questionable; 50 if determination met provisions of state law and policy);
writdet = the case's score on the adequacy of the written determination (0 if completely wrong, 5 if not adequate, 10 if adequate).
The overall QPI score cannot exceed 100.
The same six components are also used to calculate a "modified pass-fail" QPI score. To obtain the modified pass-fail score, the law and policy (lawpol) component is linked to the first four components in the following way. If a case receives fewer than 10 points for adequacy of claimant information (clmtinfo), adequacy of employer information (empinfo), adequacy of other information (othinfo), or provision of opportunity for rebuttal (rebutprv), then the score on law and policy cannot exceed 30 (that is, whether the case meets the provisions of state law and policy are at best "questionable"). For example, if a case scores 5 (inadequate) or 0 (not available or missing) on adequacy of claimant information, then the score on law and policy could be at most 30, and the sum of the six components would be at most 75. Only when the first four components receive scores of 10 is the law and policy component scored independently.
A QPI summary score of 80 or less is considered failing. Table 2-1 (panel A) shows a cross-tabulation of the accuracy of separation denials by whether the denial determination passed or failed QPI. (Panel A of Table 2-1 is repeated from Table 4-14 of the Final Report.) Of the 902 separation denial cases that were both investigated by DCA and had QPI appraisals, 603 were determined proper denials by DCA and passed QPI. Also, 54 denial cases were determined improper denials by DCA and failed QPI. But 218 cases that were proper denials (as determined by DCA) failed QPI, and 27 cases that were improper denials (as determined by DCA) passed QPI.
Table 2-2 (panel A) shows cross-tabulations of the accuracy of nonseparation denials by whether the denial determination passed or failed QPI. The findings are similar to those for separation denials: Of the 895 nonseparation denial cases that were both investigated by DCA and had QPI appraisals, 607 were determined proper denials by DCA and passed QPI. Also, 78 nonseparation denial cases were determined improper denials by DCA and failed QPI. But 145 cases that were proper denials (as determined by DCA) failed QPI, and 65 cases that were improper denials (as determined by DCA) passed QPI.
Two main conclusions follow from panels A of Tables 2-1 and 2-2. First, the QPI is only weakly correlated with the findings of the DCA investigations. A high proportion of erroneous denials pass QPI and a high proportion of proper denials fail QPI. Second, the QPI gives an excessively negative view of the extent to which denials are erroneous.
It is natural to ask whether either of these problems could be solved or mitigated simply by lowering the QPI pass/fail threshold that is, by considering denial determinations with a QPI score of less than 80 to be passing. Panel B of Table 2-1 shows that, if the QPI pass/fail threshold were lowered to 65, only 14 percent of the separation denial determinations would fail QPI much closer to the 9 percent error rate found for separation denials by the DCA investigations. However, lowering the QPI threshold also raises the number and proportion of erroneous separation denials that pass QPI. With a threshold of 80, one-third of the erroneous separation denials pass QPI, whereas with a threshold of 65, nearly two-thirds (63 percent) of erroneous separation denials pass QPI.
Panel B of Table 2-2 tells a similar story for nonseparation denials. If the QPI threshold were lowered to 70 for nonseparation determinations, only 17 percent of the nonseparation denial determinations would fail QPI very close to the 16 percent error rate found for nonseparation denials by the DCA investigations. But again, lowering the QPI threshold raises the number and proportion of erroneous denials that pass QPI. With a threshold of 80, 45 percent of the erroneous nonseparation denials pass QPI, whereas with a threshold of 70, 59 percent of erroneous nonseparation denials pass QPI.
We conclude that lowering the QPI pass/fail threshold would not be a satisfactory way of modifying the QPI so as to improve its performance and a measure of the accuracy of non monetary denials. By correcting one problem with the QPI the excessively negative impression the QPI gives of denial determinations one would simply increase the number and proportion of erroneous denials that pass QPI.
2.2. Eliminating Apparently Misleading Elements from the QPI
The findings shown in panels A of Tables 2-1 and 2-2 show that, in addition to giving an overly negative view of denial determinations, the QPI is only weakly correlated with the findings of DCA investigations. It is also natural to ask whether the QPI could be modified in some way so as to improve its correlation with the DCA findings. For example, if some components of the QPI are highly correlated with the DCA findings, then perhaps these components could be used (and the others eliminated) so as to improve the QPI as a performance indicator.
Results presented in the Denied Claims Accuracy Pilot Project Final Report (pp. 89-91 and Table 4-18) show that three of the six components of the QPI claimant information, employer information, and rebuttal opportunity provided tend to be negatively related to the outcome of the DCA investigation. These basic results are repeated in Table 2-3, which shows probit regressions of the DCA outcome (proper or erroneous denial) on the individual components of the QPI. (A separate equation is estimated for separation and nonseparation denials. Table 2-3 differs somewhat from Table 4-18 the Final Report because is based on probit analysis, with coefficients transformed so that they can be interpreted as discrete linear changes).
A positive coefficient in Table 2-3 indicates that a high score on the QPI component is positively related to a correct determination, and conversely. The main conclusion from Table 2-3 is that separation denials that received a high score on the claimant information or rebuttal opportunity provided components of the QPI were actually more likely to be found in error by the DCA investigation. Also, nonseparation denials that received a high score on the claimant information component of the QPI (and, to some extent, the employer-information and rebuttal-opportunity-provided components) were more likely to be found in error by the DCA investigation. These conclusions are strongest for the adequacy of claimant information, whose coefficient is negative and significant at the 5-percent level in both the separation and nonseparation equations.
The results in Table 2-3 suggest that the QPI would perform better if some of its components those that are negatively correlated with the findings of the DCA investigation were eliminated. A simple method of testing whether the QPI could be modified to give results that more closely resemble the DCA results is to compute a modified QPI score that relies only on the three components of the QPI scoring that are positively correlated with the outcome of the DCA investigation: other information, law and policy, and the written determination.
Deleting the three components of the QPI that appear to be negatively correlated with the outcome of the DCA investigation yields the following modified QPI score (QPIm):
QPIm = (othinfo + lawpol + writdet) (10/7),where the notation is the same as above. Note that in computing the modified QPI, the three included components must be weighted by (10/7) in order for the score to range from 0 to 100. (Perfect scores on othinfo, lawpol, and writdet would sum to 70. Multiplying 70 by 10/7 yields 100.)
Panel C of Tables 2-1 and 2-2 shows crosstabs of this modified QPI with findings of the DCA investigations. With a pass/fail threshold of 80 (as with the conventional QPI) the modified QPI gives results that are essentially the same as the results from the conventional QPI compare panels A and C in Table 2-1 for separation denials, and compare panels A and C in Table 2-2 for nonseparation denials. That is, eliminating the components of the QPI that are negatively related to the findings of the DCA investigations does not improve the performance of the QPI.
Would choosing a different threshold for the modified QPI help? Panel D of Tables 2-1 and 2-2 crosstabulates the DCA outcomes against the modified QPI but this time using a QPI pass/fail threshold of 65. The results in panel D of Table 2-1 (for separation denials) are quite similar to those in panel C (which used a threshold of 70 with the conventional QPI). And similarly, the results in panel D of Table 2-2 (for nonseparation denials) are quite similar to those in panel C (which used a threshold of 65 with the conventional QPI). Lowering the modified QPI pass/fail threshold can bring the total proportion of denials that fail QPI closer to the proportion of denials that are in error, but only by increasing the number and proportion of erroneous denials that pass QPI. (Recall that this was the outcome when the threshold was lowered for the conventional QPI again, compare panels A and B in Tables 2-1 and 2-2.)
The conclusion is that a simple attempt to modify the QPI eliminating components of the QPI that are negatively correlated with the findings of the DCA investigation does not improve the performance of the QPI.
2.3. Discriminant Analysis of the QPI and Denied Claims Accuracy
An alternative method of examining whether the QPI scoring can be modified to mimic (or predict) the findings of the DCA investigation is to compute a discriminant function based on the QPI's components. The goal here is to form a linear combination of the components of the QPI such that the linear combination can be used to discriminate between correct and erroneous denials. This could be accomplished using a canned discriminant analysis package; however, because discriminant analysis is a variant of standard regression analysis, it is straightforward to develop the approach in a regression context. (This also has the advantage of making clear what is being done.)
2.3.1. The Discriminant Function and Assignment of Scores.
Consider using the sample of cases for which both DCA investigations were conducted and QPI scorings were performed to estimate the following regression model :
In (1), DCA is a zeroone indicator of whether the denial was correct or erroneous (1 if
correct, 0 if erroneous), clmtinfo, empinfo, othinfo, rebutprv, lawpol, and writdet are the
components of the QPI as already defined, the b's are regression coefficients to be
estimated, and e is a random error term. Applying ordinary least squares to (1) yields
estimates of the b's, which can be used as weights of a linear discriminant function:
where L denotes the discriminant function and the b's are least-squares estimates of
the b's in equation (1). These b's are weights that maximize the ability to discriminate
between correct and erroneous denials. [Some practitioners advocate using a
nonlinear transformation of the estimated coefficients in the discriminant analysis, as
described in the appendix to this chapter. However, the results will be essentially
similar to those described here.]
Once the weights of the discriminant function have been obtained, equation (2) can be used to compute a score for each denial case. This is done by substituting the values of the QPI components of each case into the discriminant function. Each case will have its own score, which can be referred to as Li. Cases with higher Li scores should be more likely to have been correct denials than cases with lower Li scores.
Next, a cut-off score, L*, must be chosen so that each case can be classified as either a "correct" denial or an "erroneous" denial based in its score. Cases with Li greater than the cut-off are classified as "correct," whereas those with Li less than the cut-off are classified as "erroneous." There are several possible ways of choosing a cut-off, but the simplest is to choose L* so that the proportions of cases assigned to "correct" and "erroneous" status based on the scores (Li) are the same as the proportions of correct and erroneous cases based on the DCA investigations. We follow this procedure below, but also check the sensitivity of the results to selection of the cut-off by examining a lower cut-off, as was done in Tables 2-1 and 2-2.
The final step is to compare the correspondence between the assignments based on the discriminant analysis with the findings of the DCA investigations. This can be done with the same type of 2-by-2 matrix that was used in Tables 2-1 and 2-2. To summarize, discriminant analysis of the QPI scoring proceeds in the following steps:
2.3.2. Results. Table 2-4 displays the results of the procedure outlined above for separation denials, and Table 2-5 does the same for nonseparation denials. The regression estimates that underlie the discriminant function scores used in Tables 2-4 and 2-5 are similar to those shown in Table 2-3, and hence are not displayed. (They differ from the estimates in Table 2-3 only because they are estimated by ordinary least squares and include, in addition to the six QPI components already discussed, a seventh whether appeal information was provided to the claimant).
Panel A of Table 2-4 crosstabulates the discriminant function pass/fail score based on all QPI components (with a pass/fail threshold of 0.75) against the findings of the DCA investigation. (A cut-off of 0.75 assigns roughly the same proportions of separation denials to "pass" and "fail" status as the proportions of correct and erroneous cases found in the DCA investigations.) Panel B of Table 2-4 repeats panel B of Table 2-1 and is the proper comparison for Panel A of Table 2-4 because it shows the crosstabulation of QPI pass/fail scores (with a pass/fail threshold of 65) against the findings of the DCA investigation. (Recall that with a QPI pass/fail threshold of 65, only 14 percent of the separation denial determinations fail QPI, which is as close as we were able to bring the conventional QPI to the 9 percent error rate found for separation denials by the DCA investigations.)
Comparing the results in panels A and B of Table 2-4 suggests that the discriminant function approach does succeed in reducing the number and proportion of proper separation denials that QPI fails. However, the discriminant analysis does not significantly alter the number or proportion of erroneous denials that QPI passes (the first rows of panels A and B of Table 2-4 are essentially similar.) This suggests again that the DCA investigations uncover information about cases that are essential to the propriety of the case and that cannot be discovered through a simple QPI case review.
Panels A and B of Table 2-5 are analogous to panels A and B of Table 2-4, but for nonseparation denials. Panel A crosstabulates the discriminant function pass/fail score based on all QPI components (with a pass/fail threshold of 0.71) against the findings of the DCA investigation. A cut-off of 0.71 assigns roughly the same proportions of nonseparation denials to "pass" and "fail" status as the proportions of correct and erroneous cases found in the DCA investigations. Panel B of Table 2-5 repeats panel B of Table 2-2 for comparison because it shows the crosstabulation of QPI pass/fail scores (with a pass/fail threshold of 70) against the findings of the DCA investigation. (With a QPI pass/fail threshold of 70, 17 percent of the nonseparation denials fail QPI, which is close the 16 percent error rate found for separation denials by the DCA investigations.)
The conclusions for nonseparation denials are similar to those for separation denials. The discriminant function approach reduces somewhat the number and proportion of proper nonseparation denials that QPI fails (although far less than was true for separation denials; compare the second rows of panels A and B of Table 2-5.) However, the discriminant analysis does not significantly alter the number or proportion of erroneous denials that QPI passes (the first rows of panels A and B of Table 2-4 are essentially the same.) Again, the DCA investigations appear to uncover important information about cases that cannot be discovered through a QPI review.
Recall that up to three of the components of the discriminant function used to obtain the scores in panel A of Tables 2-4 and 2-5 are negatively related to the accuracy of the denial adequacy of claimant information, adequacy of employer information, and provision of rebuttal opportunity. Accordingly, it might make sense to drop those components from the discriminant function. Doing so will not improve the performance of the scores that are based on the discriminant function (in fact, just the opposite). But it seems difficult to justify inclusion of components in a discriminant function that, although they should in principle be positively related to the outcome in question, are negatively related to the outcome.
Panels C of Tables 2-4 and 2-5 display crosstabulations the discriminant function pass/fail scores based only on adequacy of other information, whether the determination met provisions of state law and policy, and adequacy of the written determination. (Cutoffs of 0.71 assign roughly the same proportions of separation denials to "pass" and "fail" status as the proportions of correct and erroneous cases found in the DCA investigations.) Panels D of Tables 2-4 and 2-5 repeat panel D of Tables 2-1 and 2-2 for comparison.
Comparison of the results in panels C and D of Tables 2-4 and 2-5 suggests that dropping the components of QPI that are negatively related to the accuracy of denials yields a discriminant function (in panels C) that gives essentially similar results as a QPI modified to drop those components (panels D). Also, the modified discriminant function performs less well (or no better) than does the discriminant function based on all QPI components (panels A). This latter result is not surprising because the modified discriminant function omits information that is correlated with the probability of correct denial (albeit in a perverse way, as discussed above).
To summarize, the results shown in Tables 2-4 and 2-5 suggest that using a discriminant analysis of QPI components to construct case scores can result in some improvement of the QPI's performance. In particular, the approach can reduce the number and proportion of proper denials that fail QPI. However, discriminant analysis cannot reduce significantly the number or proportion of erroneous denials that QPI passes. Under the best circumstances that is, using a discriminant analysis that takes maximum advantage of the information contained in the components of the QPI QPI is capable of classifying at most 41 percent of erroneous nonmonetary denials as failing. Because the goal of both Quality Assurance and Quality Control is to uncover and diagnose errors in the system so that they can be corrected, the ability of QPI to identify correctly only about 40 percent of all erroneous nonmonetary denials is a clear shortcoming that suggests again the importance of conducting intensive field investigations like those carried out during the DCA pilot project.
Rather than use the estimated coefficients of equation (1) directly to obtain a discriminant function, some practitioners advocate using the following nonlinear transformation of the coefficients (b's) in equation (1) to obtain the weights (b's) in equation (2). (See, for example, Kleinbaum, Kupper, and Muller 1988, pp. 566572.) Use the squared multiple correlation coefficient (R2) obtained in estimating equation (1) to compute the Mahalanobis generalized measure of the distance between the correctly and erroneously denied samples (D2):
where b'i denotes the estimated coefficient of the i'th right-hand-side variable in equation (1).
Once the weights of the discriminant function have been obtained, equation (2) can be used to compute a score for each denial case, as described in the text.
State Unemployment Insurance agencies make decisions about the eligibility and amount of UI benefits to be paid (if any) to each UI claimant. Errors can be of two types. First, a worker who is in fact eligible may be erroneously denied benefits. Such errors (rejecting an eligible claimant) have been the main concern of the DCA pilot project. Second, a claimant who is ineligible for UI benefits may be found eligible for and receive benefits (or, relatedly, a claimant may receive a higher benefit amount than he or she is eligible for). This latter type of error (accepting an ineligible claimant) has been investigated since the late-1980s under the Benefit Accuracy Measurement program (BAM, previously known as Benefits Quality Control, or BQC).
Section 3.1 compares the denial error rates found by the DCA pilot project with the overpayment error rates found in the same five pilot states during the same time period under the BAM program. The main purpose is to examine differences between denial and payment error rates. Section 3.2 attempts to account for the differences between denial and payment error rates. For example, are there differences between denial and payment errors in undetectable issues or incorrect decisions made by the agency?
3.1. Erroneous Denials and Payment Errors
The top panel of Table 3-1 reviews the DCA Pilot data on erroneous denials from September 1997 through August 1998. In the five pilot states taken together, 16 percent of monetary denials were erroneous, nearly 9 percent of separation denials were erroneous, and 15 percent of nonseparation denials were erroneous.
The middle and bottom panels of Table 3-1 display the payment error rates (and the frequencies from which the rates are derived) in the five pilot states during the same time period. These figures come from BAM records. Two kinds of overpayment rate are shown. The middle panel shows "total overpayments," which are positive payments made to UI claimants who the BAM investigation determined should have received a payment of zero in the key week. The bottom panel shows the sum of total and partial overpayments, which are total overpayments plus payments that are too large, according to the BAM investigation.
Note that, in computing the overpayment error rates, the total number of BAM cases is used as a base (that is, the denominator of the overpayment error rates is the total number of BAM cases). The reason is that BAM has a single sampling frame of paid cases and investigates all eligibility issues for each payment (because each payment was issued after monetary, separation, and nonseparation eligibility had been determined). The universe consists of actions that affirmed the claimant's eligibility under all three criteria. This differs from the DCA pilot, in which there was a separate sampling frame for each type of denial.
The total overpayment measure is a closer analog to erroneous denials than is the sum of total and partial overpayments. The reason is that denial entails no payment when the correct payment was positive. The obverse of denial is total overpayment a positive when no payment should have been made. Also, many partial overpayments involve only a small sum (even a dollar or two). Accordingly, most of the comparisons below are between erroneous denials and total overpayments.
The middle panel of Table 3-1 shows that total overpayment rates are sharply lower than erroneous denial rates. In the five states taken together, the total overpayment rate on monetary determinations was 0.6 percent, compared with a monetary denial error rate of 16 percent. That is, the monetary denial error rate was nearly 27 times the total overpayment rate on monetary determinations.
Similarly, the total overpayment rate on separation and nonseparation determinations exceeded the separation and nonseparation denial rates, although the differences are less striking than for monetary determinations. The total overpayment rate on separation determinations was 1.2 percent, compared with a separation denial error rate of almost 9 percent. The total overpayment rate on nonseparation determinations was 5.6 percent, compared with a nonseparation denial error rate of 15 percent.
The gap between total overpayment rates and denial error rates holds for all five pilot states and for all three types of claim, with just two exceptions. In South Carolina and West Virginia, the nonseparation denial error rate is similar to the total overpayment rate on nonseparation determinations. South Carolina has the highest overpayment rate on nonseparation determinations, reflecting what appears to be a complicated treatment of disqualifying income. West Virginia, on the other hand, has by far the lowest rate of erroneous nonseparation denials, and a rate of overpayments on nonseparation determinations that is close to the average of the five pilot states. With these two exceptions, the behavior of the five pilot states is consistent with a far stronger aversion to overpayment than to erroneous denial. This aversion to making overpayments relative to erroneously denying benefits is consistent with keeping benefit payments down.
Although the comparison between erroneous denials and the sum of total and partial overpayments is less striking, it leads to the same general conclusion: State agencies are far more wary of making overpayments than of incorrectly denying benefits. Note that under 10 percent of all overpayments on monetary determinations (that is, the sum of total and partial overpayments) are total overpayments. (The total overpayment rate is 0.6 percent, whereas the total and partial overpayment rate is 6.9 percent.). But nearly all overpayments on separation determinations are total overpayments. (The total overpayment rate is 1.2 percent, whereas the total and partial overpayment rate is 1.3 percent.). Roughly 60 percent of all overpayments on nonseparation determinations are total overpayments. (The total overpayment rate is 5.6 percent, whereas the total and partial overpayment rate is 9.4 percent.).
3.2. Why the Differences between Overpayment and Denial Error Rates?
The main finding of the preceding section is that total overpayment rates are sharply lower than erroneous denial rates. This suggests that agencies place a higher priority on avoiding overpayments to ineligible claimants than on improperly denying benefits to eligible claimants.
However, the findings require further explanation. In particular, what mechanism underlies the low frequency of overpayments, relative to erroneous denials? Do overpayments involve more situations that are difficult to detect, and do erroneous denials involve a greater tendency to misapply adequate information (for example, by taking an incorrect action, failing to identify an issue, or not following procedures)?
Table 3-2 takes a first step in addressing these questions by tabulating prior agency action of total overpayments by state and type of determination. For total overpayments on monetary, separation, and nonseparation determinations, the table shows the percentage (and number) of cases that could not be detected, were already being resolved, and on which adequate information was misapplied (through an incorrect action, failure to identify an issue, or by not following procedures). (These categories and their interpretation are described in more detail in chapter 1.) The table displays figures for each of the five pilot states individually and for the five states aggregated. Note that Table 3-2 has the same form as Table 1-1, which displays the same type of data for erroneous monetary, separation, and nonseparation denials.
Comparison of the findings is facilitated by Table 3-3, which shows prior agency action on erroneous denials (from Table 1-1) and on total overpayments (from Table 3- 2) for the five DCA pilot states aggregated. Table 3-3 gives the striking impression that a large majority of total overpayments involve errors that could not be detected by normal agency procedures. Over three-quarters of total overpayments on monetary and nonseparation determinations were "undetectable," and about 60 percent of total overpayments on nonseparation determinations were "undetectable." The percentages of erroneous denials that were undetectable are far lower: 39 percent for erroneous monetary denials, 20 percent for erroneous separation denials, and 22 percent for erroneous nonseparation denials.
It follows that erroneous denials are far more likely to result from errors of judgment that is, misapplication of adequate information than is the case for total overpayments. Table 3-3 suggests that this is the case. Roughly 30 percent of erroneous monetary denials result from incorrect agency action, failure to identify an issue, or failure to follow procedures (that is, misapplication of adequate information); this contrasts with 14 percent of total overpayments on monetary determinations resulting from these causes. Similarly, over 70 percent of erroneous separation denials result from misapplication of adequate information, in contrast to 36 percent of total overpayments on separation determinations. And nearly 70 percent of erroneous nonseparation denials result from misapplication of adequate information, in contrast to about 21 percent of total overpayments on nonseparation determinations.
Overall, the findings of this and the previous section suggest that agencies have quite low total overpayment rates, especially on monetary and separation determinations. Also, the total overpayment errors that do occur tend to be caused not by human error rather, they are difficult to detect. This latter is not an unexpected finding: Claimants have an interest in providing any pertinent information indicating that they are eligible for benefits and in concealing information suggesting that they are ineligible. As a result, agencies can be expected to have more information in the case of an erroneous denial than in the case of an overpayment. (However, the inference that agencies can be expected to have more information in the case of erroneous denials makes it surprising that overpayment rates are so much lower than erroneous denial rates.)
In contrast, rates of erroneous denials are far higher than rates of total overpayment. Moreover, erroneous denials are far more likely to result from errors of judgment than are total overpayments. Whereas most total overpayments can be considered "undetectable," most separation and nonseparation denial errors result from misapplying adequate information.
3.3. Claimant Characteristics and Denial Probabilities
As discussed at the beginning of this chapter, when an individual claims UI benefits, one of four events can occur:The third possibility is the type of error that is the focus of a DCA investigation, and the fourth possibility is the type of error that is the focus of the BAM program.
This section models these unconditional probabilities using ordinary least squares (OLS) equations that control for the characteristics of each claimant. The goal is to examine the role, if any, that demographic and other characteristics of claimants play in the determinations process and its outcome. (More sophisticated modeling techniques are available, and may be appropriate, for examining these issues. In particular, because the determinations process is one that has multiple possible outcomes, multinomial logit would be an appropriate technique. Accordingly, the analysis presented here should be considered preliminary.)
Four models are estimated. In the first, the sample of eligible claimants from
BAM is pooled with the sample of claimants from the DCA pilot project who were
eligible (as discovered by the DCA investigation) but who had received an erroneous
monetary determination. (Claimants who received a total overpayment are dropped
from the BAM sample because they are ineligible. Claimants who received a partial
overpayment are retained because, even though they received a partial overpayment,
they were eligible.) The following model is then estimated:
where DMi equals 1 if claimant i was denied benefits for monetary reasons, 0
otherwise, X1 through XK denote characteristics of the claimant and the claim, beta0
through betaK are linear coefficients to be estimated by OLS, and ui is an error term
assumed to be random. Note that, because the sample used in estimation includes
only eligible claimants, the DMi indicator denotes not just denial, but erroneous
monetary denial. The comparison being made in this model is between (a) eligible
claimants who were correctly determined eligible for benefits, and (b) eligible
claimants who were erroneously denied benefits.
Similar models are estimated for separation and nonseparation denials. For the model of separation denials, the sample of eligible claimants from BAM is pooled with the sample of claimants from the DCA pilot project who were eligible (as determined by the DCA investigation) but who had received an erroneous separation determination. For the model of nonseparation denials, the sample of eligible claimants from BAM is pooled with the sample of claimants from the DCA pilot project who were eligible but had received an erroneous nonseparation determination. (In creating the samples used to estimate these models, claimants who received a total overpayment are dropped from the BAM sample because they are ineligible. However, claimants who received a partial overpayment are retained.)
A fourth model is estimated for total overpayment. For this model, a sample of
ineligible claimants is constructed by pooling all properly denied claimants from the
DCA pilot sample with all claimants who received a total overpayment from the BAM
sample. The model estimated is then:
where OTi equals 1 if claimant i received a total overpayment, 0 otherwise, X1 through
XK denote characteristics of the claimant and the claim, g0 through gK are coefficients
estimated by OLS, and ei is random error. The comparison being made in this model
is between (a) ineligible claimants who were properly determined to be ineligible (and
so received no benefits), and (b) ineligible claimants who were improperly determined
to be eligible (and so received benefits they should not have received).
Table 3-4 displays the results of the models just described. In the first column, the sample of eligible UI claimants is used to estimate the probability of erroneous monetary denial. The results suggest that men and Hispanics who are monetarily eligible for benefits are significantly more likely to be denied (erroneously) than are others. Claimants whose correct WBA and potential duration of benefits are relatively high are significantly less likely to be erroneously denied for monetary reasons. This is not surprising claimants whose correct WBA and potential duration are high are not the "borderline" cases that are likely to be error-prone. Finally, the probability of erroneous monetary denial appears to be lower in New Jersey than in the other four pilot states, other things equal.
The second column of Table 3-4 displays the results of estimating a model in which an indicator for erroneous separation denial sample is regressed on the same independent variables. (The sample of eligible UI claimants from BAM is pooled with the erroneous separation denials from the DCA pilot.) These results suggest that the older a claimant, the less likely he or she is to receive an erroneous separation denial. Claimants whose correct WBA and potential duration of benefits are relatively high are also less likely to receive an erroneous separation denial. (This result makes less sense in the case of separation denials than for monetary denials. Although it seems likely that conditions of separation are related to a worker's earnings history low earnings workers are more likely to quit or be discharged for cause it should be no more difficult to determine the conditions of separation for a lower-wage/lower-benefit worker than for a higher-wage/higher-benefit worker. The finding suggests that adjudicators may incorrectly use a relatively weak earnings history as an indication that a claimant does not meet the separation conditions for eligibility.) Finally, the probability of erroneous separation denial appears to be higher in Wisconsin than in the other four pilot states, other things equal.
In the third column of Table 3-4, the sample of eligible UI claimants is used to estimate the probability of erroneous nonseparation denial. The results suggest that blacks are significantly more likely to receive an erroneous nonseparation denial than are others. As is true of monetary and separation denials, claimants whose WBA and potential duration of benefits are relatively high are significantly less likely to receive an erroneous nonseparation denial. (The comments about this result in the context of separation denials also apply here.) Finally, the probability of erroneous nonseparation denial appears to be lower in New Jersey and West Virginia than in the other three pilot states, other things equal.
The right-most column of Table 3-4 displays the results of estimating a model in which an indicator of total overpayment is regressed on characteristics of the claimant and other characteristics of the claim. (For this regression, the sample of ineligible UI claimants from the DCA pilot is pooled with the BAM sample of ineligible claimants who received a total overpayment.) The results suggest that ineligible men and American Indians (of whom there are very few in this sample) are more likely than others to receive a total overpayment. Claimants whose correct potential duration is relatively long are also more likely to receive a total overpayment, but the claimant's WBA (relative to the state maximum WBA) has no discernible impact of the probability of total overpayment. That is, claimants who are at the maximum potential duration, but whose WBA is below the maximum, are more likely than others to receive a total overpayment. Finally, compared with Wisconsin, total overpayments are more frequent in South Carolina and less frequent in New Jersey and West Virginia.
Overall, the results in Table 3-4 suggest four main conclusions. First, and understandably, claimants whose earnings history puts them near the minimum threshold of benefit eligibility are more likely to receive an erroneous monetary denial than are other claimants. Second, it appears that adjudicators may incorrectly use a relatively weak earnings history as an indicator that a claimant does not meet the separation or nonseparation conditions for eligibility. Third, claimants who are at the maximum potential duration, but whose WBA is below the maximum, are more likely than others to receive a total overpayment. Fourth, there is some evidence in the data that individual characteristics such as race and gender may play a role in erroneous denials and total overpayments. These results are not strong; however, they should alert agencies to a potential problem and should be investigated in future studies of overpayments and erroneous denials.
Errors in determining eligibility have financial consequences for claimants and UI trust funds. This chapter combines information on error rates gathered in the denied claims pilot project (DCA) with other data to estimate the dollar impacts of erroneous denials.
Estimating the benefits not paid to claimants as a result of erroneous denials poses significant analytic problems. For erroneous denials related to monetary and separation determinations, there is no initial payment or associated payment stream because the administrative decisions found the claimant ineligible. The unobserved counterfactual a payment stream covering a succession of weeks within a benefit year never took place. The dollar cost to the claimant is the weekly benefit amount (or WBA, which can be calculated using correct information on base period earnings) times the unobserved number of weeks in benefit status. Estimating the unobserved weeks in benefit status presents a challenge that is addressed in this and the next chapter.
The chapter is divided into four sections. Section 4.1 identifies several important issues that need to be addressed in developing cost estimates. Section 4.2 describes three alternative approaches for making cost estimates. Section 4.3 reviews penalties associated with benefit denials and discusses duration in benefit status. Section 4.4 presents two sets of estimates of the benefits lost due to erroneous denials. Estimates based on the experiences of the pilot states are used to derive national totals.
4.1. Issues in Estimating Lost Benefits
Several issues must be addressed in deriving estimates of the benefits lost due to erroneous denials. Four are treated here: self-correction of initial administrative errors, the interconnectedness of error corrections, estimating the benefits lost per case, and aggregation issues.
4.1.1. Self-correction. Initial errors in denying benefits may be corrected by the normal operations of the UI system. Such "self-correction" was addressed in both pilot projects and both analyses showed self-corrections to be common, especially for monetary determinations. It would seem that estimates of lost benefits should include only cases where initial agency errors would not be corrected through routine administrative procedures.
An important issue (raised in chapter 3 of the Final Report) is the length of the interval between the claim date and the date for measuring the accuracy of monetary determinations. Determining the appropriate length of this interval is a key operational consideration for the eventual implementation of DCA measurement in the states. All but one pilot state recommended that the sampling of monetary denials be delayed for ten work days from the date that the claim was filed, in order to avoid including in the samples to be investigated claims that were initially denied but that will be redetermined in the normal course of the determination process.
It seems useful to distinguish between initial errors and final errors, where the latter constitute a smaller total due to agency self-corrections, employers' actions, and appeals. Table 4-1 summarizes the error rates (both unadjusted and adjusted for selfcorrections) for the two pilot projects. For both pilot projects the figures displayed are simple averages for the five pilot states. In both pilots, self-correction was larger for monetary issues and separation issues than for nonseparation issues. For the former pair of issues more than one-quarter of initial errors were "corrected" by the combined effects of agency procedures, employer actions, and appeals.
There is suggestion in Table 4-1 that overall denial error rates declined between the two periods and by more for monetary and separation issues than for nonseparation issues (where error rates may even have increased). However, because only one state participated in both pilot projects, conclusions regarding a possible decrease in error rates between the two time periods cannot be drawn.
One approach to estimating the benefits lost due to erroneous denials would be to recognize that errors, even when corrected, entail costs to the claimant in delayed payments. If benefits are eventually paid, however, these payments should be captured by BAM (because BAM covers all paid weeks). Using this logic, erroneous denials that are eventually corrected are already part of BAM, and to avoid double counting, they should not be included in the DCA estimates. Thus, it seems that the appropriate approach would examine just the forgone benefits associated with final ("adjusted") errors as displayed in the middle column in Table 4-1. Accordingly, the estimates of lost benefits presented below use only "final" error cases.
4.1.2. Interface Among the Errors. The process of claim, determination, and payment involves a sequence of administrative decisions. For a new initial claim, the sequence is roughly: (a) a monetary determination, (b) the possibility of a separation determination, and (c) the possibility of one or more nonseparation determinations. Receipt of benefits requires positive decisions on all three administrative decisions.
While the preceding sequence oversimplifies the actual process, it provides a useful framework for considering the effects of erroneous denials. If a claimant is erroneously denied on a monetary issue which is then corrected, there could still be a separation denial and/or nonseparation denial. In developing accurate estimates of the benefits lost due to erroneous denials, the interrelations implied by this sequence must be recognized.
This issue was addressed in the DCA Final Report (Woodbury and Vroman 1999, section 5.2.2). There, two situations were noted in which incorrect denials would not lead to payments: (a) an erroneous denial where the reason was incorrect but the decision was correct and (b) correction of an erroneous denial that would be followed by a correct denial at a later stage of the payment process (for example, correction of an erroneous monetary denial could be followed by a denial on a separation issue and/or a nonseparation issue).
The earlier report noted that denials that were correct but for the wrong reason (that is, the first of the above two situations) were observed for 3 percent of monetary denials, 15 percent of separation denials and 14 percent of nonseparation denials. Thus, the impact of these situations could be estimated directly and would be expected to have minor implications for the magnitude of lost benefits.
The second situation is more problematical due to the analytic framework of the DCA Pilot Project, which investigated a case only with respect to one of the three issues monetary, separation, or nonseparation. Not all corrected monetary determinations would lead to a payment, and not all corrected separation determinations would lead to a payment (at least for a full period in benefit status). In the cost estimates presented in section 4.4, we attempt to take account of this difficulty in the design of the DCA Pilot; however, the corrections are only rough.
4.1.3. Benefits Lost per Case. The procedure for estimating benefits lost (that is, not paid) focuses on benefits lost per erroneously denied claim. For a claimant erroneously denied benefits, this loss is the product of estimated time in benefit status and the weekly benefit amount (WBA). Of the two elements that determine benefits lost per case, the WBA is known once the correct information on base period earnings has been obtained. (In both pilot projects, the WBA of those erroneously denied was lower than for those who received benefits. The differential was about 20 percent in the 1997-98 pilot.) What is not known is the time in benefit status. For individuals erroneously denied, benefit duration must be estimated.
One way to estimate benefit duration is to use statewide (or national) average duration for beneficiaries during the same period as that covered by the pilot project; that is, fiscal year 1998. This method has been used by Skrable (1999) and, with modifications discussed in section 4.3, is used in section 4.4 below.
As noted in chapter 5 of the DCA Final Report, an alternative way of estimating average benefit duration is to derive a statistical estimate of the average duration for each person erroneously denied benefits in the pilot states and then average the statistical estimates. Unfortunately, the data needed to derive such estimates are available in only one of the pilot states South Carolina. In chapter 5, the South Carolina data are used to obtain such estimates. The implications of those estimates for lost benefits are also derived in chapter 5.
4.1.4. Aggregation Issues. After identifying an erroneous denial and estimating the resulting benefits lost, those losses must be aggregated to the universe of cases. Aggregation involves at least three issues. The first is to identify the universe of similar cases. Second, aggregation may entail summing over time periods; for example, from a week to a full calendar year. Third, there are questions of how to aggregate results from the five DCA pilot states to national totals.
To identify the universe of similar cases it seems appropriate to use data from administrative reports routinely submitted by the states. For monetary determinations the needed data are included in the ETA 218 reports. These data record the total number of monetary determinations and the number of claims with sufficient wage credits. Those with insufficient credits can serve as the universe for incorrect monetary denials. During fiscal year 1998 there were 10.78 million monetary determinations and 1.17 million findings of insufficient wage credits. Hence, 1.17 million is a universe count of denials on monetary issues.
Data submitted by states in ETA 207 reports for the same period show that there were 3.42 million determinations on separation issues and 1.86 million denials. Nonseparation determinations and denials from these same reports totaled 4.28 million and 2.39 million respectively in fiscal year 1998. Thus, universe counts for separation and nonseparation denials are 1.86 million and 2.39 million. The error rates estimated in the five pilot states can be applied to the national denial totals to estimate the national number of errors of each type.
Aggregation by time period must also be recognized. Quality control measurement systems operate using a key week concept. The findings from a key week need to be aggregated over time to estimate annual dollar losses due to payment errors. This is true of both denial errors and payment errors.
In order to assess the national consequences of denied claim errors, it is necessary to aggregate the results from the five pilot states to the U.S. as a whole. Two questions arise in connection with obtaining national totals from the experience of the five pilot states: First, are the average benefits lost per case in the pilot states representative of the national average? Second, are there other peculiarities of the pilot states for example, the mix of denied cases across the three issues (monetary, separation, and nonseparation) that need to be recognized in developing national estimates?
To illustrate one aspect of the problem, it is instructive to review data from the pilot states on the average weekly wage and the average weekly benefit. One potential element in aggregation would be to adjust for differences in the average weekly wage (hence WBA) between the pilot states and the U.S. average. In 1998, the national AWW for all covered employees was $610.43. The employment-weighted AWW for the five DCA states was $610.26, a deviation of only 0.03 percent from the national average. This virtual equality, however, did not translate into equality of WBAs. In calendar year 1998, the national average WBA was $202.29 whereas the average WBA (weighted by weeks compensated) for the five pilot states was $231.33, a difference of 14.4 percent. The difference reflects the importance of the high WBA in New Jersey and its share of weeks compensated in the weighting for the pilot states.
4.2. Three Approaches to Estimating Lost Benefits
The dollar costs of denied claims can be estimated in three ways: (1) the key week approach, (2) the benefit year approach, and (3) a hybrid approach. Each is described presently.
4.2.1. The Key Week Approach. This approach, which would copy the approach followed in BAM, would concentrate on benefits lost in a single week due to an erroneous denial. For both monetary and separation issue errors, the weekly benefit loss would be given by the WBA. For most nonseparation issue errors, the benefit loss would be the WBA, but this can be modified in some instances. A disqualifying or deductible income penalty would be assessed as denial of benefits for one or more weeks either with or without a reduction in the Maximum Benefit Amount (MBA). Penalties would vary according to the exact income source and according to state law. (Tables 410A and 410B of the "Comparison of State Unemployment Insurance Laws" show the type of penalty applied to pension benefits and other types of employee compensation subject to disqualifying and deductible income penalties.)
After the erroneous denials have been corrected, the interfaces between the issues would still need to be considered. Thus, as noted in section 4.1, not all corrected monetary denials would be followed by a payment because of possible denials on separation and nonseparation issues. (It is also possible for a claimant to remain monetarily ineligible even after a monetary determination error has been corrected for example, the error may have been failure of an employer to report wages during the base period, but base period earnings may remain insufficient to meet the monetary eligibility criteria.)
A problem in following the key week approach is the absence of an observed payment stream for erroneous monetary and separation denials. Payments never started, so there is no stream of weeks compensated from which to draw a sample. Because there is no payment stream, one cannot sample from different points (weeks) in the stream as in BAM. Thus, a focus on key weeks (as in BAM), would make it impossible to impute lost benefits due to erroneous monetary and separation denials. (For nonseparation errors, there is no such problem because, as with BAM cases, nonseparation errors can occur during any individual week in benefit status, and hence can be sampled.)
After assigning a cost per key week to all erroneous denials (typically the WBA), there would remain the question of how to aggregate these costs to statewide annual totals. A direct approach would be to multiply total weekly costs by 52, which would convert a "week" into a year. Although simple, we are reluctant to propose this as a method for aggregating to statewide annual totals. Discussions with BAM professional staff in the pilot project states and at the national Office of Workforce Security suggest that no convincing aggregation procedure exists. These discussions lead us to conclude that errors on monetary and separation determinations are essentially tied to the case or person, not to a key week.
4.2.2. The Benefit Year Approach. This approach examines the consequences of erroneous administrative decisions within a framework where the time unit is the benefit year. Errors are modeled as having consequences that span several weeks. This approach to estimating lost benefits explicitly recognizes benefit duration and the associated stream of payments that did not occur.
Under this approach, the WBAs calculated for each of the three determination issues are the same as under the key week approach. The interfaces between the three issues also need to be explicitly treated. As noted in section 4.1, benefit duration could be modeled in different ways (the use of statewide average durations or projected durations based on a regression equation methodology are the main possibilities). The durational approach is appropriate even for nonseparation issues because many of these erroneous disqualifications have multi-week penalties (see section 4.3 below).
The universes for sampling the three types of cases are as follows:
An implication of the preceding is that denials for separation issues can be expected to have shorter penalty periods than denials for monetary reasons because separation denials are applied to both new initial claims and additional claims. Penalty periods for nonseparation errors should be shortest because they may occur at any point in a benefit spell rather than at the start. (For further discussion of penalty periods, see section 4.3.)
4.2.3. A Hybrid Approach. The BAM (key week) approach to estimating lost benefits draws samples from the universe of paid claims. This can be applied directly to just one type of issue in DCA, nonseparation determinations. In erroneous denials involving monetary and separation determinations, there is no series of weeks in benefit status where key week sampling can be applied. The hybrid approach follows BAM in the treatment of nonseparation issues but a benefit year approach for the other two issues (monetary and separation determinations).
For all three issues, the determination of the WBA is the same as in the key week and benefit year approaches described above. The interfaces among the three types of denials would be treated in a similar manner under the hybrid approach. However, for both monetary and separation errors, a duration in benefit status would be assigned in arriving at the estimate of lost benefits. As noted above, the duration estimates could be taken from statewide averages (with adjustments as described below) or could be based on a regression methodology utilizing BAM micro data (see chapter 5). Because many of the procedures to be followed under the three approaches are identical, it would be interesting to know the sensitivity of findings to the choice of approach.
Denials lead to penalties that delay or reduce (sometimes to zero) the benefits paid to claimants. It is useful to review briefly the penalties for each type of denial. As will be seen, the penalties vary by issue and by state for separation and nonseparation determinations.
The loss to the individual claimant due to an erroneous denial, regardless of the issue, is the weekly benefit amount (WBA) times the weeks of UI benefits that would have been paid had a correct determination been made. There is little uncertainty surrounding the WBA but much uncertainty as to the duration in benefit status. For both separation and nonseparation determinations, the claimant will have satisfied the states monetary eligibility criteria so that the WBA is established. For monetary determinations, the WBA will also be known after correct information on the claimants base period earnings has been assembled.
For the key week and benefit year approaches to cost estimation discussed in section 4.2, it is necessary to estimate the length of time the claimant would have received benefits during the current benefit year, had the correct eligibility decision been made. Under the benefit year approach, duration must be estimated for all three types of errors while the hybrid approach requires duration estimates for monetary and separation denials. Only the key week approach could (potentially) make estimates without reference to benefit duration.
Although estimating duration is highly uncertain for individual recipients, there are systematic differences in benefit duration for the three types of denials. Table 4-2 presents a taxonomy to help illustrate differences in the durational consequences of the three types of erroneous denials.
The rows of Table 4-2 identify administrative activities for the three types of determinations (monetary, separation and nonseparation). For each type of determination, the table shows the universe of claims subject to the determination, the administrative decision (outcome), and the penalty for a denial. The columns identify three types of claims that need to be distinguished because of differing durational consequences: new initial claims, additional initial claims, and continued claims. (Note that the table simplifies by omitting interstate and transitional claims.) The body of Table 4-2 shows the interface between type of claim and type of determination. Monetary and separation determinations are applied to new initial claims. Separation determinations are also applied to additional initial claims (second and later claims in a given benefit year). Nonseparation determinations are applied only to continued claims.
A monetary determination is made for all new initial claims. The vast majority of new initial claims arise from a separation due to lack of work, a quit, or a discharge. (In what follows, we focus on discharges due to ordinary misconduct, not flagrant or aggravated misconduct.) For quits and discharges, the UI agency adjudicates the separation to determine eligibility. Adjudication occurs in roughly one-fourth of initial claims (that is, both new and additional initial claims, according to ETA 207 reports on the number of determinations and unpublished data counts of new spells).
A denial for separation reasons usually implies that benefits will not be received for the duration of the current spell of unemployment. (Exceptions arising from disqualifications for specific periods are discussed below.) A durational penalty also applies for a monetary determination when the claimant is found to have insufficient wage credits. However, on average, the duration of benefits associated with a monetary denial (that is, due to insufficient wage credits) tends to be longer than the duration associated with a separation denial. The reason is apparent in Table 4-2: many separation denials are applied to additional initial claims, in which claimants have already used up a substantial share of their MBA with an earlier claim (or claims).
4.3.1. Monetary denials. These are the most straightforward. When the claimant is found to have insufficient earnings in the base period, the penalty is complete exclusion from benefit status. The claimant does not establish a benefit year. If there is a later separation and a subsequent claim for benefits, the claimant may collect benefits following the later separation; however, for the denied claims project, this future event lies beyond the scope of analysis.
4.3.2. Separation denials. For all practical purposes there are two separation issues: voluntary quits and misconduct. In FY 1998 determinations on voluntary quits totaled 1.484 million and determinations of misconduct totaled 1.830 million. Combined, they accounted for 97.3 percent of all separation determinations for the year. (These totals come from ETA 207 quarterly reports.)
Penalties for separation issues are essentially of two kinds. The first is disqualification from receiving benefits for the duration of the current spell of unemployment a "durational" disqualification. A claimant subject to such a durational disqualification must requalify for benefits (by working for a specified period of time and/or earning a specified amount) in order to be eligible for benefits in the event of a later involuntary job separation. The second type of penalty is delay of benefit receipt for a specified number of weeks, usually (but not always) with a corresponding reduction in the MBA. In general, a claimant subject to such a disqualification may wait out the specified period of disqualification, reopen the claim for benefits, and receive benefits.
Table 4-3 displays penalties for each of the five pilot states on the two separation issues. For a voluntary quit, four of the five pilot states disqualify the claimant for the duration of the current spell and impose a requalification requirement. Nebraska is the exception, delaying benefits for the week of the claim and 7 to 10 subsequent weeks and reducing the MBA by a corresponding amount. (More than one such penalty may be assessed in Nebraska if a claimant quit from more than one base period employer.)
Requalification requirements raise another issue in the estimation of lost benefits. A subsequent separation may have implications for an earlier erroneous denial. An erroneously denied claimant could lose benefits from a later separation if the separation occurs before the requalification period has been satisfied. Although such situations exist, they are probably rare, and we do not take account of them in the cost estimates presented.
In four of the five pilot states, the penalty for a misconduct discharge is delay of benefit payments along with reduction of the MBA. In New Jersey (the exception), benefits are delayed for the week of the claim plus the subsequent 5 weeks; the MBA is not reduced. In Nebraska and South Carolina, there is discretion in the number of weeks by which benefits are delayed and reduced (that is, the penalty is of variable duration). In Wisconsin, the claimant is disqualified for the duration of unemployment. In both West Virginia and Wisconsin, benefit reductions can be recovered by meeting a requalification requirement. This could be relevant for second and later separations that occur within a given benefit year.
The penalties on separation issues for the pilot states, as summarized in Table 4-3, need to be viewed in a national perspective. Of the 53 UI programs, 49 disqualify a claimant who quit voluntarily for the duration of unemployment. Thus, for voluntary quits, the penalties in the pilot states roughly reflect the national situation. However, in 40 of 53 UI programs, a claimant discharged for misconduct is disqualified for the duration of unemployment. (Of the pilot states, Wisconsin is the only one that disqualifies a discharged claimant for the duration of unemployment.) That is, the pilot states greatly understate the prevalence of durational disqualifications for misconduct. Aggregated to national totals, the benefits lost due to erroneous misconduct denials in the pilot states would understate the corresponding benefit lost nationwide.
Table 4-3 introduces complexities that need to be considered in analyzing the costs of erroneous denials. The main point of the information in Table 4-3 is that the penalties associated with individual disqualifications vary considerably by issue and state. To treat all penalties on separation denials as if they were durational would be a simplification of reality, especially for misconduct issues.
4.3.3. Nonseparation denials. The previous discussion about variation in disqualification penalties has even more force when applied to nonseparation disqualifications. The UI reporting system (ETA 207 reports) explicitly identifies five nonseparation issues: (a) able and available for work, (b) disqualifying or deductible income, (c) refusal of suitable work, (d) reporting requirements, and (e) profiling. In addition, there is a sixth catch-all category ("other"). During FY 1998 there were 4.3 million nonseparation determinations with the two largest categories being able and available (1.360 million) and disqualifying or deductible income (1.012 million). All of these disqualifications are applied to weeks of continued claims arising from both new and additional initial claims.
Most disqualifications for nonseparation violations are of two types [(U.S. Department of Labor, Comparison of State Unemployment Insurance Laws, tables 400 (able and available), 404 (refusal of suitable work), and 410A and 410B (disqualifying and deductible income)]. Able and available disqualifications are generally for the week of the violation with no reduction in the MBA. Thus, a claimant who exhausted benefits could collect the full MBA, even with the penalty (that is, there would only be a delay in receipt of benefits). Violation of reporting requirements, refusal of suitable work, and profiling violations generally have penalties that cover several weeks, often the remainder of the current unemployment spell. (The "other" category, because it is a catch-all, has more than a single potential disqualification depending on the issue.)
If the claimant exhausts benefits for which he or she is eligible, single week penalties that delay payments without reducing the MBA do not ultimately result in lost benefits. For claimants who do not exhaust their MBA, the penalty represents a one week loss of benefits.
As noted, many penalties for nonseparation issues apply for multiple weeks or for the remaining duration of the unemployment spell (for example, violation of reporting requirements or refusal of suitable work). The determination leading to the penalty can occur for any week in which benefits are claimed. These weeks may follow either a new initial claim or an additional initial claim. For the latter, the penalty will typically last for fewer weeks than if it followed a new initial claim.
Disqualifying or deductible income penalties either reduce benefits in weeks when the income is received (hence delaying benefit receipt) or reduce the MBA by the amount of the alternative income source, up to the full MBA. (In cases where benefits are reduced by less than the MBA, benefit payments are also delayed.) These penalties cover workers' compensation, wage continuation payments, severance pay, vacation pay, pension benefits. State approaches differ widely, and the penalties associated with disqualifying income accounted for about 25 percent of all nonseparation disqualifications in fiscal year 1998 (ETA 207 data).
National estimates of the benefits lost due to erroneous denials must make assumptions about three questions discussed in section 4.1: self-correction, the interface among the errors, and the benefits lost per case. The estimates presented in this report all use the benefit year approach discussed in section 4.2. We begin with a discussion of the estimates that are similar to those prepared by Skrable (1999), and then modify those estimates to check the sensitivity of his estimates to certain assumptions.
4.4.1. Number of erroneous denials. Skrable starts with an imputation of the number of erroneous monetary, separation, and nonseparation denials in the United States, based on UI financial data and the 1998 DCA results. Table 4-4 displays the calculations. During 1998, there were roughly 10.8 million monetary determinations, 15.9 million separation determinations, and 117.6 million nonseparation determinations in the United States (see the notes to Table 4-4 for sources). Of the monetary determinations, approximately 1.2 million (or 10.9 percent) resulted in denial of benefits. Of the separation determinations, 3.4 million (or 21.4 percent) were adjudicated, and 1.9 million (or 11.7 percent) resulted in denial. Of the nonseparation determinations, 4.3 million (or 3.6 percent) were adjudicated, and 2.4 million (or 2.0 percent) resulted in denial.
The 1998 DCA Pilot Project found that, in the five pilot states, 16.0 percent of the monetary denials, 8.7 percent of the separation denials, and 15.0 percent of the nonseparation denials were erroneous. The 1998 pilot also found that a significant proportion of erroneous denials were corrected either by the UI agencies or through the appeals process. After adjusting for these "self-corrections," the error rates are 11.2 percent for monetary denials, 6.4 percent for separation denials, and 12.9 percent for nonseparation denials.
Assuming that the adjusted error rates estimated in the five pilot states are representative of error rates throughout the United States, they can be applied to the number of denials in the United States to obtain imputations of the number of erroneous denials nationally. This is done in the bottom row of Table 4-4: the imputed number of erroneous monetary denials for the United States, 131,264, is calculated by multiplying the total number of denials in the U.S. by 0.112, and similarly for separation and nonseparation denials. [These figures differ somewhat from those in Skrable (1999), apparently due to a different accounting of self-corrections.]
Skrable (1999) further reduces the number of erroneous monetary denials because only about 75 percent of all correct monetary determinations result in a first payment. There are two reasons for this. First, many monetarily eligible claimants turn out to be ineligible for nonmonetary reasons. Second, not all monetarily eligible claimants receive a first payment; that is, they end their claim because they find a job quickly or drop out of the labor force. After accounting for these factors, the imputation is that roughly 99,000 claimants who would have received benefits were erroneously denied benefits for monetary reasons. [We assume that all erroneous separation and nonseparation denials would have resulted in benefits being paid. As a result, no similar reductions are made for erroneous separation or nonseparation denials.]
The imputed number of erroneous monetary, separation, and nonseparation denials that would have resulted in benefits being paid are shown both in the bottom row of Table 4-4 and in the first column of Table 4-5, which is used to display further development of the denial cost estimates.
4.4.2. Average weekly benefit amount. It is also necessary to assign an average weekly benefit amount (WBA) to each of the three types of erroneous denials. BAM data for FY 1998 show that the average WBA of claimants nationally was $199.18. However, Skrable noted that the average WBA for erroneously denied claimants (as determined by the DCA investigation) fell below the average WBA of eligible claimants in BAM data from the pilot states. Accordingly, the average benefits lost by erroneously denied claimants would be less than the average WBA of paid claimants by what Skrable calls a relative WBA factor, which can be defined as:
For erroneous monetary denials, this relative WBA factor is 0.859; for erroneous separation denials, it is 0.866; and for erroneous nonseparation denials, it is 0.909 (see the column headed "relative WBA factor" in Table 4-5, panels A and B.) 4.4.3. Average duration of benefits received by erroneously denied claimants. Finally, for each type of denial, it is necessary to use an estimate of the average number of weeks of benefits that would have been received by erroneously denied claimants. Estimates of the number of weeks of benefits lost due to erroneous monetary, separation, and nonseparation determinations are discussed in turn.
First, the average number of weeks of benefits lost due to each erroneous monetary denial can be imputed as the average number of weeks compensated per first payment (that is, the sum of first and subsequent spells of benefit receipt within a given benefit year). This assumes that erroneously denied claimants are similar to correctly determined eligible claimants. In FY 1998, this average number of weeks compensated was 14.2 in the United States (ETA 5159 reports) and is used in both panels A and B of Table 4-5 as an imputation of the number of weeks of benefits lost due to an erroneous monetary denial.
Second, how many weeks of benefits were lost by the typical claimant who was denied for separation reasons? The average number of weeks compensated overstates this because separation determinations are made on additional initial claims (as well as on new initial claims), and additional initial claims result in shorter spells of benefit receipt. During 1998, 61.2 percent of all initial claims were new initial claims, and 38.8 percent were additional initial claims (ETA 5159 reports). In 1990- 1993, across fifteen states for which special survey data are available (Battelle Memorial Institute 1999, Table 7-1), the average benefit duration for first spells of unemployment was about twice the duration of subsequent spells (the mean first spell was 13.18 weeks long, and the mean subsequent spell was 6.16 weeks long).
These figures allow a rough imputation of the average number of weeks of benefits lost by a claimant denied for separation reasons. Suppose that the duration of subsequent spells in 1998 was still roughly half that of first spells, as found by Battelle Memorial Institute (1999, Table 7-1) for 1991-93. Then the average number of compensated weeks associated with an additional initial claim would be about 7.1 (half of 14.2). If all separation denials were for the duration of unemployment, the mean duration of a separation denial would be 11.4 weeks [= (0.61214.2) + (0.3887.1)]. (This also assumes that the rates of adjudication for reasons of separation are the same for new and additional claims.) The figure would be somewhat lower if account were taken of the fact that some separation denials are for fixed periods that are shorter than the duration of the spell. In any case, panel B of Table 4-5 uses 11.4 weeks as the imputed number of weeks of benefits lost due to an erroneous separation denial.
Third, how many weeks of benefits were lost by the typical claimant who was denied for nonseparation reasons? Earlier estimates of the loss in benefits from nonseparation disqualifications have assumed the penalty to be one week per disqualification (Belle and Casey 1988, Skrable 1999). This assumption is made in panel A of Table 4-5.
As discussed above, able and available disqualifications are for one week (the week of the violation). However, other nonseparation disqualifications, such as those for violating reporting requirements, refusal of suitable work, and (in many cases) disqualifying and deductible income, involve penalties that cover several weeks or the the remainder of the current unemployment spell. It seems important to use an estimate of the weeks of benefits lost due to a nonseparation denial that takes account of this fact.
Table 4-6 shows figures that have been used to derive such estimates. The first row shows the average number of weeks compensated per first payment in the five pilot states (individually and aggregated) and in the United States (ETA 5159 reports). The imputed duration of a multi-week nonseparation penalty (third row) is based on three assumptions: First, the average nonseparation penalty is assessed at the midpoint of a spell of benefit receipt (so that the average multi-week penalty cuts off half the compensated weeks of unemployment). Second, a nonseparation penalty is equally likely to be assessed on new initial and additional initial claims. Third, the average duration of a subsequent spell of unemployment is half that of a first spell (as assumed above). These second two assumptions can be summarized by a "duration adjustment" (DURADJ), which can be written as:
DURADJ = P + (1-P)(0.5)
where P denotes the proportion of all initial claims that are new initial claims (that is, first spells, based on U.S. Department of Labor 5159 reports), and 0.5 is the duration of the average subsequent spell relative to the average first spell. (Intuitively, the duration adjustment reduces the duration of a penalty assessed on a subsequent spell of unemployment by one-half and weights the number of penalties assessed on first and subsequent spells appropriately using P.) The imputed duration of an average multi-week penalty (third row) is then the product of one-half the average weeks of benefit receipt and the duration adjustment.
Finally, the imputed average duration of all nonseparation penalties is the weighted average of the durations of multi-week and one-week penalties (bottom row of Table 4-6). For example, 42.3 percent of all nonseparation penalties in the United States were multi-week, so the imputed average of all nonseparation penalties was 2.91 = [(5.510.423) + (1.000.577)]. Panel B of Table 4-5 uses this imputation for the number of weeks of benefits lost due to an erroneous separation denial.
4.4.4. Calculation of benefits lost by erroneously denied claimants. The benefits lost due to erroneous denials can now be calculated as the product of three factors:
Hence, for claimants erroneously denied for monetary reasons, the total benefits lost
(LOSSm) during FY 1998 is imputed as:
and analogously for separations and nonseparation denials.
Panel A of Table 4-5 shows estimates similar to those developed by Skrable (1999), who assumed that a separation denial results in lost benefits of 14.2 weeks (that is, the average number of weeks compensated per first payment) and that a nonseparation denial results in the loss of one week of benefits. Based on these assumptions, the loss of benefits resulting from erroneous denials amounts to $583 million.
The above discussion (section 4.4.3) suggests that, for two reasons, using 14.2 weeks (the average number of weeks compensated per first payment) as the average duration of a separation denial will produce an upward-biased estimate of the losses that result from erroneous separation denials. First, fixed length disqualifications (especially on misconduct issues) would tend to make the penalty period shorter than the full duration of unemployment. Second, many separation penalties are applied to additional initial claims (rather than new initial claims), which are necessarily associated with relatively short spells of benefit receipt.
Also, using 1 week as the average duration of a nonseparation denial may produce a downward-biased estimate of the losses that result from erroneous nonseparation denials. As discussed above, many nonseparation denials involve multi-week penalties.
Accordingly, panel B of Table 4-5 shows revised estimates of the losses due to erroneous denials that are based on the alternative assumptions that (a) the average duration of a separation denial is 11.4 weeks (rather than 14.2 weeks), based on the calculation in section 4.4.3, and (b) the average duration of a nonseparation denial is 2.91 weeks (rather than 1 week), based on the calculations in Table 4-6. Based on these latter assumptions, which we believe to be more realistic, the loss of benefits resulting from erroneous denials is somewhat higher, about $625 million (panel B).
The differences between the estimates in panels A and B not great: those estimates suggest that the benefits lost due to erroneous denials represent between 3.2 and 3.4 percent of total regular UI benefit payments by taxable and reimbursable employers combined. (During calendar year 1998, taxable employers paid $18.4 billion in benefits, while reimbursable employers paid $1.0 billion.)
The similarly of the two estimates of lost benefits reflects two offsetting effects. The benefits lost due to erroneous separation denials are $57 million lower in panel B than in panel A because a separation denial is assumed to disqualify a worker for fewer weeks of benefits in panel B. On the other hand, the benefits lost due to erroneous nonseparation denials are $99 million higher in panel B than in panel A because a nonseparation denial is assumed to disqualify a worker for more weeks of benefits in panel B.
Under the estimates in panel B, the overall average loss of benefits per erroneous denial $1,239. However, the average monetary error results in lost benefits of $2,430; the average separation error results in lost benefits of $1,966; and the average nonseparation error results in lost benefits of $527. It is clear, then, that the biggest bang for the administrative buck is likely to be found in reducing errors in monetary and separation determinations. This is true even when the average weeks of lost benefits associated with a nonseparation denial is nearly three. This conclusion implicitly assumes that investigation costs are quite similar for the three types of determinations. In a cost-benefit framework, one would need to compare the costs of investigations for the three types of determinations with the potential benefits deriving from those investigations (that is, the benefit payments received by claimants otherwise erroneously denied).
A key issue in estimating costs in the DCA project is to estimate the benefits that individuals would have received if they had not been erroneously denied. One approach is to impute these lost benefits, as in Chapter 4, from reported UI administrative data on the average WBA and number of weeks of benefits received by claimants who were determined eligible for benefits. This approach has the advantage of being relatively simple to implement using state data on actual beneficiaries from the same period as the DCA pilot.
A second approach, developed in this chapter, is to generate regression-based estimates of the benefits that would have been received by erroneously denied claimants if they had not been erroneously denied. (We also generate complementary estimates of the expected weeks of benefit receipt of erroneously denied claimants.) In brief, the procedure is as follows. First, use a sample of individuals who were correctly determined as eligible for UI benefits to estimate a benefits function. This is done by regressing total benefits received during the benefit year on the claimant's individual characteristics (such as age, gender, citizenship, ethnicity, and schooling), the claimant's usual hourly wage, weekly benefit amount, potential duration of benefits, and the season in which the initial claim was filed. Second, use the estimated coefficients of the regression equation to make imputations of the expected benefits that would have been received by claimants who were erroneously denied benefits. This is done by substituting into the estimated benefits function the average characteristics of claimants who were erroneously denied benefits. Further details are provided below.
5.1. Description of the Data and Samples
Benefit Accuracy Measurement data are appropriate to use in estimating a benefits function. They include most of the information on individual characteristics and benefit receipt that are required to estimate a benefits (or unemployment benefit duration) function. In particular, because the DCA Pilot Project was conducted in five states from September 1997 through August 1998, BAM data from the same period in one or more of the five pilot states are a natural choice for estimating the benefits function.
In order to perform the estimation, however, the BAM data file must be supplemented with an additional piece of information: the dollar amount of UI benefits received by the claimant during the benefit year. This information is not included in the BAM record because the BAM investigation applies to a specific week during a claimant's benefits year, not the full (or completed) benefit year.
The Quality Control Division in one of the pilot states South Carolina volunteered to add the needed data to its BAM records from the time period of the DCA Pilot Project. Specifically, the South Carolina group drew data on benefits paid during the full benefit year (that is, after the benefit year had ended) for each claimant in BAM batches 199736 through 199833, inclusive. These data were identified by batch and sequence number, so that they could be matched back to the BAM benefits master file and used in estimation. (This effort was carried out by Leland Teal, Bob Branham, Layne Waters, and Doug Potter of the South Carolina Employment Security Commission. We are extremely grateful for their efforts and willingness to provide these additional data.)
Table 5-1 displays descriptive statistics of the South Carolina BAM sample that is used to estimate the benefits function in the next section (see the figures in the "correct determination" column). Table 5-1 also gives descriptive statistics of the three groups of workers for whom estimates of "expected" benefits are required claimants erroneously denied for monetary, separation, and nonseparation reasons (see the figures in the "erroneous denial" columns). These latter come from the South Carolina DCA data. Note that the three samples of erroneously denied claimants are quite small there are only 44 erroneous monetary denials, 10 erroneous separation denials, and 36 erroneous nonseparation denials. The very small number of erroneous separation denials may limit the usefulness of the analysis in the case of separation denials; however, the number of erroneous monetary and nonseparation denials appears large enough to support reasonable imputations of the benefits that these claimants would have received during the full benefit year had they not been erroneously denied.
The figures in Table 5-1 suggest that, in South Carolina during the time of the DCA Pilot Project, the typical claimant who was correctly determined to be eligible was somewhat less likely to be male than the typical claimant erroneously denied for monetary (and separation) reasons, and somewhat more likely to be male than the typical claimant erroneously denied for nonseparation reasons. Also, claimants who were correctly determined to be eligible had considerably higher usual wages than claimants erroneously denied for monetary and separation reasons, and slightly higher usual wages than claimants erroneously denied for nonseparation reasons. Other differences across the four groups can be seen in Table 5-1, but the two differences just mentioned turn out to be the most important (see section 5.4).
One of the variables displayed in Table 5-1 requires further comment. The number of weeks of benefits received by each claimant in the BAM sample is estimated by dividing total benefit payments during the benefit year (the supplemental data drawn by South Carolina) by the WBA (which is in the regular BAM data). This estimate of the number of weeks of benefits received could be called the "compressed duration" of benefit receipt:
Compressed duration = (Benefits paid during the BY) / WBA
A problem in using this measure of duration is that it assumes that claimants never receive partial benefits. In that sense, compressed duration could also be thought of as equivalent full weeks of benefit payments. The compressed duration measure is used below mainly because there is no clear alternative given the available data. However, from the standpoint of estimating the dollar cost of erroneous denials, the more policyrelevant outcome is total UI benefits received by a claimant during the benefit year. Accordingly, we rely mainly on benefits paid in making inferences, and include estimates of compressed duration for comparison.
5.2. Modeling Benefits Received and Benefit Duration
5.2.1. Models estimated. There is a large empirical literature modeling UI benefits receipt and its duration (for a review, see Woodbury and Anderson 1997). Early empirical work used ordinary least squares (OLS) and regressed either benefits received or the duration of benefit receipt (in weeks) on various explanatory variables. Following this work, we estimate models of the following form:
where Y is the outcome variable (either benefits paid by the end of the benefit year or weeks of benefit received), X1 through XK represent appropriate explanatory variables, and u is a disturbance term that is assumed to be normally distributed. The b coefficients on X1 through XK provide an estimate of the relationship between the explanatory variables and the dependent variable.
A problem in using OLS to estimate equation (1) is that OLS requires the assumption that the error term (u) is normal. However, when the dependent variable is UI benefits received or the duration of benefit receipt (as in this case), the underlying distribution of u is not normal. There are two reasons for this. First, because each worker is eligible for a specified Maximum Benefit Amount (MBA), the distribution of Y (and hence u) has a spike at the MBA (and at the maximum benefit duration). Second, the empirical frequency distribution of benefits paid to claimants (and of weeks of benefit receipt) is not bell-shaped, as the normality assumption requires. Rather, it shows one spike at zero weeks of unemployment, and falling frequencies for greater durations of benefit receipt, until the spike (mentioned above) appears at the MBA. Except for the spike at the maximum, the empirical distribution looks much like an inverse exponential.
This problem (failure of the disturbance term to satisfy the normality assumption) can be addressed in an equation like (1) by making a more appropriate assumption about the distribution of u, and estimating the equation under that alternative distributional assumption. The Weibull distribution has been widely assumed in studies of jobless duration because it provides a good approximation to the empirical distribution of unemployment duration (see, for example, Lancaster 1979). Estimating equation (1) under the assumption that u has the Weibull distribution requires maximum likelihood (ML) techniques.
Whether equation (1) is estimated by OLS or ML (the Weibull case), the benefits
that would have been received by an erroneously denied worker can be imputed by
substituting the observed characteristics of that worker into the estimated equation (1).
Letting b0 though bK denote the estimated coefficients of equation (1), the imputation
of the benefits that would have been received by a claimant with characteristics X1,
X2, ... XK would be:
where Y* denotes the imputed benefit amount. Because interest focuses on the
average benefits that would have been received by each of the three groups of
erroneously denied claimants, we successively substitute the average observed
characteristics of each of the three erroneously denied groups into the estimated
benefits (or weeks paid) equations (see section 5.3 below).
5.2.2. Results of estimation. Table 5-2 displays the results of estimating four versions of equation (1). The two left-hand columns show estimated models of total benefits received during the full benefit year. The first of these is estimated by OLS, the second by ML with the Weibull assumption. The two right-hand columns show estimated models of the weeks of benefits received during the full benefit year ("compressed weeks," as discussed above). Again, the first of these is estimated by OLS, the second by ML with the Weibull assumption.
The explanatory variables included in each of the models are as follows:
Although only models with the above explanatory variables are reported in Table 5-2, several models were estimated with alternative explanatory variables in order to check the robustness of the results and the imputations based on them. In particular, several different specifications of the usual wage and WBA variables were checked. Although the alternative models vary somewhat in their explanatory power, the implied imputations of benefits that would have been received by erroneously denied claimants are insensitive to the changes in specification that were tried.
The interpretation of the estimated coefficients displayed in Table 5-2 is as follows. In the OLS benefits-paid model, each coefficient gives the estimated change in total benefits paid during the full benefit year that is associated with a unit change in the given characteristic (holding fixed the other characteristics). For example, the coefficient on the male variable, -504.54, suggests that if a man and a woman of the same age, education, citizenship, ethnicity, schooling, usual wage, UI benefits, and season of filing were compared, the man could be expected to draw about $505 less in UI benefits over the course of the full benefit year.
Similarly, in the OLS weeks paid model, each coefficient gives the estimated change in weeks of benefits paid that is associated with a unit change in the given characteristic (holding fixed the other characteristics). The coefficient on the male variable, -2.5, suggests that, given other characteristics, an average male claimant can be expected to draw about 2.5 fewer weeks of UI benefits during the benefit year than an average female claimant.
Interpretation of the coefficients in the ML/Weibull models is somewhat different: Each coefficient gives the approximate proportional change in the dependent variable that is attributable to a unit change in the explanatory variable. [The exact proportional change in the dependent variable attributable to a unit change in an explanatory variable is exp(bk) - 1. The smaller is bk, the closer this expression comes to bk.] For example, in the benefits-paid model, the coefficient on the male variable, -0.278, suggests that, given other characteristics, an average male claimant can be expected to draw about 24 percent less in UI benefits than a woman claimant with the same observable characteristics other than gender [because exp(-0.278) - 1 = -0.24]. This implies a substantially larger impact of gender on benefits received than the corresponding OLS estimate implies: 24 percent of the average benefits received by eligible claimants is about $700, nearly 40 percent larger than the $505 estimate obtained from the model estimated by OLS. Such differences highlight the importance of the distributional assumption used in estimating the model.
Although the OLS and ML/Weibull results differ in the magnitude of estimated impacts, they are in basic agreement with regard to which variables are most important in determining benefits paid and duration of benefit receipt. In all four models displayed in Table 5-2, gender (the male-female variable) and the hourly wage in the usual job are highly significant, both statistically and in the sense that their coefficients are large. In addition, a higher WBA appears to increase benefits paid (see the benefits-paid equations), and greater potential duration has a positive impact on the expected duration of benefit receipt (see the OLS weeks-paid equation). However, no other variable in the models is statistically significant at conventional levels.
In sum, the main predictors of benefits paid and weeks paid in the full benefit year are gender and and hourly wage in the usual job. Men receive less in benefits and fewer weeks of benefits than women, other things equal. And claimants with a higher usual wage receive more benefits and more in weeks of benefits, other things equal.
5.3. Expected Benefits and Benefit Duration of Erroneously Denied Claimants
The reason for estimating the models described above is to obtain imputations of the benefits that erroneously denied claimants would have received if they had been correctly determined eligible. Table 5-3 displays such imputations based on the estimates in Table 5-2. The first row shows that claimants who were correctly determined to be eligible for UI received, on average, $2,893 during the benefit year. The second row shows that, at the time of their erroneous denial, the average claimant denied for monetary reasons had received no benefits, the average claimant denied for separation reasons had received about $640, and the average claimant denied for nonseparation reasons had received $850. [Some claimants who were erroneously denied for separation reasons had received benefits at the time of their erroneous denial because they were filing an additional initial claim within the same benefit year. Although they had met the conditions of separation on their new initial claim, and hence received benefits, the conditions of separation on their additional initial claim the claim that was subject to the DCA investigation were viewed as unsatisfactory (wrongly, according to the investigation).]
The third and fourth rows of Table 5-3 show imputations of the expected benefits paid in the benefit year to claimants with the characteristics of erroneously denied claimants. Imputations are shown for claimants who were correctly determined eligible as well as for claimants who were erroneously denied for monetary, separation, and nonseparation reasons. These imputations are derived by successively substituting the average characteristics of each group of claimants into the estimated benefits models displayed in Table 5-3. (Note that the expected benefits of eligible claimants, as imputed by the Weibull model, differ from the observed average benefits received by eligible claimants. This is due to the nonlinearity of the Weibull specification, which implies that the expected benefits of the average claimant in the sample need not equal the mean benefits received by claimants in the sample.)
For erroneous monetary denials, the imputations in the third and fourth rows may be interpreted directly as the benefits lost by the average claimant erroneously denied for monetary reasons. However, for separation denials, average benefits that had been received at the time of the erroneous denial ($639.40) must be subtracted from the imputations in the third and fourth rows in order to obtain an estimate of benefits lost due to erroneous separation denials. For nonseparation denials, the imputations in the third and fourth rows should not be used to infer benefits lost due to erroneous nonseparation denials. As discussed in section 4.3, many nonseparation denials do not disqualify a claimant from receiving benefits for the remaining duration of the unemployment spell. For nonseparation determinations, then, the imputations merely show that erroneously denied claimants are similar to claimants who receive correct determinations (that is, their expected benefits and expected weeks are essentially similar to those of correctly determined claimants).
The fifth row of Table 5-3 shows that claimants who were correctly determined to be eligible for UI received, on average, 16.1 weeks of benefit payments during the benefit year. The sixth row shows that, at the time of their erroneous denial, the average claimant denied for monetary reasons had received no weeks of benefits, the average claimant denied for separation reasons had received just over 4 weeks of benefits, and the average claimant denied for nonseparation reasons had received 5 weeks benefits.
The seventh and eighth rows of Table 5-3 show imputations of the expected weeks of benefits paid to erroneously denied claimants. These are derived by substituting the average characteristics of each group of claimants into the estimated weeks paid models displayed in Table 5-2. The above discussion of how to interpret the imputations for erroneous separation and nonseparation denials applies here as well.
The following subsections elaborate on the above points.
5.3.1. Erroneous monetary denials. The imputations based on OLS estimates suggest that the average claimant erroneously denied for monetary reasons would have received benefits of $2,290 (see Table 5-3). This implies that the average erroneous monetary denials resulted in a loss of about 79 percent of the benefits received by the average claimant correctly determined eligible (that is, $2,290/$2,893 see Table 5-4). Similarly, the imputations based on the ML/Weibull estimates suggest that the average claimant erroneously denied for monetary reasons would have received benefits of $2,117, or about 78 percent of the expected benefits received by the average claimant correctly determined eligible. (The expected benefits of eligible claimants as estimated by the Weibull model, $2,720, are used in this latter calculation.) Again, these imputations can be interpreted directly as the benefits lost by claimants who the DCA investigations found were erroneously denied for monetary reasons see the summary in Table 5-4.
Imputations of the expected duration of benefit receipt suggest smaller differences between claimants erroneously denied for monetary reasons and claimants correctly determined eligible. Specifically, the OLS imputations suggest that the average claimant erroneously denied for monetary reasons would have received 14.6 weeks of benefits, about 91 percent of the average eligible claimant. Similarly, the Weibull imputations suggest that claimants erroneously denied for monetary reasons would have received 13.7 weeks of benefits, about 91 percent of the average eligible claimant (as calculated from the Weibull model). These findings are again summarized in Table 5-4.
Whereas expected benefits received by claimants erroneously denied for monetary reasons are just under 80 percent of those received by eligible claimants, the expected weeks of benefits received by claimants erroneously denied for monetary reasons are slightly over 90 percent of those for eligible claimants (see the first two columns of Table 5-4). This difference reflects the finding that claimants erroneously denied for monetary reasons were eligible for lower weekly and maximum benefit amounts (WBA and MBA) than were claimants who were correctly determined eligible (see the figures on WBA and MBA in Table 5-1). In addition, it appears that the characteristics of claimants erroneously denied for monetary reasons tend to reduce the expected weeks of benefit receipt of these claimants. As discussed further below, two factors tend to reduce the benefit duration of claimants erroneously denied for monetary reasons: Compared with other eligible claimants, they are more likely to be male, and they have lower usual wages.
It follows that, in calculating benefits lost due to erroneous denials, account must be taken of both the lower WBA and the shorter expected unemployment duration of erroneously denied claimants compared with other eligible claimants. Benefits paid in the full benefit year, which are observed directly, implicitly take account of both factors. For that reason, it may be preferable to use benefits paid rather than weeks paid in imputing benefits lost due to erroneous denials.
5.3.2. Erroneous separation denials. Imputations based on the OLS estimates suggest that the average claimant erroneously denied for separation reasons would have received benefits of $2,238, about 77 percent of the benefits received by the average eligible claimant. Imputations based on the Weibull estimates suggest that the average claimant erroneously denied for separation reasons would have received benefits of $2,154, about 79 percent of the expected benefits received by the average eligible claimant. (As before, the expected benefits of the average eligible claimant as estimated by the Weibull model, $2,720, are used in this latter calculation.)
In order to estimate the benefits lost due to erroneous separation denials, it is necessary to subtract the benefits that had been received at the time of the erroneous denial ($639) from the above imputations. Doing so yields estimates of $1,599 ($2,238 $639) and $1,515 ($2,154 $639), which amount to 71 percent and 70 percent of the imputed benefits that would have been received by these claimants over the full benefit year. This in turn implies that the benefits lost by the average claimant erroneously denied for separation reasons were about 55 percent ($1,599/$2,893) to 56 percent ($1,515/$2,720) of the average benefits received by an eligible paid claimant. These findings are summarized in the middle columns of Table 5-4.
For duration of benefit receipt, the imputations suggest smaller differences between claimants erroneously denied for separation reasons and claimants correctly determined eligible. (This was also the case with erroneous monetary denials.) The OLS imputations suggest that the average claimant erroneously denied for separation reasons would have received 14.9 weeks of benefits, about 93 percent of the average eligible claimant. The Weibull imputations suggest that claimants erroneously denied for separation reasons would have received 14.4 weeks of benefits, about 95 percent of the average eligible claimant.
Subtracting the weeks of benefits that had been received at the time of the erroneous separation denial (4.15) from the above imputations yields estimates of the weeks of benefits lost due to erroneous denials. For the OLS imputation, the estimate is 10.8 weeks (14.9 4.15), and for the Weibull imputation, the estimate is 10.3 weeks (14.4 4.15). Hence, 71 to 72 percent the imputed weeks of benefits that would have been received by these claimants over the full benefit year were lost due to an erroneous separation denial. Accordingly, the weeks of benefits lost by the average claimant erroneously denied for separation reasons amounted to about 67 percent (10.8/16.1) to 68 percent (10.3/15.1) of the average weeks received by an eligible paid claimant (see again the middle columns of Table 5-4).
Because the available sample of erroneous separation denials is so small (10), there is reason to be cautious in using these findings. However, the findings on erroneous separation denials are similar to those on erroneous monetary denials. This occurs mainly because the characteristics of claimants erroneously denied for monetary and separation reasons are similar (refer to Table 5-1).
5.3.3. Erroneous nonseparation denials. The OLS imputations suggest that the average claimant erroneously denied for nonseparation reasons would have received benefits of $2,767, about 96 percent of the benefits received by the average eligible claimant, during the full benefit year (Table 5-3). The Weibull imputations suggest that the average claimant erroneously denied for nonseparation reasons would have received benefits of $2,620, about 96 percent of the expected benefits received by the average eligible claimant. (The expected benefits of the average eligible claimant as estimated in the Weibull model, $2,720, are again used in this latter calculation.)
Regarding duration of benefit receipt, the OLS imputations suggest that the average claimant erroneously denied for nonseparation reasons would have received 16.6 weeks of benefits, 3 percent more than the average eligible claimant. The Weibull imputations suggest that claimants erroneously denied for nonseparation reasons would have received 15.35 weeks of benefits, about 2 percent more than the average eligible claimant.
Penalties for nonseparation issues vary according to the disqualifying issue. As noted in section 4.3, able and available disqualifications are generally for the week in question and entail no reduction in the MBA. The other nonseparation issues, however, may involve penalties that cover several weeks or the remainder of the current spell of unemployment. Accordingly, using the imputations of expected benefits paid during the benefit year to claimants erroneously denied for nonseparation reasons to impute the benefits lost as a result of erroneous denials is problematic at best. (This differs from the imputations for erroneous monetary and separation denials, which can be used to impute lost benefits, as discussed above.) For able and available issues, the imputations should not be used at all because the able and available penalty is one week of benefits. Also, regarding the other nonseparation issues, it could be argued that an erroneous denial may change a claimant's behavior, perhaps causing him or her to claim less than otherwise. Unfortunately, data are not available on the benefits paid (during the full benefit year) to claimants who were subject to an erroneous nonseparation denial. If such data were available, the (actual) benefits received during the full benefit year could be compared with the (imputed) benefits that would have been received to see whether an erroneous nonseparation denial did have a behavioral impact.
5.3.4. Discussion. The results suggest that, compared with the typical eligible claimant, claimants erroneously denied for monetary or separation reasons have characteristics that imply less received in benefits and somewhat shorter durations of unemployment. Specifically, the imputations suggest that claimants erroneously denied for monetary or separation reasons would have received 77 to 80 percent of the benefits received by the average eligible claimant. This implies that the benefits lost by the average claimant who is erroneously denied for monetary reasons are 77 to 80 percent of the average benefits received by an eligible paid claimant. Because some claimants erroneously denied for separation reasons had already received some benefits, the benefits lost by the average claimant erroneously denied for separation reason are 70 to 71 percent of their expected benefits (that is, the benefits that they would have received), or 55 to 56 percent of the average benefits received by an eligible paid claimant. (These figures are summarized in Table 5-4.)
In contrast, the typical claimant who is erroneously denied for nonseparation reasons is quite similar to the typical eligible claimant, and could be expected to receive a similar amount of UI benefits during a benefit year if he or she did not encounter an erroneous nonseparation determination. Specifically, the imputations suggest that the expected benefits of claimants erroneously denied for nonseparation reasons are 96 percent of the benefits of eligible claimants, and that their expected unemployment duration is 2 to 3 percent longer than that of eligible claimants. However, the average penalty for a nonseparation violation is less severe than for failure to meet the monetary and separation eligibility conditions (that is, benefits are often denied for one week rather than for the remaining duration of the unemployment spell). Because the cost to a claimant of an erroneous nonseparation denial is not the remaining benefit entitlement, the imputations cannot be used to infer directly the loss of benefits due to erroneous nonseparation denials.
Which characteristics of claimants erroneously denied for monetary and separation reasons make them prone to receive less in UI benefits than the typical eligible claimant? That is, if they had been determined eligible (as they should have been), why would they have received less UI benefits and experienced somewhat shorter unemployment durations, on average? The answer can be seen by referring again to Table 5-2. Recall that the main correlates of benefits received and weeks paid are gender (men tend to receive less in UI benefits over the benefit year and to have shorter durations) and the usual hourly wage (claimants with higher usual wage tend to receive more in UI benefits and to have longer durations). Table 5-1 shows that, compared with the typical correctly determined eligible claimant, the typical claimant who was erroneously denied for monetary or separation reasons was more likely to be a man and had a lower usual hourly wage. Both of these circumstances imply lower UI benefit receipts and shorter durations of unemployment.
In sum, claimants erroneously denied for monetary and separation reasons would have received less in UI benefits during the full benefit year (and would have received fewer weeks of benefits) than the average eligible claimant because more of them were men and because they had lower usual hourly wages.
5.4. Implications for Benefits Lost Due to Erroneous Denials
The implications of these findings are developed in Table 5-5, which repeats panels A and B from Table 4-5 and adds panels C and D, which show two alternative imputations of lost benefits based on the regression analysis. The approach taken in panels C and D follows closely that developed in section 4.4.
Panels C and D use the same figures for the number of erroneous denials and average WBA for the United States that they were in panels A and B. However, panel C uses Table 5-4's Weibull estimates of benefits lost (as a percentage of a correctly determined claimant's benefits) to adjust the benefit losses of claimants erroneously denied for monetary and separation reasons. Recall that these benefit loss estimates take account of both the lower average WBA and the shorter duration of benefits received by claimants erroneously denied for monetary and separation reasons. Accordingly, for both monetary and separation denials, the duration of the average denial is shown as 14.2 weeks in panel C because the adjustment factors used (0.778 and 0.557) adjust for both the lower WBA and shorter durations of claimants erroneously denied for monetary and separation reasons. [Note that the relative WBA factor for erroneous nonseparation denials in panel C is 0.948. This is calculated from Table 5-3 as the expected WBA of claimants who received an erroneous nonseparation denial ($2,620/15.35 = $170.70) divided by the WBA of the average correctly determined claimant ($2,720/15.1 = $180.15). This relative WBA factor is slightly higher than that used in panels A, B, and D. The denial duration for nonseparation denials in panel C is 2.9 weeks, as in panels B and D.]
Panel D uses Table 5-4's estimates of weeks of benefits lost (as a percentage of a correctly determined claimant's benefits) to adjust the benefit losses of claimants erroneously denied for monetary and separation reasons. The panel D estimates of the relative WBA factor revert to those used in panels A and B. However, the denial duration is based on the Weibull estimates. Accordingly, the denial duration of claimants erroneously denied for monetary reasons is estimated to be 12.9 weeks (14.2 0.908), and the denial duration of claimants erroneously denied for separation reasons is estimated to be 9.6 weeks (14.2 0.679).
The two sets of regression-base estimates yield virtually identical results. Both suggest that the total lost benefits due to erroneous denials amount to about $565 million, or about 3.1 percent of total regular UI benefit payments. Although slightly less than the estimates in panels A and B, these estimates are very similar to the earlier estimates and suggest that a reasonable estimate of the benefits lost due to erroneous denials is slightly above 3 percent of total regular UI benefit payments.
Sections 5.1 and 5.2 of this chapter outlined and implemented a regression strategy for estimating the benefits that erroneously denied UI claimants would have received had they been correctly determined eligible for benefits. As discussed in chapter 5 of the Denied Claims Accuracy Pilot Project Final Report and in chapter 4 above, there are various difficulties in making such imputations. In this chapter, the main practical difficulty lack of data on the full benefit-year experience of a sample of eligible claimants was overcome with the cooperation of the South Carolina Quality Control Division. The South Carolina group provided data on the full benefityear payments made to a sample of claimants who were randomly selected for investigation by the Benefit Accuracy Measurement program concurrent with the DCA Pilot Project. The supplemental data from South Carolina allow estimation of models of benefits received and the duration of benefit receipt that can serve as the basis for imputing the dollar amount of benefits that would have been received by erroneously denied claimants if they had not been denied.
Imputations based on the estimated models (see Tables 5-3 and 5-4) suggest that the benefits lost by erroneously denied claimants (as a percentage of the benefits received by a typical correctly determined claimant) amount to just under 80 percent for erroneous monetary denials and about 55 percent for erroneous separation denials. Also, the weeks of benefits lost as a percentage of the weeks of benefits received by a typical correctly determined claimant amount to 91 percent for erroneous monetary denials and 67 to 68 percent for erroneous separation denials. These findings suggest that using the average benefits (or weeks of benefits) received by eligible claimants to estimate the dollar cost of erroneous monetary and separation denials would result in upward-biased estimates of benefits lost due to erroneous monetary and separation denials.
Section 5.3 explored the implications of these estimates for the benefits lost by erroneously denied claimants. Two approaches were taken, and both suggest that the total lost benefits due to erroneous denials amount to about $565 million. This amounts to about 3.1 percent of total regular UI benefit payments, and is only slightly less than the estimates of lost benefits developed in chapter 4 (see Table 5-5).
Battelle Memorial Institute. "Dynamic Models of Unemployment Insurance Benefit Receipt: Survival Rate Analysis Report." Unemployment Insurance Occasional Paper 99-1. Washington, DC: U.S. Department of Labor, Employment and Training Administration, Unemployment Insurance Service, 1999.
Belle, Richard and Joseph Casey. "Denials Quality Control Pilot." Final Report Prepared for the Unemployment Insurance Service, Employment and Training Administration, U.S. Department of Labor. Silver Spring, MD: Applied Management Sciences, Inc., November 1988.
Kleinbaum, David G., Lawrence L. Kupper, and Keith E. Muller. Applied Regression Analysis and Other Multivariate Methods, second edition. Boston: PWS-Kent, 1988.
Lancaster, Tony. "Econometric Methods for the Duration of Unemployment." Econometrica 47 (July 1979): 939956.
Skrable, Burman. "Memorandum: The Denied Claim Accuracy Pilot Recommendations for Implementing National Measurement of Denied Claim Accuracy." Division of Performance Review, Unemployment Insurance Service, Employment and Training Administration, U.S. Department of Labor, April 21, 1999.
U.S. Department of Labor. Comparison of State Unemployment Insurance Laws. Washington, DC: U.S. Department of Labor, Employment and Training Administration, January 7, 1996, and subsequent updates.
U.S. Department of Labor. Unemployment Insurance Financial Data, ET Handbook 394. Washington, DC: U.S. Department of Labor, Employment and Training Administration, January 1999.
Woodbury, Stephen A. and Wayne Vroman. "Denied Claims Accuracy Pilot Project." Final Report Prepared for the Division of Performance Review, Unemployment Insurance Service, Employment and Training Administration, U.S. Department of Labor. Herndon, VA: PRAMM Consulting Group, Inc., May 1999.
Woodbury, Stephen A. and Rod W. Anderson. "A Two-State Evaluation of the Emergency Unemployment Compensation Program." Final Report Prepared for the Unemployment Insurance Service, Employment and Training Administration, U.S. Department of Labor, under Contract No. M-5114-5-00-97- 40. Kalamazoo, MI: W.E. Upjohn Institute, June 1997.
| State | ||||||
|---|---|---|---|---|---|---|
| Nebraska | New Jersey | South Carolina | West Virginia | Wisconsin | Five Pilot States |
|
| Erroneous Monetary Denials | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 64.7 | 8.7 | 52.3 | 55.0 | 21.6 | 39.0 |
| Was already resolving issue | 5.9 | 34.8 | 31.8 | 15.0 | 48.7 | 31.2 |
| Took incorrect action | 0.0 | 39.1 | 0.0 | 0.0 | 10.8 | 9.2 |
| Did not identify issue | 23.5 | 4.4 | 2.3 | 25.0 | 8.1 | 9.2 |
| Did not follow procedures | 5.9 | 13.0 | 13.6 | 5.0 | 10.8 | 10.6 |
| Number of cases | 17 | 23 | 44 | 20 | 37 | 141 |
| Erroneous Separation Denials | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 0.0 | 22.7 | 20.0 | 33.3 | 20.0 | 19.8 |
| Was already resolving issue | 0.0 | 13.6 | 20.0 | 0.0 | 5.0 | 8.1 |
| Took incorrect action | 37.5 | 59.2 | 20.0 | 66.7 | 65.0 | 55.8 |
| Did not identify issue | 50.0 | 4.6 | 30.0 | 0.0 | 0.0 | 9.3 |
| Did not follow procedures | 12.5 | 0.0 | 10.0 | 0.0 | 10.0 | 7.0 |
| Number of cases | 8 | 22 | 10 | 6 | 40 | 86 |
| Erroneous Nonseparation Denials | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 19.2 | 21.4 | 27.0 | 0.0 | 27.3 | 22.2 |
| Was already resolving issue | 3.9 | 3.6 | 5.4 | 7.1 | 18.2 | 8.7 |
| Took incorrect action | 65.4 | 60.7 | 35.1 | 85.7 | 38.6 | 51.0 |
| Did not identify issue | 3.9 | 10.7 | 8.1 | 7.1 | 2.3 | 6.0 |
| Did not follow procedures | 7.7 | 3.6 | 24.3 | 0.0 | 13.6 | 12.1 |
| Number of cases | 26 | 28 | 37 | 14 | 44 | 149 |
Source: Tabulated from Denied Claims Accuracy Pilot records (prior agency action code in error issue table).
Table 1-2
Prior agency action on erroneous denials, by type of denial and
whether erroneous denials passed QPI
(column percentages with number of cases)
| Failed QPI | Passed QPI | Total | |
|---|---|---|---|
| Erroneous Separation Denials | |||
| Investigation determined agency: | |||
| Could not detect issue | 11.1 | 34.6 | 18.8 |
| Was already resolving issue | 9.3 | 3.9 | 7.5 |
| Took incorrect action | 61.1 | 46.2 | 56.3 |
| Did not identify issue | 9.3 | 11.5 | 10.0 |
| Did not follow procedures | 9.3 | 3.9 | 7.5 |
| Number of cases | 54 | 26 | 80 |
| Erroneous Nonseparation Denials | |||
| Investigation determined agency: | |||
| Could not detect issue | 14.1 | 32.3 | 22.4 |
| Was already resolving issue | 5.1 | 12.3 | 8.4 |
| Took incorrect action | 61.5 | 36.9 | 50.4 |
| Did not identify issue | 7.7 | 4.6 | 6.3 |
| Did not follow procedures | 11.5 | 13.9 | 12.6 |
| Number of cases | 78 | 65 | 143 |
Source: Tabulated from Denied Claims Accuracy Pilot records (prior agency action code in error issue table).
Table 1-3
Error detection point on erroneous denials, by type of denial and whether erroneous denial passed QPI
(column percentages with frequencies in parentheses)
| Error Detection Point | |||||
|---|---|---|---|---|---|
| Verification of wages/ separation |
Claimant interview |
Third party | UI records | Row percentage | |
| Erroneous Separation Denials | |||||
| Failed QPI | 73.9 (17/23) |
57.9 (11/19) |
0.0 (0/2) |
72.2 (26/36) |
67.5 (54/80) |
| Passed QPI | 26.1 (6/23) |
42.1 (8/19) |
100.0 (2/2) |
27.8 (10/36) |
32.5 (26/80) |
| Column percentage | 28.8 (23/80) |
23.8 (19/80) |
2.5 (2/80) |
45.0 (36/80) |
100.0 (80/80) |
| Erroneous Nonseparation Denials | |||||
| Failed QPI | 55.6 (5/9) |
50.0 (16/32) |
41.7 (5/12) |
58.4 (52/89) |
54.9 (78/142) |
| Passed QPI | 44.4 (5/9) |
50.0 (16/32) |
58.3 (5/12) |
41.6 (52/89) |
45.1 (64/142) |
| Column percentage | 6.3 (9/142) |
22.5 (32/142) |
8.5 (12/142) |
62.7 (89/142) |
100.0 (142/142) |
Source: Tabulated from Denied Claims Accuracy Pilot records (prior agency action and error detection point codes in error issue table).
Table 1-4
Prior agency action on erroneous denials by detection point, erroneous separation and nonseparation denials
(column percentages with frequencies in parentheses)
| Error Detection Point | |||||
|---|---|---|---|---|---|
| Investigation determined agency: | Verification of wages/separation | Claimant interview | Third party | UI records | Row percentage |
| Erroneous Separation Denials | |||||
| Could not detect issue | 29.2 (7/24) |
28.6 (6/21) |
50.0 (1/2) |
7.7 (3/39) |
19.8 (17/86) |
| Was already resolving issue | 0.0 (0/24) |
14.3 (3/21) |
0.0 (0/2) |
10.3 (4/39) |
8.1 (7/86) |
| Took incorrect action | 58.3 (14/24) |
38.1 (8/21) |
0.0 (0/2) |
66.7 (26/39) |
55.8 (48/86) |
| Did not identify issue | 8.3 (2/24) |
0.0 (0/21) |
0.0 (0/2) |
15.4 (6/39) |
9.3 (8/86) |
| Did not follow procedures | 4.2 (1/24) |
19.1 (4/21) |
50.0 (1/2) |
0.0 (0/39) |
7.0 (6/86) |
| Column percentage | 27.9 (24/86) |
24.4 (21/86) |
2.3 (2/86) |
45.3 (39/86) |
100.0 (86/86) |
| Erroneous Nonseparation Denials | |||||
| Could not detect issue | 55.6 (5/9) |
46.9 (15/32) |
50.0 (6/12) |
6.3 (6/95) |
22.2 (33/149) |
| Was already resolving issue | 11.1 (1/9) |
0.0 (0/32) |
0.0 (0/12) |
12.6 (12/95) |
8.7 (13/149) |
| Took incorrect action | 33.3 (3/9) |
28.1 (9/32) |
50.0 (6/12) |
61.1 (58/95) |
51.0 (76/149) |
| Did not identify issue | 0.0 (0/9) |
3.1 (1/32) |
0.0 (0/12) |
8.4 (8/95) |
6.0 (9/149) |
| Did not follow procedures | 0.0 (0/9) |
21.9 (7/32) |
0.0 (0/12) |
11.6 (11/95) |
12.1 (18/149) |
| Column percentage | 6.0 (9/149) |
21.5 (32/149) |
8.1 (12/149) |
63.8 (95/149) |
100.0 (149/149) |
Source: Tabulated from Denied Claims Accuracy Pilot records (prior agency action and error detection point codes in error issue table).
Table 1-5
Percentages of erroneous denials that failed each element of the Quality Performance
Indicator review, by type of denial
(percentages with number of cases)
| QPI Element | Separation Denials |
Nonseparation Denials |
|---|---|---|
| Claimant information inadequate or missing | 34.6 | 29.4 |
| Employer information inadequate or missing | 44.4 | 11.2 |
| Other information inadequate or missing | 13.6 | 9.1 |
| Rebuttal opportunity not offered | 12.4 | 6.3 |
| Determination does not meet provisions of law and policy | 66.7 | 54.5 |
| Written determination inadequate or wrong | 34.6 | 39.9 |
| Total erroneous denials | 81 | 143 |
| Number that failed QPI | 54 | 78 |
| Percentage that failed QPI | 66.7% | 54.5% |
Source: Tabulated from Denied Claims Accuracy Pilot records (QPI table and error issue table).
| QPI Element | Separation Denials |
Nonseparation Denials |
|---|---|---|
| Claimant information inadequate or missing | 26.7 | 21.9 |
| Employer information inadequate or missing | 40.0 | 6.3 |
| Other information inadequate or missing | 0.0 | 18.8 |
| Rebuttal opportunity not offered | 6.7 | 3.1 |
| Determination does not meet provisions of law and policy | 40.0 | 34.4 |
| Written determination inadequate or wrong | 13.3 | 25.0 |
| Total "undetectable" erroneous denials | 15 | 32 |
| Number that failed QPI | 6 | 11 |
| Percentage that failed QPI | 40.0% | 34.4% |
Source: Tabulated from Denied Claims Accuracy Pilot records (QPI table and error issue table).
| A. Regular QPI: Pass/fail threshold of 80 (score > 80 required to pass) | |||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 54 (67) | 27 (33) | 81 (100) |
| Proper denial | 218 (27) | 603 (73) | 821 (100) |
| Total | 272 (30) | 630 (70) | 902 (100) |
| B. Regular QPI: Pass/fail threshold of 65 (score > 65 required to pass) | |||||
|---|---|---|---|---|---|
| QPI score | |||||
| DCA finding | fail | pass | Total | ||
| Improper denial | 30 (37) | 51 (63) | 81 (100) | ||
| Proper denial | 100 (12) | 721 (88) | 821 (100) | ||
| Total | 130 (14) | 772 (86) | 902 (100) | ||
| C. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 80 (score greater than 80 required to pass) |
|||||
|---|---|---|---|---|---|
| QPI score | |||||
| DCA finding | fail | pass | Total | ||
| Improper denial | 54 (67) | 27 (33) | 81 (100) | ||
| Proper denial | 217 (26) | 604 (74) | 821 (100) | ||
| Total | 271 (30) | 631 (70) | 902 (100) | ||
| D. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 65 (score > 65 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 32 (40) | 49 (60) | 81 (100) |
| Proper denial | 81 (10) | 740 (90) | 821 (100) |
| Total | 113 (13) | 789 (87) | 902 (100) |
Source: Tabulated from Denied Claims Accuracy Pilot records.
| A. Regular QPI: Pass/fail threshold of 80 (score > 80 required to pass) | |||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 78 (55) | 65 (45) | 143 (100) |
| Proper denial | 145 (19) | 607 (81) | 752 (100) |
| Total | 223 (25) | 672 (75) | 895 (100) |
| B. Regular QPI: Pass/fail threshold of 70 (score > 70 required to pass) | |||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 59 (41) | 84 (59) | 143 (100) |
| Proper denial | 92 (12) | 660 (88) | 752 (100) |
| Total | 151 (17) | 744 (83) | 895 (100) |
| C. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 80 (score > 80 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 78 (55) | 65 (45) | 143 (100) |
| Proper denial | 145 (19) | 607 (81) | 752 (100) |
| Total | 223 (25) | 672 (75) | 895 (100) |
| D. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 70 (score > 70 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 55 (38) | 88 (62) | 143 (100) |
| Proper denial | 73 (10) | 679 (90) | 752 (100) |
| Total | 128 (14) | 767 (86) | 895 (100) |
Source: Tabulated from Denied Claims Accuracy Pilot records.
| Type of Denial | ||
|---|---|---|
| Component of QPI | Separation | Nonseparation |
| Claimant information adequate | -0.055** (0.017) |
-0.089** (0.029) |
| Employer information adequate | 0.011 (0.029) |
-0.022 (0.045) |
| Other information adequate | 0.186** (0.087) |
0.079 (0.073) |
| Rebuttal opportunity provided | -0.040* (0.018) |
-0.005 (0.060) |
| Provisions of law and policy met | 0.196** (0.054) |
0.288** (0.063) |
| Written determination adequate | 0.039 (0.027) |
0.103** (0.040) |
| Number of observations | 902 | 895 |
| Pseudo R-squared | 0.137 | 0.113 |
* denotes significance at 5% level; ** denotes significance at 1% level.
Note: Coefficients shown indicate the discrete change in the probability of a denial being correct when the QPI component indicated is coded as satisfactory. That is, the interpretation of the coefficients is the same as in a linear probability model (estimated by ordinary least squares).
| A. Discriminant function scores based on all QPI components: Pass/fail threshold of 0.75 (score > .75 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 31 (38) | 50 (62) | 81 (100) |
| Proper denial | 51 (6) | 770 (94) | 821 (100) |
| Total | 82 (9) | 820 (91) | 902 (100) |
| B. Regular QPI: Pass/fail threshold of 65 (score > 65 required to pass) | |||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 30 (37) | 51 (63) | 81 (100) |
| Proper denial | 100 (12) | 721 (88) | 821 (100) |
| Total | 130 (14) | 772 (86) | 902 (100) |
| C. Discriminant function scores based on adequacy of other information, whether determination met provisions of state law and policy, and adequacy of written determination: Pass/fail threshold of 0.71 (score > .80 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 31 (38) | 50 (62) | 81 (100) |
| Proper denial | 79 (10) | 742 (90) | 821 (100) |
| Total | 110 (12) | 792 (88) | 902 (100) |
| D. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 65 (score > 65 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 32 (40) | 49 (60) | 81 (100) |
| Proper denial | 81 (10) | 740 (90) | 821 (100) |
| Total | 113 (13) | 789 (87) | 902 (100) |
Source: Computations from Denied Claims Accuracy Pilot records. See text for explanation.
| A. Discriminant function scores based on all QPI components: Pass/fail threshold of 0.71 (score > .71 required to pass) |
|||
| QPI score | |||
|---|---|---|---|
| DCA finding | fail | pass | Total |
| Improper denial | 58 (41) | 84 (59) | 142 (100) |
| Proper denial | 86 (11) | 665 (89) | 751 (100) |
| Total | 144 (16) | 749 (84) | 893 (100) |
| B. Regular QPI: Pass/fail threshold of 70 (score > 70 required to pass) | |||
| QPI score | |||
|---|---|---|---|
| DCA finding | fail | pass | Total |
| Improper denial | 59 (41) | 84 (59) | 143 (100) |
| Proper denial | 92 (12) | 660 (88) | 752 (100) |
| Total | 151 (17) | 744 (83) | 895 (100) |
| C. Discriminant function scores based on adequacy of other information, whether determination met provisions of state law and policy, and adequacy of written determination: Pass/fail threshold of 0.71 (score > .71 required to pass) |
|||
|---|---|---|---|
| QPI score | |||
| DCA finding | fail | pass | Total |
| Improper denial | 53 (37) | 89 (63) | 142 (100) |
| Proper denial | 71 (9) | 680 (91) | 751 (100) |
| Total | 124 (14) | 769 (86) | 893 (100) |
| D. QPI modified to omit claimant information, employer information, and provision of rebuttal opportunity; Pass/fail threshold of 70 (score > 70 required to pass) |
|||||
| QPI score | |||||
|---|---|---|---|---|---|
| DCA finding | fail | pass | Total | ||
| Improper denial | 55 (38) | 88 (62) | 143 (100) | ||
| Proper denial | 73 (10) | 679 (90) | 752 (100) | ||
| Total | 128 (14) | 767 (86) | 895 (100) | ||
Source: Computations from Denied Claims Accuracy Pilot records. See text for explanation.
Table 3-1
Error rates on denied claims (DCA Pilot) and paid claims (BAM) in DCA pilot states, September 1997 through August 1998
(error percentages with frequencies in parentheses)
| State | ||||||
|---|---|---|---|---|---|---|
| Nebraska | New Jersey | South Carolina | West Virginia | Wisconsin | Five Pilot States | |
| Erroneous Denials | ||||||
| Monetary | 10.1 (20/198) |
12.6 (23/182) |
23.4 (45/192) |
15.1 (19/126) |
18.2 (37/203) |
16.0 (144/901) |
| Separation | 4.0 (8/200) |
11.3 (22/195) |
5.0 (10/200) |
3.4 (7/208) |
19.7 (40/203) |
8.7 (871/1006) |
| Nonseparation | 14.0 (28/200) |
14.4 (28/195) |
18.5 (37/200) |
6.8 (14/206) |
21.7 (44/203) |
15.0 (151/1004) |
| Total Overpayments | ||||||
| Monetary | 0.3 (1/359) |
1.5 (7/464) |
0.2 (1/473) |
0.6 (3/465) |
0.4 (2/482) |
0.6 (14/2243) |
| Separation | 0.3 (1/359) |
1.5 (7/464) |
2.1 (10/473) |
1.7 (8/465) |
0.4 (2/482) |
1.2 (28/2243) |
| Nonseparation | 4.7 (17/359) |
2.8 (13/464) |
14.2 (67/473) |
6.5 (30/465) |
1.5 (7/482) |
5.6 (134/2243) |
| Total and Partial Overpayments | ||||||
| Monetary | 5.6 (20/359) |
25.2 (117/464) |
0.6 (3/473) |
4.1 (19/465) |
1.0 (5/482) |
6.9 (154/2243) |
| Separation | 0.6 (2/359) |
1.5 (7/464) |
2.1 (10/473) |
1.7 (8/465) |
0.4 (2/482) |
1.3 (29/2243) |
| Nonseparation | 8.9 (32/359) |
6.0 (28/464) |
17.1 (81/473) |
11.8 (55/465) |
3.1 (15/482) |
9.4 (211/2243) |
Source: Tabulated from Denied Claims Accuracy Pilot records and Benefit Accuracy Measurement Records for September 1997 through September 1998. Missing cases excluded.
| State | ||||||
|---|---|---|---|---|---|---|
| Nebraska | New Jersey |
South Carolina |
West Virginia |
Wisconsin | Five Pilot States |
|
| Total Overpayments of Monetary Determinations | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 0.0 | 85.7 | 100.0 | 100.0 | 50.0 | 78.5 |
| Was already resolving issue | 100.0 | 14.3 | 0.0 | 0.0 | 0.0 | 7.1 |
| Took incorrect action | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Did not identify issue | 0.0 | 0.0 | 0.0 | 0.0 | 50.0 | 14.3 |
| Did not follow procedures | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Number of cases | 1 | 7 | 1 | 3 | 2 | 14 |
| Total Overpayments of Separation Determinations | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 0.0 | 71.4 | 40.0 | 87.5 | 50.0 | 60.7 |
| Was already resolving issue | 0.0 | 0.0 | 10.0 | 0.0 | 0.0 | 3.4 |
| Took incorrect action | 100.0 | 14.3 | 30.0 | 0.0 | 50.0 | 21.4 |
| Did not identify issue | 0.0 | 14.3 | 0.0 | 0.0 | 0.0 | 3.6 |
| Did not follow procedures | 0.0 | 0.0 | 20.0 | 12.5 | 0.0 | 10.7 |
| Number of cases | 1 | 7 | 10 | 8 | 2 | 28 |
| Total Overpayments of Nonseparation Determinations | ||||||
| Investigation determined agency: | ||||||
| Could not detect issue | 70.6 | 76.9 | 94.0 | 33.3 | 85.7 | 75.3 |
| Was already resolving issue | 17.7 | 0.0 | 0.0 | 13.3 | 14.3 | 3.7 |
| Took incorrect action | 0.0 | 0.0 | 3.0 | 10.0 | 0.0 | 3.7 |
| Did not identify issue | 0.0 | 23.1 | 3.0 | 6.7 | 0.0 | 7.5 |
| Did not follow procedures | 11.8 | 0.0 | 0.0 | 36.7 | 0.0 | 9.7 |
| Number of cases | 17 | 13 | 67 | 30 | 7 | 134 |
Source: Tabulated from Benefit Accuracy Measurement Records for September 1997 through September 1998. Missing cases excluded.
Table 3-3| Erroneous Denials | Total Overpayments | |
|---|---|---|
| Monetary Determinations | ||
| Investigation determined agency: | ||
| Could not detect issue | 39.0 | 78.5 |
| Was already resolving issue | 31.2 | 7.1 |
| Took incorrect action | 9.2 | 0.0 |
| Did not identify issue | 9.2 | 14.3 |
| Did not follow procedures | 10.6 | 0.0 |
| Number of cases | 141 | 14 |
| Separation Determinations | ||
| Investigation determined agency: | ||
| Could not detect issue | 19.8 | 60.7 |
| Was already resolving issue | 8.1 | 3.4 |
| Took incorrect action | 55.8 | 21.4 |
| Did not identify issue | 9.3 | 3.6 |
| Did not follow procedures | 7.0 | 10.7 |
| Number of cases | 86 | 28 |
| Nonseparation Determinations | ||
| Investigation determined agency: | ||
| Could not detect issue | 22.2 | 75.3 |
| Was already resolving issue | 8.7 | 3.7 |
| Took incorrect action | 51.0 | 3.7 |
| Did not identify issue | 6.0 | 7.5 |
| Did not follow procedures | 12.1 | 9.7 |
| Number of cases | 149 | 134 |
Source: Tabulated from prior agency action codes in Denied Claims Accuracy Pilot records and Benefit Accuracy Measurement Records for September 1997 through August 1998. Missing cases excluded.
| Independent variable | Monetary denial |
Separation denial |
Nonseparation denial |
Total Overpayment |
|---|---|---|---|---|
| Age | 0.000 (0.000) |
-0.001* (0.000) |
0.000 (0.000) |
0.001 (0.000) |
| Male | 0.026* (0.011) |
0.001 (0.009) |
-0.014 (0.012) |
0.027** (0.010) |
| U.S. citizen | 0.043 (0.032) |
0.036 (0.027) |
-0.012 (0.033) |
0.014 (0.028) |
| Ethnicity: | ||||
| Black | 0.020 (0.015) |
0.019 (0.013) |
0.036* (0.016) |
0.027* (0.012) |
| Hispanic | 0.063* (0.026) |
0.025 (0.022) |
0.018 (0.027) |
0.030 (0.023) |
| American Indian | -0.048 (0.066) |
0.006 (0.054) |
-0.010 (0.070) |
0.161** (0.055) |
| Asian/ Pacific Islander | 0.041 (0.049) |
-0.042 (0.042) |
0.004 (0.051) |
-0.017 (0.045) |
| Caucasian | ref | ref | ref | ref |
| Schooling: | ||||
| less than high school | -0.002 (0.013) |
0.007 (0.011) |
-0.007 (0.014) |
-0.006 (0.012) |
| high school graduate | ref | ref | ref | ref |
| some college | -0.021 (0.014) |
-0.003 (0.012) |
-0.016 (0.015) |
0.011 (0.014) |
| college degree | 0.009 (0.017) |
0.007 (0.014) |
0.010 (0.017) |
0.010 (0.017) |
| UI benefits: | ||||
| WBA/maximum WBA | -0.068** (0.023) |
-0.042* (0.019) |
-0.051* (0.024) |
-0.015 (0.022) |
| potential duration | -0.017** (0.001) |
-0.006** (0.001) |
-0.006** (0.002) |
0.004** (0.001) |
| Spring: | ||||
| winter | -0.003 (0.014) |
-0.009 (0.012) |
-0.005 (0.015) |
-0.001 (0.013) |
| spring | -0.012 (0.014) |
-0.014 (0.012) |
-0.017 (0.015) |
0.011 (0.013) |
| summer | 0.004 (0.015) |
-0.013 (0.012) |
-0.021 (0.015) |
0.000 (0.014) |
| fall | ref | ref | ref | ref |
| State: | ||||
| Nebraska | -0.033 (0.017) |
-0.061** (0.014) |
-0.024 (0.018) |
-0.030 (0.016) |
| New Jersey | -0.045** (0.016) |
-0.042** (0.013) |
-0.047** (0.017) |
-0.043** (0.016) |
| South Carolina | 0.018 (0.017) |
-0.066** (0.014) |
-0.019 (0.017) |
0.039* (0.016) |
| West Virginia | -0.021 (0.016) |
-0.064** (0.013) |
-0.056** (0.017) |
-0.081** (0.016) |
| Wisconsin | ref | ref | ref | ref |
| Constant | 0.494** (0.050) |
0.263** (0.044) |
0.316** (0.055) |
-0.035 (0.036) |
| Observations | 2202 | 2147 | 2209 | 2675 |
| R-squared | 0.10 | 0.04 | 0.03 | 0.06 |
Notes: Standard errors in parentheses.
* denotes significance at 5% level; ** denotes significance at 1% level.
"ref" denotes reference category in a set of dummy variables.
"na" denotes variable not appropriate for inclusion in the equation.
Table 4-1
Denial error rates, unadjusted and adjusted for "self-corrections," by pilot project and
issue
| Unadjusted error rates |
Adjusted error rates |
Percentage reduction due to self-correction |
|
|---|---|---|---|
| 1980s Pilot | |||
| Monetary | 0.230 | 0.156 | 32 |
| Separation | 0.152 | 0.093 | 39 |
| Nonseparation | 0.141 | 0.113 | 20 |
| 1990s Pilot | |||
| Monetary | 0.160 | 0.112 | 30 |
| Separation | 0.087 | 0.064 | 27 |
| Nonseparation | 0.150 | 0.129 | 14 |
Sources: Estimates for the 1980s pilot taken from Table 4 of Belle and Casey (1988). Estimates for the 1990s pilot taken from Tables 4-10A, 4-10B, and 4-10C or Woodbury and Vroman (1999).
Table 4-2
Framework for viewing the duration of erroneous denials
| Type of claim | ||||
|---|---|---|---|---|
| Type of determination | New initial claims | Additional initial claims | Continued claims | |
| Monetary | ||||
| Universe subject to determinations | All claims (lack of work, quits, and discharges) | NA-a | NA-a | |
| Administrative decision | Sufficient or insufficient wage credits | NA-a | NA-a | |
| Penalty | Duration of spell | NA-a | NA-a | |
| Separation | ||||
| Universe subject to determinations | All quits and discharges | All quits and discharges | NA-b | |
| Administrative decision | Award or denial | Award or denial | NA-b | |
| Penalty | Usually, duration of spell (full entitlement) | Usually, duration of spell, (remaining entitlement) | NA-b | |
| Nonseparation | ||||
| Universe subject to determinations | NA-c | NA-c | All continued weeks claimed | |
| Administrative decision | NA-c | NA-c | Payment, deferral, or reduction of entitlement | |
| Penalty | NA-c | NA-c | Single week or reduced entitlement | |
Notes:
NA-a denotes not applicable because monetary determinations are made only for new initial claims.
NA-b denotes not applicable because separation determinations are made only for new and additional initial claims.
NA-c denotes not applicable because nonseparation determinations are made only for continued claims.
| State | Voluntary quit | Misconduct discharge |
|---|---|---|
| Nebraska | Benefits delayed and MBA reduced by 7-10 weeks | Benefits delayed and MBA reduced by 7-10 weeks |
| New Jersey | Disqualified from receiving benefits for duration of unemployment (4 weeks and 6 times WBA) | Benefits delayed for claim week + 5 weeks |
| South Carolina | Disqualified from receiving benefits for duration of unemployment (8 times WBA) | Benefits delayed and MBA reduced by 5-26 weeks |
| West Virginia | Disqualified from receiving benefits for duration of unemployment (30 days of work) | Benefits delayed and MBA reduced by 6 weeks |
| Wisconsin | Disqualified from receiving benefits for duration of unemployment (4 weeks and 4 times WBA) | Disqualified from receiving benefits for duration of unemployment (7 weeks and 14 times WBA) |
Notes: Where a requalification requirement is shown, a claimant must meet that requirement (some duration of employment and/or amount of earnings) before any benefits can be received after a subsequent involuntary separation. If no requalification requirement is shown, a claimant may wait out the specified period of disqualification, reopen the claim for benefits, and receive benefits. In Nebraska, a separate penalty is applied for each voluntary quit or discharge during the base period; these penalties are cumulative. In West Virginia, the benefit reduction for a misconduct discharge can be reversed if the claimant works 30 days during the same benefit year.
Source: U.S. Department of Labor, January 1999, tables 401 and 402, and discussions with Quality Control administrators in the pilot states.
| Type of denial | |||
|---|---|---|---|
| Monetary | Separation | Nonseparation | |
| Determinations | 10,782,000 | 15,910,000 | 117,591,000 |
| Determinations with issues (adjudications) |
na | 3,416,000 | 4,276,000 |
| Denials | 1,172,000 | 1,859,000 | 2,390,000 |
| Percentage of denials in error (from pilot, not adjusted for "self-corrections") |
16.0 | 8.7 | 15.0 |
| Percentage of denials in error (from pilot, adjusted for "self-corrections") |
11.2 | 6.4 | 12.0 |
| Estimated uncorrected errors, based on adjusted error rate |
131,264 | 118,976 | 286,800 |
| Estimated uncorrected errors (adjusted) that would result in paymenta |
98,842 | 118,976 | 286,800 |
Source: First three rows from Skrable (1999): Monetary determinations from ETA 218 reports; separation determinations are the sum of determinations on new initial claims (ETA 218 reports) and additional initial claims (ETA 5159 reports); nonseparation determinations are intrastate and interstate liable weeks claimed for UI, UCFE, and UCX (ETA 5159 reports). Estimates of denial error rates (fourth and fifth rows) from Woodbury and Vroman (1999).
benefits.Table 4-5
Estimates of benefits lost due to erroneous denials, United States, fiscal year 1998
| Number of erroneous denials | U.S. WBA (BAM) | Relative WBA factor | Denial duration (weeks) | Total losses ($ million) | Average loss per error ($) | |
|---|---|---|---|---|---|---|
| A: Estimates based on Skrable (1999) | ||||||
| Monetary | 98,842 | $199.18 | 0.859 | 14.20 | $240 | $2,430 |
| Separation | 118,976 | 199.18 | 0.866 | 14.20 | 291 | 2,449 |
| Nonseparation | 286,800 | 199.18 | 0.909 | 1.00 | 52 | 181 |
| Total | 504,618 | -- | -- | 583 | 1,156 | |
| B: Modified Estimates | ||||||
| Monetary | 98,842 | 199.18 | 0.859 | 14.20 | 240 | 2,430 |
| Separation | 118,976 | 199.18 | 0.866 | 11.40 | 234 | 1,966 |
| Nonseparation | 286,800 | 199.18 | 0.909 | 2.91 | 151 | 527 |
| Total | 504,618 | -- | -- | 625 | 1,239 | |
Notes: Number of erroneous denials come from Table 4-4. U.S. WBA is the average weekly benefit amount for fiscal year 1998 in the Benefit Accuracy Measurement data. The relative WBA factor is the ratio of the average WBA of erroneously denied claimants to the average WBA of paid claimants. (Note that this differs across the three types of denials.) Denial duration figures are described in section 4.3. Total losses are the product of the number of erroneous denials, U.S. WBA, the relative WBA factor, and denial duration. The "modified estimates" in panel B are based on denial durations described in section 4.3.
| State | |||||||
|---|---|---|---|---|---|---|---|
| Nebraska | New Jersey | South Carolina | West Virginia | Wisconsin | Five Pilot States | United States | |
| Average duration of benefit receipt (weeks) | 10.70 | 16.50 | 9.90 | 13.70 | 12.40 | 13.36 | 13.90 |
| Duration adjustment | 0.81 | 0.84 | 0.77 | 0.86 | 0.74 | 0.79 | 0.79 |
| Imputed duration of average multi-week penalty (weeks) | 4.33 | 6.91 | 3.82 | 5.91 | 4.61 | 5.28 | 5.51 |
| Multi-week penalties as percent of all penalties | 71.0 | 38.9 | 14.7 | 43.2 | 51.0 | 44.7 | 42.3 |
| Imputed average duration of all nonseparation penalties | 3.36 | 3.30 | 1.41 | 3.12 | 2.84 | 2.92 | 2.91 |
Source: Average duration of benefit receipt from U.S. Department of Labor 5159 reports. Imputed duration of multi-week penalties is one-half the average duration of benefit receipt times the duration adjustment (described in the text). The imputed average duration of all nonseparation penalties (bottom row) is the imputed duration of multi-week penalties weighed by the proportion of all nonseparation penalties that are multi-week. (Multi-week penalties as a percent of all penalties derived from U.S. Department of Labor 207 reports.)
| Erroneous denial | ||||
|---|---|---|---|---|
| Characteristic or outcome | Determination correct |
Monetary | Separation | Nonseparation |
| Benefits paid in benefit year | $2,893.12 (1,893.49) |
na | na | na |
| Weeks of benefits paid in benefit year | 16.11 (8.05) |
na | na | na |
| Proportion exhausting benefits | 0.311 (0.464) |
na | na | na |
| Age | 40.55 (11.81) |
37.60 (8.53) |
38.86 (12.60) |
45.68 (18.50) |
| Male | 0.473 (0.500) |
0.636 (0.487) |
0.500 (0.527) |
0.389 (0.494) |
| U.S. citizen | 0.995 (0.072) |
1.000 (0.000) |
1.000 (0.000) |
0.972 (0.164) |
| Ethnicity: | ||||
| Black | 0.396 (0.490) |
0.523 (0.505) |
0.400 (0.516) |
0.361 (0.487) |
| Hispanic | 0.010 (0.101) |
0.045 (0.211) |
0.000 (0.000) |
0.028 (0.167) |
| Caucasian | 0.594 (0.492) |
0.432 (0.501) |
0.600 (0.516) |
0.583 (0.500) |
| Schooling: | ||||
| less than high school | 0.237 (0.425) |
0.273 (0.451) |
0.300 (0.483) |
0.250 (0.439) |
| high school graduate | 0.455 (0.499) |
0.545 (0.504) |
0.600 (0.516) |
0.389 (0.494) |
| some college | 0.201 (0.401) |
0.114 (0.321) |
0.000 (0.000) |
0.222 (0.422) |
| college degree | 0.108 (0.311) |
0.068 (0.255) |
0.100 (0.316) |
0.139 (0.351) |
| Hourly wage in usual job | $9.84 (4.21) |
$8.63 (2.86) |
$8.08 (4.00) |
$9.38 (3.71) |
| Reservation wage | $8.32 (3.31) |
$7.69 (2.16) |
$7.62 (3.32) |
8.16 (3.29) |
| UI benefits: | ||||
| Weekly benefit amount | $177.74 (52.07) |
$154.59 (57.95) |
$150.10 (51.63) |
$164.27 (60.91) |
| Maximum benefit amount | $4,387.01 (1,517.65) |
$3,448.80 (1,586.15) |
$3,498.90 (1,681.18) |
$4,137.33 (1,655.98) |
| WBA/maximum WBA | 0.747 (0.219) |
0.650 (0.244) |
0.631 (0.217) |
0.690 (0.256) |
| potential duration (weeks) |
24.31 (3.20) |
21.33 (5.39) |
22.49 (5.40) |
24.81 (2.97) |
| Season of claim: | ||||
| winter | 0.244 (04.30) |
0.205 (0.408) |
0.400 (0.516) |
0.250 (0.439) |
| spring | 0.262 (0.440) |
0.204 (0.408) |
0.300 (0.483) |
0.222 (0.422) |
| summer | 0.219 (0.414) |
0.227 (0.434) |
0.300 (0.483) |
0.167 (0.378) |
| fall | 0.275 (0.447) |
0.364 (0.487) |
0.000 (0.000) |
0.361 (0.487) |
| Observations | 389 | 44 | 10 | 36 |
Source: Tabulated from Benefit Accuracy Measurement and Denied Claims Accuracy files for South Carolina, September 1997 through August 1998.
| Dependent variable: | Benefits paid OLS | Benefits paid ML/Weibull | Weeks paid OLS | Weeks paid ML/Weibull |
|---|---|---|---|---|
| Estimator: | ||||
| Independent variable | ||||
| Age | -0.08 (7.11) |
0.002 (0.004) |
0.007 (0.037) |
0.002 (0.004) |
| Male | -504.54** (167.51) |
-0.278** (0.090) |
-2.502** (0.866) |
-0.269** (0.090) |
| U.S. citizen | 1,127.00 (1,160.47) |
0.611 (0.744) |
7.628 (5.996) |
0.677 (0.737) |
| Ethnicity: | ||||
| Black | -161.38 (171.97) |
-0.120 (0.092) |
-1.304 (0.889) |
-0.123 (0.092) |
| Hispanic | 589.28 (779.39) |
-0.155 (0.354) |
3.488 (4.027) |
-0.169 (0.353) |
| Caucasian | ref | ref | ref | ref |
| Schooling: | ||||
| less than high school | 137.46 (206.84) |
0.049 (0.113) |
0.900 (1.069) |
0.051 (0.113) |
| high school | ref | ref | ref | ref |
| some college | 3.23 (211.02) |
0.042 (0.117) |
0.178 (1.090) |
0.029 (0.117) |
| college degree | 276.10 (269.09) |
-0.113 (0.141) |
0.876 (1.390) |
-0.120 (0.140) |
| Hourly wage in usual job | 90.03** (26.05) |
0.039** (0.015) |
0.406** (0.135) |
0.043** (0.014) |
| UI benefits: | ||||
| WBA | 12.75** (2.03) |
0.006** (0.001) |
-0.015 (0.011) |
-0.001 (0.001) |
| potential duration | 42.86 (27.09) |
0.000 (0.016) |
0.320* (0.140) |
-0.001 (0.016) |
| Season: | ||||
| winter | -151.06 (218.52) |
0.073 (0.120) |
-0.499 (1.129) |
0.063 (0.119) |
| spring | -130.20 (211.87) |
0.014 (0.115) |
-0.851 (1.095) |
-0.001 (0.114) |
| summer | -354.46 (222.33) |
-0.089 (0.120) |
-1.863 (1.149) |
-0.089 (0.120) |
| fall | ref | ref | ref | ref |
| Constant | -2,036.17 (1,380.17) |
6.314** (0.864) |
1.244 (7.131) |
2.306** (0.858) |
| Observations | 389 | 389 | 389 | 389 |
| R-squared/likelihood ratio | 0.31 | 75.84 | 0.08 | 20.81 |
Notes: Standard errors in parentheses.
* denotes significance at 5% level; ** denotes significance at 1% level.
"ref" denotes reference category in a set of dummy variables.
"na" denotes variable not appropriate for inclusion in the equation.
| Outcome / estimator | Determination correct |
Erroneous denial | ||
|---|---|---|---|---|
| Monetary | Separation | Nonseparation | ||
| Observed benefits paid: | ||||
| in benefit year | $2,893.12 (1,893.49) |
na | na | na |
| at time of erroneous denial | na | $0.00 (0.00) |
$639.40 (702.18) |
$850.17 (1,218.07) |
| Expected benefits paid in benefit year: | ||||
| OLS | $2,893.12 (1,009.91) |
$2,289.63 (1,003.72) |
$2,238.49 (1,068.06) |
$2,767.23 (1,106.14) |
| ML/Weibull | $2,720.33 (1,088.45) |
$2,116.55 (862.20) |
$2,154.40 (1,037.93) |
$2,620.27 (1,298.34) |
| Observed weeks of benefits paid: | ||||
| in benefit year | 16.11 (8.05) |
na | na | na |
| at time of erroneous denial | na | 0.00 (0.00) |
4.15 (4.34) |
5.02 (6.19) |
| Expected weeks of benefits paid in benefit year: | ||||
| OLS | 16.11 (2.30) |
14.62 (1.88) |
14.91 (2.81) |
16.64 (2.30) |
| ML/Weibull | 15.10 (2.97) |
13.71 (2.22) |
14.40 (3.10) |
15.35 (3.19) |
| Observations | 389 | 44 | 10 | 36 |
Notes: Observed benefits paid and weeks of benefits paid during the benefit year are tabulated from Benefit Accuracy Measurement and Denied Claims Accuracy files for South Carolina, September 1997 through August 1998. (Supplementary data was supplied by the QC Division of the South Carolina Employment Security Commission.) Expected benefits paid and weeks of benefits paid during the benefit year are derived from the models displayed in Table 4-2 by substituting the average characteristics of each group of workers into the estimated model and solving for the dependent variable (benefits paid or weeks of benefits paid).
| Type of erroneous denial | ||||||
|---|---|---|---|---|---|---|
| Monetary | Separation | Nonseparation | ||||
| Estimator: | OLS | Weibull | OLS | Weibull | OLS | Weibull |
| Benefits lost as percentage of correctly determined claimant's benefits |
0.791 | 0.778 | 0.553 | 0.557 | na | na |
| Number of weeks lost as percentage of correctly determined claimant's weeks of benefits |
0.908 | 0.908 | 0.668 | 0.679 | na | na |
Notes: Figures calculated from Table 5-3. Imputations for erroneous nonseparation denials are not reported because, as discussed in the text, the regression analysis does not produce estimates of benefits lost due to nonseparation denials.
Table 5-5
Estimates of benefits lost due to erroneous denials, United States, fiscal year 1998
| Number of erroneous denials |
U.S. WBA (BAM) |
Relative WBA factor |
Denial duration (weeks) |
Total losses ($ million) |
Average loss per error ($) |
|
|---|---|---|---|---|---|---|
| A: Estimates based on Skrable (1999) | ||||||
| Monetary | 98,842 | $199.18 | 0.859 | 14.20 | $240 | $2,430 |
| Separation | 118,976 | 199.18 | 0.866 | 14.20 | 291 | 2,449 |
| Nonseparation | 286,800 | 199.18 | 0.909 | 1.00 | 52 | 181 |
| Total | 504,618 | -- | -- | -- | 583 | 1,156 |
| B: Modified Estimates | ||||||
| Monetary | 98,842 | 199.18 | 0.859 | 14.20 | 240 | 2,430 |
| Separation | 118,976 | 199.18 | 0.866 | 11.40 | 234 | 1,966 |
| Nonseparation | 286,800 | 199.18 | 0.909 | 2.91 | 151 | 527 |
| Total | 504,618 | -- | -- | -- | 625 | 1,239 |
| C: Modified Using Regression-Based Estimates of Benefits Lost | ||||||
| Monetary | 98,842 | 199.18 | 0.778 | 14.20 | 218 | 2,200 |
| Separation | 118,976 | 199.18 | 0.557 | 14.20 | 187 | 1,575 |
| Nonseparation | 286,800 | 199.18 | 0.948 | 2.91 | 158 | 549 |
| Total | 504,618 | -- | -- | -- | 563 | 1,114 |
| D: Modified Using Regression-Based Estimates of Weeks Lost | ||||||
| Monetary | 98,842 | 199.18 | 0.859 | 12.90 | 218 | 2,200 |
| Separation | 118,976 | 199.18 | 0.866 | 9.60 | 197 | 1,656 |
| Nonseparation | 286,800 | 199.18 | 0.909 | 2.91 | 151 | 527 |
| Total | 504,618 | -- | -- | -- | 566 | 1,122 |
Notes: Panels A and B are repeated from Table 4-5. Estimates in panel C and D are derived from this chapter's regression-based estimates. In panel C, the duration of both monetary and separation denials is given as 14.2 because the adjustment factors used (0.778 and 0.557) take account of both the lower WBA and the shorter duration of claimants denied for monetary and separation reasons.