Skip Navigation

On January 31, 2002, the Congressional Budget Office (CBO) released The Budget and Economic Outlook: Fiscal Years 2003-2012, which presents CBO's latest projections of federal revenues and outlays for that period. Chapter 5 of that report discusses the uncertainties in CBO's baseline projection of the total budget surplus and includes a chart (reproduced here as Figure 1) illustrating how those uncertainties increase over six years. This supplementary report describes the data and methods used to construct the chart. In brief, CBO calculated measures of uncertainty using the inaccuracies in its past projections that arose from economic and technical factors, not from legislation.
 

Figure 1. Uncertainty in CBO's Projection of the Total Budget Surplus Under Current Policies

SOURCE: Congressional Budget Office.

NOTES: This figure shows the estimated likelihood of alternative projections of the surplus under current policies. The calculations are based on CBO's past track record. CBO's baseline projection falls in the middle of the darkest area. Under the assumption that current policies do not change, the probability is 10 percent that actual surpluses will fall in the darkest area and 90 percent that they will fall within the whole shaded area.

Actual surpluses will of course be affected by legislation enacted during the next five years, including decisions about discretionary spending. The effects of future legislation are not included in this figure.

Figure 1 presents CBO's baseline projection of the budget surplus as a fan of probabilities around the mean projection for fiscal years 2002 through 2007. The fan widens as the projection extends. The baseline projection falls in the middle of the highest-probability area--the darkest part of the figure. But the figure makes clear that nearby projections--other paths in the darkest part of the figure--have nearly the same probability as the baseline. Moreover, projections that are quite different from the baseline have a significant probability of being realized.⁽¹⁾

The shaded area in the figure represents the 90 percent confidence range (the range within which the actual value has a 90 percent chance of falling). CBO estimates that range on the basis of the uncertainty in its historical record of budget projections--a total of 20 baselines spanning the period from 1981 to 2001.⁽²⁾ In other words, the estimates of uncertainty presume that in the future, CBO will experience inaccuracies similar to those it experienced in the past, with about the same probability distribution of large and small inaccuracies.

The 1981-2001 sample period, however, was not typical for the post-World War II period as a whole. It contains only two recessions (those of 1981-1982 and 1990-1991), and the first recession is not well represented in the sample because only one of the baseline projections preceded it. Since World War II, by contrast, a total of nine recessions have occurred. When CBO takes into account the greater volatility of output in that entire period, the width of the fan chart increases by roughly one-third by the fifth year (see Figure 2).
 

Figure 2. Uncertainty in CBO's Projection of the Total Budget Surplus, Assuming Average Business-Cycle Activity Since 1947

SOURCE: Congressional Budget Office.

NOTE: The narrower confidence range (the outer boundary of the fan chart in Figure 1) is based on CBO's record of budget projections since 1981. The wider range represents the uncertainty in CBO's current projection if future cyclical movements of the economy follow the average pattern since 1947 rather than the less volatile pattern that has existed since 1981. The assumption is that cyclical inaccuracies will be proportionally larger if business-cycle activity is greater but that noncyclical inaccuracies will be unaffected.

CBO introduced its fan-chart presentation of uncertainty last year in The Budget and Economic Outlook: Fiscal Years 2002-2011. This year's version of the chart reflects an improvement in the analysis of uncertainty: additional research that allows inaccuracies correlated with the business cycle to be distinguished from those not correlated with the business cycle.

That distinction is a useful one, because inaccuracies in the assessment of noncyclical changes are likely to grow as the projection horizon lengthens, whereas inaccuracies correlated with the business cycle would not be expected to increase in the same way. According to CBO's estimates, cyclical inaccuracies historically have in fact been small for the first two years of a baseline, when CBO attempts to incorporate its views of the business cycle in the forecast. Those inaccuracies rise to a higher level for the later years of a projection--when CBO does not try to forecast the business cycle--but they flatten out (see Figure 3). Noncyclical inaccuracies, by contrast, increase throughout the projection period.⁽³⁾
 

Figure 3. Cyclical and Noncyclical Parts of CBO's Inaccuracies in Projecting the Primary Surplus

SOURCE: Congressional Budget Office.

NOTES: The lines in this figure show root-mean-square errors (RMSEs), a type of average that ignores the signs of individual errors and gives greater weight to larger errors. The RMSE of total inaccuracy is calculated by squaring the RMSEs of the cyclical and noncyclical parts, adding them together, and taking the square root of the sum. Thus, the combined RMSE is smaller than the sum of the two components' RMSEs.

The primary budget surplus is the difference between federal revenues and federal outlays excluding net interest.

That breakdown suggests that by the end of five years, CBO's inaccuracies in projecting the budget's bottom line have consisted, in roughly equal parts, of cyclical inaccuracies and inaccuracies in assessing economic and other noncyclical trends that underlie the budget.

The analysis also suggests that if CBO had been confronted over the past two decades with a less stable economy--one more representative of the cyclical experience of the entire post-World War II period--the cyclical component would have been roughly twice as large as the noncyclical component by the end of five years (see Figure 4).
 

Figure 4. Cyclical and Noncyclical Parts of CBO's Inaccuracies in Projecting the Primary Surplus, Assuming Average Business-Cycle Activity Since 1947

SOURCE: Congressional Budget Office.

NOTES: The lines in this figure show root-mean-square errors (RMSEs), a type of average that ignores the signs of individual errors and gives greater weight to larger errors. The RMSE of total inaccuracy is calculated by squaring the RMSEs of the cyclical and noncyclical parts, adding them together, and taking the square root of the sum. Thus, the combined RMSE is smaller than the sum of the two components' RMSEs.

This figure represents the inaccuracies in CBO's projections assuming that future cyclical movements of the economy follow the average pattern since 1947 rather than the less volatile pattern that has existed since 1981. The assumption is that cyclical inaccuracies will be proportionally larger if business-cycle activity is greater but that noncyclical inaccuracies will be unaffected.

The primary budget surplus is the difference between federal revenues and federal outlays excluding net interest.

Whether the next decade will more closely resemble the past 20 years or the entire postwar period cannot be determined in advance. However, recent research suggests that a structural change in the economy occurred in the early 1980s, which may explain why cyclical movements have been fewer and milder in the past two decades and may presage a relatively stable economy in the future. Analysts differ on the precise nature of the structural change.⁽⁴⁾ But if it persists, Figure 1 may portray the uncertainties in CBO's projection of the surplus better than the wider range in Figure 2 does.

Preparing the fan chart involved two stages. In the first stage, CBO constructed measures of its past projection inaccuracies that remove the effects of changes in legislation and other factors. In the second stage, CBO constructed probability distributions at six time horizons, beginning with the current fiscal year (the one in which the projection was made) and covering the next five years. The probability distributions were derived from a model that distinguishes between inaccuracies that appear to stem from the difficulty of forecasting the business cycle and inaccuracies that are not correlated with the business cycle and appear to stem from other causes.
 

Stage One: Constructing the Measures of Inaccuracies

Creating measures of inaccuracies in CBO's past budget projections involved adjusting those projections for several factors: legislation (including laws that affect discretionary spending) and net interest on the federal debt.

CBO subtracted from its projections of revenues and outlays the estimated effects of laws dealing with revenues or mandatory spending that were enacted after the projections were made. That adjustment was necessary because CBO's baseline projections are intended to show the expected level of the budget surplus or deficit assuming that current tax and spending policies remain the same.⁽⁵⁾ Without that adjustment, the measures of inaccuracies would include the effects of later legislation, which would run counter to the purpose of the baseline.

CBO also excluded discretionary spending from both the baseline projections and actual outlays. The effect of omitting discretionary spending is to treat all discrepancies between actual discretionary spending and baseline projections of such spending in the same way as differences resulting from other budget legislation.⁽⁶⁾ CBO decided on that treatment for two reasons: because levels of discretionary spending are determined anew each year through appropriation acts and because that treatment would permit the use of a longer historical record.

Inaccuracies in projecting net interest largely depend on inaccuracies in projecting the government's publicly held debt. That debt, in turn, is the cumulation of annual budget deficits (minus surpluses), so inaccuracies in projecting net interest depend on the cumulation of other inaccuracies in projecting the deficit or surplus. CBO therefore excluded net interest from its initial calculations of projection inaccuracies. In the subsequent step, however, it incorporated into the fan-chart calculations the effects of other misestimates on net interest.

CBO calculated inaccuracies for each year covered by the winter baseline projections that it published from 1981 through 2001. In most years, those projections were issued in January or February, although in 1996, publication was delayed until May. For reasons involving the availability of data, CBO used its July 1981 projection in place of the one published in February 1982.⁽⁷⁾ The resulting sample was small: only 20 current-year projections, declining to 15 five-year-ahead projections.⁽⁸⁾ (The sample size diminishes because projections made in the past five years can be compared with actual outcomes only through 2001.)

The estimated effects of legislation dealing with revenues or mandatory spending were taken primarily from information published in CBO's twice-yearly reports on the budget and economic outlook. Most of those reports show the multiyear budgetary effects of legislation enacted since the previous projection. For cases in which estimates were not available (as will be discussed below), substitutes were constructed.

Revenues

As required by the Congressional Budget Act of 1974, the Joint Committee on Taxation (JCT) estimates the effects of tax legislation--bills that alter income, estate and gift, excise, or payroll taxes--at the time that the legislation is being considered by the Congress.⁽⁹⁾ CBO produces estimates for legislation that affects customs duties and miscellaneous receipts that are classified as revenues.

Those estimated effects of tax legislation were used to adjust each baseline projection of revenues. For example, the projection made in January 1994 for fiscal year 1999 was adjusted downward from $1,630 billion to $1,619 billion (see Table 1). That adjustment reflected all tax laws enacted after January 1994 and through fiscal year 1999. The law with the largest budgetary impact was the Taxpayer Relief Act of 1997, which JCT estimated would reduce revenues in 1999 by $7 billion.⁽¹⁰⁾ Similar adjustments were made for the other years in the baseline projections. The differences between those adjusted projections and actual revenues represent the inaccuracies attributable to economic and technical factors (see Table 2).
 

CBO's and JCT's estimates of the effects of tax legislation are not revised after their initial publication, even though later economic and technical information might permit better estimates. (For instance, knowledge about an actual tax base, such as wages or corporate profits, in a given year would improve estimates of how a change in tax law would affect revenues.) Using unrevised data on the effects of legislation may overstate the true uncertainty of CBO's budget projections, all other things being held equal.

Outlays

The estimated effects of legislation on outlays (excluding net interest) were also taken largely from CBO's reports on the budget and economic outlook. However, as with revenues, some adjustment to that information was necessary.

• Baseline Projections of Discretionary Spending. As noted above, differences between actual and projected levels of discretionary spending were assumed to be attributable to legislation. But the July 1981 projection did not include a separate category for discretionary spending. For that baseline only, discretionary spending was approximated by adding the projections for defense, other grants to state and local governments, and other federal operations.⁽¹¹⁾
  
  
• Insufficient Details About Legislation. In some cases, the estimated effects of legislation were not published in enough detail to separate out the effects of legislation on discretionary spending. In other cases, the information was published for some but not all of the six years in the baseline budget projection. One or both of those problems applied to the following periods: August 1986 to January 1987, August 1987 to February 1988, August 1994 to January 1995, and January 1998 to August 1998. In those cases, supplemental information from CBO's files was used to estimate the needed numbers.

As with revenues, the estimated effects of legislation on outlays (including both discretionary and mandatory spending) were used to adjust each baseline projection of outlays. After removing interest payments, the differences between those adjusted projections and actual outlays are the inaccuracies attributable to economic and technical factors (see Table 3).
 

Primary Budget Surplus or Deficit

The difference between revenues and outlays excluding net interest is known as the primary budget surplus (or deficit when negative). Correspondingly, CBO's inaccuracies in projecting revenues, minus its inaccuracies in projecting noninterest outlays, equal its inaccuracies in projecting the primary surplus or deficit (see Tables 4 and 5). As described above, that calculation excludes legislative changes. In stage two, the inaccuracies in projecting the primary budget surplus or deficit were cumulated into inaccuracies in projecting publicly held debt, which were used to estimate the uncertainty of CBO's projections of net interest.
 

Stage Two: Constructing Probability Distributions

The historical record of inaccuracies in projecting the primary surplus or deficit (adjusted for legislation) presented in Table 4 forms the basis for the statistical calculations that CBO used to derive the probability distributions underlying the fan chart.

As noted above, CBO's record of projections is both short and possibly unrepresentative (in that it does not contain a historically normal number of business cycles). In the absence of a rich sample, estimates may be improved if additional information can be brought to bear. In this case, CBO used its knowledge of its forecasting procedures and of business cycles, as well as its historical record, to draw more reliable conclusions about the probability distribution of inaccuracies in its budget projections.

The Statistical Model for Inaccuracies in the Primary Surplus or Deficit

With the effects of legislation removed, CBO's past inaccuracies are closely related to inaccuracies in forecasting economic variables. Thus, they should be affected by errors in the projection of the business cycle. Forecasting the course of a business cycle over more than two years is virtually impossible, so CBO has traditionally tried to incorporate the business cycle into its economic projections in a serious way only for the current year and the budget year.⁽¹²⁾ In its projections for longer horizons, CBO simply assumes that gross domestic product (GDP) will, on average, adhere to its trend (or "potential") path.⁽¹³⁾ That assumption recognizes that, in fact, GDP will sometimes be above and sometimes below its potential, but CBO cannot forecast those boom or recession periods more than a couple of years ahead.

Given the way in which CBO makes its economic projections, its budget projections can be expected to show a certain pattern of inaccuracies. As long as CBO continues to do a reasonably good job of projecting the business cycle, that cycle should not contribute much to the inaccuracy of budget projections for the current year. For the budget year, its contribution should be slightly larger (because errors in forecasting increase with the horizon) but still modest. For later years, however, cyclical factors should loom larger. CBO assumes that by the last two years of the five-year projection horizon, GDP will be at or close to its potential level. Thus, whenever the economy is in fact above or below its potential, none of that difference will be reflected in the budget projections. Consequently, as the projection horizon lengthens, the budget misestimates that result from miscalculating the business cycle should grow in importance, until they reach their maximum in the last two years of the five-year period.

According to that analysis, the portion of budget inaccuracies attributable to the business cycle may be estimated by using the correlation between those inaccuracies and the GDP gap (the percentage difference between actual GDP and its potential value). For projections several years ahead, the level of the GDP gap is a good indicator of unexpected cyclical conditions. For projections only one or two years ahead, by contrast, the change in the GDP gap is a better indicator than the level, because the approaching levels of the gap are likely to be quite similar to the recent level.

Using the GDP gap and its change to measure unforeseen changes in cyclical conditions, CBO estimated by means of a linear regression what portion of its past inaccuracies was attributable to business cycles (see Box 1). Restrictions on the regression incorporate the exogenous information that, of the two variables, the change in the GDP gap is the main source of uncertainty over shorter horizons and the level of the gap over longer ones. For the intermediate year (the first year after the two-year forecast), both the level of the GDP gap and its change are significant sources of uncertainty.
 

The portion of the overall inaccuracies explained by the two business-cycle variables in the regression is called the cyclical part. The rest, the noncyclical part, represents the inaccuracies that result from such factors as noncyclical changes in average tax rates, capital gains realizations, the share of GDP that goes to taxpayers in high tax brackets, and federal spending for Medicare and Medicaid.⁽¹⁴⁾

CBO does not expect its projection inaccuracies to display a negative or positive bias--otherwise it would change its projections. Accordingly, CBO assumed that the probability distribution of its projection inaccuracies was centered around a zero average. That assumption is not contradicted by the data.

Calculating the Distribution of Inaccuracies from the Model

The statistical model computes coefficients that relate misestimates of the surplus or deficit (shown in Table 4) to the business-cycle variables. Given the historical pattern of the business cycle, those coefficients can be used to describe the distribution of inaccuracies that might be expected to occur simply because of the business cycle. One way to describe that distribution is through the root-mean-square error (RMSE), a kind of average error that ignores the signs of individual errors and gives extra weight to large errors.⁽¹⁵⁾ The model assumes that the RMSE of the cyclical part of misestimates will rise to a plateau (see Figure 3).

That model does not account for all of a given projection inaccuracy, however. What is left, the noncyclical part, also has a distribution that can be summarized by its RMSE. Like the cyclical component, that part of a misestimate rises as the projection horizon lengthens, but it does not plateau (see Figure 3). For simplicity, CBO assumes that the noncyclical influences captured in the residual from the regression are independent of the cyclical component at each horizon.⁽¹⁶⁾ That assumption is nearly correct and makes little difference to the results.⁽¹⁷⁾

The estimated RMSEs for the cyclical and noncyclical parts can be combined to form an estimate of the RMSE for overall budget misestimates. Two RMSEs are combined by squaring each of them, adding those squares together, and taking the square root of the sum. That calculation yields a combined RMSE that is less than the sum of the two component RMSEs (see Figure 3).

The model's estimate of the distribution of budget misestimates appears generally consistent with CBO's past record. Out of 105 past projection inaccuracies for the primary surplus or deficit in fiscal years 1981 through 2001, only 3 percent fall outside the calculated 90 percent confidence range--a range that ought, in a large enough sample, to encompass 90 percent of the observations (see Figure 5).
 

Figure 5. CBO's Past Inaccuracies in Projecting the Primary Surplus, Compared with the Constructed 90 Percent Confidence Range

SOURCE: Congressional Budget Office.

NOTE: Each thin line represents the actual inaccuracies of the set of projections made in a given year. The thick lines represent the 90 percent confidence range constructed from CBO's statistical model for inaccuracies. That range encompasses nearly all of CBO's past record.

The inaccuracies in projecting the primary surplus or deficit for a given year have so far been formulated as a percentage of that year's actual revenues. For each year in the projection through 2007, the estimated RMSE of the inaccuracies can be converted into dollars by multiplying it by CBO's current baseline projection of total revenues.

Uncertainty in Projections of the Total Surplus

The uncertainty range for CBO's projection of the total surplus or deficit (shown in Figure 1) requires information about how the predicted inaccuracies in the primary budget (the budget excluding net interest) will affect the government's debt-service costs. Those inaccuracies are run through a simple debt-service model that tracks how inaccuracies in projecting surpluses or deficits translate into inaccuracies in projecting debt; the model applies an interest rate that is a weighted average of CBO's current baseline projections of rates on three-month Treasury bills and 10-year Treasury notes. That model is an approximation of the model that CBO uses for its budget projections.

The extent to which projection inaccuracies for the primary surplus are correlated across horizons is important for the computation of debt-service costs. When those inaccuracies are highly correlated, they have a large accumulated effect on outstanding debt, and the associated change in the government's interest burden is large. In calculating the probability distribution of projection inaccuracies for the total surplus (including net interest), CBO assumed that the cyclical and noncyclical parts will continue to have the same correlation structure as in the past.⁽¹⁸⁾

The percentiles for the total surplus that are used to draw the fan chart are computed by multiplying the values associated with the various percentiles for the standard normal distribution by the calculated RMSE of the probability distribution of the total surplus at different horizons. Those percentiles are shown in Table 6.
 

CBO will continue its efforts to refine these calculations. It welcomes suggestions for improving the methodology.

1. Technically, the probability density is highest near the baseline and falls off for more distant projections.

2. The projections are those made in July 1981 and CBO's winter projections (usually published in January) from 1983 through 2001. Insufficient data were available to use either projections made before 1981 or the projection made in early 1982. In the cases of the two years surrounding the 1981 projection, available data about the effects of legislation on changes in CBO's baseline budget projections were insufficient, and discretionary spending was not reported separately. As discussed in the following section, those data are important because the measures of inaccuracy used in this analysis were constructed by removing the effects of legislation, including discretionary spending (along with interest payments). The baseline budget projections that CBO made before 1980 were not comparable with later ones, because the early economic assumptions represented targets rather than projections.

3. CBO did not begin making 10-year projections until 1996. Before that, its baseline typically extended for five years beyond the current year. Because there are not yet any uncertainty measures for the sixth through the tenth year, this analysis focuses on a five-year projection horizon.

4. Although there seems to be general agreement in the recent economics literature that the growth of output has become more stable and that the expansion phases of business cycles are likely to be longer in the future than in the past, economists disagree about the causes of that increased stability. Those disagreements concern the importance of factors such as monetary policy, financial markets and institutions, inflation, supply shocks, and the behavior of inventory investment. For discussions of those and other points, see Margaret M. McConnell and Gabriel Perez-Quiros, "Output Fluctuations in the United States: What Has Changed Since the Early 1980's?" American Economic Review, vol. 90, no. 5 (December 2000), pp. 1464-1476; Olivier Blanchard and John Simon, "The Long and Large Decline in U.S. Output Volatility," Brookings Papers on Economic Activity, no. 1 (2001), pp. 135-174; and Marcelle Chauvet and Simon Potter, "Recent Changes in the U.S. Business Cycle," The Manchester School, vol. 69, no. 5 (special issue 2001), pp. 481-508.

5. For more information about the purpose of CBO's baseline and the rules that govern its construction, see Congressional Budget Office, The Budget and Economic Outlook: Fiscal Years 2003-2012 (January 2002), Chapter 1.

6. In CBO's usual analyses of changes in its projections since the previous baseline, CBO allocates a small proportion of any changes in assumptions about discretionary outlays to the categories of economic or technical revisions (ibid., Box 5-1). In CBO's classifications, economic revisions are ones that stem from changes in the agency's economic forecast, and technical revisions are ones that cannot be attributed to new legislation or to changes in the components of the economic forecast.

7. Specifically, CBO did not have enough information in its files to include the estimated effects of legislation enacted between February 1982 and February 1983. Much better data were available for the slightly longer period of July 1981 through February 1983.

8. The sample size could have been doubled by including the updated projections that CBO typically publishes in the summer, but those updates are closely related to the winter baselines and do not really offer additional information useful for calculating inaccuracies.

9. See Section 201(f) of the Congressional Budget Act of 1974 (as amended), 2 U.S.C. 601(f).

10. See Congressional Budget Office, The Economic and Budget Outlook: An Update (September 1997), p. 36.

11. See Congressional Budget Office, Baseline Budget Projections: Fiscal Years 1982-1986 (July 1981), p. 38.

12. In relation to CBO's baseline, the current year is the fiscal year in which the projection is made and the budget year is the following fiscal year (the one for which the budget is under consideration). Years beyond the budget year are referred to as out-years.

13. See Congressional Budget Office, CBO's Method for Estimating Potential Output: An Update (August 2001).

14. See CBO, The Budget and Economic Outlook: Fiscal Years 2003-2012, Chapters 3 and 4.

15. The RMSE is calculated by squaring each projection inaccuracy, averaging the squares, and taking the square root of the result. (For distributions with a mean of zero, it is equal to the standard deviation.) The RMSE forms the basis for CBO's calculation of the fan chart. Roughly speaking, a band of plus or minus one RMSE from a projection encompasses about two-thirds of the likely variation--that is, the outcome is likely to be within one RMSE of the estimate about two-thirds of the time. Other confidence intervals in the fan chart are calculated from RMSEs.

16. The fitted part and the residual from the regression are taken, respectively, to be the cyclical and noncyclical parts of the projection inaccuracies. By construction, those two parts are uncorrelated for the whole regression sample, which pools the inaccuracies for the six different horizons, but they have sample correlations different from zero at individual forecast horizons.

17. Because the sample of projections is small, to estimate the distribution of inaccuracies with any confidence, CBO assumed that the inaccuracies shown in Table 4 were generated by a normal distribution. The sample kurtosis and skewness of the inaccuracies are consistent with that assumption. The assumption of a normal distribution is not rejected at any of the horizons for either of those statistical measures at any conventional significance level.

18. The uncertainty that arises from the impact on net interest increases the RMSE of the probability distribution of projection inaccuracies. However, it does not alter the assumption that inaccuracies are normally distributed, because the changes in debt-service costs are a linear function of the current and past changes in the primary budget. The RMSE of the total surplus, in fact, is computed using that linear relationship.