Much of the work of crafting and enacting public policy is dependent on predictions of the outcomes of the policy being considered. Such outcomes can’t actually be known with certainty—as any responsible researcher understands—yet there’s a drive to arrive at the concrete forecasts often demanded by legislators. A perhaps apocryphal story about Lyndon Johnson illustrates the pressure economists can feel to draw strong conclusions. The tale has an economist describing the uncertainty of his forecast to LBJ, and offering a range of values rather than unsupported precision. “Ranges are for cattle,” Johnson is said to have replied. “Give me a number.”
True or not, the anecdote as recounted by Charles F. Manski in Public Policy in an Uncertain World goes a long way towards explaining how the modest findings of cautious researchers become the bold assertions of self-sure politicians. In Manski’s experience, economic consultants understand policy makers to be “either psychologically unwilling or cognitively unable to cope with uncertainty.” And so researchers feel compelled to offer precise predictions that they understand may not be credible.
Following John Kenneth Galbraith’s popularization of the term “conventional wisdom,” Manski offers “conventional certitude” to describe predictions that are generally accepted as true but that may not warrant the certainty they’re afforded. He uses the Congressional Budget Office (CBO) as a case study of conventional certitude in action.
The CBO was established in the Congressional Budget Act of 1974, which includes language usually interpreted as mandating the CBO to provide precise predictions of the budgetary impact of pending legislation. These CBO “scores” are passed on to Congress without accompanying measures of uncertainty, as Manski explains, even though the legislation they assess is often far too complex to allow for the precise forecasting of budgetary implications. And so everyone involved takes for granted what they should surely understand to be fudged, and CBO scores are widely accepted. Even today they’re eagerly awaited by both sides of the aisle in our dysfunctionally contentious Congress.
The Patient Protection and Affordable Care Act of 2010 (aka “Obamacare”), writes Manski, well illustrates the treatment of CBO scores:
Throughout the legislative process, Congress and the media paid close attention to the scores of alternative bills considered by various congressional committees. A culminating event occurred on March 18, 2010, when the CBO, assisted by staff of the Joint Committee on Taxation (JCT), provided a preliminary score for the combined consequences of the Patient Protection and Affordable Care Act and the Reconciliation Act of 2010. CBO director Douglas Elmendorf wrote to House of Representatives Speaker Nancy Pelosi: “CBO and JCT estimate that enacting both pieces of legislation… would produce a net reduction of changes in federal deficits of $138 billion over the 2010-2019 period as a result of changes in direct spending and revenue.”
Anyone seriously contemplating the many changes to federal law embodied in this legislation should recognize that the $138 billion prediction of deficit reduction can be no more than a very rough estimate. However, the twenty-five-page letter from Elmendorf to Pelosi expressed no uncertainty and did not document the methodology generating the prediction.
Media reports largely accepted the CBO scores as fact, the hallmark of conventional certitude. For example, a March 18, 2010, New York Times article documenting how CBO scoring was critical in shaping the legislation reported: “A preliminary cost estimate of the final legislation, released by the Congressional Budget Office on Thursday, showed that the President got almost exactly what he wanted: a $940 billion price tag for the new insurance coverage provisions in the bill, and the reduction of future federal deficits of $138 billion over 10 years.” The Times article did not question the validity of the $940 and $138 billion figures.
Manski further illustrates the oddity of CBO scores and their acceptance by noting Elmendorf’s subsequent reluctance to project beyond the office’s standard ten-year window. As Elmendorf put it in a letter to Paul Ryan, “A detailed year-by-year projection, like those that CBO prepares for the 10- year budget window, would not be meaningful over a longer horizon because the uncertainties involved are simply too great.” Elmendorf listed a number of factors preventing long term certainty, all of which, Manski notes, apply just as well to the ten year prediction. “Longer-term predictions may be more uncertain than shorter-term ones,” Manski explains, “but it is not reasonable to set a discontinuity at ten years, with certitude expressed up to that point and uncertainty only beyond it.”
And so, Manski concludes, despite certainly understanding the CBO’s ten-year prediction to be but a rough estimate, Elmendorf “felt compelled to adhere to the established CBO practice of expressing certitude when providing ten-year predictions, which play a formal role in the congressional budget process.” And, despite what he sees as the CBO’s “admirable reputation for impartiality,” Manski calls for a move away from the reliance on conventional certitude:
I would like to believe that Congress will make better decisions if the CBO provides it with credible predictions of budgetary impacts. Whether or not this is a reasonable expectation, I worry that someday sooner or later the existing social contract to take CBO scores at face value will break down. Conventional certitudes that lack foundation cannot last indefinitely. I think it better for the CBO to preemptively act to protect its reputation than to have some disgruntled group in Congress or the media declare that the emperor has no clothes.
Manski’s prescription is a shift to “credible interval scoring,” rather than “incredible certitude.” And one of the book’s larger arguments is that this recommendation should be broadly applied, and that those who rely on or report on policy analysis need to better understand its assumptions and limitations. Analysts and policy makers need to head back to the range, essentially, right alongside LBJ’s cattle.
