What is the value of dynamic scoring for legislators?

When Congress considers legislation, nonpartisan agencies provide estimates of the law’s potential economic effects to policymakers, a process known as “scoring.” In recent decades, analysts at the Congressional Budget Office and Joint Committee on Taxation have developed models that incorporate complex feedback effects, going beyond conventional scoring techniques. These “dynamic scoring” methods can produce significantly different estimates of a law’s economic impact, but there are tradeoffs in applying them in different policy areas. On this episode Doug Elmendorf and Glenn Hubbard join Wendy Edelberg, director of The Hamilton Project, for a discussion on the costs and benefits of dynamic scoring.

• Listen to the Brookings Podcast on Economic Activity on Apple, Spotify, or wherever you like to get podcasts.
• Learn about other Brookings podcasts from the Brookings Podcast Network.
• Sign up for the podcasts newsletter for occasional updates on featured episodes and new shows.
• Send feedback email to [email protected].

[music]

EBERLY: I’m Jan Eberly, the James R. and Helen D. Russell Professor of Finance at Northwestern University.

STEINSSON: And I’m Jón Steinsson, Chancellor’s Professor of Economics at the University of California, Berkeley.

EBERLY: We’re the coeditors of the Brookings Papers on Economic Activity, a semiannual academic conference and journal that pairs rigorous research with real time policy analysis to address the most urgent economic challenges of the day.

STEINSSON: And this is the Brookings Podcast on Economic Activity, where we share conversations with leading economists on the research they do and how it will affect economic policy.

EBERLY: Behind every piece of legislation there’s a team of experts using the tools of their trade to evaluate the impact of that legislation becoming law. The paper discussed in today’s episode, “Dynamic scoring: A progress report on why, when, and how,” is focused on the behind-the-scenes work done by economists to estimate the law’s effect on the federal budget deficit, also known as budget scoring. The authors, Doug Elmendorf of Harvard University, Glenn Hubbard of Columbia Business School, and Heidi Williams of Dartmouth College, look at how more complex scoring models or dynamic scoring could enhance the information provided to Congress as they consider legislative proposals. Elmendorf and Hubbard are interviewed here by Wendy Edelberg, director of the Hamilton Project at Brookings.

STEINSSON: I’m really looking forward to this discussion. While budget scoring may seem a little bit in the weeds, it’s a crucial input into the legislative process. How will proposed new legislation affect the federal budget? Clearly, this is something that Congress needs to have a good handle on in order to make good decisions. But estimating this often depends on a lot of assumptions, some of which are highly controversial. So, how far we should go down that road is a big question.

EBERLY: It is a really tricky issue. There’s a clear tradeoff. You’ll likely get better estimates, but there’s also a greater risk of losing consensus in the process.

STEINSSON: The authors are making a really interesting argument in their paper. In some areas, they contend, our knowledge of certain behavioral responses has become sufficiently solid that it makes sense to incorporate more of them into the budget scoring process.

EBERLY: These tradeoffs are crucial to incorporating the best knowledge in the field while still maintaining integrity and broad support for the budget process. So, now let’s hear from the authors and Wendy.

EDELBERG: Super. Thank you, Jan and Jón. This is Wendy Edelberg, and I am delighted to be here with Glenn Hubbard. Hi, Glenn.

HUBBARD: Hi. How are you?

EBERLY: And Doug Elmendorf. Hey, Doug.

ELMENDORF: Hello.

EDELBERG: And I should say, Doug was my one-time boss at CBO, hired me into CBO to be director of the Macroeconomic Analysis Division. And I was very happy to work for Doug. Happy to be talking to him today.

ELMENDORF: And I was earlier your boss at the Federal Reserve Board.

EDELBERG: That’s true. A recidivist boss.

ELMENDORF: I’ve been following you around, Wendy.

EDELBERG: So, this is a real delight for me. And to be talking about dynamic scoring, which is something that I thought a little bit about in my time at CBO.

So, Doug, why don’t we start with you? For our listeners who aren’t familiar with budget scoring, let alone dynamic scoring, can you start by explaining the goals and methods of what we’ll call conventional scoring rather than the less fortunate way that it’s described as static scoring, and then move on to how that’s different from dynamic scoring?

[3:56]

ELMENDORF: Well, thank you. The basic idea here is that when Congress is considering potential changes in budget policy, it asks either the Congressional Budget Office or the staff of the Joint Committee on Taxation to provide an estimate of how a proposed change in policy would affect the federal budget. And essentially, CBO covers changes in spending and JCT covers changes in the tax code. And these estimates are then used by Congress as it refines legislation, as it debates legislation. And also, the estimates are revealed publicly in many cases. So, they affect the public perception of legislation that’s being considered as important what’s in these estimates from the agencies.

These agencies are nonpartisan. They do not recommend policy changes. They simply help Congress understand if Congress does A, the following things will happen to the budget. And if Congress does B, a different set of things will happen to the budget. The agencies have skilled economists, budget analysts to help produce the best estimates they can.

The conventional estimates include a lot of behavioral responses. That is to say, if you change a benefit program or a provision of the tax code, then people or businesses will change their behavior. And a lot of those behavioral changes are included in estimates because they matter for the budgetary effects. But there is a longstanding convention that behavioral changes that would affect overall macroeconomic figures—so, GDP, overall, national income, labor supply, productivity—those kinds of overall economic measures are held fixed in the estimating process. So, if a proposal might change one of those things, that effect is actually not presented in the estimate by CBO or JCT in the conventional scoring process. Dynamic scoring would broaden the set of effects that are reported in the estimates.

EDELBERG: So, Glenn, let me ask you to flesh out then just what then is a dynamic score in its fully formed existence?

[6:16]

HUBBARD: Sure. To pick up the categories that Doug had led off with, it would begin with just the classic static score—what is the direct spending or revenue effect of a policy change? Then micro behavioral effects. And then the dynamic score is the additional macroeconomic feedback. This is important because most policymakers are doing big or maybe even moderate policies because they think there is some sort of effect on the economy, on GDP, on productivity, or labor supply. And in some cases, like the immigration case we talk about in the paper, there could be counterintuitive effects that policymakers need to know. So, the information value of dynamic scoring is is high. And the credibility of nonpartisan agencies like the Congressional Budget Office or the Joint Committee gives policymakers sort of unvarnished information that they can use in their decisions. So, we feel it’s a core part of the information they need.

EDELBERG: That’s super. And just so that people understand. Can you just give us like a quick thirty second explanation of when CBO is involved and when the Joint Committee on Taxation, JCT, is involved?

[7:29]

HUBBARD: Pretty simple. Not entirely, but pretty simple. CBO spending, JCT Tax. There are some borders of what’s a tax and what spending, but to a first approximation, I think listeners can say the CBO handles the spending part and the JCT the tax part.

EDELBERG: Excellent. So, you guys and your coauthor, Heidi Williams, have a great discussion in the paper about how CBO incorporated dynamic effects into its scores of some previous proposed immigration reforms. That seems like a great example to me of how including these dynamic effects can be critical to a score being informative. Can one of you talk us through that? Probably, Doug, because I know he was there for at least one of these important scores.

[8:14]

ELMENDORF: So, I was. But this first was done at CBO in 2006. There was a comprehensive immigration proposal Congress was considering. And the whole idea of this proposal was that it would change the number of immigrants to the country. And under those circumstances, CBO and Congress at the time decided it wouldn’t make sense to try to hold fixed the number of people in the country in the estimate if the whole point of the legislation would be to change the number of people in the country. And so, CBO and JCT included in their estimate at the time, the direct effects of adding people to the U.S. population.

Later, when I was director, Wendy, you were leading the macroeconomic analysis, there was another comprehensive immigration proposal. This was 2013, I guess. And again, we felt it just made no sense to talk about the effects of the proposal without allowing for this change in the U.S. population to be part of the estimate. So, we built the direct effects of the change in population into the estimate we provided.

We also provided a separate additional analysis that brought in even more effects of the immigration proposal, effects like changes in productivity that might arise because of a change in the number of high-skilled people who came to the country, effects like that.

And so, when the Congress is interested, and when CBO and JCT are given enough time to do more complicated estimates or complicated analysis, then the agencies can do that and can provide that information. And immigration is a case where we did and where it mattered for the estimates, as we explained in the paper.

[10:09]

HUBBARD: I think what Doug is saying is very important because immigration was a case in which the sign could even change depending on income that you thought immigrants were going to earn and then pay taxes on in the United States. So, if you’re a policymaker trying to assess revenue and spending effects, overall budget effects, you would need to know that.

ELMENDORF: Let me emphasize this estimating process is not meant to either facilitate more immigration or to reduce immigration. Those are policy choices that are up to the Congress, the elected representatives of the people. The agencies are just trying to provide information so that Congress can make a fully informed choice.

EDELBERG: Doug, you talked about the time and energy in putting together that dynamic estimate, and I was part of that time and that energy. So, I certainly remember those late nights. And maybe that’s the context for my next question, which is I can certainly see the case for using dynamic scoring a lot more than we do. So, we talked about an important circumstance when CBO used dynamic scoring, but there’s lots of other circumstances. In fact, the vast majority of scores that the CBO puts out don’t use dynamic scoring. So, maybe this is a question to you, Glenn. Can you walk through some of the reasons for not using dynamic scoring that maybe you think have some validity to them?

[11:33]

HUBBARD: Well, sure. There are a number of arguments people make. One, to build on something you just said, Wendy, is when a change is de minimus in terms of its overall economic effects. Another argument people make that I don’t think Heidi and Doug and I find as compelling is just that there’s uncertainty and therefore you shouldn’t do it, because where there’s uncertainty other places, too, in the micro effects. So, that doesn’t sound so compelling.

Time and budget is an issue. The CBO staff and the JCT staff are extremely smart and extremely hard working, and we all have only so many hours in a day. In the paper we offer suggestions of how to get around that, but that’s a legitimate concern.

Another concern people raise that somewhat legitimate is politics, are politics involved here. But it’s Washington. It’s the Congress. Our job as technocrats isn’t to engage in politics, it’s just to provide the best information we can. But those are all reasons that people might offer.

EDELBERG: And in amongst those reasons you mentioned uncertainty around dynamic scoring. And it sounded like you were saying that that was an argument that people make against dynamic scoring that does not persuade you. So, I want to ask more about arguments that you hear that people will say dynamic scoring is a terrible idea. Uncertainty is definitely one of those ideas, that it sounds like you don’t find persuasive. Are there other ideas that you guys took some pains to knock down in the paper?

[13:07]

HUBBARD: I would just mention a couple. On uncertainty, it’s not that uncertainty is not important in macro effects. It definitely is. The question is, is it more of a problem there than it is in micro estimates? So, suppose we were doing a big tax reform that Congress might have thought would increase GDP. We may also be uncertain about the micro elements of that. What’s the response of individuals’ work effort to a tax reform or saving or investment? Those are all uncertain, too. And yet we wouldn’t say we wouldn’t do them.

On the issue of time and budget. One of the ways—and CBO and JCT both already do this well—to get around that is to develop heuristics and rules of thumb that say for this kind of policy, here’s an approximate feedback. That enables the very talented staffs of both agencies to be very nimble in their policy responses.

[14:03]

ELMENDORF: Let me add a few other concerns and how we thought about them in the paper. One other concern that’s raised about dynamic scoring, and Glenn alluded to this, is that maybe the effects would be very small. One has to be careful about that argument, though. It’s true that CBO and JCT can’t do huge analysis of every single proposal they see. But sometimes, even for small proposals, the dynamic effects can matter consequentially for the budgetary impact.

And immigration would be another example, even a small immigration proposal. So, for a fairly small increase or decrease in the number of immigrants, not accounting for the change in population in an estimate could still make that estimate a poor one on its own merits, regardless of whether the change is large enough to be detectable in the measurements of the macro economy. Just for an individual proposal, getting the estimate right or as right as can be done might involve bringing in the fact that there would be a change in population.

It’s also true about size of the effects of these proposals that sometimes the real information is that something would have only a small effect. When proposals are brought to the Congress, advocates tend to think that will have dramatically good effects. Opponents often will claim they will have dramatically bad effects. And there’s value in having these objective agencies bring evidence to bear on how big or small an effect might be. So, it’s true that it’s hard to do dynamic scoring for everything, but also in picking what to do it for, there are tradeoffs in deciding to focus only on large bills, which is the case sometimes, versus trying to cast a wider net using the heuristics that Glenn talked about.

EDELBERG: Let’s talk a little more about deciding when to do it. So, if dynamic scoring can’t be implemented for every piece of proposed legislation—I think you guys have both described arguments that persuade me on that front—so, if it can’t be implemented for every piece of legislation, what process do you imagine for deciding what is conventional and what is dynamic?

So, just to tell you what’s in my mind a little bit, Doug talked about policymakers, some policymakers who think positive economic effects will be huge. And so, the dynamic score will be much more favorable to them than a conventional score. And they may be rather uninclined actually to hear that they’re wrong and would actually prefer to have their conventional score, and then to just go out and tout the conventional score does not actually include all of the dynamic effects that I think are actually in train if we pass this piece of legislation. And similarly, an opponent of a piece of legislation might think the dynamic effects are terrible and they don’t want to hear from CBO that the negative effects are only modest and so they too would prefer a conventional score.

So, with that as backdrop—at least that’s my bias from my time talking to policymakers about whether or not they wanted a dynamic score—are there ideas that you want to put on the table for possible ways forward for when a dynamic score should be done by the agencies and when a conventional score should be done?

[17:25]

HUBBARD: Well, I think it’s a great question, Wendy. But I would put it in the here and now and going forwards. In the here and now, we argue that we should be doing more of this precisely in part for the reasons you raise. If people are seeking convenience, our job as technocrats is not to provide that. It’s to provide information. But going forward, what are other areas that are very promising? We talk about some in the paper, like immigration, but we look at others that are very important, like permitting where we may not know enough. So, it’s a way of having the agencies reach out to outside experts to say, where is the state of applied research? What do we know now that we didn’t know 5 or 10 years ago that we might be able to incorporate? So, I think it’s both the here and now and going forward.

[18:08]

ELMENDORF: Yes. And another aspect of this decision-making that we describe in the paper is that Congress will have to make ongoing assessments of the possibilities, as Glenn has described them. This is not something that CBO or JCT could do on their own. Their job is to serve the Congress. So, Congress has to think about what information it wants, drawing on the advice that CBO and JCT can offer about what they are capable of doing.

And so, we envision this going forward as an ongoing process of discussion and deliberation by the key committees of Congress with these agencies to understand as evidence changes, as the policy backdrop changes, as the capabilities of the agencies change, what would make sense in different ways at different points in time?

EDELBERG: Let’s talk more about capabilities, because that seems so critically important to dynamic scoring being used. The agencies, CBO and JCT, have to be able to do this dynamic analysis and do it quickly. And I was there building some of the capabilities, Glenn, that you talked about, some of those rules of thumb, those heuristics. And I’m really proud of the strides that CBO made when I was there and has continued to make since I left. It’s really heartening. But what advice do you offer CBO and JCT? Where should they be pushing their capabilities, their skill sets, where they can do analysis and do it quickly in order to make dynamic scoring more potentially part of the process going forward?

[19:53]

HUBBARD: I think it’s two things, Wendy, to figure out what are the biggest areas where it can be promising. That’s where outside experts, the CBO already has a panel of advisors, you know, getting input from the research community on what we know. But then I think it really is doubling down on the efforts of rules of thumb and heuristics. Nobody expects that the CBO could build a dynamic general equilibrium model for every problem on the planet. That’s not the goal. The goal would be to use research to get to rules of thumb and then to ask researchers where there are new areas.

[20:27]

ELMENDORF: Exactly. One CBO report that we highlight in our paper was report a few years ago on the effects of increasing federal infrastructure spending and what that would do to the economy and the budget over time. And in that report, CBO lays out really a modular structure, or here are the key inputs that are needed about the effects of infrastructure spending on productivity. The timing of those effects. The results depend on how an increase in spending might be financed. So, CBO lays out a structure. It then applies that structure to the particular case of potential increases in federal infrastructure spending.

But that same structure could be applied potentially to other sorts of changes in federal spending. For example, we talk in the paper about federal spending for research and development.

So, that report on infrastructure spending not only was a wonderful dynamic analysis of that proposal, it essentially laid out the structure for the way one might approach other policies of generally similar sorts in the future. So, that’s an example of the kind of modeling that can be done that helps us to develop these heuristics that Glenn was speaking of.

EDELBERG: I am guessing that if there are listeners out there who, before tuning in to this, had never heard of dynamic scoring, perhaps had never even heard of any kind of budget scoring, they’re listening to this thinking, what do you mean all scores aren’t dynamic? Like, all scores should be dynamic. It just seems logically obvious that this is all of the information that should be incorporated in the information that Congress gets when it thinks about the budgetary consequences of a proposed piece of legislation.

But I am guessing that there are other listeners who had heard of dynamic scoring and just sort of have a vague sense that, wait, wasn’t I supposed to hate dynamic scoring? There’s some reason why it’s political and I’m supposed to be on the side of this political fight to hate dynamic scoring. Or on the flip side, if this is a political issue and my team’s supposed to like it. How on earth did this thing become political? How did politics get caught up in whether or not dynamic scoring is a good thing or a bad thing?

[22:54]

HUBBARD: See, I don’t think this needs to be a political discussion. In the game show Jeopardy! you give the answer and guess the question. So, if the answer is “dynamic scoring,” I would say what is part of the information set that Congress needs? Congress is inherently political, and members of Congress may do what they wish, but it’s far better for the process to have expert advice from highly respected, nonpartisan agencies in the mix, politics notwithstanding.

EDELBERG: Okay. So, Glenn, that was a very reasoned, thoughtful answer. However, I don’t think it met some of our listeners where they are, which is they think it’s political and now they don’t quite remember why. Like, let’s meet them where they are.

[23:41]

HUBBARD: I’ll give an example on both sides. And so, what if I told you I thought tax cuts paid for themselves? And the reason they pay for themselves is because I do a tax reform. Well, let’s take the most recent tax reform, the Tax Cut and Jobs Act in 2017. So, as I said, that’s going to create so much economic activity, doesn’t cost the government anything.

Alternatively, on spending, I could say, you know, it’s such a good idea to invest in early childhood education for children—that kind of thing pays for itself. And so, I think the political voices on both sides may make arguments like that. There are surely feedback effects from early childhood education, just as there are from tax cuts on investment. Saying they pay for themselves is probably an extreme. It is an extreme. And so, to have respected agencies like the CBO or the JCT saying you’ll get some back, but please don’t think there’s a free lunch. I think that’s helpful.

[24:37]

ELMENDORF: I agree completely with Glenn. It’s true that the issue of dynamic scoring, I think, has received most of the popular attention, to the extent it’s received any popular attention, through discussions of tax cuts. When I was at CBO as an analyst in 1994 and 1995, this issue came to the fore in part because Republicans had taken a majority in the House of Representatives for the first time in some decades and tax cuts were on the agenda.

But I think it’s really unfortunate that this issue came to popular attention just through that set of policies, because in fact, as Glenn says, one could think about it completely symmetrically with certain spending proposals that would also be good for growth, as tax cuts might be, also would generally not pay for themselves as tax cuts would not. And so, I think the popular perception may have been somewhat skewed by the particular set of issues that were on the front of people’s minds when this issue has broken through to the broader public. But it’s really not about any particular set of policies.

I think in general, not including effects on labor supply and saving and productivity will tend to give less credit to proposals that are boosting economic growth. Because it’s these extra growth effects that are being left out in a conventional score. So, in that sense, proposals that would boost growth—either on the spending or tax side, either up or down, depending on one’s perspective—those proposals for growth are the ones that are getting shortchanged to some extent and conventional scoring.

EDELBERG: I want to end with a bit of a wild card question, which is the whole nature of this really excellent paper that the three of you have written is around this idea that agencies have the capability of giving excellent, well thought out information to Congress and that Congress should use that information to act most effectively. That suggests to me some optimism that policymakers really are making decisions, or could potentially be making decisions, based on the analytical information provided to them by, as Glenn called them, technocrats.

So, I’m guessing that the two of you must have some level of optimism to have gone through the time and energy of writing this paper that good information helps the policy process. And you gave some thought to how Congress can get better information. So, I invite both of you to leave our audience with what I’m hoping are your optimistic reasons for why you thought this was worth your time.

[27:38]

HUBBARD: Well, I’m always an optimist, so it’s a fixed effect. So, my comments may be taken with a grain of salt. But I think that most members of Congress are responsive to information about policy. So, they start with priors, to be sure. I’m not naive about that, but I think they are responsive. Now, not all, but many. In some of the cases we’ve mentioned, the fact that dynamic scores can even change the sign of an argument may make a policymaker scratch his or her head before pulling a lever. So, I think there’s reason to be optimistic. There will always be some who just don’t care about analysis, but I honestly believe they’re in the minority.

[28:14]

ELMENDORF: So, I’m an optimist, too. But my optimism was in many ways reinforced by my time as director of CBO. I found that many, many members with whom I spoke were interested in the analysis that CBO was doing. That was not the only factor that affected the decisions, nor should it have been. Budget analysts are the first people to say there’s a lot more to policy decision-making than just a budget analysis or a budget estimate. So, the members of Congress I worked with had a lot of factors that they were taking into account in the decisions that they made. And some of those factors seemed to me more rational in a technocratic sense than others.

But I found an awful lot of people on the Hill who wanted to understand the estimates the CBO was making. And I think my colleagues on the Joint Tax Committee staff would say the same thing. And it’s the job of the technocrats to provide that information. And then, sure enough, elected decision-makers will get to make their decisions. And then we hold them accountable as a society for decisions that they make. And that’s been a pretty effective system in our country, and I’m optimistic about its strong continuation.

EDELBERG: Well, just selfishly, thank you because you’ve made me more optimistic. So, if nothing else, I will take that. Thank you so much to Doug, thank you, Glenn.

[music]

And thank you for authoring this paper with Heidi and then having this conversation with me. I greatly enjoyed it.

HUBBARD: Thanks. A pleasure.

ELMENDORF: Thank you very much, Wendy.

STEINSSON: Once again, I’m Jón Steinsson.

EBERLY: And I’m Jan Eberly.

STEINSSON: And this has been the Brookings Podcast on Economic Activity. Thanks to our guests for this great conversation and be sure to subscribe to get notifications about new releases of this podcast.

EBERLY: The Brookings Podcast on Economic Activity is produced by the Brookings Podcast Network. Learn more about this and our other podcasts at Brookings dot edu slash podcasts. Send feedback to podcasts at Brookings dot edu, and find out more about the Brookings Papers on Economic Activity online at Brookings dot edu slash B P E A.

STEINSSON: Thanks to the team that makes this podcast possible, including Kuwilileni Hauwanga, supervising producer; co-producers Fred Dews and Chris Miller; audio engineer, Gastón Reboredo; show art was designed by Katie Merris at Brookings; and promotional support comes from our colleagues in Brookings Communications.