Imagine being a chemist in the Middle Ages. Your colleagues are all working on fanciful tasks like trying to develop a philosopher’s stone to grant immortality or turning lead into gold. You continually point out to them that all of their attempts have pretty much failed completely. In fact, based on your own research, you have strong reason to believe that the path they are taking is going completely the wrong direction. They aren’t just failing because they haven’t found the right formula to transform metal into gold, but because the task they have set out to do cannot be done. You urge them to abandon their efforts and focus on other areas, but they don’t seem to listen. Instead, their response: “well sure we haven’t been able to turn lead into gold yet, but can you do any better? I don’t see any gold in your hands either.”
You can probably already see where I’m going with this metaphor. Replace chemist with economist and turning lead into gold with DSGE macroeconomics and you’ll have a good sense of what criticizing macro feels like. I don’t think it’s an exaggeration to say that every critique of macro is invariably going to be met by some form of the same counterargument. Of course the model isn’t perfect, but we’re doing our best. We’re continually adding the features to the benchmark model that you claim we are missing. Heterogeneity, financial markets, even behavioral assumptions. They’re coming. And if you’re so smart, come up with something better. It takes a model to beat a model.
I won’t deny there is some truth to this argument. Just because a model is unrealistic, just because it’s missing some feature of reality, doesn’t mean it isn’t useful. It doesn’t mean it isn’t a reasonable first step on the path to something better. The financial crisis didn’t prove that the methods of macroeconomics are wrong. Claims that nobody in the macro profession is asking interesting questions or trying to implement interesting ideas are demonstrably false. We certainly don’t want criticisms of macro to lead to less study of the topic. The questions are too important. The potential gains from solving problems like business cycles or economic growth too great. And if we don’t have anything better, why not keep pushing forward? Why not take the DSGE apparatus as far as we possibly can?
But what I, and many others, have tried to show, is that the methods of macroeconomics are severely constrained by assumptions with questionable theoretical or empirical backing. The foundation that modern macroeconomics is building on is too shaky to support the kinds of improvements that we hope it will eventually make. Now, you can certainly argue that I am wrong and that DSGE models are perfectly capable of answering the questions we ask. That’s quite possible. But if I am not wrong about the flaws in the method, then we shouldn’t need a new model to think about giving up on the current one. When somebody points out a flaw in the foundation of a building, the proper response is not to keep building until they come up with a better solution. It’s to knock the building down and focus all effort on finding that solution.
I can confidently say that if a better alternative to DSGE exists, I will not be the one to develop it. I am nowhere near smart or creative enough to do that. I don’t think any one person is. What I have been trying to do is convince others that it’s worth devoting a little bit more of our research efforts to exploring other methods and to challenge the fundamental assumptions that have held a near monopoly on macroeconomic research for the last 40 years. The more people that focus on finding a model to beat the current model, the better chance we have to actually find one. As economists, we should at the very least be open to the idea that competition is good.
Where do we start? I think agent based simulation models offer one potential path. The key benefit of moving toward a simulation model over a mathematical one is that concerns about tractability are much less pressing. Many of the most concerning assumptions of standard macro models are made because without them the model becomes unsolvable. With an agent based model, it is much easier to incorporate features like heterogeneity and diverse behavioral assumptions and just let the simulation sort out what happens. Equilibrium in an agent based model is not an assumption, but an emergent result. The downside is that an ABM cannot produce nice closed form analytical solutions. But in a world as complex as ours I think restricting ourselves to only being able to answer questions that allow for a closed form solution is a pretty bad idea – it’s looking for your keys under the streetlight because that’s where the light is.
Maybe even more important than developing specific models to challenge the DSGE benchmark is to try to introduce a little more humility into the modeling process. As I’ve discussed before, there isn’t much of a reason to put any stock in quantitative predictions of models that we know bear little resemblance to reality. Estimating the effects of policy to a decimal point is just not something we are capable of doing right now. Let’s stop pretending we can.
A day may come when the old guard of macroeconomics convinces this starry eyed graduate student to give up his long battle against the evils of DSGE models in macroeconomics. But it is not this day.
Shockingly, it seems like many top economists have not yet discovered my superior critique of macroeconomics (because obviously if they had they would be convinced to stop defending DSGE). Instead, we get statements like:
Macroeconomic policy questions involve trade-offs between competing forces in the economy. The problem is how to assess the strength of those forces for the particular policy question at hand. One strategy is to perform experiments on actual economies. Unfortunately, this strategy is not available to social scientists. The only place that we can do experiments is in dynamic stochastic general equilibrium (DSGE) models
That’s from the recent paper “On DSGE Models” written by three prominent DSGE modelers, Lawrence Christiano, Martin Eichenbaum, and Mathias Trabandt (who I’ll call CET). As you might suspect, I disagree. And I expect their defense will be about as effective at shifting the debate as my critique was (and since my readership can be safely rounded down to 0, that’s not a compliment). Unlike Olivier Blanchard’s recent thoughts on DSGE models, which conceded that many of the criticisms of DSGE models actually contained some truth, CET leave no room for alternatives. It’s DSGE or bust and “people who don’t like dynamic stochastic general equilibrium (DSGE) models are dilettantes.” So we’re off to a good start.
But let’s avoid the ad hominem as much as we can and get to the economics. Beyond the obviously false statement that DSGE models are the only models where we can do experiments, CET don’t offer much we haven’t heard before. Their story is by now pretty standard. They begin by admitting that yes, of course RBC models with their emphasis on technology shocks, complete markets, and policy ineffectiveness were woefully inadequate. But we’re better now! Macroeconomics has come a long way in the last 35 years! They then proceed to provide answers to the common criticisms of DSGE modeling.
Worried DSGE models don’t include a role for finance? Clearly you’ve never heard of Carlstrom and Fuerst (1997) or Bernanke et al. (1999) which include financial accelerator effects. Maybe you’re more concerned about shadow banking? Gertler and Kiyotaki (2015) have you covered. Zero lower bound? Please, Krugman had that one wrapped up all the way back in 1998. Want a role for government spending? Monetary policy? Here’s 20 models that give you the results you want.
Essentially, CET try to take everything that critics of DSGE models say is missing and show that actually many researchers do include these features. This strategy is common in any rebuttal to attacks on DSGE models. Every time somebody points out a flaw in one class of models (representative agent models, rational expectations models, models that use HP-filtered data, complete markets, etc.) they point to another group of models that purports to solve these problems. In doing so they miss the point of these critiques entirely.
The problem with DSGE models is not that they are unable to explain specific economic phenomenon. The problem is that they can explain almost any economic phenomenon you can possibly imagine and we have essentially no way to decide which models are better or worse than others except by comparing them to data that they were explicitly designed to match. It’s true there were models written before the recession that contained features that looked a lot like those in the crisis. We just had no reason to look at those models over the hundreds of other ones that had entirely different implications.
Whatever idea you can dream up, you can almost be sure that somebody has written a DSGE model to capture it. Too much of what DSGE models end up being is mathematical justifications for ideas people have already worked out intuitively in their minds (often stripped of much of the nuance that made the idea interesting in the first place). All the DSGE model itself adds is a set of assumptions everybody knows are false that generate those intuitive results. CET do nothing to address this criticism.
Take CET’s defense of representative agent models. They say “It has been known for decades that restrictions like (1)[the standard Euler equation] can be rejected, even in representative agent models that allow for habit formation. So, why would anyone ever use the representative agent assumption? In practice analysts have used that assumption because they think that for many questions they get roughly the right answer.”
Interesting. If they already knew the right answer, what was the model for again? Everybody agrees the assumptions are completely bogus, but it gets the result we wanted so who cares? That’s really the argument they want to make here?
But this is the game we play. Despite CET’s claims to the contrary, I am almost certain that most macroeconomics papers begin with the result. Once they know what they want to prove, it becomes a matter of finagling a model that sounds somewhat like it could be related to how an actual economy works and produces the desired result (and when this task can’t be done, it becomes a “puzzle”). The recession clearly demonstrated the importance of finance on the economy? Simple. Let’s write a DSGE model where finance is important. If in the end the model actually shows that finance is unimportant rewrite it until you get the answer you want.
Almost every DSGE macroeconomics paper follows pretty much the same outline. First, they present some stylized facts from macroeconomic data. Next, they review the current literature and explain why it is unable to fit those facts. Then they introduce their new model with slightly different assumptions that can fit the facts and brag about how well the model (that they designed specifically to fit the facts) actually fits the facts. Finally they do “experiments” using their model to show how different policies could have changed economic outcomes.
In my view, macroeconomics should be exactly the opposite. Don’t bother trying to exactly match macroeconomic aggregates for the United States economy with a model that looks nothing like the United States economy. Have a little more humility. Instead, start by getting the assumptions right. Since we will never be able to capture all of the intricacies of a true economy, the model economy should look very different from a real economy. However, if the assumptions that generate that economy are realistic, it might still provide answers that are relevant for the real world. A model that gets the facts right but the assumptions wrong probably does not.
I spent 15 posts arguing that the DSGE paradigm gets the assumptions spectacularly wrong. CET provide many examples of people using these flawed assumptions to try to give us answers to many interesting questions. They do not, however, provide any reason for us to believe those answers.
But, then again, I am but a dilettante, so you probably shouldn’t believe me either.
Trump’s recently proposed plan to cut corporate taxes has opened a debate about whether corporate tax cuts are good for workers. Opponents of the plan argue that it will only help corporations increase their profit while supporters believe a large portion of the benefits will accrue to workers through increased wages. I don’t want to comment on that debate. Instead, I want to discuss a point made by John Cochrane in his attempt to prove that a lower corporate tax can increase wages (responding to a post by Greg Mankiw). You can find his post here. Please read it before continuing.
Cochrane shows in a simple model that a decrease in taxes will cause an increase in wages of where is the current tax rate. In other words, if the tax rate is 1/3, a decrease in taxes of one dollar increases wages by $1.50. Cochrane then says something that I find incredibly misleading:
“This is also a lovely little example for people who decry math in economics. At a verbal level, who knows? It seems plausible that a $1 tax cut could never raise wages by more than $1. Your head swims. A few lines of algebra later, and the argument is clear. You could never do this verbally.”
There are two issues with Cochrane’s statement. The first is that it is pretty easy to prove that a $1 tax cut could raise wages by more than $1. Assume the only two inputs are labor and capital and profits are zero. Assume the rental price of capital is fixed. Any change in taxes must therefore cause a change in wages. If production doesn’t change, this change has to be the exact amount of the tax (otherwise profit would change). Now assume that the tax caused some deadweight loss so lowering it will also increase production. Again wages increase so they must have increased more than one for one with the increase in tax.
Now you might say those assumptions are a bit ridiculous and I would agree. But Cochrane actually used the exact same assumptions (and more). He just hid them behind some math. And that brings me to the second problem with Cochrane’s statement. A few lines of algebra later, the argument is actually not clear at all unless you already know what’s going on (even Greg Mankiw admits he doesn’t have intuition for the result in the post that Cochrane is expanding on).
Let’s take Cochrane’s “proof” piece by piece and outline the assumptions he needed to get his result.
He writes: the production technology is
To just write this line, we need three strong assumptions
Assumption 1: There are only two productive inputs, labor and capital
Assumption 3: The aggregate production function exhibits constant returns to scale (multiplying each of its inputs by some factor also multiplies its output by that factor)
The last assumption is necessary to write the function in its form and also ensures that firms have zero profit.
Next we have that firms maximize
Again, we are implicitly making more assumptions here
Assumption 4: Firms maximize profits every period (the setup of the problem guarantees that this behavior also maximizes lifetime profits, but another model might not have that property)
Assumption 5: All workers get paid the same wage, which is taken as given by individual firms (i.e. labor markets are perfectly competitive)
Assumption 6: The rental rate of capital is exogenously set. Mankiw set up the problem as a small open economy so that the interest rate (the price of capital) is constant. Note that the US is obviously not a small open economy.
Continuing, the firm’s first order conditions are
Again, more assumptions
Assumption 7: Workers get paid their marginal product (technically this one follows from 4 and 5 above so maybe I shouldn’t count it).
Assumption 8: Firms know their production function as well as the marginal products of labor and capital.
Assumption 9: Wages are fully flexible and can be changed at any time.
Assumption 10: Capital can move costlessly between countries.
I’ll stop there but I’m sure there are plenty more (including the assumptions of no involuntary unemployment, no money of any kind, and that the economy is always in equilibrium – assumptions common to many macro models). My point in doing this exercise is to demonstrate that in order to even begin to write an economic model using math you need to make strong assumptions. Without them the problem quickly becomes either impossible to solve or impossible to interpret. By hiding these assumptions (either intentionally or not) behind fancy equations, they often go unnoticed.
Nobody that criticizes math in economics is literally criticizing the use of algebra or calculus to provide intuition about an economic result. What we criticize are the restrictions that using math places on the economic problem. Mises described math in economics as a “vicious method, starting from false assumptions and leading to fallacious inferences. Its syllogisms are not only sterile; they divert the mind from the study of the real problems and distort the relations between the various phenomena.”
I can’t help but think that diverting the mind from the real problem is exactly what’s happening here. The corporate tax debate is really about the incidence of the tax. Do workers bear most of the burden, or does it primarily serve to prevent monopoly profits and rents? Cochrane’s example avoids this question by assumption. Rather than being a nail in the coffin for people who want less math in economics, it serves as a perfect example of why those criticisms exist. In some ways math provides clarity over verbal reasoning, but it can also be deceiving. Behind the formal logic and the proofs is a fragile set of assumptions that in many cases drive the results.
P.S. I don’t usually agree with Paul Krugman but I think he gets this one right in this post. He also shows which assumptions are driving the result, and that they are not ones that make much sense.
P.P.P.S. Casey Mulligan claims Krugman and Summers still get it wrong. I haven’t fully wrapped my head around his argument. What was Cochrane saying about algebra making everything clear?
P.P.P.P.S. I’m still in favor of cutting the corporate tax precisely because it is so hard to determine the incidence. Even if we want to stop monopoly profits (and I’m not sure that we do), it seems better to me to just focus on preventing monopolies.
In macroeconomics, research almost always follows a similar pattern. First, the economist comes up with a question. Maybe they look at data and generate some stylized facts about some aspect of the economy. Then they set out to “explain” these facts using a structural economic model (I put explain in quotes because this step usually involves stripping away everything that made the question interesting in the first place). Using their model, they can then make some predictions or do some policy analysis. Finally, they write a paper describing their model and its implications.
There is nothing inherently wrong with this approach to research. But there are some issues. The first is that every paper looks exactly the same. Every paper needs a model. Sometimes papers adapt existing models, but they need enough difference to be a contribution on their own (but not so much difference that you leave the narrow consensus of modern macroeconomic methodology). Rarely, if ever, is there any attempt to compare models, to evaluate their failures and successes. It’s always: previous papers missed this and that feature while mine includes it.
This kind of iterative modeling can give the illusion of progress, but it really just represents sideways movement. The questions of macroeconomics haven’t really changed much in the last 100 years. What we have done is develop more and more answers to those questions without really making any progress on figuring out which of those answers is actually correct. Thousands of answers to a question is in many ways no better than none at all.
I don’t think it’s too much of a mystery why macroeconomics looks this way. Everybody already knows how an evaluation of our current answers to macroeconomic questions would go. The findings: we don’t know anything and all our models stink. I’d be surprised if even 10% of economists would honestly suggest a policymaker to carry out the policy that their papers suggest.
Academics still need to publish of course so they change the criteria that describes good macroeconomic research. Rarely is a paper evaluated on how well it answers an economic question. Instead, what matters is the tool used to answer the question. An empirical contribution without a model will get yawns in a macro seminar. A new mathematical contribution that uses a differential equations derived from a heat diffusion equation from physics? Mouths will be watering.
The claim is that these tools can then be used by other researchers as we continue to get closer and closer to the truth. The reality is that they are used by other researchers, but they only use the tools to develop their own slightly “better” tool in their own paper. In other words, the primary consumer of economic papers is economists who want to write papers. Widely cited papers are seen as better. Why? Because they helped a bunch of other people write their own papers? When does any of this research start to actually be helpful to people who aren’t responsible for creating it? Should we measure the quality of beef by how many cows it can feed?
Again there is an easy explanation for why economists are the only ones who can read economics papers. They would be completely unintelligible to anybody else. Reading and understanding the mechanism behind a macroeconomic paper is often a herculean task even for a trained economist. A non-expert has no chance. There could be good reasons for this complexity. I don’t expect to be able to open a journal on quantum mechanics and get anything out of it. But there is one enormous difference between physics and economics models. The physics ones actually work.
Economics didn’t always look this way. Read a paper by Milton Friedman or Armen Alchian. Almost no equations, much less the giant dynamic systems in models today. Does anybody think modern economic analysis is better than the kind done by those two?
The criticism of doing economics in words rather than math is that it is harder to be internally consistent. An equation has fewer interpretations than a sentence. I’m sympathetic to that argument. But I think there are better ways to add transparency to economics than by writing everything out in math that requires 20 years of school to understand. There are ways to formalize arguments other than systems of equations, ways to explain the mechanism that generates the data other than structural DSGE models.
The problem with purely verbal arguments is that you can easily lose your train of logic. Each sentence can make sense on its own but completely contradict another piece of the argument. Simultaneous systems of equations can prevent this kind of mistake. They are just one way. Computer simulations can provide the same discipline. Let’s say I have some theory about the way the world works. If I can design a computer simulation that replicates the kind of behavior I described in words, doesn’t that prove that my argument is logically consistent? It of course doesn’t mean I am right, but neither does a mathematical model. Each provides a complete framework that an outside observer can evaluate and decide whether its assumptions provide a useful view of the world.
The rest of the profession doesn’t seem to agree with me that a computer simulation and a system of equations serve the same purpose. I’m not exactly sure why. One potential worry is that it’s harder to figure out what’s actually happening in a computer simulation. With a system of equations I can see exactly which variables affect others and the precise channel of each kind of change. In a simulation, outcomes are emergent. Maybe I develop a simulation where an increase in taxes causes output to fall. Simply looking at the rules I have given each agent of how to act might not tell me why that fall occurred. It might be some complex interaction between these agents that generates that result.
That argument makes sense, but I think it only justifies keeping mathematical models rather than throwing out computer models. They serve different purposes. And computer models have their own advantages. One, which has yet to be explored in any serious way, is the potential for visual results. Imagine that the final result of an economic paper was not a long list of greek symbols and equals signs, but rather a full moving mini economic world. Agents move around, trade with each other. Firms set prices, open and close. Output and unemployment rise and fall. A simple version of such a model is the “sugarscape” model of Axtell and Epstein which creates a simple world where agents search for and trade sugar in order to survive.
Now imagine a much more complicated version of that that looked a lot more like a real economy. Rather than being able to “see” the relationships between variables in an equation, I could literally see how agents act and interact visually. My ideal world of economic research would not be writing papers, but creating apps. I want to download your model on my computer and play with it. Change the parameter values, apply different shocks, change the number and types of agents. And then observe what happens. Will this actually tell us anything useful about the economy? I’m not sure. But I think it’s worth a try. (I’m currently trying to do it myself. Hopefully I can post a version of it here soon)
Part 14 in a series of posts on modern macroeconomics. This post concludes my criticisms of the current state of macroeconomic research by tying all of my previous posts to Hayek’s warning about the pretense of knowledge in economics
Russ Roberts, host of the excellent EconTalk podcast, likes to say that you know macroeconomists have a sense of humor because they use decimal points. His implication is that for an economist to believe that they can predict growth or inflation or any other variable down to a tenth of a percentage point is absolutely ridiculous. And yet economists do it all the time. Whether it’s the CBO analysis of a policy change or the counterfactuals of an academic macroeconomics paper, quantitative analysis has become an almost necessary component of economic research. There’s nothing inherently wrong with this philosophy. Making accurate quantitative predictions would be an incredibly useful role for economics.
Except for one little problem. We’re absolutely terrible at it. Prediction in economics is really hard and we are nowhere near the level of understanding that would allow for our models to deliver accurate quantitative results. Everybody in the profession seems to realize this fact, which means nobody believes many of the results their theories produce.
Let me give two examples. The first, which I have already mentioned in previous posts, is Lucas’s result that the cost of business cycles in theoretical models is tiny – around one tenth of one percent of welfare loss compared to an economy with no fluctuations. Another is the well known puzzle in international economics that trade models have great difficulty producing large gains from trade. A seminal paper by Arkolakis, Costinot, and Rodriguez-Clare evaluates how our understanding of the gains from trade have improved from the theoretical advances of the last 10 years. Their conclusion: “so far, not much.”
How large are the costs of business cycles? Essentially zero. How big are the gains from trade? Too small to matter much. That’s what our theories tell us. Anybody who has lived in this world the last 30 years should be able to immediately take away some useful information from these results: the theories stink. We just lived through the devastation of a large recession. We’ve seen globalization lift millions out of poverty. Nobody believes for a second that business cycles and trade don’t matter. In fact, I’m not sure anybody takes the quantitative results of any macro paper seriously. For some reason we keep writing them anyway.
In the speech that provided the inspiration for the title of this blog, Hayek warned economists of the dangers of the pretense of knowledge. Unlike the physical sciences, where controlled laboratory experiments are possible and therefore quantitative predictions might be feasible, economics is a complex science. He argues,
The difficulties which we encounter in [complex sciences] are not, as one might at first suspect, difficulties about formulating theories for the explanation of the observed events – although they cause also special difficulties about testing proposed explanations and therefore about eliminating bad theories. They are due to the chief problem which arises when we apply our theories to any particular situation in the real world. A theory of essentially complex phenomena must refer to a large number of particular facts; and to derive a prediction from it, or to test it, we have to ascertain all these particular facts. Once we succeeded in this there should be no particular difficulty about deriving testable predictions – with the help of modern computers it should be easy enough to insert these data into the appropriate blanks of the theoretical formulae and to derive a prediction. The real difficulty, to the solution of which science has little to contribute, and which is sometimes indeed insoluble, consists in the ascertainment of the particular facts
Hayek (1989) – The Pretence of Knowledge
In other words, it’s not that developing models to explain economic phenomena is especially challenging, but rather that there is no way to collect sufficient information to apply any theory quantitatively. Preferences, expectations, technology. Any good macroeconomic theory would need to include each of these features, but each is almost impossible to measure.
Instead of admitting ignorance, we make assumptions. Preferences are all the same. Expectations are all rational. Production technologies take only a few inputs and outputs. Fluctuations are driven by a single abstract technology shock. Everyone recognizes that any realistic representation of these features would require knowledge far beyond what is available to any single mind. Hayek saw that this difficulty placed clear restrictions on what economists could do. We can admit that every model needs simplification while also remembering that those simplifications constrain the ability of the model to connect to reality. The default position of modern macroeconomics instead seems to be to pretend the constraints don’t exist.
Many economists would probably agree with many of the points I have made in this series, but it seems that most believe the issues have already been solved. There are a lot of models out there and some of them do attempt to deal directly with some of the problems I have identified. There are models that try to reduce the importance of TFP as a driver of business cycles. There are models that don’t use the HP-Filter, models that have heterogeneous agents, models that introduce financial frictions and other realistic features absent from the baseline models. For any flaw in one model, there is almost certainly another model that attempts to solve it. But in solving that single problem they likely introduce about ten more. Other papers will deal with those problems, but maybe they forget about the original problem. For each problem that arises, we just introduce a new model. And then we take those issues as solved even though they are solved by a set of models that potentially produce conflicting results and with no real way to differentiate which is more useful.
Almost every criticism I have written about in the last 13 posts of this series can be traced back to the same source. Macroeconomists try to do too much. They haven’t heeded Hayek’s plea for humility. Despite incredible simplifying assumptions, they take their models to real data and attempt to make predictions about a world that bears only a superficial resemblance to the model used to represent it. Trying to answer the big questions about macroeconomics with such a limited toolset is like trying to build a skyscraper with only a hammer.
Part 13 in a series of posts on modern macroeconomics. Previous posts in this series have pointed out many problems with DSGE models. This post aims to show that these problems have either been (in my opinion wrongly) dismissed or ignored by most of the profession.
“If you have an interesting and coherent story to tell, you can tell it in a DSGE model. If you cannot, your story is incoherent.”
The above quote comes from a testimony given by prominent Minnesota macroeconomist V.V. Chari to the House of Representatives in 2010 as they attempted to determine how economists could have missed an event as large as the Great Recession. Chari argues that although macroeconomics does have room to improve, it has made substantial progress in the last 30 years and there is nothing fundamentally wrong with its current path.
Of course, not everybody has been so kind to macroeconomics. Even before the crisis, prominent macroeconomists had begun voicing some concerns about the current state of macroeconomic research. Here’s Robert Solow in 2003:
The original impulse to look for better or more explicit micro foundations was probably reasonable. It overlooked the fact that macroeconomics as practiced by Keynes and Pigou was full of informal microfoundations. (I mention Pigou to disabuse everyone of the notion that this is some specifically Keynesian thing.) Generalizations about aggregative consumption-saving patterns, investment patterns, money-holding patterns were always rationalized by plausible statements about individual–and, to some extent, market–behavior. But some formalization of the connection was a good idea. What emerged was not a good idea. The preferred model has a single representative consumer optimizing over infinite time with perfect foresight or rational expectations, in an environment that realizes the resulting plans more or less flawlessly through perfectly competitive forward-looking markets for goods and labor, and perfectly flexible prices and wages.
There is, however, such a thing as too much convergence. To caricature, but only slightly: A macroeconomic article today often follows strict, haiku-like, rules: It starts from a general equilibrium structure, in which individuals maximize the expected present value of utility, firms maximize their value, and markets clear. Then, it introduces a twist, be it an imperfection or the closing of a particular set of markets, and works out the general equilibrium implications. It then performs a numerical simulation, based on calibration, showing that the model performs well. It ends with a welfare assessment.
Such articles can be great, and the best ones indeed are. But, more often than not, they suffer from some of the flaws I just discussed in the context of DSGEs: Introduction of an additional ingredient in a benchmark model already loaded with questionable assumptions. And little or no independent validation for the added ingredient. Olivier Blanchard (2008) – The State of Macro
And more recently, the vitriolic critique of Paul Romer:
Would you want your child to be treated by a doctor who is more committed to his friend the anti-vaxer and his other friend the homeopath than to medical science? If not, why should you expect that people who want answers will keep paying attention to economists after they learn that we are more committed to friends than facts. Romer (2016) – The Trouble with Macroeconomics
These aren’t empty critiques by no-name graduate student bloggers. Robert Solow and Paul Romer are two of the biggest names in growth theory in the last 50 years. Olivier Blanchard was the chief economist at the IMF. One would expect that these criticisms would cause the profession to at least strongly consider a re-evaluation of its methods. Looking at the recent trends in the literature as well as my own experience doing macroeconomic research, it hasn’t. Not even close. Instead, it seems to have doubled down on the DSGE paradigm, falling much closer to Chari’s point of view that “there is no other game in town.”
But it’s even worse than that. Taken at its broadest, sticking to DSGE models is not too restrictive. Dynamic simply means that the models have a forward looking component, which is obviously an important feature to include in a macroeconomic model. Stochastic means there should be some randomness, which again is probably a useful feature (although I do think deterministic models can be helpful as well – more on this later). General equilibrium is a little harder to swallow, but it still provides a good measure of flexibility.
Even within the DSGE framework, however, straying too far from the accepted doctrines of macroeconomics is out of the question. Want to write a paper that deviates from rational expectations? You better have a really good reason. Never mind that survey and experimental evidence shows large differences in how people form expectations, or the questionable assumption that everybody in the model knows the model and takes it as truth, or that the founder of rational expectations John Muth later said:
It is a little surprising that serious alternatives to rational expectations have never been proposed. My original paper was largely a reaction against very naive expectations hypotheses juxtaposed with highly rational decision-making behavior and seems to have been rather widely misinterpreted.
Quoted in Hoover (2013) – Rational Expectations: Retrospect and Prospect
Using an incredibly strong assumption like rational expectations is accepted without question. Any deviation requires explanation. Why? As far as I can tell, it’s just Kevin Malone economics: that’s the way it has always been done and nobody has any incentive to change it. And so researchers all get funneled into making tiny changes to existing frameworks – anything truly new is actively discouraged.
Now, of course, there are reasons to be wary of change. It would be a waste to completely ignore the path that brought us to this point and, despite their flaws, I’m sure there have been some important insights generated by the DSGE research program (although to be honest I’m having trouble thinking of any). Maybe it really is the best we can do. But wouldn’t it at least be worth devoting some time to alternatives? I would estimate that 99.99% of theoretical business cycle papers published in top 5 journals are based around DSGE models. Chari wasn’t exaggerating when he said there is no other game in town.
Wouldn’t it be nice if there were? Is there any reason other than tradition that every single macroeconomics paper needs to follow the exact same structure, to draw from the same set restrictive set of assumptions? If even 10% of the time and resources devoted to tweaking existing models was instead spent on coming up with entirely new ways of doing macroeconomic research I have no doubt that a viable alternative could be found. And there already are some (again, more on this later), but they exist only on the fringes of the profession. I think it’s about time for that to change.
Part 12 in a series of posts on modern macroeconomics. This post draws inspiration from Axel Leijonhufvud’s distinction between models and theories in order to argue that the current procedure of performing macroeconomic research is flawed.
In a short 1997 article, Axel Leijonhufvud gives an interesting argument that economists too often fail to differentiate between models and theories. In his words,
For many years now, ‘model’ and ‘theory’ have been widely used as interchangeable terms in the profession. I want to plead the case that there is a useful distinction to be made. I propose to conceive of economic ‘theories’ as sets of beliefs about the economy and how it functions. They refer to the ‘real world’ – that curious expression that economists use when it occurs to them that there is one. ‘Models’ are formal but partial representations of theories. A model never encompasses the entire theory to which it refers.
Leijonhufvud (1997) – Models and Theories
The whole article is worth reading and it highlights a major problem with macroeconomic research: we don’t have a good way to determine when a theory is wrong. Standard economic theory has placed a large emphasis on models. Focusing on mathematical models ensures that assumptions are laid out clearly and that the implications of those assumptions are internally consistent. However, this discipline also constrains models to be relatively simple. I might have some grand theory of how the economy works, but even if my theory is entirely accurate, there is little chance that it can be converted into a tractable set of mathematical equations.
The result of this discrepancy between model and theory makes it very difficult to judge the success or failure of a theory based on the success or failure of a model. As discussed in an earlier post, we begin from the premise that all models are wrong and therefore it shouldn’t be at all surprising when a model fails to match some feature of reality. When a model is disproven along some margin, there are two paths a researcher can take: reject the theory or modify the assumptions of the model to better fit the theory. Again quoting Leijonhufvud,
When a model fails, the conclusion nearest to hand is that some simplifying assumption or choice of empirical proxy-variable is to blame, not that one’s beliefs about the way the world works are wrong. So one looks around for a modified model that will be a better vehicle for the same theory.
Leijonhufvud (1997) – Models and Theories
A good example of this phenomenon comes from looking at the history of Keynesian theory over the last 50 years. When the Lucas critique essentially dismantled the Keynesian research program in the 1980s, economists who believed in Keynesian insights could have rejected that theory of the world. Instead, they adopted neoclassical models but tweaked them in such a way that produced results that were still consistent with Keynesian theory. New Keynesian models today include rational expectations and microfoundations. Many hours have been spent converting Keynes’s theory into fancy mathematical language, adding (sometimes questionable) frictions into the neoclassical framework to coax it into giving Keynesian results. But how does that help us understand the world? Have any of these advances increased our understanding of economics beyond what Keynes already knew 80 years ago? I have my doubts.
Macroeconomic research today works on the assumption that any good theory can be written as a mathematical model. Within that class of theories, the gold standard are those that are tractable enough to give analytical solutions. Restricting the set of acceptable models in this way probably helps throw out a lot of bad theories. What I worry is that it also precludes many good ones. By imposing such tight constraints on what a good macroeconomic theory is supposed to look like we also severely constrain our vision of a theoretical economy. Almost every macroeconomic paper draws from a tiny selection of functional forms for preferences and production because any deviation quickly adds too much complexity and leads to systems of equations that are impossible to solve.
But besides restricting the set of theories, speaking in models also often makes it difficult to understand the broader logic that drove the creation of the model. As Leijonhufvud puts it, “formalism in economics makes it possible to know precisely what someone is saying. But it often leaves us in doubt about what exactly he or she is talking about.” Macroeconomics has become very good at laying out a set of assumptions and deriving the implications of those assumptions. Where it often fails is in describing how we can compare that simplified world to the complex one we live in. What is the theory that underlies the assumptions that create an economic model? Where do the model and the theory differ and how might that affect the results? These questions seem to me to be incredibly important for conducting sound economic research. They are almost never asked.
Part 11 in a series of posts on modern macroeconomics. The discussion to this point has focused on the early contributions to DSGE modeling by Kydland and Prescott. Obviously, in the last 30 years, economic research has advanced beyond their work. In this post I will argue that it has advanced in the wrong direction.
In Part 3 of this series I outlined the Real Business Cycle (RBC) model that began macroeconomics on its current path. The rest of the posts in this series have offered a variety of reasons why the RBC framework fails to provide an adequate model of the economy. An easy response to this criticism is that it was never intended to be a final answer to macroeconomic modeling. Even if Kydland and Prescott’s model misses important features of a real economy doesn’t it still provide a foundation to build upon? Since the 1980s, the primary agenda of macroeconomic research has been to build on this base, to relax some of the more unrealistic assumptions that drove the original RBC model and to add new assumptions to better match economic patterns seen in the data.
New Keynesian Models
Part of the appeal of the RBC model was its simplicity. Driving business cycles was a single shock: exogenous changes in TFP attributed to changes in technology. However, the real world is anything but simple. In many ways, the advances in the methods used by macroeconomists in the 1980s came only at the expense of a rich understanding of the complex interactions of a market economy. Compared to Keynes’s analysis 50 years earlier, the DSGE models of the 1980s seem a bit limited.
What Keynes lacked was clarity. 80 years after the publication of the General Theory, economists still debate “what Keynes really meant.” For all of their faults, DSGE models at the very least guarantee clear exposition (assuming you understand the math) and internal consistency. New Keynesian (NK) models attempt to combine these advantages with some of the insights of Keynes.
The simplest distinction between an RBC model and an NK model is the assumptions made about the flexibility of prices. In early DSGE models, prices (including wages) were assumed to be fully flexible, constantly adjusting in order to ensure that all markets cleared. The problem (or benefit depending on your point of view) with perfectly flexible prices is that it makes it difficult for policy to have any impact. Combining rational expectations with flexible prices means monetary shocks cannot have any real effects. Prices and wages simply adjust immediately to return to the same allocation of real output.
To reintroduce a role for policy, New Keynesians instead assume that prices are sticky. Now when a monetary shock hits the economy firms are unable to change their price so they respond to the higher demand by increasing real output. As research in the New Keynesian program developed, more and more was added to the simple RBC base. The first RBC model was entirely frictionless and included only one shock (to TFP). It could be summarized in just a few equations. A more recent macro model, The widely cited Smets and Wouters (2007), adds frictions like investment adjustment costs, variable capital utilization, habit formation, and inflation indexing in addition to 7 structural shocks, leading to dozens of equilibrium equations.
Building on a Broken Foundation
New Keynesian economists have their heart in the right place. It’s absolutely true that the RBC model produces results at odds with much of reality. Attempting to move the model closer to the data is certainly an admirable goal. But, as I have argued in the first 10 posts of this series, the RBC model fails not so much because it abstracts from the features we see in real economies, but because the assumptions it makes are not well suited to explain economic interactions in the first place.
One potential improvement of NK models over their RBC counterparts is less reliance on (potentially misspecified) technology shocks as the primary driver of business cycle fluctuations. The “improvement,” however, comes only by introducing other equally suspect shocks. A paper by V.V. Chari, Patrick Kehoe, and Ellen McGrattan argues that four of the “structural shocks” in the Smets-Wouters setup do not have a clear interpretation. In other words, although the shocks can help the model match the data, the reason why they are able to do so is not entirely clear. For example, one of the shocks in the model is a wage markup over the marginal productivity of a worker. The authors argue that this could be caused either by increased bargaining power (through unions, etc.), which would call for government intervention to break up unions, or through an increased preference for leisure, which is an efficient outcome and requires no policy. The Smets-Wouters model can say nothing about which interpretation is more accurate and is therefore unhelpful in prescribing policy to improve the economy.
I mentioned above that the main goal of the New Keynesian model was to revisit the features Keynes talked about decades ago using modern methods. In today’s macro landscape, microfoundations have become essential. Keynes spoke in terms of broad aggregates, but without understanding how those aggregates are derived from optimizing behavior at a micro level they fall into the Lucas critique. But although the NK model attempts to put in more realistic microfoundations, many of its assumptions seem at odds with the ways in which individuals actually act.
Take the assumption of sticky prices. It’s probably a valid assumption to include in an economic model. With no frictions, an optimizing firm would want to change its price every time new economic data became available. Obviously that doesn’t happen as many goods stay at a constant price for months or years. So of course if we want to match the data we are going to need some reason for firms to hold prices constant. The most common way to achieve this result is called “Calvo Pricing” in honor of a paper by Guillermo Calvo. And how do we ensure that some prices in the economy remain sticky? We simply assign each firm some probability that they will be unable to change their price. In other words, we assume exactly the result we want to get.
Of course, economists have recognized that Calvo pricing isn’t really a satisfactory final answer, so they have worked to microfound Calvo’s idea through an optimization problem. The most common method, proposed by Rotemberg in 1982, is called the menu cost model. In this model of price setting, a firm must pay some cost every time it changes its prices. This cost ensures that a firm will only change its price when it is especially profitable to do so. Small price changes every day no longer make sense. The term menu cost comes from the example of a firm needing to print out new menus every time it changes one of its prices, but it can be applied more broadly to encompass every cost a firm might incur by changing a price. More specifically, price changes could have an adverse effect on consumers, causing some to leave the store. It could require more advanced computer systems, more complex interactions with suppliers, competitive games between firms.
But by wrapping all of these into one “menu cost,” am I really describing a firm’s problem. A “true” microfoundation would explicitly model each of these features at a firm level with different speeds of price adjustment and different reasons causing firms to hold prices steady. What do we gain by adding optimization to our models if that optimization has little to do with the problem a real firm would face? Are we any better off with “fake” microfoundations than we were with statistical aggregates. I can’t help but think that the “microfoundations” in modern macro models are simply fancy ways of finagling optimization problems until we get the result we want. Interesting mathematical exercise? Maybe. Improving our understanding of the economy? Unclear.
Other additions to the NK model that purport to move the model closer to reality are also suspect. As Olivier Blanchard describes in his 2008 essay, “The State of Macro:”
Reconciling the theory with the data has led to a lot of un- convincing reverse engineering. External habit formation—that is a specification of utility where utility depends not on consumption, but on consumption relative to lagged aggregate consumption—has been introduced to explain the slow adjustment of consumption. Convex costs of changing investment, rather than the more standard and more plausible convex costs of investment, have been introduced to explain the rich dynamics of investment. Backward indexation of prices, an assumption which, as far as I know, is simply factually wrong, has been introduced to explain the dynamics of inflation. And, because, once they are introduced, these assumptions can then be blamed on others, they have often become standard, passed on from model to model with little discussion
In other words, we want to move the model closer to the data, but we do so by offering features that bear little resemblance to actual human behavior. And if the models we write do not actually describe individual behavior at a micro level, can we really still call them “microfounded”?
And They Still Don’t Match the Data
In Part 10 of this series, I discussed the idea that we might not want our models to match the data. That view is not shared by most of the rest of the profession, however, so let’s judge these models on their own terms. Maybe the microfoundations are unrealistic and the shocks misspecified, but, as Friedman argued, who cares about assumptions as long as we get a decent result? New Keynesian models also fail in this regard.
Forecasting in general is very difficult for any economic model, but one would expect that after 40 years of toying with these models, getting closer and closer to an accurate explanation of macroeconomic phenomena, we would be able to make somewhat decent predictions. What good is explaining the past if it can’t help inform our understanding of the future? Here’s what the forecasts of GDP growth look like using a fully featured Smets-Wouters NK model
Forecast horizons are quarterly and we want to focus on the DSGE prediction (light blue) against the data (red). One quarter ahead, the model actually does a decent job (probably because of the persistence of growth over time), but its predictive power is quickly lost as we expand the horizon. The table below shows how poor this fit actually is.
The values, which represent the amount of variation in the data accounted for by the model are tiny (a value of 1 would be a perfect match to the data). Even more concerning, the authors of the paper compare a DSGE forecast to other forecasting methods and find it cannot improve on a reduced form VAR forecast and is barely an improvement over a random walk forecast that assumes that all fluctuations are completely unpredictable. Forty years of progress in the DSGE research program and the best we can do is add unrealistic frictions and make poor predictions. In the essay linked above, Blanchard concludes “the state of macro is good.” I can’t imagine what makes him believe that.
Part 10 in a series of posts on modern macroeconomics. This post deals with a common defense of macroeconomic models that “all models are wrong.” In particular, I argue that attempts to match the model to the data contradict this principle.
In my discussion of rational expectations I talked about Milton Friedman’s defense of some of the more dubious assumptions of economic models. His argument was that even though real people probably don’t really think the way the agents in our economic models do, that doesn’t mean the model is a bad explanation of their behavior.
Friedman’s argument ties into a broader point about the realism of an economic model. We don’t necessarily want our model to perfectly match every feature that we see in reality. Modeling the complexity we see in real economies is a titanic task. In order to understand, we need to simplify.
Early DSGE models embraced this sentiment. Kydland and Prescott give a summary of their preferred procedure for doing macroeconomic economic research in a 1991 paper, “The Econometrics of the General Equilibrium Approach to Business Cycles.” They outline five steps. First, a clearly defined research question must be chosen. Given this question, the economist chooses a model economy that is well suited to answering it. Next, the parameters of the model are calibrated to fit based on some criteria (more on this below). Using the model, the researcher can conduct quantitative experiments. Finally, results are reported and discussed.
Throughout their discussion of this procedure, Kydland and Prescott are careful to emphasize that the true model will always remain out of reach, that “all model economies are abstractions and are by definition false.” And if all models are wrong, there is no reason we would expect any model to be able to match all features of the data. An implication of this fact is that we shouldn’t necessarily choose parameters of the model that give us the closest match between model and data. The alternative, which they offered alongside their introduction to the RBC framework, is calibration.
What is calibration? Unfortunately a precise definition doesn’t appear to exist and its use in academic papers has become somewhat slippery (and many times ends up meaning I chose these parameters because another paper used the same ones). But in reading Kydland and Prescott’s explanation, the essential idea is to find values for your parameters from data that is not directly related to the question at hand. In their words, “searching within some parametric class of economies for the one that best fits some set of aggregate time series makes little sense.” Instead, they attempt to choose parameters based on other data. They offer the example of the elasticity of substitution between labor and capital, or between labor and leisure. Both of these parameters can be obtained by looking at studies of human behavior. By measuring how individuals and firms respond to changes in real wages, we can get estimates for these values before looking at the results of the model.
The spirit of the calibration exercise is, I think, generally correct. I agree that economic models can never hope to capture the intricacies of a real economy and therefore if we don’t see some discrepancy between model and data we should be highly suspicious. Unfortunately, I also don’t see how calibration helps us to solve this problem. Two prominent econometricians, Lars Hansen and James Heckman, take issue with calibration. They argue
It is only under very special circumstances that a micro parameter such as the intertemporal elasticity of substitution or even a marginal propensity to consume out of income can be “plugged into” a representative consumer model to produce an empirically concordant aggregate model…microeconomic technologies can look quite different from their aggregate counterparts. Hansen and Heckman (1996) – The Empirical Foundations of Calibration
Although Kydland and Prescott and other proponents of the calibration approach wish to draw parameters from sources outside the model, it is impossible to entirely divorce the values from the model itself. The estimation of elasticity of substitution or labor’s share of income or any other parameter of the model is going to depend in large part on the lens through which these parameters are viewed and each model provides a different lens. To think that we can take an estimate from one environment and plug it into another would appear to be too strong an assumption.
There is also another more fundamental problem with calibration. Even if for some reason we believe we have the “correct” parameterization of our model, how can we judge its success? Thomas Sargent, in a 2005 interview, says that “after about five years of doing likelihood ratio tests on rational expectations models, I recall Bob Lucas and Ed Prescott both telling me that those tests were rejecting too many good models.” Most people, when confronted with the realization that their model doesn’t match reality would conclude that their model was wrong. But of course we already knew that. Calibration was the solution, but what does it solve? If we don’t have a good way to test our theories, how can we separate the good from the bad? How can we ever trust the quantitative results of a model in which parameters are likely poorly estimated and which offers no criterion for which it can be rejected? Calibration seems only to claim to be an acceptable response to the idea that “all models are wrong” without actually providing a consistent framework for quantification.
But what is the alternative? Many macroeconomic papers now employ a mix of calibrated parameters and estimated parameters. A common strategy is to use econometric methods to search over all possible parameter values in order to find those that are most likely given the data. But in order to put any significance on the meaning of these parameters, we need to take our model as the truth or as close to the truth. If we estimate the parameters to match the data, we implicitly reject the “all models are wrong” philosophy, which I’m not sure is the right way forward.
And it gets worse. Since estimating a highly nonlinear macroeconomic system is almost always impossible given our current computational constraints, most macroeconomic papers instead linearize the model around a steady state. So even if the model is perfectly specified, we also need the additional assumption that we always stay close enough to this steady state that the linear approximation provides a decent representation of the true model. That assumption often fails. Christopher Carroll shows that a log linearized version of a simple model gives a misleading picture of consumer behavior when compared to the true nonlinear model and that even a second order approximation offers little improvement. This issue is especially pressing in a world where we know a clear nonlinearity is present in the form of the zero lower bound for nominal interest rates and once again we get “unpleasant properties” when we apply linear methods.
Anyone trying to do quantitative macroeconomics faces a rather unappealing choice. In order to make any statement about the magnitudes of economic shocks or policies, we need a method to discipline the numerical behavior of the model. If our ultimate goal is to explain the real world, it makes sense to want to use data to provide this discipline. And yet we know our model is wrong. We know that the data we see was not generated by anything close to the simple models that enable analytic (or numeric) tractability. We want to take the model seriously, but not too seriously. We want to use the data to discipline our model, but we don’t want to reject a model because it doesn’t fit all features of the data. How do we determine the right balance? I’m not sure there is a good answer.