A Different Kind of Economic Modeling

In macroeconomics, research almost always follows a similar pattern. First, the economist comes up with a question. Maybe they look at data and generate some stylized facts about some aspect of the economy. Then they set out to “explain” these facts using a structural economic model (I put explain in quotes because this step usually involves stripping away everything that made the question interesting in the first place). Using their model, they can then make some predictions or do some policy analysis. Finally, they write a paper describing their model and its implications.

There is nothing inherently wrong with this approach to research. But there are some issues. The first is that every paper looks exactly the same. Every paper needs a model. Sometimes papers adapt existing models, but they need enough difference to be a contribution on their own (but not so much difference that you leave the narrow consensus of modern macroeconomic methodology). Rarely, if ever, is there any attempt to compare models, to evaluate their failures and successes. It’s always: previous papers missed this and that feature while mine includes it.

This kind of iterative modeling can give the illusion of progress, but it really just represents sideways movement. The questions of macroeconomics haven’t really changed much in the last 100 years. What we have done is develop more and more answers to those questions without really making any progress on figuring out which of those answers is actually correct. Thousands of answers to a question is in many ways no better than none at all.

I don’t think it’s too much of a mystery why macroeconomics looks this way. Everybody already knows how an evaluation of our current answers to macroeconomic questions would go. The findings: we don’t know anything and all our models stink. I’d be surprised if even 10% of economists would honestly suggest a policymaker to carry out the policy that their papers suggest.

Academics still need to publish of course so they change the criteria that describes good macroeconomic research. Rarely is a paper evaluated on how well it answers an economic question. Instead, what matters is the tool used to answer the question. An empirical contribution without a model will get yawns in a macro seminar. A new mathematical contribution that uses a differential equations derived from a heat diffusion equation from physics? Mouths will be watering.

The claim is that these tools can then be used by other researchers as we continue to get closer and closer to the truth. The reality is that they are used by other researchers, but they only use the tools to develop their own slightly “better” tool in their own paper. In other words, the primary consumer of economic papers is economists who want to write papers. Widely cited papers are seen as better. Why? Because they helped a bunch of other people write their own papers? When does any of this research start to actually be helpful to people who aren’t responsible for creating it? Should we measure the quality of beef by how many cows it can feed?

Again there is an easy explanation for why economists are the only ones who can read economics papers. They would be completely unintelligible to anybody else. Reading and understanding the mechanism behind a macroeconomic paper is often a herculean task even for a trained economist. A non-expert has no chance. There could be good reasons for this complexity. I don’t expect to be able to open a journal on quantum mechanics and get anything out of it. But there is one enormous difference between physics and economics models. The physics ones actually work.

Economics didn’t always look this way. Read a paper by Milton Friedman or Armen Alchian. Almost no equations, much less the giant dynamic systems in models today. Does anybody think modern economic analysis is better than the kind done by those two?

The criticism of doing economics in words rather than math is that it is harder to be internally consistent. An equation has fewer interpretations than a sentence. I’m sympathetic to that argument. But I think there are better ways to add transparency to economics than by writing everything out in math that requires 20 years of school to understand. There are ways to formalize arguments other than systems of equations, ways to explain the mechanism that generates the data other than structural DSGE models.

The problem with purely verbal arguments is that you can easily lose your train of logic. Each sentence can make sense on its own but completely contradict another piece of the argument. Simultaneous systems of equations can prevent this kind of mistake. They are just one way. Computer simulations can provide the same discipline. Let’s say I have some theory about the way the world works. If I can design a computer simulation that replicates the kind of behavior I described in words, doesn’t that prove that my argument is logically consistent? It of course doesn’t mean I am right, but neither does a mathematical model. Each provides a complete framework that an outside observer can evaluate and decide whether its assumptions provide a useful view of the world.

The rest of the profession doesn’t seem to agree with me that a computer simulation and a system of equations serve the same purpose. I’m not exactly sure why. One potential worry is that it’s harder to figure out what’s actually happening in a computer simulation. With a system of equations I can see exactly which variables affect others and the precise channel of each kind of change. In a simulation, outcomes are emergent. Maybe I develop a simulation where an increase in taxes causes output to fall. Simply looking at the rules I have given each agent of how to act might not tell me why that fall occurred. It might be some complex interaction between these agents that generates that result.

That argument makes sense, but I think it only justifies keeping mathematical models rather than throwing out computer models. They serve different purposes. And computer models have their own advantages. One, which has yet to be explored in any serious way, is the potential for visual results. Imagine that the final result of an economic paper was not a long list of greek symbols and equals signs, but rather a full moving mini economic world. Agents move around, trade with each other. Firms set prices, open and close. Output and unemployment rise and fall. A simple version of such a model is the “sugarscape” model of Axtell and Epstein which creates a simple world where agents search for and trade sugar in order to survive.

Now imagine a much more complicated version of that that looked a lot more like a real economy. Rather than being able to “see” the relationships between variables in an equation, I could literally see how agents act and interact visually. My ideal world of economic research would not be writing papers, but creating apps. I want to download your model on my computer and play with it. Change the parameter values, apply different shocks, change the number and types of agents. And then observe what happens. Will this actually tell us anything useful about the economy? I’m not sure. But I think it’s worth a try. (I’m currently trying to do it myself. Hopefully I can post a version of it here soon)

What’s Wrong With Modern Macro?

After 15 posts, 17,795 words, and about 9 months, my series of posts on the problems with modern macro is finally complete. If anyone cares, here is the list of all the posts in order.

Part 1: Before Modern Macro – Keynesian Economics

Part 2: The Death of Keynesian Economics: The Lucas Critique, Microfoundations, and Rational Expectations

Part 3: Real Business Cycle and the Birth of DSGE Models

Part 4: How Did a “Measure of our Ignorance” Become the Cause of Business Cycles?

Part 5: Filtering Away All Our Problems

Part 6: The Illusion of Microfoundations I: The Aggregate Production Function

Part 7: The Illusion of Microfoundations II: The Representative Agent

Part 8: Rational Expectations Aren’t so Rational

Part 9: Carrying on the Torch of the Market Socialists

Part 10: All Models are Wrong, Except When We Pretend They Are Right

Part 11: Building on a Broken Foundation

Part 12: Models and Theories

Part 13: No Other Game in Town

Part 14: A Pretense of Knowledge in Macroeconomics

Part 15: Where Do We Go From Here?

If you made it through all of them I’m not sure if I should congratulate you or feel sorry for you, but either way thanks for reading.

What’s Wrong With Modern Macro? Part 14 A Pretense of Knowledge in Macroeconomics

Part 14 in a series of posts on modern macroeconomics. This post concludes my criticisms of the current state of macroeconomic research by tying all of my previous posts to Hayek’s warning about the pretense of knowledge in economics

Russ Roberts, host of the excellent EconTalk podcast, likes to say that you know macroeconomists have a sense of humor because they use decimal points. His implication is that for an economist to believe that they can predict growth or inflation or any other variable down to a tenth of a percentage point is absolutely ridiculous. And yet economists do it all the time. Whether it’s the CBO analysis of a policy change or the counterfactuals of an academic macroeconomics paper, quantitative analysis has become an almost necessary component of economic research. There’s nothing inherently wrong with this philosophy. Making accurate quantitative predictions would be an incredibly useful role for economics.

Except for one little problem. We’re absolutely terrible at it. Prediction in economics is really hard and we are nowhere near the level of understanding that would allow for our models to deliver accurate quantitative results. Everybody in the profession seems to realize this fact, which means nobody believes many of the results their theories produce.

Let me give two examples. The first, which I have already mentioned in previous posts, is Lucas’s result that the cost of business cycles in theoretical models is tiny – around one tenth of one percent of welfare loss compared to an economy with no fluctuations. Another is the well known puzzle in international economics that trade models have great difficulty producing large gains from trade. A seminal paper by Arkolakis, Costinot, and Rodriguez-Clare evaluates how our understanding of the gains from trade have improved from the theoretical advances of the last 10 years. Their conclusion: “so far, not much.”

How large are the costs of business cycles? Essentially zero. How big are the gains from trade? Too small to matter much. That’s what our theories tell us. Anybody who has lived in this world the last 30 years should be able to immediately take away some useful information from these results: the theories stink. We just lived through the devastation of a large recession. We’ve seen globalization lift millions out of poverty. Nobody believes for a second that business cycles and trade don’t matter. In fact, I’m not sure anybody takes the quantitative results of any macro paper seriously. For some reason we keep writing them anyway.

In the speech that provided the inspiration for the title of this blog, Hayek warned economists of the dangers of the pretense of knowledge. Unlike the physical sciences, where controlled laboratory experiments are possible and therefore quantitative predictions might be feasible, economics is a complex science. He argues,

The difficulties which we encounter in [complex sciences] are not, as one might at first suspect, difficulties about formulating theories for the explanation of the observed events – although they cause also special difficulties about testing proposed explanations and therefore about eliminating bad theories. They are due to the chief problem which arises when we apply our theories to any particular situation in the real world. A theory of essentially complex phenomena must refer to a large number of particular facts; and to derive a prediction from it, or to test it, we have to ascertain all these particular facts. Once we succeeded in this there should be no particular difficulty about deriving testable predictions – with the help of modern computers it should be easy enough to insert these data into the appropriate blanks of the theoretical formulae and to derive a prediction. The real difficulty, to the solution of which science has little to contribute, and which is sometimes indeed insoluble, consists in the ascertainment of the particular facts
Hayek (1989) – The Pretence of Knowledge

In other words, it’s not that developing models to explain economic phenomena is especially challenging, but rather that there is no way to collect sufficient information to apply any theory quantitatively. Preferences, expectations, technology. Any good macroeconomic theory would need to include each of these features, but each is almost impossible to measure.

Instead of admitting ignorance, we make assumptions. Preferences are all the same. Expectations are all rational. Production technologies take only a few inputs and outputs. Fluctuations are driven by a single abstract technology shock. Everyone recognizes that any realistic representation of these features would require knowledge far beyond what is available to any single mind. Hayek saw that this difficulty placed clear restrictions on what economists could do. We can admit that every model needs simplification while also remembering that those simplifications constrain the ability of the model to connect to reality. The default position of modern macroeconomics instead seems to be to pretend the constraints don’t exist.

Many economists would probably agree with many of the points I have made in this series, but it seems that most believe the issues have already been solved. There are a lot of models out there and some of them do attempt to deal directly with some of the problems I have identified. There are models that try to reduce the importance of TFP as a driver of business cycles. There are models that don’t use the HP-Filter, models that have heterogeneous agents, models that introduce financial frictions and other realistic features absent from the baseline models. For any flaw in one model, there is almost certainly another model that attempts to solve it. But in solving that single problem they likely introduce about ten more. Other papers will deal with those problems, but maybe they forget about the original problem. For each problem that arises, we just introduce a new model. And then we take those issues as solved even though they are solved by a set of models that potentially produce conflicting results and with no real way to differentiate which is more useful.

Almost every criticism I have written about in the last 13 posts of this series can be traced back to the same source. Macroeconomists try to do too much. They haven’t heeded Hayek’s plea for humility. Despite incredible simplifying assumptions, they take their models to real data and attempt to make predictions about a world that bears only a superficial resemblance to the model used to represent it. Trying to answer the big questions about macroeconomics with such a limited toolset is like trying to build a skyscraper with only a hammer.

What’s Wrong With Modern Macro? Part 13 No Other Game in Town

Part 13 in a series of posts on modern macroeconomics. Previous posts in this series have pointed out many problems with DSGE models. This post aims to show that these problems have either been (in my opinion wrongly) dismissed or ignored by most of the profession. 

“If you have an interesting and coherent story to tell, you can tell it in a DSGE model. If you cannot, your story is incoherent.”

The above quote comes from a testimony given by prominent Minnesota macroeconomist V.V. Chari to the House of Representatives in 2010 as they attempted to determine how economists could have missed an event as large as the Great Recession. Chari argues that although macroeconomics does have room to improve, it has made substantial progress in the last 30 years and there is nothing fundamentally wrong with its current path.

Of course, not everybody has been so kind to macroeconomics. Even before the crisis, prominent macroeconomists had begun voicing some concerns about the current state of macroeconomic research. Here’s Robert Solow in 2003:

The original impulse to look for better or more explicit micro foundations was probably reasonable. It overlooked the fact that macroeconomics as practiced by Keynes and Pigou was full of informal microfoundations. (I mention Pigou to disabuse everyone of the notion that this is some specifically Keynesian thing.) Generalizations about aggregative consumption-saving patterns, investment patterns, money-holding patterns were always rationalized by plausible statements about individual–and, to some extent, market–behavior. But some formalization of the connection was a good idea. What emerged was not a good idea. The preferred model has a single representative consumer optimizing over infinite time with perfect foresight or rational expectations, in an environment that realizes the resulting plans more or less flawlessly through perfectly competitive forward-looking markets for goods and labor, and perfectly flexible prices and wages.

How could anyone expect a sensible short-to-medium-run macroeconomics to come out of that set-up?
Solow (2003) – Dumb and Dumber in Macroeconomics

Olivier Blanchard in 2008:

There is, however, such a thing as too much convergence. To caricature, but only slightly: A macroeconomic article today often follows strict, haiku-like, rules: It starts from a general equilibrium structure, in which individuals maximize the expected present value of utility, firms maximize their value, and markets clear. Then, it introduces a twist, be it an imperfection or the closing of a particular set of markets, and works out the general equilibrium implications. It then performs a numerical simulation, based on calibration, showing that the model performs well. It ends with a welfare assessment.

Such articles can be great, and the best ones indeed are. But, more often than not, they suffer from some of the flaws I just discussed in the context of DSGEs: Introduction of an additional ingredient in a benchmark model already loaded with questionable assumptions. And little or no independent validation for the added ingredient.
Olivier Blanchard (2008) – The State of Macro

And more recently, the vitriolic critique of Paul Romer:

Would you want your child to be treated by a doctor who is more committed to his friend the anti-vaxer and his other friend the homeopath than to medical science? If not, why should you expect that people who want answers will keep paying attention to economists after they learn that we are more committed to friends than facts.
Romer (2016) – The Trouble with Macroeconomics

These aren’t empty critiques by no-name graduate student bloggers. Robert Solow and Paul Romer are two of the biggest names in growth theory in the last 50 years. Olivier Blanchard was the chief economist at the IMF. One would expect that these criticisms would cause the profession to at least strongly consider a re-evaluation of its methods. Looking at the recent trends in the literature as well as my own experience doing macroeconomic research, it hasn’t. Not even close. Instead, it seems to have doubled down on the DSGE paradigm, falling much closer to Chari’s point of view that “there is no other game in town.”

But it’s even worse than that. Taken at its broadest, sticking to DSGE models is not too restrictive. Dynamic simply means that the models have a forward looking component, which is obviously an important feature to include in a macroeconomic model. Stochastic means there should be some randomness, which again is probably a useful feature (although I do think deterministic models can be helpful as well – more on this later). General equilibrium is a little harder to swallow, but it still provides a good measure of flexibility.

Even within the DSGE framework, however, straying too far from the accepted doctrines of macroeconomics is out of the question. Want to write a paper that deviates from rational expectations? You better have a really good reason. Never mind that survey and experimental evidence shows large differences in how people form expectations, or the questionable assumption that everybody in the model knows the model and takes it as truth, or that the founder of rational expectations John Muth later said:

It is a little surprising that serious alternatives to rational expectations have never been proposed. My original paper was largely a reaction against very naive expectations hypotheses juxtaposed with highly rational decision-making behavior and seems to have been rather widely misinterpreted.
Quoted in Hoover (2013) – Rational Expectations: Retrospect and Prospect

Using an incredibly strong assumption like rational expectations is accepted without question. Any deviation requires explanation. Why? As far as I can tell, it’s just Kevin Malone economics: that’s the way it has always been done and nobody has any incentive to change it. And so researchers all get funneled into making tiny changes to existing frameworks – anything truly new is actively discouraged.

Now, of course, there are reasons to be wary of change. It would be a waste to completely ignore the path that brought us to this point and, despite their flaws, I’m sure there have been some important insights generated by the DSGE research program (although to be honest I’m having trouble thinking of any). Maybe it really is the best we can do. But wouldn’t it at least be worth devoting some time to alternatives? I would estimate that 99.99% of theoretical business cycle papers published in top 5 journals are based around DSGE models. Chari wasn’t exaggerating when he said there is no other game in town.

Wouldn’t it be nice if there were? Is there any reason other than tradition that every single macroeconomics paper needs to follow the exact same structure, to draw from the same set restrictive set of assumptions? If even 10% of the time and resources devoted to tweaking existing models was instead spent on coming up with entirely new ways of doing macroeconomic research I have no doubt that a viable alternative could be found. And there already are some (again, more on this later), but they exist only on the fringes of the profession. I think it’s about time for that to change.

What’s Wrong With Modern Macro? Part 12 Models and Theories

Part 12 in a series of posts on modern macroeconomics. This post draws inspiration from Axel Leijonhufvud’s distinction between models and theories in order to argue that the current procedure of performing macroeconomic research is flawed.

In a short 1997 article, Axel Leijonhufvud gives an interesting argument that economists too often fail to differentiate between models and theories. In his words,

For many years now, ‘model’ and ‘theory’ have been widely used as interchangeable terms in the profession. I want to plead the case that there is a useful distinction to be made. I propose to conceive of economic ‘theories’ as sets of beliefs about the economy and how it functions. They refer to the ‘real world’ – that curious expression that economists use when it occurs to them that there is one. ‘Models’ are formal but partial representations of theories. A model never encompasses the entire theory to which it refers.
Leijonhufvud (1997) – Models and Theories

The whole article is worth reading and it highlights a major problem with macroeconomic research: we don’t have a good way to determine when a theory is wrong. Standard economic theory has placed a large emphasis on models. Focusing on mathematical models ensures that assumptions are laid out clearly and that the implications of those assumptions are internally consistent. However, this discipline also constrains models to be relatively simple. I might have some grand theory of how the economy works, but even if my theory is entirely accurate, there is little chance that it can be converted into a tractable set of mathematical equations.

The result of this discrepancy between model and theory makes it very difficult to judge the success or failure of a theory based on the success or failure of a model. As discussed in an earlier post, we begin from the premise that all models are wrong and therefore it shouldn’t be at all surprising when a model fails to match some feature of reality. When a model is disproven along some margin, there are two paths a researcher can take: reject the theory or modify the assumptions of the model to better fit the theory. Again quoting Leijonhufvud,

When a model fails, the conclusion nearest to hand is that some simplifying assumption or choice of empirical proxy-variable is to blame, not that one’s beliefs about the way the world works are wrong. So one looks around for a modified model that will be a better vehicle for the same theory.
Leijonhufvud (1997) – Models and Theories

A good example of this phenomenon comes from looking at the history of Keynesian theory over the last 50 years. When the Lucas critique essentially dismantled the Keynesian research program in the 1980s, economists who believed in Keynesian insights could have rejected that theory of the world. Instead, they adopted neoclassical models but tweaked them in such a way that produced results that were still consistent with Keynesian theory. New Keynesian models today include rational expectations and microfoundations. Many hours have been spent converting Keynes’s theory into fancy mathematical language, adding (sometimes questionable) frictions into the neoclassical framework to coax it into giving Keynesian results. But how does that help us understand the world? Have any of these advances increased our understanding of economics beyond what Keynes already knew 80 years ago? I have my doubts.

Macroeconomic research today works on the assumption that any good theory can be written as a mathematical model. Within that class of theories, the gold standard are those that are tractable enough to give analytical solutions. Restricting the set of acceptable models in this way probably helps throw out a lot of bad theories. What I worry is that it also precludes many good ones. By imposing such tight constraints on what a good macroeconomic theory is supposed to look like we also severely constrain our vision of a theoretical economy. Almost every macroeconomic paper draws from a tiny selection of functional forms for preferences and production because any deviation quickly adds too much complexity and leads to systems of equations that are impossible to solve.

But besides restricting the set of theories, speaking in models also often makes it difficult to understand the broader logic that drove the creation of the model. As Leijonhufvud puts it, “formalism in economics makes it possible to know precisely what someone is saying. But it often leaves us in doubt about what exactly he or she is talking about.” Macroeconomics has become very good at laying out a set of assumptions and deriving the implications of those assumptions. Where it often fails is in describing how we can compare that simplified world to the complex one we live in. What is the theory that underlies the assumptions that create an economic model? Where do the model and the theory differ and how might that affect the results? These questions seem to me to be incredibly important for conducting sound economic research. They are almost never asked.

What’s Wrong With Modern Macro? Part 11 Building on a Broken Foundation

Part 11 in a series of posts on modern macroeconomics. The discussion to this point has focused on the early contributions to DSGE modeling by Kydland and Prescott. Obviously, in the last 30 years, economic research has advanced beyond their work. In this post I will argue that it has advanced in the wrong direction.

In Part 3 of this series I outlined the Real Business Cycle (RBC) model that began macroeconomics on its current path. The rest of the posts in this series have offered a variety of reasons why the RBC framework fails to provide an adequate model of the economy. An easy response to this criticism is that it was never intended to be a final answer to macroeconomic modeling. Even if Kydland and Prescott’s model misses important features of a real economy doesn’t it still provide a foundation to build upon? Since the 1980s, the primary agenda of macroeconomic research has been to build on this base, to relax some of the more unrealistic assumptions that drove the original RBC model and to add new assumptions to better match economic patterns seen in the data.

New Keynesian Models

Part of the appeal of the RBC model was its simplicity. Driving business cycles was a single shock: exogenous changes in TFP attributed to changes in technology. However, the real world is anything but simple. In many ways, the advances in the methods used by macroeconomists in the 1980s came only at the expense of a rich understanding of the complex interactions of a market economy. Compared to Keynes’s analysis 50 years earlier, the DSGE models of the 1980s seem a bit limited.

What Keynes lacked was clarity. 80 years after the publication of the General Theory, economists still debate “what Keynes really meant.” For all of their faults, DSGE models at the very least guarantee clear exposition (assuming you understand the math) and internal consistency. New Keynesian (NK) models attempt to combine these advantages with some of the insights of Keynes.

The simplest distinction between an RBC model and an NK model is the assumptions made about the flexibility of prices. In early DSGE models, prices (including wages) were assumed to be fully flexible, constantly adjusting in order to ensure that all markets cleared. The problem (or benefit depending on your point of view) with perfectly flexible prices is that it makes it difficult for policy to have any impact. Combining rational expectations with flexible prices means monetary shocks cannot have any real effects. Prices and wages simply adjust immediately to return to the same allocation of real output.

To reintroduce a role for policy, New Keynesians instead assume that prices are sticky. Now when a monetary shock hits the economy firms are unable to change their price so they respond to the higher demand by increasing real output. As research in the New Keynesian program developed, more and more was added to the simple RBC base. The first RBC model was entirely frictionless and included only one shock (to TFP). It could be summarized in just a few equations. A more recent macro model, The widely cited Smets and Wouters (2007), adds frictions like investment adjustment costs, variable capital utilization, habit formation, and inflation indexing in addition to 7 structural shocks, leading to dozens of equilibrium equations.

Building on a Broken Foundation

New Keynesian economists have their heart in the right place. It’s absolutely true that the RBC model produces results at odds with much of reality. Attempting to move the model closer to the data is certainly an admirable goal. But, as I have argued in the first 10 posts of this series, the RBC model fails not so much because it abstracts from the features we see in real economies, but because the assumptions it makes are not well suited to explain economic interactions in the first place.

New Keynesian models do almost nothing to address the issues I have raised. They still rely on aggregating firms and consumers into representative entities that offer only the illusion of microfoundations rather than truly deriving aggregate quantities from realistic human behavior. They still assume rational expectations, meaning every agent believes that the New Keynesian model holds and knows that every other agent also believes it holds when they make their decisions. And they still make quantitative policy proposals by assuming a planner that knows the entire economic environment.

One potential improvement of NK models over their RBC counterparts is less reliance on (potentially misspecified) technology shocks as the primary driver of business cycle fluctuations. The “improvement,” however, comes only by introducing other equally suspect shocks. A paper by V.V. Chari, Patrick Kehoe, and Ellen McGrattan argues that four of the “structural shocks” in the Smets-Wouters setup do not have a clear interpretation. In other words, although the shocks can help the model match the data, the reason why they are able to do so is not entirely clear. For example, one of the shocks in the model is a wage markup over the marginal productivity of a worker. The authors argue that this could be caused either by increased bargaining power (through unions, etc.), which would call for government intervention to break up unions, or through an increased preference for leisure, which is an efficient outcome and requires no policy. The Smets-Wouters model can say nothing about which interpretation is more accurate and is therefore unhelpful in prescribing policy to improve the economy.

Questionable Microfoundations

I mentioned above that the main goal of the New Keynesian model was to revisit the features Keynes talked about decades ago using modern methods. In today’s macro landscape, microfoundations have become essential. Keynes spoke in terms of broad aggregates, but without understanding how those aggregates are derived from optimizing behavior at a micro level they fall into the Lucas critique. But although the NK model attempts to put in more realistic microfoundations, many of its assumptions seem at odds with the ways in which individuals actually act.

Take the assumption of sticky prices. It’s probably a valid assumption to include in an economic model. With no frictions, an optimizing firm would want to change its price every time new economic data became available. Obviously that doesn’t happen as many goods stay at a constant price for months or years. So of course if we want to match the data we are going to need some reason for firms to hold prices constant. The most common way to achieve this result is called “Calvo Pricing” in honor of a paper by Guillermo Calvo. And how do we ensure that some prices in the economy remain sticky? We simply assign each firm some probability that they will be unable to change their price. In other words, we assume exactly the result we want to get.

Of course, economists have recognized that Calvo pricing isn’t really a satisfactory final answer, so they have worked to microfound Calvo’s idea through an optimization problem. The most common method, proposed by Rotemberg in 1982, is called the menu cost model. In this model of price setting, a firm must pay some cost every time it changes its prices. This cost ensures that a firm will only change its price when it is especially profitable to do so. Small price changes every day no longer make sense. The term menu cost comes from the example of a firm needing to print out new menus every time it changes one of its prices, but it can be applied more broadly to encompass every cost a firm might incur by changing a price. More specifically, price changes could have an adverse effect on consumers, causing some to leave the store. It could require more advanced computer systems, more complex interactions with suppliers, competitive games between firms.

But by wrapping all of these into one “menu cost,” am I really describing a firm’s problem. A “true” microfoundation would explicitly model each of these features at a firm level with different speeds of price adjustment and different reasons causing firms to hold prices steady. What do we gain by adding optimization to our models if that optimization has little to do with the problem a real firm would face? Are we any better off with “fake” microfoundations than we were with statistical aggregates. I can’t help but think that the “microfoundations” in modern macro models are simply fancy ways of finagling optimization problems until we get the result we want. Interesting mathematical exercise? Maybe. Improving our understanding of the economy? Unclear.

Other additions to the NK model that purport to move the model closer to reality are also suspect. As Olivier Blanchard describes in his 2008 essay, “The State of Macro:”

Reconciling the theory with the data has led to a lot of un- convincing reverse engineering. External habit formation—that is a specification of utility where utility depends not on consumption, but on consumption relative to lagged aggregate consumption—has been introduced to explain the slow adjustment of consumption. Convex costs of changing investment, rather than the more standard and more plausible convex costs of investment, have been introduced to explain the rich dynamics of investment. Backward indexation of prices, an assumption which, as far as I know, is simply factually wrong, has been introduced to explain the dynamics of inflation. And, because, once they are introduced, these assumptions can then be blamed on others, they have often become standard, passed on from model to model with little discussion

In other words, we want to move the model closer to the data, but we do so by offering features that bear little resemblance to actual human behavior. And if the models we write do not actually describe individual behavior at a micro level, can we really still call them “microfounded”?

And They Still Don’t Match the Data

In Part 10 of this series, I discussed the idea that we might not want our models to match the data. That view is not shared by most of the rest of the profession, however, so let’s judge these models on their own terms. Maybe the microfoundations are unrealistic and the shocks misspecified, but, as Friedman argued, who cares about assumptions as long as we get a decent result? New Keynesian models also fail in this regard.

Forecasting in general is very difficult for any economic model, but one would expect that after 40 years of toying with these models, getting closer and closer to an accurate explanation of macroeconomic phenomena, we would be able to make somewhat decent predictions. What good is explaining the past if it can’t help inform our understanding of the future? Here’s what the forecasts of GDP growth look like using a fully featured Smets-Wouters NK model

Source: Gurkaynak et al (2013)

Forecast horizons are quarterly and we want to focus on the DSGE prediction (light blue) against the data (red). One quarter ahead, the model actually does a decent job (probably because of the persistence of growth over time), but its predictive power is quickly lost as we expand the horizon. The table below shows how poor this fit actually is.

Source: Edge and Gurkaynak (2010)

The R^2 values, which represent the amount of variation in the data accounted for by the model are tiny (a value of 1 would be a perfect match to the data). Even more concerning, the authors of the paper compare a DSGE forecast to other forecasting methods and find it cannot improve on a reduced form VAR forecast and is barely an improvement over a random walk forecast that assumes that all fluctuations are completely unpredictable. Forty years of progress in the DSGE research program and the best we can do is add unrealistic frictions and make poor predictions. In the essay linked above, Blanchard concludes “the state of macro is good.” I can’t imagine what makes him believe that.

What’s Wrong With Modern Macro? Part 10 All Models are Wrong, Except When We Pretend They Are Right

Part 10 in a series of posts on modern macroeconomics. This post deals with a common defense of macroeconomic models that “all models are wrong.” In particular, I argue that attempts to match the model to the data contradict this principle.

In my discussion of rational expectations I talked about Milton Friedman’s defense of some of the more dubious assumptions of economic models. His argument was that even though real people probably don’t really think the way the agents in our economic models do, that doesn’t mean the model is a bad explanation of their behavior.

Friedman’s argument ties into a broader point about the realism of an economic model. We don’t necessarily want our model to perfectly match every feature that we see in reality. Modeling the complexity we see in real economies is a titanic task. In order to understand, we need to simplify.

Early DSGE models embraced this sentiment. Kydland and Prescott give a summary of their preferred procedure for doing macroeconomic economic research in a 1991 paper, “The Econometrics of the General Equilibrium Approach to Business Cycles.” They outline five steps. First, a clearly defined research question must be chosen. Given this question, the economist chooses a model economy that is well suited to answering it. Next, the parameters of the model are calibrated to fit based on some criteria (more on this below). Using the model, the researcher can conduct quantitative experiments. Finally, results are reported and discussed.

Throughout their discussion of this procedure, Kydland and Prescott are careful to emphasize that the true model will always remain out of reach, that “all model economies are abstractions and are by definition false.” And if all models are wrong, there is no reason we would expect any model to be able to match all features of the data. An implication of this fact is that we shouldn’t necessarily choose parameters of the model that give us the closest match between model and data. The alternative, which they offered alongside their introduction to the RBC framework, is calibration.

What is calibration? Unfortunately a precise definition doesn’t appear to exist and its use in academic papers has become somewhat slippery (and many times ends up meaning I chose these parameters because another paper used the same ones). But in reading Kydland and Prescott’s explanation, the essential idea is to find values for your parameters from data that is not directly related to the question at hand. In their words, “searching within some parametric class of economies for the one that best fits some set of aggregate time series makes little sense.” Instead, they attempt to choose parameters based on other data. They offer the example of the elasticity of substitution between labor and capital, or between labor and leisure. Both of these parameters can be obtained by looking at studies of human behavior. By measuring how individuals and firms respond to changes in real wages, we can get estimates for these values before looking at the results of the model.

The spirit of the calibration exercise is, I think, generally correct. I agree that economic models can never hope to capture the intricacies of a real economy and therefore if we don’t see some discrepancy between model and data we should be highly suspicious. Unfortunately, I also don’t see how calibration helps us to solve this problem. Two prominent econometricians, Lars Hansen and James Heckman, take issue with calibration. They argue

It is only under very special circumstances that a micro parameter such as the intertemporal elasticity of substitution or even a marginal propensity to consume out of income can be “plugged into” a representative consumer model to produce an empirically concordant aggregate model…microeconomic technologies can look quite different from their aggregate counterparts.
Hansen and Heckman (1996) – The Empirical Foundations of Calibration

Although Kydland and Prescott and other proponents of the calibration approach wish to draw parameters from sources outside the model, it is impossible to entirely divorce the values from the model itself. The estimation of elasticity of substitution or labor’s share of income or any other parameter of the model is going to depend in large part on the lens through which these parameters are viewed and each model provides a different lens. To think that we can take an estimate from one environment and plug it into another would appear to be too strong an assumption.

There is also another more fundamental problem with calibration. Even if for some reason we believe we have the “correct” parameterization of our model, how can we judge its success? Thomas Sargent, in a 2005 interview, says that “after about five years of doing likelihood ratio tests on rational expectations models, I recall Bob Lucas and Ed Prescott both telling me that those tests were rejecting too many good models.” Most people, when confronted with the realization that their model doesn’t match reality would conclude that their model was wrong. But of course we already knew that. Calibration was the solution, but what does it solve? If we don’t have a good way to test our theories, how can we separate the good from the bad? How can we ever trust the quantitative results of a model in which parameters are likely poorly estimated and which offers no criterion for which it can be rejected? Calibration seems only to claim to be an acceptable response to the idea that “all models are wrong” without actually providing a consistent framework for quantification.

But what is the alternative? Many macroeconomic papers now employ a mix of calibrated parameters and estimated parameters. A common strategy is to use econometric methods to search over all possible parameter values in order to find those that are most likely given the data. But in order to put any significance on the meaning of these parameters, we need to take our model as the truth or as close to the truth. If we estimate the parameters to match the data, we implicitly reject the “all models are wrong” philosophy, which I’m not sure is the right way forward.

And it gets worse. Since estimating a highly nonlinear macroeconomic system is almost always impossible given our current computational constraints, most macroeconomic papers instead linearize the model around a steady state. So even if the model is perfectly specified, we also need the additional assumption that we always stay close enough to this steady state that the linear approximation provides a decent representation of the true model. That assumption often fails. Christopher Carroll shows that a log linearized version of a simple model gives a misleading picture of consumer behavior when compared to the true nonlinear model and that even a second order approximation offers little improvement. This issue is especially pressing in a world where we know a clear nonlinearity is present in the form of the zero lower bound for nominal interest rates and once again we get “unpleasant properties” when we apply linear methods.

Anyone trying to do quantitative macroeconomics faces a rather unappealing choice. In order to make any statement about the magnitudes of economic shocks or policies, we need a method to discipline the numerical behavior of the model. If our ultimate goal is to explain the real world, it makes sense to want to use data to provide this discipline. And yet we know our model is wrong. We know that the data we see was not generated by anything close to the simple models that enable analytic (or numeric) tractability. We want to take the model seriously, but not too seriously. We want to use the data to discipline our model, but we don’t want to reject a model because it doesn’t fit all features of the data. How do we determine the right balance? I’m not sure there is a good answer.

What’s Wrong With Modern Macro? Part 8 Rational Expectations Aren't so Rational

Part 8 in a series of posts on modern macroeconomics. This post continues the criticism of the underlying assumptions of most macroeconomic models. It focuses on the assumption of rational expectations, which was described briefly in Part 2. This post will summarize many of the points I made in a longer survey paper I wrote about expectations in macroeconomics that can be found here.

What economists imagine to be rational forecasting would be considered obviously irrational by anyone in the real world who is minimally rational
Roman Frydman (2013) – Rethinking Economics p. 148

Why Rational Expectations?

Although expectations play a large role in many explanations of business cycles, inflation, and other macroeconomic data, modeling expectations has always been a challenge. Early models of expectations relied on backward looking adaptive expectations. In other words, people form their expectations by looking at past trends and extrapolating them forward. Such a process might seem plausible, but there is a substantial problem with using a purely adaptive formulation of expectations in an economic model.

For example, consider a firm that needs to choose how much of a good to produce before it learns the price. If it expects the price to be high, it will want to produce a lot, and vice versa. If we assume firms expect today’s price to be the same as tomorrow’s, they will consistently be wrong. When the price is low, they expect a low price and produce a small amount. But the low supply leads to a high price in equilibrium. A smart firm would see their errors and revise their expectations in order to profit. As Muth argued in his original defense of rational expectations, “if the prediction of the theory were substantially better than the expectations of the firms, then there would be opportunities for ‘the insider’ to profit from the knowledge.” In equilibrium, these kinds of profit opportunities would be eliminated by intelligent entrepreneurs.

The solution proposed by Muth and popularized in macro by Lucas, was to simply assume that agents had the same model of the economy as the economist. Under rational expectations, an agent does not need to look at past data to make a forecast. Instead, their expectations are model based and forward looking. If an economist can detect consistent errors in an agent’s forecasting, rational expectations assumes that the agents themselves can also detect these errors and correct them.

What Does the Data Tell Us?

The intuitive defense of rational expectations is appealing and certainly rational expectations marked an improvement over previous methods. But if previous models of expectations gave agents too little cognitive ability, rational expectations models give them far too much. Rational expectations leaves little room for disagreement between agents. Each one needs to instantly jump to the “correct” model of the economy (which happens to correspond to the one created by the economist) and assume every other agent has made the exact same jump. As Thomas Sargent put it, rational expectations leads to a “communism of models. All agents inside the model, the econometrician, and God share the same model.”

The problem, as Mankiw et al note, is that “the data easily reject this assumption. Anyone who has looked at survey data on expectations, either those of the general public or those of professional forecasters, can attest to the fact that disagreement is substantial.” Branch (2004) and Carroll (2003) offer further evidence that heterogeneous expectations play an important role in forecasting. Another possible explanation for disagreement in forecasts is that agents have access to different information. Even if each agent knew the correct model of the economy, having access to private information could lead to a different predictions. Mordecai Kurz has argued forcefully that disagreement does not stem from private information, but rather different interpretations of the same information.

Experimental evidence also points to heterogeneous expectations. In a series of “learning to forecast” experiments, Cars Hommes and coauthors have shown that when agents are asked to forecast the price of an asset generated by an unknown model, they appear to resort to simple heuristics that often differ substantially from the rational expectations forecast. Surveying the empirical evidence surrounding the assumptions of macroeconomics as a whole, John Duffy concludes “the evidence to date suggests that human subject behavior is often at odds with the standard micro-assumptions of macroeconomic models.”

“As if” Isn’t Enough to Save Rational Expectations

In response to concerns about the assumptions of economics, Milton Friedman offered a powerful defense. He agreed that it was ridiculous to assume that agents make complex calculations when making economic decisions, but claimed that that is not at all an argument against assuming that they made decisions as if they knew this information. Famously, he gave the analogy of an expert billiard player. Nobody would ever believe that the player planned all of his shots using mathematical equations to determine the exact placement, and yet a physicist who assumed he did make those calculations could provide an excellent model of his behavior.

The same logic applies in economics. Agents who make forecasts using incorrect models will be driven out as they are outperformed by those with better models until only the rational expectations forecast remains. Except this argument only works if rational expectations is actually the best forecast. As soon as we admit that people use various models to forecast, there is no guarantee it will be. Even if some agents know the correct model, they cannot predict the path of economic variables unless they are sure that others are also using that model. Using rational expectations might be the best strategy when others are also using it, but if others are using incorrect models, it may be optimal for me to use an incorrect model as well. In game theory terms, rational expectations is a Nash Equilibrium, but not a dominant strategy (Guesnerie 2010).

Still, Friedman’s argument implies that we shouldn’t worry too much about the assumptions underlying our model as long as it provides predictions that help us understand the world. Rational expectations models also fail in this regard. Even after including many ad hoc fixes to standard models like wage and price stickiness, investment adjustment costs, inflation indexing, and additional shocks, DSGE models with rational expectations still provide weak forecasting ability, losing out to simpler reduced form vector autoregressions (more on this in future posts).

Despite these criticisms, we still need an answer to the Lucas Critique. We don’t want agents to ignore information that could help them to improve their forecasts. Rational expectations ensures they do not, but it is far too strong. Weaker assumptions on how agents use the information to learn and improve their forecasting model over time retain many of the desirable properties of rational expectations while dropping some of its less realistic ones. I will return to these formulations in a future post (but read the paper linked in the intro – and here again – if you’re interested).


What Does it Mean to Be Rational?

One of the most basic assumptions that underlies economic analysis is that people are rational. Given a set of possible choices, an individual in an economic model always chooses the one they think will give them the largest net benefit. This assumption has come under attack by many within the profession as well as by outsiders.

Some of these criticisms are easy to reject. For example, it is common for non-economists to make the jump from rationality to selfishness and denounce economics for assuming all humans only care about themselves. Similar criticisms point to economics for being too materialistic when real people actually value intangible goods as well. Neither of these arguments hits the mark. When an economist talks of rationality, they mean it in the broadest possible way. If donating to charity or giving gifts or going to poor countries to build houses are important for you, economics does not say you are wrong to choose those things. Morality or duty can have just as powerful an impact on your choices as material desires. Economics makes no distinction between these categories. You choose what you prefer. The reason you prefer it is irrelevant for an economist.

More thoughtful criticisms argue that even after defining preferences correctly, people sometimes make decisions that actively work against their own interests. The study of these kinds of issues with decision-making  has come to be called “behavioral economics.” One of the most prominent behavioral economists, Richard Thaler, likes to use the example of the Herzfeld Caribbean Basin Fund. Despite its NASDAQ ticker CUBA, the fund has little to do with the country Cuba. Nevertheless, when Obama announced an easing of Cuban trade restrictions in 2014, the stock price increased 70%. A more recent example occurred when investors swarmed to Nintendo stock after the release of Pokemon Go before realizing that Nintendo doesn’t even make the game.

But do these examples show that people are irrational? I don’t think so. There needs to be a distinction between being rational and being right. Making mistakes doesn’t mean that your original goal wasn’t to maximize your own utility, it just means that the path you chose was not the best way to accomplish that goal. To quote Ludwig von Mises, “human action is necessarily always rational” because “the ultimate end of action is always the satisfaction of some desires” (Human Action, p. 18). He gives the example of doctors 100 years ago trying to treat cancer. Although the methods used at that time may appear irrational by modern standards, those methods were what the doctors considered to be the best given the knowledge available at the time. Mises even applies the same logic to emotional actions. Rather than call emotionally charged actions irrational, Mises instead chooses to argue that “emotions disarrange valuations”  (p. 16). Again, an economist cannot differentiate between the reasons an individual chooses to act. Anger is as justifiable as deliberate calculation.

That is not to say that the behavioralists don’t have a point. They just chose the wrong word. People are not irrational. They just happen to be wrong quite often. The confusion is understandable, however, because rationality itself is not well defined. Under my definition, rationality only requires that agents make the best decision possible given their current knowledge. Often economic models go a step further and make specific (and sometimes very strong) assumptions about what that knowledge should include. Even in models with uncertainty, people are usually expected to know the probabilities of events occurring. In reality, people often face not only unknown events, but unknowable events that could never be predicted beforehand. In such a world, there is no reason to expect that even the most intelligent rational agent could make a decision that appears rational in a framework with complete knowledge.

Roman Frydman describes this point nicely

After uncovering massive evidence that contemporary economics’ standard of rationality fails to capture adequately how individuals actually make decisions, the only sensible conclusion to draw was that this standard was utterly wrong. Instead, behavioral economists concluded that individuals are less than fully rational or are irrational
Rethinking Expectations: The Way Forward For Macroeconomics, p. 148-149

Don’t throw out rationality. Throw out the strong knowledge assumptions. Perhaps the worst of these is the assumption of “rational expectations” in macroeconomics. I will have a post soon arguing that they are not really rational at all.

What’s Wrong With Modern Macro? Part 7 The Illusion of Microfoundations II: The Representative Agent

Part 7 in a series of posts on modern macroeconomics. Part 6 began a criticism of the form of “microfoundations” used in DSGE models. This post continues that critique by emphasizing the flaws in using a “representative agent.” This issue has been heavily scrutinized in the past and so this post primarily offers a synthesis of material found in articles from Alan Kirman and Kevin Hoover (1 and 2). 

One of the key selling points of DSGE models is that they are supposedly derived from microfoundations. Since only individuals can act, all aggregates must necessarily be the result of the interactions of these individuals. Understanding the mechanisms that lie behind the aggregates therefore seems essential to understanding the movements in the aggregates themselves. Lucas’s attack on Keynesian economics was motivated by similar logic. We can observe relationships between aggregate variables, but if we don’t understand the individual behavior that drives these relationships, how do we know if they will hold up when the environment changes?

I agree that ignoring individual behavior is an enormous problem for macroeconomics. But DSGE models do little to solve this problem.

Ideally, microfoundations would mean modeling behavior at an individual level. Each individual would then make choices based on the current economic conditions and these choices would aggregate to macroeconomic variables. Unfortunately, putting enough agents to make this exercise interesting is challenging to do in a mathematical model. As a result, “microfoundations” in a DSGE model usually means assuming that the decisions of all individuals can be summarized by the decisions of a single “representative agent.” Coordination between agents, differences in preferences or beliefs, and even the act of trading that is the hallmark of a market economy are eliminated from the discussion entirely. Although we have some vague notion that these activities are going on in the background, the workings of the model are assumed to be represented by the actions of this single agent.

So our microfoundations actually end up looking a lot closer to an analysis of aggregates than an analysis of true individual behavior. As Kevin Hoover writes, they represent “only a simulacrum of microeconomics, since no agent in the economy really faces the decision problem they represent. Seen that way, representative-agent models are macroeconomic, not microfoundational, models, although macroeconomic models that are formulated subject to an arbitrary set of criteria.” Hoover pushes back against the defense that representative agent models are merely a step towards truly microfounded models, arguing that not only would full microfoundations be infeasible but also that many economists do take the representative agent seriously on its own terms, using it both for quantitative predictions and policy advice.

But is the use of the representative agent really a problem? Even if it fails on its promise to deliver true “microfoundations,” isn’t it still an improvement over neglecting optimizing behavior entirely? Possibly, but using a representative agent offers only a superficial manifestation of individual decision-making, opening the door for misinterpretation. Using a representative agent assumes that the decisions of one agent at a macro level would be made in the same way as the decisions of millions of agents at a micro level. The theoretical basis for this assumption is weak at best.

In a survey of the representative agent approach, Alan Kirman describes many of the problems that arise when many agents are aggregated into a single representative. First, he presents the theoretical results from Sonnenschein (1972), Debreu (1974), and Mantel (1976), which show that even with strong assumptions on the behavior of individual preferences, the equilibrium that results by adding up individual behavior is not necessarily stable or unique.

The problem runs even deeper. Even if we assume aggregation results in a nice stable equilibrium, worrying results begin to arise as soon as we start to do anything with that equilibrium. One of the primary reasons for developing a model in the first place is to see how it reacts to policy changes or other shocks. Using a representative agent to conduct such an analysis implicitly assumes that the new aggregate equilibrium will still correspond to decisions of individuals. Nothing guarantees that it will. Kirman gives a simple example of a two-person economy where the representative agent’s choice makes each individual worse off. The Lucas Critique then applies here just as strongly as it does for old Keynesian models. Despite the veneer of optimization and rational choice, a representative agent model still abstracts from individual behavior in potentially harmful ways.

Of course, macroeconomists have not entirely ignored these criticisms and models with heterogeneous agents have become increasing popular in recent work. However, keeping track of more than one agents makes it nearly impossible to achieve useful mathematical results. The general process for using heterogeneous agents in a DSGE model then is to first prove that these agents can be aggregated and summarized by a set of aggregate equations. Although beginning from heterogeneity and deriving aggregation explicitly helps to ensure that the problems outlined above do not arise, it still imposes severe restrictions on the types of heterogeneity allowed. It would be an extraordinary coincidence if the restrictions that enable mathematical tractability also happen to be the ones relevant for understanding reality.

We are left with two choices. Drop microfoundations or drop DSGE. The current DSGE framework only offers an illusion of microfoundations. It introduces optimizing behavior at an aggregate level, but has difficulty capturing many of the actions essential to the workings of the market economy at a micro level. It is not a first step to discovering a way to model true microfoundations because it is not a tool well-suited to analyzing the behavior of more than one person at a time. Future posts will explore some models that are.