What’s Wrong With Modern Macro? Part 10 All Models are Wrong, Except When We Pretend They Are Right

Part 10 in a series of posts on modern macroeconomics. This post deals with a common defense of macroeconomic models that “all models are wrong.” In particular, I argue that attempts to match the model to the data contradict this principle.

In my discussion of rational expectations I talked about Milton Friedman’s defense of some of the more dubious assumptions of economic models. His argument was that even though real people probably don’t really think the way the agents in our economic models do, that doesn’t mean the model is a bad explanation of their behavior.

Friedman’s argument ties into a broader point about the realism of an economic model. We don’t necessarily want our model to perfectly match every feature that we see in reality. Modeling the complexity we see in real economies is a titanic task. In order to understand, we need to simplify.

Early DSGE models embraced this sentiment. Kydland and Prescott give a summary of their preferred procedure for doing macroeconomic economic research in a 1991 paper, “The Econometrics of the General Equilibrium Approach to Business Cycles.” They outline five steps. First, a clearly defined research question must be chosen. Given this question, the economist chooses a model economy that is well suited to answering it. Next, the parameters of the model are calibrated to fit based on some criteria (more on this below). Using the model, the researcher can conduct quantitative experiments. Finally, results are reported and discussed.

Throughout their discussion of this procedure, Kydland and Prescott are careful to emphasize that the true model will always remain out of reach, that “all model economies are abstractions and are by definition false.” And if all models are wrong, there is no reason we would expect any model to be able to match all features of the data. An implication of this fact is that we shouldn’t necessarily choose parameters of the model that give us the closest match between model and data. The alternative, which they offered alongside their introduction to the RBC framework, is calibration.

What is calibration? Unfortunately a precise definition doesn’t appear to exist and its use in academic papers has become somewhat slippery (and many times ends up meaning I chose these parameters because another paper used the same ones). But in reading Kydland and Prescott’s explanation, the essential idea is to find values for your parameters from data that is not directly related to the question at hand. In their words, “searching within some parametric class of economies for the one that best fits some set of aggregate time series makes little sense.” Instead, they attempt to choose parameters based on other data. They offer the example of the elasticity of substitution between labor and capital, or between labor and leisure. Both of these parameters can be obtained by looking at studies of human behavior. By measuring how individuals and firms respond to changes in real wages, we can get estimates for these values before looking at the results of the model.

The spirit of the calibration exercise is, I think, generally correct. I agree that economic models can never hope to capture the intricacies of a real economy and therefore if we don’t see some discrepancy between model and data we should be highly suspicious. Unfortunately, I also don’t see how calibration helps us to solve this problem. Two prominent econometricians, Lars Hansen and James Heckman, take issue with calibration. They argue

It is only under very special circumstances that a micro parameter such as the intertemporal elasticity of substitution or even a marginal propensity to consume out of income can be “plugged into” a representative consumer model to produce an empirically concordant aggregate model…microeconomic technologies can look quite different from their aggregate counterparts.
Hansen and Heckman (1996) – The Empirical Foundations of Calibration

Although Kydland and Prescott and other proponents of the calibration approach wish to draw parameters from sources outside the model, it is impossible to entirely divorce the values from the model itself. The estimation of elasticity of substitution or labor’s share of income or any other parameter of the model is going to depend in large part on the lens through which these parameters are viewed and each model provides a different lens. To think that we can take an estimate from one environment and plug it into another would appear to be too strong an assumption.

There is also another more fundamental problem with calibration. Even if for some reason we believe we have the “correct” parameterization of our model, how can we judge its success? Thomas Sargent, in a 2005 interview, says that “after about five years of doing likelihood ratio tests on rational expectations models, I recall Bob Lucas and Ed Prescott both telling me that those tests were rejecting too many good models.” Most people, when confronted with the realization that their model doesn’t match reality would conclude that their model was wrong. But of course we already knew that. Calibration was the solution, but what does it solve? If we don’t have a good way to test our theories, how can we separate the good from the bad? How can we ever trust the quantitative results of a model in which parameters are likely poorly estimated and which offers no criterion for which it can be rejected? Calibration seems only to claim to be an acceptable response to the idea that “all models are wrong” without actually providing a consistent framework for quantification.

But what is the alternative? Many macroeconomic papers now employ a mix of calibrated parameters and estimated parameters. A common strategy is to use econometric methods to search over all possible parameter values in order to find those that are most likely given the data. But in order to put any significance on the meaning of these parameters, we need to take our model as the truth or as close to the truth. If we estimate the parameters to match the data, we implicitly reject the “all models are wrong” philosophy, which I’m not sure is the right way forward.

And it gets worse. Since estimating a highly nonlinear macroeconomic system is almost always impossible given our current computational constraints, most macroeconomic papers instead linearize the model around a steady state. So even if the model is perfectly specified, we also need the additional assumption that we always stay close enough to this steady state that the linear approximation provides a decent representation of the true model. That assumption often fails. Christopher Carroll shows that a log linearized version of a simple model gives a misleading picture of consumer behavior when compared to the true nonlinear model and that even a second order approximation offers little improvement. This issue is especially pressing in a world where we know a clear nonlinearity is present in the form of the zero lower bound for nominal interest rates and once again we get “unpleasant properties” when we apply linear methods.

Anyone trying to do quantitative macroeconomics faces a rather unappealing choice. In order to make any statement about the magnitudes of economic shocks or policies, we need a method to discipline the numerical behavior of the model. If our ultimate goal is to explain the real world, it makes sense to want to use data to provide this discipline. And yet we know our model is wrong. We know that the data we see was not generated by anything close to the simple models that enable analytic (or numeric) tractability. We want to take the model seriously, but not too seriously. We want to use the data to discipline our model, but we don’t want to reject a model because it doesn’t fit all features of the data. How do we determine the right balance? I’m not sure there is a good answer.

Free Will, Morality, and Libertarianism How can you be held responsible for something that wasn't your fault?

An astute reader sees an apparent contradiction between my last post and my post on free will, asking “how can a libertarian reconcile no real choice with the importance of being free to choose?”

It’s a great question. If free will doesn’t exist, if every single action is pre-determined, can we even have a consistent concept of morality? I made a distinction between actions made freely and those forced by the state, but isn’t that distinction meaningless in a deterministic world? When all actions are at some level outside an individual’s control, is there any difference between the direct coercion of government and more indirect factors influencing the decision (like genetics, education, religion, etc.) that are also completely removed from the realm of free choice? The answers to these questions are far from obvious and I certainly don’t pretend to have a perfect response, but I hope this post will clarify the way I think about the issue. My ideas here are heavily influenced by (who else) Hayek’s discussion of similar topics in chapter 5 of The Constitution of Liberty.

First, it is important to understand my earlier defense of determinism. The key point is that every action can be traced to a chain of previous events. Going back far enough in each person’s chain, some link will be the result of an event that is outside of their direct control. And if I know all of the links in this causal chain, if I know everything that has ever influenced an individual, I can predict with absolute certainty their next move. Their “choice” was determined long before they are required to make it.

If we accept this argument, then at the moment of a decision nothing can be done. There is no way that a given individual would have made any decision different than the one they made. We could replay the same history a million times and get the same result in every single trial.

But here’s the problem. Go back to the example I gave in the last post. A person is trying to decide whether to give money to the poor (let’s call him Bob). Note that I will consider giving to the poor to be a good thing. If for some reason you disagree with that assessment, replace “giving to the poor” with any action you consider moral and the argument should still go through.

Now assume Bob exists in two universes (A and B). In each universe, Bob has had almost the exact same experience. He has the same parents, the same teachers, read all the same books. As a result, in each universe he has developed a system of values which teaches him to care for his fellow human beings. Now introduce one difference between the two universes. Universe A has a government which forces Bob to give to the poor through taxation. In Universe B, Bob is free to do as he pleases. Of course, we know  that Bob is not really free to choose. If he chooses to give to the poor, it is only because he grew up in a society that taught him that that was the right thing to do, and only because of his upbringing that he has any desire to do the right thing at all. Universe B Bob doesn’t choose to give to the poor any more than Universe A Bob does. Both only give due to the influence of others. How can we say one is more moral?

Consider George. George also exists in both universes, but he has had a different experience than Bob. Where Bob was taught to live a life of compassion, George only cares about his own material well being. Help the poor? How does that help George? In Universe A, George still has to give to the poor. The government forces him to give against his wishes. And it makes him angry. He works hard to earn his money, why should he give to those who don’t? He sees Bob gives to the poor as well, but he believes it is only because the government forces him to do so.

In Universe B, George doesn’t give to the poor. His values tell him that you get exactly what you deserve in life and he acts on those values. There is no government to force him to do otherwise. And yet he sees Bob give to the poor anyway. Maybe he just dismisses Bob as too stupid to realize that his money won’t help them, that the poor need to help themselves. But maybe Bob’s actions give him pause. Maybe they form a new link in George’s causal chain. Maybe he questions his decision, and even though he could never have changed his choice at that moment, he might think about the situation differently next time. Maybe his system of values begins to change.

Phrased in this way, we begin to see a real distinction between the decision to give in each universe. It is true that in both universes Bob would have given to the poor. Our reason for calling Bob’s actions moral cannot be that he himself could have made another choice at the moment of his decision – without free will, he really couldn’t have. But we can compare Bob’s choices to those of another individual. If we replace Bob with George in Universe A, the result is the same – both give to the poor, and we can’t judge morality because nobody could have acted differently. In Universe B, however, Bob and George are allowed to act differently in the same situation. Their decision tree has two branches. Neither will choose any branch other than the one already pre-determined by their life experiences, but the existence of the branches matters because somebody else could have.

So now we can give an answer to the original question. If nobody can truly make choices of their own volition, why does choice matter for morality?

Because if a choice is available, even if each individual will always make the same choice, another might have acted differently.

But even if you buy the argument above, a question still remains. We might agree that Bob made a moral decision in the example above even though it wasn’t truly his choice, but does that mean that George is responsible for his actions? Can we blame George for not giving to the poor? After all, it’s not his fault that he didn’t have Bob’s life. Here I defer to Hayek:

Strictly speaking, it is nonsense to say, as is so often said, that “it is not a man’s fault that he is as he is,” for the aim of assigning responsibility is to make him different from what he is or might be. If we say that a person is responsible for the consequences of an action, this is not a statement of fact or an assertion about causation. The statement would, of course, not be justifiable if nothing he “might” have done or omitted could have altered the result. But when we use words like “might” or “could” in this connection, we do not mean that at the moment of his decision something in him acted otherwise than was the necessary effect of causal laws in the given circumstances. Rather, the statement that a person is responsible for what he does aims at making his actions different from what they would be if he did not believe it to be true.

We assign responsibility to a man, not in order to say that as he was he might have acted differently, but in order to make him different.The Constitution of Liberty, p. 137-138

What do we want from our society? Do we want to move from Universe B to Universe A, from a world where people are free to make their own value judgements to one where they are not given any choice, where we claim to know what is best for them? Or do we want to convert people from Georges to Bobs, to convince them to buy into the system, convince them that the ideals we aim for are ones worth striving to achieve? In my view, a free society gives us the best chance of achieving the latter goal. Only a free society allows us a choice, and even if our choice is set long before we make it, knowing that other individuals could have made another choice remains important.

Social Cooperation Is the free market argument in need of rebranding?

The standard free market analysis places the individual at its center. As Adam Smith famously noted in 1776, although they act in their own self interest, an individual in a free market is “led by an invisible hand to promote an end which was no part of his intention.” And it is generally argued that competition is the driving force behind the benefits of the market process. Entrepreneurs constantly search for new opportunities to make profit, and as a result they find more efficient ways to provide goods to consumers.

Phrased in this way, the free market argument tends to evoke images of Social Darwinism – the best rise to the top, and the weak are left behind. Competition implies a constant struggle between market participants to seek their own benefit at the expense of others. This vision often leads critics to argue that the free market ideal generates an uncaring society. If everybody acts only in their own self interest, there is no room for cooperative behavior that is essential for human interaction. Morality, emotion, personal connections – none of it matters. The free market places “profit over people.”

This critique stems from a wildly incorrect reading of the free market argument.

Go back to Adam Smith. At the center of his work is the idea of division of labor. A market economy thrives not because individuals work in isolation. Instead, it depends entirely on the relationships between individuals, focusing each person’s talents on an activity where they possess a comparative advantage.

Perhaps the best illustration of the role of cooperation in a market economy is Leonard Read’s famous essay I, Pencil (there is also an excellent video inspired by the essay). Read points out that no individual on their own knows how to make even something as simple as a pencil. The production process requires dozens of firms and hundreds of workers each performing specialized tasks with little knowledge of the final product. There is no planner describing how to make a pencil and yet through the actions of individuals as well as the interactions between individuals, the production process arises spontaneously. An individual acting alone would quickly fail in a market economy.

Ludwig von Mises’s famous treatise Human Action, a comprehensive analysis of the working of the free market system, was almost given a different titleSocial Cooperation. Although Mises dropped this alternate title, the theme that markets depend as much on cooperation between individuals as they do on individual action itself runs throughout the book. Mises notes:

Within the frame of social cooperation there can emerge between members of society feelings of sympathy and friendship and a sense of belonging together. These feelings are the source of man’s most delightful and most sublime experiences. They are the most precious adornment of life; they lift the animal species man to the heights of a really human existence.
Human Action p. 144

A free market, in Mises’s view, doesn’t destroy relationships between individuals, but instead fosters these feelings. Even if we take the idea of “Social Darwinism” seriously, even if we admit that all individuals are driven by the desire to fight for their own survival, that doesn’t lead us to a world of selfishness (in a narrow sense) because “the most adequate means of improving his  condition is social cooperation and the division of labor” (Human Action, p. 176).

But the argument that markets and morals are inconsistent faces an even deeper flaw. In a market economy we have a choice. Of course we can choose to think only of ourselves, to put money over family, to value material goods over relationships. But that has absolutely nothing to do with the free market itself. Nothing in the market argument says that I should only care about wealth. If you want to put other priorities first, nobody in a free market has any right to stop you (the catch is that you also don’t have any right to make other people pay you).

A free market doesn’t place any moral judgement on the actions of individuals. It is perfectly consistent with both a savage society where everybody fights for their narrow self interest and ignores others as well as a responsible one where we care for our fellow humans. It is up to each of us as individuals to choose to live our lives morally (but of course, this choice is only an illusion).

What is the alternative? The only clear alternative I can see is to use the state to try to impose your morals on others. By enacting laws that force people to behave morally, maybe we can create a more caring society.

Such a system seems doomed to have the opposite effect. Let’s say you believe that redistribution of wealth is important. Poor people aren’t poor because they didn’t work hard. They just had bad luck. It’s the responsibility of the rich to help these people out. I am sympathetic to this reasoning. However, by forcing people to give up their wealth through taxation, we change the equation from one of responsibility to one of coercion. Rather than giving to the poor out of some sense of moral duty, I give because I don’t want to go to jail. Is attempting to legislate morality in this way more likely to generate a caring society or a resentful one? Respect between classes, or class warfare?

The free market argument should not marry itself to the individual. It is true that all actions must at their core come from individual decisions, but the market only works through the relationships between individuals. Human Action is only half of the story. Social Cooperation is equally important. By obscuring this fact, defenders of markets concede too much. Emphasizing efficiency and the incredible material progress society has made since adopting a market system is fine, but we can’t ignore the moral argument. Morality can’t be imposed. It has to be a choice. And only a free society offers that choice. “Liberty is an opportunity for doing good, but this is so only when it is also an opportunity for doing wrong” (Hayek, The Constitution of Liberty, p. 142).