Modelling versus theory

While looking for something completely unrelated, I came across this 2007 Econ Journal Watch paper Model Building versus Theorizing: The Paucity of Theory in the Journal of Economic Theory. From the abstract:

We argue that a model may qualify as theory only if it purports to answer three questions: Theory of what?, Why should we care?, What merit in your explanation? We examine the 66 regular articles appearing in the 2004 issues of Journal of Economic Theory—“the leading journal in economic theory” —and apply the three requirements. … We find that 27 articles fail the first test (Theory of what?) and 58 articles fail at least one of the three requirements. Thus, 88 percent of the articles do not qualify as theory. … We contend that the journal’s claim to scientific status is doubtful, as well as the very title of the journal. A truer title would be, Journal of Economic Model Building. More generally, we challenge calling model building “theory.”

While I would be reluctant to get drawn into an argument about what is theory, Klein and Romero’s argument about the lack of relevance of much economic modelling is important. They go on to say:

All stakeholders should be concerned that scholarly prestige will be leveraged in a way that feeds mere scholasticism, rather than real contributions to science, learning, and culture. Even if scholastic arts did not distort thought and understanding, they certainly might divert them from the things that matter more. If JET—and many other outlets—consists mainly of crafts that lack integrity as explanation, it does not deserve much prestige within the enterprise we call economics. This article, then, speaks to all stakeholders — elected officials, taxpayers, tuition payers, donors, university administrators, faculty, students, and other citizens concerned about the character and content of economics. …

Our concern is to challenge the semantics that hold that every model is (or entails) theory. We maintain that scientific culture understands theory to entail requirements of importance and usefulness. Theory must serve real purposes of the science, thus, arguably meriting attention from the scientific community.

Of Klein and Romero’s three questions, I usually worry about the third. Economic models are often not tested and even less commonly discarded. Klein and Romero point to some evidence.

It is possible that other economists take published models and subsequently supply the commitment to empirical relevance necessary to graduate the models to theory (Hausman 1992, 273). Whether such graduation occurs is a question calling for further research, but investigations by Philip Coelho and James McClure (2005; 2007) suggest that few models graduate to theory. In one investigation, Coelho and McClure identify the JET articles published in 1980 and containing at least five lemmas. They find that there were 12 such articles. They then investigate the articles that cite those 12 papers. As of June 2006, there were 237 articles that cite the 12 JET articles. They report that of 237, only nine utilize data. Of the nine, only two articles attempt a direct empirical assessment of the model’s results, and zero render a judgment of “accept” or “reject.

I sometimes fear that a lot of the work integrating economics and evolutionary biology might fall at this hurdle (particularly the discarding bit), as there are more theorists than experimentalists in this area. But thankfully there is so much relevant empirical work being done in evolutionary biology, anthropology, experimental psychology and behavioural economics that it’s going to be hard to maintain a poor theory in this area without somebody pulling it apart.