Predicting replication

The Behavioural Economics Replication Project:

This project will provide evidence of how accurately peer prediction markets can forecast replication of scientific experiments in economics.

In order to incentivize prediction market activity, and collect evidence on actual replication, eighteen (18) prominently published studies in experimental economics were chosen for trading in prediction markets, followed by replication. They are laboratory studies, using student participants, that were published in the American Economic Review (AER) or in the Quarterly Journal of Economics (QJE) in the years 2011 to 2014, testing specific hypotheses using between-subjects designs.

It is neat that a prediction market will be opened up to allow experimental economists to bet on which studies will be replicated. How much faith do those familiar with the workings of academia have in these studies?

Looking at the 18 studies, I suspect there will be a higher rate of replication than occurred in last year’s special issue of Social Psychology, but it wouldn’t surprise me to see a majority fail to replicate.