Behavioral science policy recommendations early in the pandemic were LARGELY CORRECT, if you ignore those that were not
In late April 2020, a group of behavioural scientists (Van Bavel et al., 2020) published a paper in Nature Human Behaviour, “Using social and behavioural science to support COVID-19 pandemic response”. They provided a range of suggestions for policy makers.
The paper sparked some debates about the readiness of behavioural science to inform the pandemic response. One of these critiques was in an article by IJzerman et al. (2020), which advised caution when applying behavioural science to policy.
A new article in Nature (Ruggeri et al., 2023) has reviewed the policy recommendations in that April 2020 paper. Part of the abstract reads:
In April 2020, an influential paper proposed 19 policy recommendations (‘claims’) detailing how evidence from behavioural science could contribute to efforts to reduce impacts and end the COVID-19 pandemic. Here we assess 747 pandemic-related research articles that empirically investigated those claims. We report the scale of evidence and whether evidence supports them to indicate applicability for policymaking. Two independent teams, involving 72 reviewers, found evidence for 18 of 19 claims, with both teams finding evidence supporting 16 (89%) of those 18 claims.
It’s a tick for that 2020 paper.
But, I wondered how long it would take for someone to extend this finding about a specific set of claims in ONE paper to a claim that “behavioural science was right!”.
It didn’t take long. Here’s one statement from the lead author of the Nature paper:
Behavioral science policy recommendations early in the pandemic were LARGELY CORRECT. Our global collaboration in @Nature covers 747 studies with an average sample size over 16,000! Evidence supports 16 of 19 claims, with lessons for science & policy. https://t.co/OzS7njsrzv pic.twitter.com/3xoGQHBJFB
— Kai Ruggeri (@kairuggeri) December 13, 2023
A quick peruse of twitter can find others of a similar vein.
It takes a relatively short memory to forget that, whatever the merits of this single paper, the behavioural science community was the source of plenty of rubbish.
Here’s one classic from a co-author of both the 2020 paper and the Nature review, Cass Sunstein, published on 29 February 2020:
At this stage, no one can specify the magnitude of the threat from the coronavirus. But one thing is clear: A lot of people are more scared than they have any reason to be. They have an exaggerated sense of their own personal risk.
Based on an experiment involving 156 undergraduate students receiving electric shocks, Sunstein suggested people exhibited “probability neglect” in worrying about the coming pandemic.
Four weeks later he was writing: “This Time the Numbers Show We Can’t Be Too Careful”. No acknowledgment that the earlier prognostication might not have been on the mark. He just moved on as if nothing had happened.
I posted more about these claims here and here.
The Van Bavel et al. article was a good target for review. Praise its sobriety.
But a review of a single paper is not a foundation for a general claim that the behavioural science community covered itself in glory. That would require a much broader study of what came out of it. That includes thought bubbles from Cass Sunstein and other behavioural scientists on public platforms. It includes outputs of the various behavioural insights teams, many of which were central to government pandemic responses. It includes the broader body of published behavioural science recommendations.
Such a broad review is a considerable effort. (Reviewing the single paper in such depth was impressive). But it wouldn’t take long to see that a general claim of correctness is overreach. Just look at some of the claims by those authors themselves.