Cass Sunstein and Reid Hastie’s Wiser: Getting Beyond Groupthink to Make Groups Smarter
Cass Sunstein and Reid Hastie’s Wiser: Getting Beyond Groupthink to Make Groups Smarter is not an exciting read. However, it is a good catalogue of group decision-making research (leading to this post to also be somewhat of a catalogue) and worth reading for an overview.
The book’s theme is that group decisions are often better than individual decisions, but that groups have weaknesses that can impair outcomes. Much of the analysis of failures in group decision-making follows a similar theme to the research into individual judgement and decision-making, in that the research has generated a long list of “biases” that groups are subject to. Most of the book, however, focuses on getting better decisions, and a lot of these (thankfully) don’t rest on identification of particular biases.
Two types of groups
Sunstein and Hastie look at two types of groups - statistical and deliberating groups.
In a statistical group, members give their inputs individually. Those inputs are then aggregated. Think voting (which works well as long as the majority is right).
There is no shortage of material about the wisdom of statistical groups. The story of Francis Galton, where he had people estimate the weight of an ox, is a classic example. The average of the individual predictions was right on the mark.
In deliberating groups, individuals provide input during deliberations. Those inputs can affect and be affected by the inputs of other group members. People aim to influence others. People might change their minds.
Even if most members of a group have the wrong answer or belief, you can picture a scenario where reason and discussion allow the right answer emerge. That is sometimes the case, but the evidence is that deliberating groups do not necessarily converge on the truth.
In one experiment, people answered questions individually before answering those same questions in groups. If the majority of the group knew the correct answer to a problem, the group’s decision was correct 79% of the time. (It’s impressive that the incorrect minority were able to derail the group 21% of the time.) If the majority of the group answered a question incorrectly when answering individually, the group converged on the right answer only 44% of the time. The result of this dynamic was that the average group decision was better, but only marginally so, than the average individual (66% versus 62%).
As a result, it may be easier to simple elicit people’s individual views and average them (or combine in some other novel way) than go through the effort of the group discussion. A statistical group may be a more efficient solution.
Why deliberating groups go wrong (or right)
Why do we get results such as this? Sunstein and Hastie describe plenty of problems that can derail deliberating groups. Group decisions can be poor due to both the rational conduct of group members and because of their “biases”. Here are a few problems that can occur for “rational” reasons:
Informational signals: It is sensible to take into account what others have said in a group deliberation. If you know Jane is knowledgeable and has good judgment, hearing that she supports a project is evidence that can affect your support. But if she is wrong, she can derail the group. Seeing other people make errors can also provide “social proof” to an error.
Self-censorship: People tend not to give information contradicting their preferred outcome. In one study of over 500 mock jury trials, the experimenters never once observed someone giving information in this circumstance.
Reputational cascades: People might know what is right (or what they think is right), but they go along with the group or certain members of the group due to concern for their reputation or standing.
Then there are the “irrational” (a lot of these points are based on single studies, so take with a grain of salt):
Deliberating groups are more likely to escalate commitment to a failing course of action. They are also more susceptible to the sunk cost fallacy, the consideration of past costs that should be irrelevant to the decision about future action
Groups can amplify the representativeness heuristic, where we judge probability based on resemblance or similarity
People in deliberating groups have more unrealistic “overconfidence” (looking at the abstract of the paper cited for this point - I can’t access the full paper - I think they are talking about over-precision)
Groups are more vulnerable to framing effects, varying their decision based on how a choice is framed (although looking at the paper Sunstein and Hastie cite, it states that there is little consistency between studies)
Group deliberation can make both groups and the individuals in those groups more extreme
Shared information has a disproportionate effect on group members. If information is distributed so that key material is unshared (held by only a few group members), this can cause deliberating groups to perform worse.
That said, deliberating groups can temper some biases:
Groups tend to rely less on the availability heuristic - a heuristic by which we judge probability by how easily examples readily come to mind. The heuristic is tempered possibly because the group members have different memories. Across the group the available memories may be somewhat more realistic. That said, groups can be subject to availability cascades. An idea held by one person can spread through the group, eventually producing a widespread belief.
Groups have a lower tendency to anchor, the over-reliance on the first piece of information with which they are presented (even if it is irrelevant to the decision at hand)
Groups tend to have reduced hindsight bias, possibly because not everyone revises their views in the same way
Groups tend to have reduced egocentric biases, the belief that others think like you. A group typically has a wider set of tastes to draw on, so you are more likely to have someone point out that your tastes are not shared.
Improving deliberation
The most interesting part of the book is when Sunstein and Hastie turn to their tactics to improve group decision. There are two groups of tactics: those designed to improve deliberation, and alternative decision-making methods. A common threads to these is diversity, although this is “not necessarily along demographic lines, but in terms of ideas and perspectives.”
They list eight ways to avoid problems in deliberating groups: (1) inquisitive and self-silencing leaders; (2) “priming” critical thinking (although we have seen how the priming literature is holding up); (3) rewarding group success (incentives are important, particularly to counter self-censorship and reputational cascades); (4) role assignment; (5) perspective changing; (6) devil’s advocates; (7) red teams; and (8) the Delphi method. A few are worth mentioning.
Role assignment involves giving people discrete roles, such as labelling someone as an “expert”. The purpose is to bring out unshared information by making it clear that the individual expert has a role to play.
Devil’s advocacy involves appointing some group members to deliberately advocate against the group’s inclinations. Sunstein and Hastie suggest that the research behind devil’s advocates is mixed. There is some evidence that devil’s advocacy can be helpful and can enhance group performance. But it requires genuine dissent. If the dissent is insincere (which is often the case if the role is assigned), people discount the dissent accordingly. The advocate also has little to gain by zealously challenging the dominant view. This means it may be better for groups to encourage real dissent.
Sunstein and Hastie are more optimistic about red teaming, the creation of a team tasked with criticising or defeating the preferred solution or plan. I can see how they might be occasionally useful, such as in mock trials, but it wasn’t clear where their optimism came from as they provided little evidence in support.
One option I find useful is the Delphi method. You ask people to state their opinions anonymously and independently before deliberation. These opinions are then made available to others. It is effectively a secret ballot plus reasons, and provides a basis for hidden information to emerge without reputational or informational cascades. Several rounds of this process can be held as the group converges on a solution. It’s a great way to flush out doubts and dissent.
Better decisions without deliberation
Much of the book is dedicated to methods to arrive at good decisions outside of, rather than within, the deliberation process. These include design thinking (as a way of eliciting as much information and as many ideas as possible), cost-benefit analysis, asking the public (public comment or consultation), tournaments, prediction markets, and harnessing experts. Some of these are effectively statistical groups with different models for combining inputs.
Unsurprisingly given Sunstein’s background, the authors are positive on cost benefit analysis. Having seen some cost-benefit sausages being made for government decision-making, I don’t quite share the same optimism, but can see the benefits in the right place.
Sunstein and Hastie are also boosters of use of tournaments. The dispersion of competitors leads to independence in inputs. Their winner take all nature incentivises divergent strategies. They can promote elite performance at the top of competitor’s capabilities.
A question not addressed in the book is to what extent tournaments can be scaled and be a widely used solution. There is a waste of resources inherent in tournaments - the input of the losing teams. A Kaggle competition uses a massive amount of data science capability, far more than the “prize”. At the moment, many candidates are happy to input this effort as there are other benefits, such as reputation. Could it be the standard way of doing things? In the case of government tournaments, they would want to pick the projects of most value to avoid over-stretching the resource.
As a tournament example, Sunstein and Hastie were underwhelmed by the IARPA prediction tournament, where teams competed to predict political and economic events. They felt that the winning solution from the Good Judgement Project was more focused on reducing noise and bias, rather than developing game changing methods that increase signal (tough crowd). (See my post on Superforecasting for more on that tournament.) Maybe the new hybrid forecasting tournament might be more to their liking.
The final technique I’ll note is effective harnessing of experts. This could be using experts who use statistics to develop accurate predictions or make decisions (often in turn drawing on other sources). It could involve identifying fields where expert knowledge is genuine (as identified in the work of Gary Klein). When doing this, however, it is often best to look at statistical groups of experts, rather than to chase a single expert. The average of experts is likely the best prediction. And there is no need to weight for an expert’s confidence in developing that average - it has no correlation with their accuracy.
Postscript 1: Sunstein and Hastie explore the question of collective intelligence (the “c factor”). That deserves to be the subject of another post.
Postscript 2: Sunstein and Hastie talk of “eureka” problems, where the right answer is clear to all once announced. Groups are good at these. They give the “trivial” example of “Why are manhole covers round?” Because “if they were almost any other shape, a loose cover could shift orientation and fall through the hole, potentially causing damage and injuries.” Is that really the logic behind their design? Or is this just a benefit? (I ask not just because most manhole covers in Australia are square or rectangular, and I have never seen a cover fall through the hole.) This example is famous as being used in Microsoft job interviews, but it is a question more focused on making the interviewer feel clever than actually predicting, say, good job performance.