Not the jam study again

Go to any behavioural science conference, event or presentation, and there is a high probability you will hear about “the jam study”. Last week’s excellent MSiX was no exception, with at least three references I can recall. The story is wonderfully simple and I have, at times, been mildly sympathetic to the idea. However, it is time for this story to be retired or heavily qualified with the research that has occurred in the intervening years.

As a start, what is the jam study? In 2000, Mark Lepper and Sheena Iyengar published their findings (ungated pdf) about the response of consumers to displays of jam in an upmarket grocery store. Their paper also contained similar experiments involving choice of chocolate and essay questions, but those experiments have not gained the same reputation.

On two Saturdays, they set up tasting displays of either six or 24 jars of jam. Consumers could taste as many jams as they wished, and if they approached the tasting table, also received a $1 discount coupon to buy the jam. For attracting initial interest, the large display of 24 jams did a better job, with 60 per cent of people who passed the display stopping. Forty per cent stopped at the six jam display. But only three per cent of those who stopped at the 24 jam display purchased any of the jam, compared with almost 30 per cent who stopped at the six jam display.

This result has been one of the centrepieces of the argument that more choice is not necessarily good. The larger display seemed to reduce consumer motivation to buy the product. The theories around this concept and the associated idea that more choice does not make us happy are often labelled the choice overload hypothesis or the paradox of choice.

Fast-forward 10 years to another paper, this one by Benjamin Scheibehenne, Rainer Greifeneder and Peter Todd. They surveyed the literature on the choice overload hypothesis – there is plenty. And across the basket of studies, evidence of choice overload does not emerge so clearly. In some cases, choice increases purchases. In others it reduces them. Scheibehenne and friends determined that the mean effect size of changing the number of choices across the studies was effectively zero.

More pointedly, the reviewed studies included a few attempts to replicate the jam study results. An experiment using jam in an upscale German supermarket found no effect. Other experiments found no effect of choice size using chocolates or jelly beans. There were small differences in study design between these and the original jam study (as original authors are always quick to point out when replications fail), but if studies are so sensitive to study design and hard to replicate, it seems foolhardy to extrapolate the results of the original study too far.

That is not to say that there is not something interesting going on here. Scheibehenne and friends suggest that there may be a set of restrictive conditions under which choice overload occurs. These conditions might involve the complexity (and not the size) of the choice, the lack of dominant alternatives, assortment of options, time pressure or the distribution of product quality. These considerations are not issues of the size of the choice itself but the way the choice is undertaken. And since the jam study appears tough to replicate, these conditions might be particularly narrow. Still, the common refrain of making it easy for customers – as recommended for dealing with choice overload issues – holds for most of them. But they suggest different and more subtle solutions than simply reducing choice.

There are a lot of interesting studies floating around on choice overload – from decisions about turning off life-support (ungated pdf) to retirement savings (ungated pdf) – and the message is not always the same. But reading through them makes it clear that the jam study is just the tip of an iceberg and not necessarily representative of what lies beneath.

Finally, when Tim Harford wrote about these studies several years ago, he pointed out another often neglected argument about the importance of choice. It is only because we have choice that we are offered any good products at all, with companies incentivised to compete for us as customers. Even if choice has negative consequences, a world without choice might be worse.

5 thoughts on “Not the jam study again

  1. I entirely agree with you on this. I had heard that the jam study was not replicated in a different setting, and I certainly don’t think believe choice reduction is always a good idea. It is also difficult to test in a supermarket, because any brand with a larger range will dominate shelf-space and may be bought more for this reason alone. One crisp manufacturer in the UK often introduces new flavours as much to dominate the shelf as for reasons of consumer choice (most people buy one of just three flavours, regardless of what else is on offer).

    The importance of the finding that choice reduction can sometimes increase sales, however, is that both economists and normal people may never even consider the possibility that this may be true, and will unquestioningly assume that you will always sell more if you offer more things. The fact that this instinctive assumption may be wrong on occasion is a really important corrective.

    Let’s say you are Apple. You currently offer iPads with 16, 32 and 64 gigabytes of storage, at ascending prices. It is suddenly possible to offer a version with 128. Should you do this?

    “I don’t know” is the right answer. But many people would say “yes, definitely.”

    Sometimes science works by expanding the range of our ignorance!

  2. Jason – if the 2000 study is hard to replicate now, could it also be due to 2014 consumers being desensitised to the 24 vs 6 display differences? Sort of the consequence of ongoing choice overload maybe?

    1. Maybe, but if I was going to make that argument, I’d use the word “adapting” rather than “being desensitised”. However, having a quick scan down the dates of the studies in the Scheibehenne and colleagues paper, there doesn’t seem to be a pattern of later papers having less choice overload. There are plenty of papers from 2003 and 2004 where consumption went up with choice.

  3. An implicit assumption in the background when labeling choice overload as bad is that the ultimate goal should be to increase consumption as much as possible. This is of course true for businesses, but should this be the goal for society as a whole as well? If a simple choice overload can make us buy less of something, then one must ask the question whether that something is really essential in our lives. [OK, philosophy mode off.]

  4. What’s interesting in the food and beverage side of CPG is a massive proliferation in total SKUs per brand and this dates back about 5 to 10 years. Look at fast-growing brands like Amy’s Kitchen which now has upwards of 400 SKUs on the aisles. Amy’s Kitchen is not an outlier here. Brands you have never heard of (Clover Organics) place hundreds of products into stores, where they used to place just a dozen or so a decade ago. Because we’re all primadonnas now in our rapidly-shifting tastes. Because it’s easier to do under digital transformation. Because it’s profitable to constantly test new products. Coke is testing 100s of new products around the world every quarter.

    On the B2B services and high-end software side, you see this at the enterprise level – I have lost count of how many products and solutions Salesforce sells. To be fair, they can afford sloppy product-suite-positioning because of their market position. IE, they can afford to have too many products. But still, they are never going back to 3 or 4 jars of jam.

    I will say this though, the “rule of three” still applies to pricing on a single offer.

    I will also say that it’s helpful when you have more than three products/solutions to group them.

Comments welcome

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s