Teacher expectations and self-fulfilling prophesies
I first came across the idea of teacher expectations turning into self-fulfilling prophesies more than a decade ago, in Steven Covey’s The 7 Habits of Highly Effective People:
One of the classic stories in the field of self-fulfilling prophecies is of a computer in England that was accidently programmed incorrectly. In academic terms, it labeled a class of “bright” kids “dumb” kids and a class of supposedly “dumb” kids “bright.” And that computer report was the primary criterion that created the teachers’ paradigms about their students at the beginning of the year.
When the administration finally discovered the mistake five and a half months later, they decided to test the kids again without telling anyone what had happened. And the results were amazing. The “bright” kids had gone down significantly in IQ test points. They had been seen and treated as mentally limited, uncooperative, and difficult to teach. The teachers’ paradigms had become a self-fulfilling prophecy.
But scores in the supposedly “dumb” group had gone up. The teachers had treated them as though they were bright, and their energy, their hope, their optimism, their excitement had reflected high individual expectations and worth for those kids.
These teachers were asked what it was like during the first few weeks of the term. “For some reason, our methods weren’t working,” they replied. “So we had to change our methods.” The information showed that the kids were bright. If things weren’t working well, they figured it had to be the teaching methods. So they worked on methods. They were proactive; they worked in their Circle of Influence. Apparent learner disability was nothing more or less than teacher inflexibility.
I tried to find the source for this story, and failed. But what I did find was a similar concept called the Pygmalion effect, and assumed that Covey’s story was a mangled or somewhat made-up telling of that research.
What is the Pygmalion effect? It has appeared in my blog feed twice in the past two weeks. Here’s a slice from the first, by Shane Parrish at Farnam Street, describing the effect and the most famous study in the area:
The Pygmalion effect is a psychological phenomenon wherein high expectations lead to improved performance in a given area. Its name comes from the story of Pygmalion, a mythical Greek sculptor. Pygmalion carved a statue of a woman and then became enamored with it. Unable to love a human, Pygmalion appealed to Aphrodite, the goddess of love. She took pity and brought the statue to life. The couple married and went on to have a daughter, Paphos.
…
Research by Robert Rosenthal and Lenore Jacobson examined the influence of teachers’ expectations on students’ performance. Their subsequent paper is one of the most cited and discussed psychological studies ever conducted.
Rosenthal and Jacobson began by testing the IQ of elementary school students. Teachers were told that the IQ test showed around one-fifth of their students to be unusually intelligent. For ethical reasons, they did not label an alternate group as unintelligent and instead used unlabeled classmates as the control group. It will doubtless come as no surprise that the “gifted” students were chosen at random. They should not have had a significant statistical advantage over their peers. As the study period ended, all students had their IQs retested. Both groups showed an improvement. Yet those who were described as intelligent experienced much greater gains in their IQ points. Rosenthal and Jacobson attributed this result to the Pygmalion effect. Teachers paid more attention to “gifted” students, offering more support and encouragement than they would otherwise. Picked at random, those children ended up excelling. Sadly, no follow-up studies were ever conducted, so we do not know the long-term impact on the children involved.
The increases in IQ were 8 IQ points for the control group, and 12 points for those who were “growth spurters”. (The papers describing the study - from 1966 (pdf) and 1968 (pdf) - are somewhat thin on the experimental methodology, but it seems the description used in the study was “growth spurters” or high scorers in a “test for intellectual blooming”).
I always took the Pygmalion effect with a grain of salt. Most educational interventions have little to zero effect - particularly over the long-run - even when they involve far more than giving a label.
As it turns out, the story is not as clean as Parrish and others typically tell it. There have been battles over the Pygmalion effect since the original paper, with failed replications, duelling meta-analyses and debates about what the Pygmalion effect actually is.
Bob C-J discusses this at The Introduction to the New Statistics (HT: Slate Star Codex - the second appearance of the Pygmalion effect in my feed). Here is a cut of Bob C-J’s summary of these battles:
The original study was shrewdly popularized and had an enormous impact on policy well before sufficient data had been collected to demonstrate it is a reliable and robust result.
Critics raged about poor measurement, flexible statistical analysis, and cherry-picking of data.
That criticism was shrugged off.
Replications were conducted.
The point of replication studies was disputed.
Direct replications that showed no effect were discounted for a variety of post-hoc reasons.
Any shred of remotely supportive evidence was claimed as a supportive replication. This stretched the Pygmalion effect from something specific (an impact on actual IQ) to basically any type of expectancy effect in any situation…. which makes it trivially true but not really what was originally claimed. Rosenthal didn’t seem to notice or mind as he elided the details with constant promotion of the effect. …
Multiple rounds of meta-analysis were conducted to try to ferret out the real effect; though these were always contested by those on opposing sides of this issue. …
Even though the best evidence suggests that expectation effects are small and cannot impact IQ directly, the Pygmalion Effect continues to be taught and cited uncritically. The criticisms and failed replications are largely forgotten.
The truth seems to be that there are expectancy effects–but:
that there are important boundary conditions (like not producing real effects on IQ)
they are often small
and there are important moderators (Jussim & Harber, 2005).
The Jussim and Harber paper (pdf) Bob C-J references provides a great discussion of the controversy. (Bob C-J also recommends Social Perception and Social Reality: Why Accuracy Dominates Bias and Self-Fulfilling Prophecy by Lee Jussim). Here’s a section of the abstract:
This article shows that 35 years of empirical research on teacher expectations justifies the following conclusions: (a) Self-fulfilling prophecies in the classroom do occur, but these effects are typically small, they do not accumulate greatly across perceivers or over time, and they may be more likely to dissipate than accumulate; (b) powerful self-fulfilling prophecies may selectively occur among students from stigmatized social groups; (c) whether self-fulfilling prophecies affect intelligence, and whether they in general do more harm than good, remains unclear, and (d) teacher expectations may predict student outcomes more because these expectations are accurate than because they are self-fulfilling.
That paper contains some amusing facts about the original Rosenthal and Jacobson study. Some students had pre-test IQ scores near zero, others near 200, yet “the children were neither vegetables nor geniuses.” Exclude scores outside of the range 60 to 160, and the effect disappears. Five of the “bloomers” had increases of over 90 IQ points. Again, exclude these five and the effect disappears. The original study is basically worthless. While there is something to the effect of teacher expectations on students, the gap between the story telling and reality is rather large.