May 11, 2017

How will stereotype threat get through the replication crisis?

The replication crisis within psychology has shown that rather many findings from previous research cannot be trusted. Some cynics see this as a reason to write of all of psychology and to not take it seriously any longer. I find that illogical and unwise. The replication crisis is a result of a failure of the methodical and statistical quality of old studies and of too limited attention for replication studies and negative findings. The solution cannot be to throw out the whole idea of a scientific approach to psychological topics. Non-scientific attempts to build psychological knowledge are even much weaker methodologically and statistically and even more negligent of contrary evidence and thus even much more unreliable. The solution must be to strengthen scientific psychological research by improving its methods and practices (read here how Carol Dweck is very seriously doing this). The current situation makes it necessary to critically check all psychological findings of the past. We can't automatically assume that research findings from the past can be trusted.

Stereotype threat: strong effects on performance

One of the topics I am curious about is the research into stereotype threat. Briefly, stereotype threat is the phenomenon that when people are reminded of a stereotype about the social, ethnic, gender, or age group to which they belong their performance may be seriously undermined. That you have to think about how people of 'your group' are stereotyped keeps you from focusing fully on the task you are doing. Your stress arousal grows, you start to monitor your own performance while doing the task, and you are busy trying to suppress negative feelings. Research by Claude Steele and Joshua Aronson (see here) and by others has suggested that the effects of stereotype threat can be substantial.

Reasons for doubt

As far as I know there has not yet been a large replication study of the classic strereotype threat studies. But there are some indications that the replication crisis could affect the status of the concept somewhat. Flore & Wicherts (2015) did a meta-analysis in which they estimated the effects of stereotype threat on math performance of girls. They did find a small effect but also wrote that the studies showed large variation in outcomes and that it is likely that the effect is inflated by publication bias. Another sign that there may be some problems in earlier studies can be found in a blog post by Schimmack (2017). In this post he statistically reanalyzed four studies which are mentioned in the most frequently cited stereotype threat publication, Steele & Aronson (1995). For this purpose he used two new techniques, the R-Index and the Test of Insufficient Variance (TIVA). Based on his analysis he concludes that the evidence for stereotype threat in this publication is relatively weak and that there is reason for doubt about the replicability of these findings.


The topic of stereotyping is important. I think stereotyping, and categorizing people in general, can cause lots of individual suffering and societal damage. It seems plausible that stereotypes can have negative effects in performance situations. But are those effects as strong as the previous research has suggested? I think it is only a matter of time before big replication studies will take place into stereotype threat. I expect that a large scale replication study will find a stereotype effect. But based on the information mentioned above, I will not be surprised if it is weaker than previous studies have suggested.

No comments:

Post a Comment

Enter your email address:

Delivered by FeedBurner