December 5, 2011

Do recent publications prove Anders Ericsson and colleagues wrong about the importance of deliberate practice? No.

About deliberate practice
I have written much about deliberate practice. Researchers have demonstrated there is a lack of evidence for the claim that natural ability is the main factor behind top performance. They have found out that what is crucial instead is the amount of time the individual has practiced and the specific way in which he or she has practiced (read more about deliberate practice here and here). Recently, two articles were published on the relative importance of deliberate practice and 'talent' for achieving high levels of performance. First, there was Deliberate Practice Is Necessary but Not Sufficient to Explain Individual Differences in Piano Sight-Reading Skill by E. Meinz and D. Hambrick. Second, there was Deliberate Practice: Necessary But Not Sufficient by G. Campitelli and F. Gobet. Do these articles shed a new light on how important deliberate practice is? Do they call for a return to the idea that innate abilities are, in the end, more important? Did these authors prove Anders Ericsson et al. wrong? No. I'll explain why.

Two new articles: deliberate practice necessary but not enough?
The authors of the first article report they found that deliberate practice accounted for nearly half of the total variance in piano sight-reading performance but that there was an incremental positive effect of working memory capacity (WMC). They go on to say that and there was no evidence that deliberate practice reduced this effect and that WMC is highly general, stable, and heritable, and that these results thus call into question the view that expert performance is solely a reflection of deliberate practice. The authors of the second article report on the basis of a survey of chess players that there is strong evidence that abundant deliberate practice is necessary (but not sufficient). The authors also review evidence showing that other factors play a role in chess skill: general cognitive abilities, sensitive period, handedness, and season of birth.

Here are some comments by me:
  1. Deliberate practice is supported: As blogger Cal Newport points out here the first paper actually emphasizes the necessity of deliberate practice for high achievement. The research shows that the impact of deliberate practice is much stronger than the impact of working memory which only yields only a minor improvement for an already well-practiced player (the authors found that this memory capacity accounts for less than 7% of a player’s ability at this task.). 
  2. Working memory turns out to be malleable: In addition to this, Ben Healey, in a comment to the Carl Newport's post, justly points out that the authors claim that working memory is highly general, stable, and heritable stands to debate: "there is evidence emerging that this might not be the case. For instance, a recent controlled trial by Jeaggi et. al. found that an n-back training exercise can improve working memory capacity."
  3. Misrepresentation of the literature: The second article states that "The claim of the DP [deliberate practice] framework is that such behavior is both necessary and sufficient in order to achieve high levels of expert performance." This is surprising statement. They do not support this statement with a specific reference and I think I know why. Nowhere in the literature have I ever encountered the claim by Anders Ericsson that deliberate practice is sufficient to achieve high levels of expert performance. On the contrary, on several occasions, he acknowledged the likelihood of other personal and environmental determinants. What Campitelli and Gobet do here seems to be a strawman argument, an argument based on misrepresentation of an opponent's position. (Readers, if I am wrong and you can find actually find examples of Ericsson having said that deliberate practice is sufficient, please send it to me and I will publish it here). 
  4. Methodical weaknesses: A weakness of the first article is that most of the musicians in the study were not yet skilled sight readers. It is generally acknowledged that performance of beginners in any performance domain correlates with already present abilities. Once an expert level is achieved, however, basic abilities no longer correlate significantly with performance. My understanding from the literature is that acquired skills overtake abilities as competence grows. The second article suffers from several other weaknesses among which: 1) the authors do not define deliberate practice accurately enough, 2) they use anonymous self-reports which make it impossible to establish whether deliberate practice was actually used and to which extent, 3) they do mention several factors which correlate with high achievement but do not convincingly demonstrate that deliberate practice was not a mediating factor. 
Conclusion: Some people have suggested to me on the basis of these articles 1) that deliberate practice is actually not so important and that Anders Ericsson and his colleagues have overstated their case for deliberate practice, and 2) that talent is actually still a major determinant of high achievement. But these articles do not justify these arguments. Sure, much future research is needed to better understand which factors affect skill and knowledge development. The deliberate practice framework will surely be refined. But the importance of deliberate practice still stands firmly, as does the statement that talent is overrated. 

8 comments:

  1. You're absolutely right!

    ReplyDelete
  2. I haven't read the primary sources yet but from this review I tend to agree, these articles seem to be arguing things that people take away from the expertise research sometimes but not so much what Ericsson et al have been arguing. The expertise researchers have helped define the domain of structured activity and measures of ability and as a result have given us valuable insights into what it takes to internalize the practice of the activity and improve at it. I don't see that fundamental insight seriously challenged by any of this other work yet. Ericsson doesn't argue against the role of implicit learning or make extreme claims about deliberate practice being enough to become an expert, he mostly seems to support the idea that the structure of an activity can't just be grasped all at once but takes time and refinement to achieve, and that there is a critical role for deliberate attention in that process. I don't know of any serious challenge to that model. Barth and his colleagues who study unaware influences on thinking make similar points about the necessity of deliberate attention at various points in thinking and learning.

    ReplyDelete
  3. Hi Todd, thanks, indeed. In Scientific American there was an article by Scott Barry Kaufman who raises some additional criticisms to Hambrick and Meinz: http://bit.ly/n8u0QH

    ReplyDelete
  4. Thanks for this reference, Scott is a great contributor to this literature, I've been finding a number of interesting articles and book chapters by him.

    ReplyDelete
  5. Thanks for posting this Coert, and also for making it very visible via google search. Meinz and Hambrick are basically creating a fictitious mini-feud that doesn't exist in order to generate interest (read: funding) in their own approach to studying human ability. And the reason that this feud doesn't exist is because their not even really talking the same language as Ericsson.

    The deliberate practice framework is merely a model for understanding constraints on those development of skill. As an experimental researcher in the psychonomic tradition, Ericsson and several of the other researchers who have studied deliberate practice are interested in deducing lawful relationships between the presence or absence of certain conditions (the constraints outlined in deliberate practice theory) and mental processes (read: not working memory capacity, but rather working memory operations) and metrics of observed domain performance. Ericsson's goal is merely to understand the interrelationships between these constraints as they come together in support of expert performance, and therefore he has no need to make any predictions about how much working memory capacity or IQ correlates with expert performance- that is the realm of *psychometrics*. So comparing their research to deliberate practice research is like comparing apples to oranges. Yes, they're both fruit, but if I ask for apple juice and the waiter hands me something orange I'm going to send it back to the kitchen. And if they come back with a glass that is half orange and half apple (e.g., a regression equation with working memory and deliberate practice in it), I am going to spill it on the floor and demand a refund. Seriously- have you ever tried mixing apple juice and orange juice. Blech!

    Incidentally, the feud between psychomomics and psychometrics is over 50 years old now, and runs far deeper than any of these little "tiffs" that make their way into the headlines. Robert Sternberg wrote about this as far back as his dissertation and the turf war shows no signs of letting up. Unfortunately, the typical consumer of this research doesn't know anything about this, and therefore lacks the necessary context in order to understand what the deliberate practice vs. talent debate is really about-- turf, grants, and jobs.

    ReplyDelete
  6. Coert:

    All of your points are false.

    1. Jaeggi's study was debunked. Numerous failures to reproduce. Look it up.
    2. Ericsson's deliberate practice papers also used self-reports, so by your logic they are just as methodologically flawed.
    3. Actually Ericsson did very definitively say that talent doesn’t matter at all:
    Our theoretical framework can also provide a sufficient account of the major facts about the nature and scarcity of exceptional performance. Our account does not depend on scarcity of innate ability (talent)…. We attribute the dramatic differences in performance between experts and amateurs–novices to similarly large differences in the recorded amounts of deliberate practice (p. 392, emphasis added).
    4. Numerous studies have found that deliberate practice accounts for only 20-30% of the variance in performance. Look it up.

    ReplyDelete
  7. Dear anonymous: what is your name?

    Here is my point by point reply:

    1) Re Jeaggi (malleability of working memory): what study are you talking about? Here is a meta-analysis which shows a small but significant effect of n-back training on working memory: http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-et-al2014_nback-training-gf-meta-analysis.pdf

    2) As far as I know, whenever Ericsson et al have tried to use objective measures as much as possible. Contrary to other researchers, Ericsson et a, when using self-reports, they have used additional methods from other sources (such as diaries and parent or coach reports) to verify the self-report information.

    3) I do not agree that quote can be summarized as: “Talent does not matter at all.” Here is a quote by Ericsson from 1993: “Although we are reluctant to accept individual differences in innate abilities (talent) and any important role of these differences in determining expert performance, we do not rule out the importance of individual differences in general.” Recently he said: “There is a fundamental difference between claiming that there is evidence for the complete absence of genetic influences on elite performance in sport and claiming that no current evidence exists for such genetic influences.

    4) Ericsson has replied to this comment in 2016 in three separate papers. Look it up.

    ReplyDelete

Enter your email address:

Delivered by FeedBurner