A new paper inPsychological Science(.htm) reports a failure to replicate Study 2 of Ariely and Wertenbroch’s influential article entitled, “Procrastination, Deadlines, and Performance: Self-Control by Precommitment.” The original study, published inPsychological Sciencein 2002 (.htm), found that people performed better on a set of tasks when each task had its own externally imposed deadline than when people set their own deadlines or faced a single last-day deadline for all tasks. The paper has had a lasting influence. It has been assigned reading in many economics and psychology courses, and has more than 2,100 citations on Google Scholar.
Because this paper has been so influential, it is worthwhile to take a close look at the original study to try to understand why it did not replicate. We did that. This post – and the next one – is about what we found.
About 20 years ago, on April 20, 2006, one of the authors of the forthcoming replication, Kyle Hyndman, received the original data files in an email sent from[email protected][1]. And about 3 years ago, on August 9, 2023, a week after Francesca Gino sued us for $25 million, we received an out-of-the-blue email from Hyndman in which he sent those files to us. We performed quick analyses of the data, and then had a conversation with Hyndman and his co-author, Alberto Bisin. In that conversation, they told us they were going to conduct a replication, and, finding ourselves busy with the lawsuit, we left it at that.
We recently learned that their replication was forthcoming inPsychological Science. And upon reading Footnote 14 of their paper, we also learned this:
This motivated us to return to this paper and fully analyze the original data for the two main studies. We conclude that the data in Studies 1 and 2 were tampered with. In two posts, we present the evidence that led us to this conclusion. Today’s post focuses on the study that failed to replicate (Study 2), and our next post is about Study 1.
Our assessment that the data were tampered with are based entirely on the analyses presented in our posts. Readers can review the evidence and draw their own conclusions.
To the best of our knowledge, Klaus Wertenbroch has never had access to any version of the data for any of the studies. And, we believe it is thanks to him that we do. When Kyle Hyndman reached out to the authors back in 2006, Klaus replied with this email [2]:
OurResearchBoxcontains the data and code to reproduce all of the results in this post.
The Study That Did Not Replicate: Study 2 of Ariely and Wertenbroch (2002)As noted above, Ariely and Wertenbroch explored how deadlines influence performance. In a context in which people had multiple tasks to perform, the authors hypothesized that people would perform better in the face of evenly spaced deadlines for those tasks, rather than when they were all due at the end.
The experiment involved an incentivized proofreading task. Each participant received three 10-page documents, each containing 100 “grammatical and spelling errors” (p. 222). Participants were tasked with finding and correcting those errors.
Sixty participants were randomly assigned to one of three conditions, exactly 20 participants in each condition:
Condition 1. Evenly Spaced Deadlines.One document was due each week, so after 7, 14, and 21 days.Condition 2. Set Your Own Deadlines.Participants chose their own deadlines (within 21 days).Condition 3. Last Day Deadline.All three documents were due on the final (21st) day.