Pre-registration & Analysis Plans

Pre-registration means writing down what you'll test and how you'll analyze it before you look at your data—like a chef committing to a recipe before cooking, rather than inventing the recipe afterward to match whatever came out of the oven. Without this commitment, it's easy to convince yourself you found something real when you're just describing random patterns that happened to look interesting. Ask this: When I hear a surprising claim, was the prediction made before or after the results were known?

How Pre-registration Works

Pre-registration creates a permanent record of your research plan before data collection begins. Think of it as placing your bets before the dice roll—you document your specific hypothesis, what variables you'll measure, how you'll analyze the data, and what patterns would count as evidence for your theory. This commitment separates genuine prediction from convenient storytelling.

Why It Matters

When researchers explore data without a pre-registered plan, they face subtle traps. They might try different ways of grouping participants, exclude certain data points that "don't fit," or test dozens of relationships and report only the ones that look interesting. Each choice seems reasonable, but together they dramatically increase the chance of false positives—findings that look significant but are just coincidences. The same data that generated the hypothesis cannot fairly test it.

Real-World Applications

Medical research: A drug trial. After the study, researchers notice the drug also helps with anxiety—but they acknowledge this as exploratory, requiring separate confirmation, rather than claiming it as a proven effect. Business decisions: A marketing team pre-registers that a new ad campaign will be judged successful if it increases click-through rates by 5%. When sales also increase, they treat it as a bonus observation rather than the primary test. Personal decisions: Before starting a new productivity system, you write down what counts as success (tasks completed per hour) rather than moving the goalposts afterward based on how you feel.

Making It Practical

You don't need formal platforms to benefit from this principle. Before starting any project, write down: what you're testing, what counts as evidence, and what you'll do with ambiguous results. The act of committing to a plan—even privately—helps you recognize when you're genuinely learning versus when you're simply rationalizing after the fact.

Pre-registration and analysis plans address the replication crisis in psychology and other sciences by reducing 'researcher degrees of freedom'—the many seemingly arbitrary decisions researchers make during analysis that can influence results. Registered Reports, a publishing format where studies are peer-reviewed before data collection, have emerged as a powerful implementation of this principle.

• Nosek et al. (2018): Highlighting the need for better research practices including pre-registration [1]
• Chambers et al. (2014): Introduced Registered Reports as a publication format; studies published this way show dramatically higher replication rates because methods are evaluated before results are known [2]
• Simmons et al. (2011): Showed that 'undisclosed flexibility' in data collection and analysis allows researchers to present almost any data set as statistically significant, earning their paper the memorable title 'False-Positive Psychology' [3]

Pre-registration is not a panacea. Poorly designed studies can still be pre-registered, and registration doesn't fix fundamental measurement flaws. Some argue it stifles exploratory research, but most experts distinguish confirmatory from exploratory work—both are valuable, but they require different standards. Pre-registration can be gamed through vague hypotheses or post-hoc rationalizations, leading to more specific 'registered reports' formats. Additionally, pre-registration assumes research follows a linear plan-execute model, but real science often involves iterative learning; balancing rigor with flexibility remains an ongoing challenge.