Method notes · 01
What preregistration prevents — and what it cannot prevent
A timestamped analysis plan closes some escape routes, but it does not turn a weak question, noisy measurement, or dishonest report into strong evidence.
In brief
- It separates predictions made before the data from explanations invented after them.
- It makes undisclosed outcome switching and optional stopping harder.
- It does not guarantee good measurement, adequate power, honest execution, or a correct model.
The problem is freedom after the fact
Most datasets permit more than one reasonable analysis. A researcher can exclude one awkward group, try a different outcome, stop after a promising week, or notice a pattern nobody predicted. Each choice may sound defensible on its own. The danger appears when the choices are made after their consequences are visible and only the successful path is reported.
Preregistration writes down the question, outcome, exclusions, stopping rule, statistical test, and decision threshold while the answers are still unknown. Later exploration remains allowed. It simply has to travel under its own name, so a reader can distinguish a test from a discovery that needs a new test.
A lock, not a certificate
A locked plan can faithfully preserve a bad plan. The sample may be too small, the outcome may not represent the claim, participants may infer the answer, or the model may assume independence where none exists. Preregistration exposes those decisions early; it does not make them wise.
Nor does a timestamp prove that the procedure was followed. That requires a results report that lists deviations, open data that can be checked, and analysis code that reproduces the published figures. The protocol is one member of an audit trail, not the whole trail.
How the registers use it
Every register here has a versioned protocol in the public repository. The page prints a SHA-256 digest of that file. When collection opens, the digest is frozen with the opening time; a later amendment is appended and dated instead of silently replacing the filed text.
The stopping rule matters as much as the hypothesis. Results are interpreted only at the milestones named in advance, and a quiet or dramatic interim result cannot create a new finish line. That is why a protocol page can look uneventful: before collection, the absence of a result is the evidence that the order of operations is intact.
Sources
- The preregistration revolution
Nosek et al., Proceedings of the National Academy of Sciences 115(11), 2018 · Peer reviewed · DOI 10.1073/pnas.1708274114 · SRC-FEBFD2E5 - Preregistration
Center for Open Science, OSF documentation · Institutional source · SRC-99B5BDF2 - Registered Reports
Center for Open Science · Institutional source · SRC-08A548E9 - False-positive psychology
Simmons, Nelson & Simonsohn, Psychological Science 22(11), 2011 · Peer reviewed · DOI 10.1177/0956797611417632 · SRC-CF7F6955