The Pros and Cons of Preregistration

Preregistration innoculates researchers against the myriad of biases that all humans inevitably succumb to: hindsight bias, confirmation bias, motivated reasoning, the bias blindspot, and many more. Without preregistration, researchers are not attended to the fact that they are cherry-picking among hypotheses or among likelihood functions. Without preregistration, the probability that an ESP researcher reports the absence of ESP is about as low as the probability that a mathematical psychologist reports that the data undercut their pet model and support that of their rival (has it ever happened?). As a Ulysses contract, however, preregistration may tie the researcher to the mast a little too tightly: when the data contain unexpected patterns this demands a different analysis than was originally foreseen, and the penalty of classifying the new analysis as "exploratory" is overly harsh. There is considerable promise in two alternative Ulysses contracts: analysis blinding and the mini-multi-analysts approach. The feasibility of these contracts will be discussed.