Specifying meaningful joint hypotheses across studies: Bayesian evidence synthesis revisited

Bayesian evidence synthesis (BES) is a method for evaluating the empirical support for a common theory across a set of heterogeneous studies. Evidence is aggregated by multiplying Bayes factors from the individual studies, yielding the product Bayes factor (PBF). The PBF numerator aggregates evidence for the joint predicted hypothesis, which states that the study-specific predicted hypothesis holds in each study. This hypothesis reflects the belief that the theory holds across different contexts, treating each study-specific predicted hypothesis as an instance of the theory. However, since Bayes factors are relative measures of evidence, interpreting support for the joint predicted hypothesis via the PBF requires a clear understanding of the alternative against which it is compared. This joint alternative hypothesis arises “automatically” in the PBF denominator when multiplying the individual Bayes factors and states that the study-specific alternative hypothesis holds in each study. However, due to its implicit, bottom-up construction, this joint alternative is at risk of lacking a clear substantive or theoretical interpretation, which makes it unclear how results from BES can be understood. In this project, we investigate how different constellations of study-specific alternative hypotheses give rise to distinct joint alternative hypotheses. We focus on three common types of alternative hypotheses at the study-level: the null, the complement, and the unconstrained alternative. We develop principled recommendations for constructing meaningful joint alternative hypotheses that support clear interpretation of BES and are useful in the context of theory-testing.