Identifying context effect sweet spots: There’s an app for that!

Context effects, wherein the introduction of a third option can seemingly alter the preference relation between two other choice options, are pervasive in decision making. While decades of research have supported their existence, there has been some difference in claims regarding exactly which occur, and under which circumstances. Two specific limitations of previous work may be preventing a fuller understanding of these behaviors: studies tend to use a few characteristic stimuli to test each effect, and they analyze results based on aggregate choice proportions. The former was remedied in a recent study by Dumbalska et al. (2020) in PNAS, which used 32 stimuli spanning a two-dimensional attribute space. We address the latter concern here. In particular, we demonstrate a novel and powerful way of investigating context effects, within subjects, based on relatively simple assumptions about individual choice patterns. The framework translates hypotheses about preference or indifference on each choice problem into probabilistic models characterized by inequality constraints on binary choice probabilities. While ostensibly these models form convex polytopes in a 32-dimensional space, which would seem computationally unwieldy, it turns out to be computationally efficient to calculate Bayes factors for their empirical performance by treating them as cross-products of line-segments. We offer an easy-to-use and openly available web application allowing other researchers to test, virtually instantaneously, their own sets of hypotheses on the data from Dumbalska et al. (2020).