Close
This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

How to know what you should know: Implications of the choice of prior distribution on the behavior of adaptive design optimization

Authors
Sabina J. Sloman
Carnegie Mellon University ~ Department of Social and Decision Sciences
Daniel Cavagnaro
California State University, Fullerton ~ Department of Information Systems and Decision Sciences
Stephen Broomell
Carnegie Mellon University ~ School of Decision Sciences
Daniel Oppenheimer
Carnegie Mellon University, United States of America
Abstract

Adaptive design optimization (ADO) is a state-of-the-art technique for designing experiments for cognitive modeling (Cavagnaro, Myung, Pitt, and Kujala, 2010). ADO dynamically identifies stimuli that, in expectation, yield the most information about the hypothetical construct of interest (e.g., parameters of a cognitive model). To calculate this expectation, ADO leverages the modeler’s existing knowledge, specified in the form of a prior distribution. “Informative” priors, constructed on the basis of domain knowledge or previous data, have the potential to align the prior with the empirical distribution in the participant population, thereby making ADO maximally efficient. However, if the informative prior is inaccurate, i.e., “misinformative,” then ADO may be led astray, leading to wasted trials and lower efficiency. To play it safe, many researchers turn to “uninformative” priors. Yet, priors chosen on the basis of their predictive agnosticism rather than insight are also unlikely to align with the population distribution, possibly making them equally inefficient. In on-going work, we investigate the consequences of informative, misinformative and uninformative prior distributions on the efficiency of experiments using ADO.

Tags

Keywords

adaptive design optimization
optimal experimental design
cognitive modeling
Discussion
New

There is nothing here yet. Be the first to create a thread.

Cite this as:

Sloman, S., Cavagnaro, D., Broomell, S., & Oppenheimer, D. (2022, July). How to know what you should know: Implications of the choice of prior distribution on the behavior of adaptive design optimization. Paper presented at Virtual MathPsych/ICCM 2022. Via mathpsych.org/presentation/848.