This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Model-based wisdom of the crowd for sequential decisions

Michael Lee
University of California, Irvine ~ Cognitive Sciences
Jeff Coon
University of California, Irvine ~ Cognitive Sciences
Bobby Thomas
University of California, Irvine ~ Cognitive Sciences
Holly Westfall
University of California, Irvine ~ Cognitive Sciences

We use cognitive models to apply the wisdom of the crowd to three sequential decision making problems: bandit problems, optimal stopping problems, and the Balloon Analogue Risk Task (BART). In each of these problems, people make a sequence of choices under uncertainty, with individual differences in decision making that depend on different attitudes toward risk. Each of the problems also has a known optimal decision-making strategy. Standard methods for the wisdom of the crowd, based on taking the modal behavior, are generally not applicable to these problems, because of their sequential nature. For example, the state-space of a bandit problem can be so large that, even for a large crowd of people, there will be game states that no individual encountered, and so there is no behavior to aggregate. We solve this problem using cognitive models, and inferring individual-level parameter values that predict what each individual would do for each sequential decision. The mode of these model-based predicted decisions then defines a crowd decision whose performance can be compared to individual and optimal performance.



wisdom of the crowd
optimal stopping
bandit problems


Cognitive Modeling
Decision Making
Bayesian Modeling
Ranked voting Last updated 3 years ago

Hi Michael, Really fantastic talk. The next time I go to Disneyland, I'm going to make sure to check in with you as to which rides are the best to go to :). I was wondering whether you've considered analyzing real-world ranked voting data from political voting (I believe this is a case of sequential "wisdom" of the crowds). It's not obvious t...

Prof. Joe Austerweil 2 comments

Thanks for the excellent talk. We wonder how you would estimate the Wisdom of the crowds when people provide a range / interval of their judgement (and assuming this range differs per person and with time). From our experience people will find it hard to pick a precise number c but will be able to come up with an interval (“my confidence that it’ll...

Prof. Gerit Pfuhl 1 comment
Exploitation or perseveration? Last updated 3 years ago

Hi Michael, Great talk. One terminological quibble --- in the last third of the talk, on bandit problems, you mention that a behavioral crowd approach will be overly influenced by "exploitation-biased" participants. I'm not convinced these participants are necessarily overly exploitative -- they may simply be perseverating. (The sequence of ...

Dr. Beth Baribault 2 comments
Cite this as:

Lee, M., Coon, J., Thomas, B., & Westfall, H. A. (2020, July). Model-based wisdom of the crowd for sequential decisions. Paper presented at Virtual MathPsych/ICCM 2020. Via