Probability & Randomness Judgement

In a random generation task, participants are asked to randomly generate a sequence of items (e.g., from numbers 1-10). Past work conclusively established that human random generation is flawed, and participants’ sequences become less random the faster they are asked to produce them (Towse, 1998). These results have been interpreted as the result of items being generated according to simple schemas (e.g., add one or subtract one) with effortful inhibition of typical outputs, and so faster sequences lead to more stereotyped behaviour (Jahanshahi et al., 2006). However, we have recently reinterpreted random generation as drawing samples for inference: people’s internal sampling process resembles algorithms used in computer science, such as Markov Chain Monte Carlo (Castillo et al., 2023). One empirically-verified prediction of this approach is that participants can randomly generate examples from non-uniform distributions, such as the distribution of UK heights. If that is the case, then what are the causes for people’s more stereotyped random generation under speeded conditions? Is it that at higher production speeds people generate fewer samples between utterances, leading to differences in the resulting sequences? Or does the sampling process change qualitatively when a speed threshold is reached, either in terms of parameters or even structure? We asked participants to randomly produce UK lifespans both at 40 and 80 items per minute, and compared the sequences they produced to several computational models. We assessed how well characteristic features of the sampling algorithm that have been informative in previous experiments changed under speeded conditions. We found large individual differences (which previous research focusing on average trends has not identified), with some participants being more random in the faster sequence, contrary to previous findings. Our results provide insight into the noise and individual variability in cognition, and will help develop better computational models of human inference and decision-making.

This is an in-person presentation on July 20, 2023 (11:00 ~ 11:20 UTC).

Repeated forecasts of changing targets are a key aspect of many everyday tasks, from predicting the weather to financial markets. A particularly simple and informative instance of such moving targets are random walks: sequences of values in which each point is a random movement from only its preceding value, unaffected by any previous points. Moreover, random walks often yield basic rational forecasting solutions in which predictions of new values should repeat the most recent value, and hence replicate the properties of the original series. In previous experiments, however, we have found that human forecasters do not adhere to this standard, showing systematic deviations from the properties of a random walk such as excessive volatility and extreme movements between subsequent predictions. We suggest that such deviations reflect general statistical signatures of human cognition displayed across multiple tasks, offering a window into underlying cognitive mechanisms. Using these deviations as new criteria, we here explore several cognitive models of forecasting drawn from various approaches developed in the existing literature, including Bayesian, error-based learning, autoregressive and sampling mechanisms. These models are contrasted with human data from two experiments to determine which best accounts for the particular statistical features displayed by participants. We find support for sampling models in both aggregate and individual fits, suggesting that these variations are attributable to the use of inherently stochastic prediction systems. We thus argue that variability in predictions is primarily driven by computational noise within the decision making process, rather than "late'' noise at the output stage.

This is an in-person presentation on July 20, 2023 (11:20 ~ 11:40 UTC).

People’s probability judgments are both biased and variable. When asked to judge the probability of binary events, e.g., whether it will rain or not, there is a bias away from extreme values. In addition, there is substantial variability when judgments of the same question are repeated, even when no new information has been presented. This combination of bias and variability has been best explained by sampling-based models. Variability is neatly explained by people basing their probability judgments on randomly recalled or simulated events. Bias though is not an inherent property of random samples, so bias is introduced through noisy counting of samples (e.g., Probability Theory Plus Noise; Costello & Watts, 2014) or by application of a generic prior over probabilities themselves to improve judgment accuracy for small numbers of samples (e.g., Bayesian Sampler; Zhu, Sanborn, & Chater, 2020). These two mechanisms make equivalent predictions for average judgments but are distinguished by their predictions for the relationship between the judgment mean and variance. Using, a recent regression-based technique, Sundh, Zhu, Chater, and Sanborn (in press) found empirical evidence for a generic prior. But the flexibility of the prior was not tested – can it adapt, particularly to environments in which probabilities are not symmetrically distributed (e.g., there are more small, or large, probabilities).? Here we expand the regression-based technique to allow it to identify either symmetric or asymmetric generic priors. Applied to four previous experiments in which participants make repeated probability judgments, the recovered generic prior was close to symmetric. These previous experiments however asked participants to judge event distributions that were themselves symmetric, so to provide a better test, we ran two new experiments in which the distribution of probabilities to judge were asymmetric. We again found that the prior was close to symmetric, suggesting that perhaps the mind has symmetry constraints, the generic prior reflects long-term experience, or that the generic prior is not represented at all but implemented “procedurally” by fixed a process of regression to the mean.

This is an in-person presentation on July 20, 2023 (11:40 ~ 12:00 UTC).

1. Background & Research question: Polarization is a complex and multifaceted issue that has gained increasing attention in recent years (e.g., Bail et al., 2018). But is society really as polarized as often assumed by popular media? Answering this question is no trivial matter given the lack of a clear and agreed-upon definition and measurement of the phenomenon. This heterogeneity can make it difficult to establish the true extent of polarization in society and to design effective interventions to reduce it. Our research contributes to the understanding of polarization by addressing the following two research questions: (1) To what degree does polarization manifest itself in the specific area of risk perceptions regarding Covid-19 measures? (2) How robust are our conclusions when we compare different operationalizations of polarization? 2. Methods & analytic pipeline: The debate around the appropriateness of Covid-19 measures often centers around the potential consequences on both public health and the economy. In order to investigate how individuals' risk perceptions differ depending on the perspective they take, we used a mixed-design study with two between-subjects conditions: One group answered questions from a health perspective and the other group from a finance perspective. Both groups were presented with the same three scenarios: (1) a lockdown scenario, (2) a mandatory Covid-19 certificates scenario, and (3) a vaccine mandate scenario. Importantly, we asked participants to report their risk perceptions regarding consequences both for themselves and society at large, as we believe that people differ in their perceptions regarding these consequences. This study design led to twelve unique combinations of between- and within-subjects conditions. In line with a pre-determined stopping rule, we collected data from a 768 participants in the United States using Amazon Mechanical Turk and followed best-practice recommendations for quality control of online samples (i.e., bot, VPN, and comprehension checks). As the main operationalization of polarization in participants' risk perceptions, we focused on the bimodality coefficient (BC; Lelkes, 2016). Using the runjags package in R, we conducted a preregistered Bayesian parameter estimation to estimate the BC's posterior distribution. In a novel approach, we defined the data-generating process of participants' risk perceptions as a beta distribution, which can assume a uniform, bimodal, or unimodal form. In addition to the BC, we computed seven further polarization indices to estimate the agreement between different measures. 3. Results: Regarding our first research question, we found that six out of the twelve unique combinations indicate credible polarization based on our pre-defined region of practical equivalence (ROPE) for the bimodality coefficient. Specifically, we found four cases of credible polarization in the finance condition and two in the health condition. In the finance condition, it was the mandatory certificate, lockdown, and vaccine mandate for one's financial situation and the vaccine mandate for others' financial situation that were polarized. In the health condition, it was the mandatory certificate and vaccine mandate for one's own health situation that were polarized. It is also notable that the other distributions are relatively uniformly distributed, indicating a high degree of variation and lack of agreement even in the non-polarized distributions of participants' risk perceptions. Regarding our second research question, we compared the posterior estimate of the bimodality coefficient to seven other operationalizations of polarization and found that there is relatively strong agreement between measures. The average absolute correlation (i.e., disregarding the sign) between measures is 0.58. 4. Conclusions & Significance of research: In conclusion, our results suggest that there is credible polarization in regards to certain Covid-19 measures, specifically those that have personal consequences. Moreover, we find that different measures of polarization tend to agree, at least regarding the relatively uniformly distributed data that we observed. Our research highlights the importance of considering the context in which polarization is measured and how it is conceptualized. Furthermore, these findings have important implications for public policy: They suggest that interventions aimed at reducing polarization should focus on addressing risks that individuals may perceive for themselves. In a second study, we use precisely these insights to implement an intervention based on one-on-one interactions between individuals with differing risk perceptions.

This is an in-person presentation on July 20, 2023 (12:00 ~ 12:20 UTC).

Leading theories of subjective probability judgments (SPs) model SPs in terms of the support, or strength of evidence, assigned to a focal hypothesis relative to the support of alternative hypotheses. These theories assume that each hypothesis elicits a fixed level of support regardless of the other hypotheses under consideration. Contradicting this idea, recent research on SPs has found context effects – changes in support for one hypothesis based on the other hypotheses under consideration (Cai & Pleskac, 2023). However, these results were obtained using artificial stimuli in laboratory settings. Do context effects in belief occur in naturalistic forecasting environments? To investigate this, we conducted a study where N = 113 participants judged the likelihood of the final ranking of the men’s NCAA basketball teams one month out. The study occurred in two phases. First, participants were asked to map 50 basketball teams onto a two-dimensional space using a Spatial Arrangement method. Then, based on their mental representations, we presented customized triplets of teams designed to elicit context effects in each participant and asked them, across 180 trials, to judge the probability of one team ranking higher than the other two in the NCAA’s final rankings. Our findings suggest that similarity and attraction effects can occur in this naturalistic environment, and there is some evidence of a compromise effect. These results invalidate the support invariance principle, which rules out a large class of psychological theories of subjective probability judgments that assume this principle. Furthermore, they suggest that belief and preference construction may be driven by similar processes.

This is an in-person presentation on July 20, 2023 (12:20 ~ 12:40 UTC).