Symposium: Computational Models Of Confidence And Metacognition

The ability to introspect and communicate confidence about one's thoughts, actions, and perceptions is a defining feature of human behavior. In addition, there is evidence for substantial interindividual variation of metacognitive abilities from clinical, educational and personality research. To better understand these interindividual differences and underlying mechanisms, there is increasing interest in inferring latent metacognitive parameters from empirical datasets involving confidence ratings. These parameters broadly fall in two categories: metacognitive biases and metacognitive inefficiencies. In this talk, I will present the ReMeta toolbox as a flexible framework to enable such inferences. Researchers can specify a generative model (e.g., expected metacognitive noise distribution) and metacognitive bias and noise parameters at different levels of the processing hierarchy. In particular, researchers can indicate whether metacognitive noise is more likely to occur at the readout stage (e.g., a metacognitive module reading out sensory evidence) or at the report level. I will demonstrate parameter and model recovery results for the default models of the toolbox and provide validation with empirical data. Current opportunities and remaining challenges are discussed.

This is an in-person presentation on July 22, 2024 (11:40 ~ 12:00 CEST).

Metacognition refers to the ability to understand and reflect on one's own cognitive processes. Model-based analysis approaches have been developed to allow for quantification and separation of latent metacognitive processes. This is important because metacognition is influenced by multiple potentially orthogonal characteristics including metacognitive sensitivity (the degree to which confidence dissociates between correct and incorrect decisions), metacognitive bias (the absolute level of confidence reported regardless of objective accuracy), and by 1st-order task performance itself. Many studies have shown that both clinical and sub-clinical symptoms of psychopathology are associated with systematic metacognitive alterations. For instance, anxiety and depression are associated with metacognitive bias towards low confidence. However, results concerning metacognitive sensitivity have been mixed, potentially due to difficulties in its measurement such as confounding by first-order task performance. A recently developed process model of metacognition (ReMeta) may provide a more optimal means of estimating both sensitivity and bias independently of first-order task performance. Here, we investigated relationships between psychiatric symptom dimensions and perceptual metacognition in a large general population sample (N>1000) using measures derived from ReMeta. The results confirmed robust relationships between symptom dimensions and metacognitive bias: An anxious-depression symptom dimension was characterised by systematic underconfidence in perceptual decisions, whereas a compulsivity symptom dimension was was characterised by systematic overconfidence. In contrast, little to no evidence was found for any symptom-related alterations in metacognitive noise/sensitivity. Sub-clinical psychiatric symptom dimensions are not associated with a reduction in metacognitive insight but rather with changes in the absolute levels of confidence reported.

This is an in-person presentation on July 22, 2024 (12:00 ~ 12:20 CEST).

Metacognitive bias, particularly positive evidence bias, is a useful tool for studying metacognition since it allows a clear dissociation between confidence behaviors and the accuracy of primary perceptual processes. Our motivating question for this study asks to what extent these kinds of metacognitive biases are similar or generalizable across different types of tasks. We have created a novel online library of four 3-Alternative Forced Choice perceptual (PTs) and cognitive/value-based (C/VTs) reaction time behavioral tasks to collect within-subject measures of performance (“T1”) and confidence (“T2”) within 14 distinct conditions that are shared across all tasks. We use this library to characterize metacognitive behavioral profiles (BPs) for each subject in each task, i.e. quantitative “fingerprint”-like relationships between performance and confidence. To quantify each BP we built a choice frequency distribution for each presented choice defining T1-BP for each condition within each task. For T2-BP for each condition within each task, we calculated the average confidence rating conditioned on choice. We computed T1-BPs and T2-BPs in this way for all tasks and for all conditions. Then, we quantified the similarity in BPs (defined by the combination of T1-BPs and T2-BPs) within every condition but across tasks using sum of squared error (SSE) as a dissimilarity metric. To test for systematic differences, we calculated mean SSE1 and SSE2 for all pairs of tasks, across all conditions. We found less dissimilarity within PTs T1-BPs and C/VTs T2-BPs and more dissimilarity within PTs T2-BPs and C/VTs T1-BPs.

This is an in-person presentation on July 22, 2024 (12:20 ~ 12:40 CEST).

In an environment rife with perceptual ambiguity and minimal external feedback, decision confidence plays a vital role in the adaptive control of behaviour. Despite the functional significance of decision confidence, the computational processes underlying confidence judgements in perceptual decisions have yet to be clearly characterised and remain the topic of ongoing debate. To better understand these mechanisms, in this study we sought to address the extent to which prior knowledge informs confidence. Contrary to previous research, we did not require participants to internalise an arbitrary, context-specific prior distribution. Instead, we used a novel psychophysical paradigm which allowed us to capitalise on probability distributions of low-level image features in natural scenes, which are well-known to influence perception. Participants reported the subjective upright of naturalistic image target patches, and then reported their confidence in their orientation responses. We used modelling to relate the probability distributions of low-level image features in natural scenes to the probability distributions of the same low-level features in the targets. As expected, we found that participants’ orientation judgements were consistent with an internalised prior of natural scene statistics. Critically, these same distributions also predicted participants’ confidence judgements. Our findings highlight the importance of using naturalistic task designs that capitalise on existing, long-term priors to further our understanding of the computational basis of confidence.

This is an in-person presentation on July 22, 2024 (12:40 ~ 13:00 CEST).

Recent years have seen a substantial proliferation of static models of confidence and metacognition. The most widely used model, although mostly implicitly assumed by metacognition researchers without empirical testing, is the Independent Truncated Gaussian model (ITG). ITG is the basis of the popular meta-d′/d′ method used to quantify metacognitive ability. However, previous modelling studies of perceptual confidence have not included ITG in formal model comparisons. The present study compares model fit of ITG to seven different alternative models of confidence and metacognition all derived from signal detection theory in a reanalysis of four previously published experiments and one new experiment, (i) a masked orientation discrimination task, (ii) a random-dot motion discrimination task, (iii) a low contrast orientation discrimination task, (iv) a dot numerosity discrimination task, and (v) a low contrast number discrimination task. I show that in all five experiments, alternative models provide a better fit than ITG: In the dot numerosity discrimination task, the best fit is achieved by the signal detection rating model. In the other four experiments, the best fit is achieved by either the weighted evidence and visibility model or the logistic weighted evidence and visibility model, implying that at least two sources of evidence are required to account for perceptual confidence, one related to the discrimination judgment, and one related to the reliability of the perceptual evidence. I discuss implications for the measurement of metacognition.

This is an in-person presentation on July 22, 2024 (15:20 ~ 15:40 CEST).

The computation of confidence judgments in decision-making crucially depends on decision time. This study delves into the role of accumulation time in sequential sampling models of confidence, exploring optimal confidence computation in a Bayesian observer model and fitting empirical data to dynamical confidence models. The formal analysis of the posterior probability of being correct highlights that an optimal observer discounts final available evidence by the accumulation time. Additionally, optimal confidence incorporates evidence about task difficulty when such evidence is available independent of the evidence about stimulus category. We introduce the dynamical visibility, time, and evidence model (dynaViTE), which assumes post-decisional evidence accumulation and accumulation of independent information about stimulus visibility and specifically incorporates accumulation time in confidence determination. We fitted the dynaViTE model to data across four experiments, including three experiments with difficulty manipulations and one experiment with both manipulation of discriminability and a speed-accuracy trade-off manipulation. DynaViTE shows a good fit to observed data, accounting for all observed data patterns. Quantitative model comparisons suggest that human observers leverage accumulation time as a pivotal factor in confidence computation. Significantly, accumulation time affected confidence, even when the observed correlation between response times and confidence was small. The weight participants assigned to the three components - evidence, visibility, and accumulation time - varied considerably across the examined experiments. This variability suggests that individuals can adapt their calculations to the specific situation. Currently, the conditions modulating these weights remain unclear. We present potential mechanisms based on Bayesian confidence and the dynaViTE model.

This is an in-person presentation on July 22, 2024 (15:40 ~ 16:00 CEST).

Accurate decisions tend to be both confident and fast. Nonetheless, there are relatively few models that can simultaneously address this three-way relationship, especially for single stage decisions where participants indicate both their choice and their confidence. Extending on a common decision architecture of the linear ballistic accumulator framework, three models have been proposed – 1) a Multiple Threshold Race model which instantiates the Balance-of-Evidence hypothesis where confidence is determined through the difference between accumulated evidence for competing options (e.g., Reynolds, Osth, Kvam, & Heathcote, in revision), 2) a newly developed Confidence Accumulator model which assumes that confidence itself is accumulated independently for each confidence option, and 3) a newly developed Timing model which assumes confidence can be derived from subjective time. To test these three confidence architectures, we ran two experiments manipulating the length of the confidence rating scale across 2-, 4-, or 6-options in a recognition memory task along with a perceptual task. Different models were compared that made different allowance for how the length of the confidence scale affected model parameters. While both model classes found that thresholds were affected by the length of the scale, drift rates were only minimally affected. Implications for models of confidence and response time will be discussed.

This is an in-person presentation on July 22, 2024 (16:00 ~ 16:20 CEST).

When making a decision, individuals tend to be more accurate when they report higher confidence in their decision. This observation led to the proposal that confidence reflects the posterior probability of making a correct response, given relevant data. In the drift diffusion model framework, this probability is determined by both the total amount of evidence sampled and accumulation time. To adequately compute confidence, one thus needs to learn the proper mapping from their readout of their decision process (i.e. evidence and time) to the corresponding probability of the decision being correct (further referred to as confidence mapping). The exact mechanical underpinnings of this learning process are still mostly unknown, as current computational models of confidence often implicitly assume this learning to be terminated. In this work, we present a new modelling framework where the confidence mapping is instead continuously updated according to feedback. We tested this model as well as the proposal that humans learn the confidence mapping from feedback in two perceptual decision making experiments where participants were alternating between two different feedback contexts. As predicted, confidence ratings progressively increased (resp. decreased) after switching into high (resp. low) feedback blocks, while objective performance (i.e. accuracy and reaction time) was not affected by feedback. Importantly, our learning model was able to precisely capture this evolution of confidence over time. Overall, this work highlights the importance of taking into account the dynamics of the computation of confidence, and sheds new lights on how confidence biases and other metacognitive inefficiencies may emerge.

This is an in-person presentation on July 22, 2024 (16:20 ~ 16:40 CEST).

Decision confidence refers to a person’s subjective judgement of the accuracy of their decision. It is thought to be a key (meta)-cognitive function to evaluate performance and adjust future behaviour. It has been suggested that decision confidence arises as the result of a post-decisional evidence accumulation process. Recently, the error positivity (Pe) signal recorded using response-locked EEG was proposed as a neural correlate of this post-decision evidence accumulation process (Desender et al., 2021), given that it has already been associated with error detection, confidence rating, and future behaviour adjustments. However, direct evidence for a link between single-trial Pe and post-decision evidence accumulation is currently lacking. In this EEG study, we jointly fit the Pe signal and choice-response times (choice-RTs) with an extended drift-diffusion model. This model has additional boundaries for confidence that allow us to model the post-decisional accumulation process. We hypothesized that the amplitude of the Pe signal would predict the confidence rating of the subject, such that the higher the amplitude of the Pe signal, the lower the confidence. We also expected that the higher the amplitude of the Pe signal, the slower the post-decision evidence accumulation process would be. We used two approaches for modeling: the chi-square method, and a Bayesian likelihood-free parameter estimation method (Bayesflow; Radev et al., 2022) which leverages invertible neural networks. We found weak positive relationships between Pe signal amplitude and post-decision evidence accumulation. We talk about these results in context of the literature on decision confidence.

This is an in-person presentation on July 22, 2024 (16:40 ~ 17:00 CEST).