Symposium: Deep Learning And Simulation-Based Inference For Computational Cognitive Modeling

In social sciences, the transition from verbal theories to computational models presents a significant challenge, primarily due to the difficulty in quantifying complex human behaviors into discrete parameters. Traditional analytical tools, such as Principal Component Analysis (PCA), have long been the preferred method for reducing high dimensional data into low-dimensional spaces. However, these linear approaches often fall short in capturing the intricate relationships that characterize human behavior. This talk introduces a novel application of variational auto-encoders (VAEs) combined with clustering techniques to provide an alternative way to discover and construct latent dimensions for behavioral problems. Unlike PCA, VAEs have the unique advantage of modeling non-linear relationships, thereby providing a deeper, more accurate interpretation of data. Mainly, our aim is to provide a tool that can distill critical dimensions for specific behavioral problems with an application on experimental data from a novel procrastination task. However, the implications of this research extend beyond individual domains, supporting a broader application. The ability to discern non-linear relationships and hidden structures can thereby enhance understanding and parametrization of various behavioral phenomena, ultimately leading to the development of better predictive models and theoretical frameworks. As such, this talk aims to evaluate machine learning as a tool for exploratory data analysis with an eye toward inspiring richer and more complete psychological theories.

This is an in-person presentation on July 22, 2024 (11:40 ~ 12:00 CEST).

Sensitivity analyses are a useful tool for ensuring the robustness of computational workflows in psychology and beyond. However, they are typically forgone, as they require numerous model refits and become downright infeasible for models where even a single model fit can be computationally expensive. In this talk, we present a framework for efficiently integrating different types of sensitivity analyses into simulation-based inference with deep learning (aka amortized Bayesian inference). Our method enables efficient (i) prior and likelihood sensitivity analysis by training a single neural network for all prior and likelihood configurations of interest, (ii) data sensitivity and multiverse analysis by leveraging the fast inference of trained neural networks, and (iii) model misspecification detection by measuring the agreement within a deep ensemble of neural networks. We present experiments on representative models that underscore the effectiveness of our approach for both parameter estimation and model comparison tasks. Our results suggest that integrating sensitivity analysis into amortized Bayesian workflows is a promising step towards reliable and robust inference.

This is an in-person presentation on July 22, 2024 (12:00 ~ 12:20 CEST).

Cognitive processes undergo various fluctuations and transient states across different temporal scales. Superstatistics are emerging as a flexible framework for incorporating such non-stationary dynamics into existing cognitive model classes. In this work, we provide the first experimental validation of superstatistics and formal comparison of four non-stationary diffusion decision models in a specifically designed perceptual decision-making task. Task difficulty and speed-accuracy trade-off were systematically manipulated to induce expected changes in model parameters. To validate our models, we assess whether the inferred parameter trajectories align with the patterns and sequences of the experimental manipulations. To address computational challenges, we present novel deep learning techniques for amortized Bayesian estimation and comparison of models with time-varying parameters. Our findings indicate that transition models incorporating both gradual and abrupt parameter shifts provide the best fit to the empirical data. Moreover, we find that the inferred parameter trajectories closely mirror the sequence of experimental manipulations. Posterior re-simulations further underscore the ability of the models to faithfully reproduce critical data patterns. Accordingly, our results suggest that the inferred non-stationary dynamics may reflect actual changes in the targeted psychological constructs. We argue that our initial experimental validation paves the way for the widespread application of superstatistics in cognitive modeling and beyond.

This is an in-person presentation on July 22, 2024 (12:40 ~ 13:00 CEST).

Amortized Bayesian inference (ABI), a subset of simulation-based inference (SBI) fueled by neural networks, has rapidly grown in popularity across diverse scientific fields. Summary statistics are an essential dimensionality reduction component of ABI workflows and most methods to-date rely either on hand-crafted (i.e., based on domain expertise) or end-to-end learned summary statistics. In this work, we explore hybrid methods to harness the complementary strengths of both sources. Our first method directly conditions a neural approximator on both types of summary statistics, thereby extending traditional end-to-end approaches in a straightforward way. Our second method employs an auxiliary generative model to learn a latent summary representation that is statistically independent from the expert summaries. We explore various aspects of our hybrid methodology across different experiments and model instances, including active learning, perfect domain expertise and imperfect artificial experts represented by pre-trained neural networks. Our empirical results suggest that hybrid representations can improve parameter estimation and model comparison in settings of scientific interest, warranting the viability of an ``expert-in-the-loop'' approach. The performance gains are especially promising in scenarios with low to medium simulation budgets.

This is an in-person presentation on July 22, 2024 (15:20 ~ 15:40 CEST).

In Bayesian inference, (joint) posterior distributions should always exist for parameters of a model given real priors. In the extreme case, a completely unidentified model, one that does not provide any inference about the data whatsoever, will result in the joint posterior exactly equaling the joint prior. There exist many useful (neuro-)cognitive models in the fields of mathematical psychology and cognitive neuroscience for which only some parameters are unidentifiable while others are identifiable. This is especially true for new models and extensions of existing models. I show from experience in multiple studies why simulation-based Bayesian inference using artificial neural networks is extremely useful for exploring and testing these (un)identified models. I show examples from Drift-Diffusion Models with internal noise, models of stop-signal cognition, and confidence judgments. I show that simulation-based Bayesian inference using the package BayesFlow is often much easier to use for unidentified models than Probabilistic Programming Languages that use MCMC algorithms (e.g. Stan and JAGS). I will show examples of using neural data to make unidentified (neuro-)cognitive models identifiable in order to measure condition- and individual-differences in cognition. Specifically, I will show how the joint posterior space of unidentified cognitive models will shrink towards some meaningful posteriors when neural data is also explained by neuro-cognitive models. I suggest a future in which the unidentifiability of (neuro-)cognitive models is systemically explored with simulation-based Bayesian inference.

This is an in-person presentation on July 22, 2024 (15:40 ~ 16:00 CEST).

A room-oriented immersive system (ROIS) augments dynamic virtual environments for multiple people to experience in a shared physical space. User movement patterns in ROIS provide context for observing emergent attention with two intersecting challenges. On the one hand, computational modeling of motion-induced attention in virtual environments has privileged single-user, head-centered experiences, leaving out the shared experiences that ROIS provides. On the other hand, while attention modeling has gathered sustained interest in multi-agent simulation, existing simulation environments often use simplified spatial configurations, compromising the agents’ perceptual viability in the physical world. To address the challenges above, we introduce TogetherFlow, a modeling framework for distributed and emergent multi-agent spatial attention using Bayesian simulation-based inference. In this framework, we represent agent attention as a spatial probability distribution based on proximity and orientation to the surrounding audiovisual objects in game-based virtual navigation environments. For the forward problem, a cluster analysis of agent attention simulates moment-to-moment attentional distributions along global navigation trajectories using variational Bayes. For the inverse problem, a generative neural network recovers the attentional distribution given the global trajectories, and its performance is extensively validated using principled Bayesian methods. Through a large-scale simulation study of models using different agent representations and movement mechanisms, we observe how cross-modal motion perception influences the emergence of agent attention and how it introduces self-organizing behaviors in ROIS. This observation will further allow us to design adaptive interfaces for augmenting the virtual environments.

This is an in-person presentation on July 22, 2024 (16:00 ~ 16:20 CEST).

A notable characteristic of Bayesian statistics lies in its capability to integrate prior knowledge into the modeling process. While this feature holds substantial potential, its practical implementation is far from straightforward. Expressing prior knowledge involves formulating prior distributions for the model parameters during the modeling process. This requires both the ability to interpret the model parameters and the skill to translate prior knowledge into valid distributions. In response to this challenge, we suggest a simulation-based prior elicitation method that translates expert knowledge into corresponding priors, regardless of the underlying data-generating model. Depending on the analyst's preference, our method allows for learning parameters of a pre-specified prior distribution family via stochastic gradient descent or a joint prior distribution over all model parameters via normalizing flows that accounts for correlation between model parameters.

This is an in-person presentation on July 22, 2024 (16:20 ~ 16:40 CEST).

Evidence-accumulation models (EAMs) are popular tools for analyzing speeded decisions in two-alternative forced choice tasks because they allow researchers to make inferences about the decision process. Estimating EAM parameters from empirical data can be computationally expensive, especially in a Bayesian framework. To lower this barrier, researchers have proposed to amortize these costs by incorporating deep neural networks (DNNs) into the estimation procedure leading to amortized Bayesian inference (ABI). The DDNs are trained to learn the relationship between model parameters and model simulations to predict faithful posterior distributions for empirical data. Importantly, the DNNs can learn such relationships for variable data sets, model configurations, and other contexts. Previous research has shown that experimental design factors can impact the faithfulness of traditional methods for estimating EAM parameters. While ABI has shown many promising results for EAMs both in simulations and in empirical data, little work has been done on how experimental design factors affect the performance of ABI. By conducting a comprehensive simulation study, we aim to establish a benchmark for the generalizability of different DNN architectures for EAMs across common experimental designs. Moreover, we will assess their generalizability on empirical data by comparing ABI to traditional parameter estimation methods. With our results, we hope to shine a light on the sensitivity of ABI to experimental design factors for EAMs. These insights could be used to further improve current ABI approaches and as a reference for practitioners for designing their experiments.

This is an in-person presentation on July 22, 2024 (16:40 ~ 17:00 CEST).