Single-trial estimates of sequential sampling models parameters are not just noisy but can also be biased

Traditionally, researchers estimate parameters of sequential sampling models (SSMs) from repeated choices across different conditions. Crucially, differences in parameters across conditions are interpreted as shifts in the underlying cognitive processes: For example, lower decision thresholds under high time pressure are interpreted as decreased cautiousness. Recent work has explored whether the parameters of SSMs can be estimated at a more detailed, single-trial level as well, to infer shifts in cognitive processes in subsequent trials. Such a more detailed window on decision-making processes has exciting applications. For example, by correlating single-trial estimates to neuroimaging data, we can relate specific brain areas to cognitive processes that may vary from trial to trial and not merely across conditions. The present work highlights some important limitations of such a powerful approach. First, we reproduce earlier work and show that single-trial estimates of SSM parameters are extremely noisy. We also show that single-trial SSM parameter estimates can be highly biased by the outcome of a choice. For example, single-trial estimates of the rate of evidence accumulation in incorrect choices are severely underestimated when compared to the generating single-trial parameter (and vice versa for correct choices). We will show how these problems can pollute the cognitive interpretation of single-trial parameters and can be exacerbated by correlations to process data. Finally, we offer a potential solution where SSMs that incorporate more information about trial-to-trial differences (e.g., stimulus or feedback properties) produce more reliable single-trial estimates.