Close
This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Analyzing variability in instance-based learning model predictions using recurrence quantification analysis

Authors
Dr. Erin McCormick
Air Force Research Laboratory
Dr. Leslie Blaha
Air Force Office of Scientific Research
Prof. Cleotilde (Coty) Gonzalez
Carnegie Mellon University ~ Social and Decision Sciences Department
Abstract

Model variability is important: if systematic variation in model predictions does not reflect systematic variation in human behavior, the model's ability to describe, predict, and explain behavior is in question. We demonstrate a method to compare variation in model predictions to variation in human behavior in a dynamic decision making task. Dynamic decisions are a sequence of inter-dependent choices in changing environments, where human choices may systematically change over time. We can characterize these changes with a qualitative and quantitative visual analytics approach, recurrence quantification analysis (RQA). RQA visualizes (with recurrence plots) and describes (with recurrence statistics) recurring states in sequences of observations. We compared human choice sequences in a dynamic decision making task to predictions of an instance-based learning (IBL) model, a memory-based model of choice with two parameters (noise and decay). Specifically, we generated predictions using two parameterizations of the IBL model: one using default noise and decay parameters from the ACT-R cognitive architecture, another using the average of noise and decay parameters from IBL models fit to human data at the individual level. We compared the recurrence statistic distributions of the human data and both parameterizations. We find ACT-R default parameters predict more decision makers with less trial-to-trial change in choices than in human data. In contrast, the averaged parameters predict more decision makers with more trial-to-trial change in choices than in human data. RQA provides new tools for assessing model predictions, and a new source of evidence for demonstrating that models successfully characterize sequences of human choice.

Tags

Keywords

Recurrence quantification analysis
Dynamic decision making
Model variability
Choice sequences
Visual analytics

Topics

Cognitive Modeling
Decision Making
Model Analysis and Comparison
Other
Discussion
New

Recurrence-plot.tk is the great resource/jumping off point for recurrence quantification analysis: http://www.recurrence-plot.tk/rqa.php A reference paper with more details: http://www.recurrence-plot.tk/marwan_PhysRep2007.pdf (Marwan, Romano, Thiel, Kurths. (2007). Recurrence plots for the analysis of complex systems. Physics Reports, 438, 237-3...

Dr. Erin McCormick 0 comments
Cite this as:

McCormick, E. N., Blaha, L., & Gonzalez, C. (2020, July). Analyzing variability in instance-based learning model predictions using recurrence quantification analysis. Paper presented at Virtual MathPsych/ICCM 2020. Via mathpsych.org/presentation/104.