An integrated cognitive model of visuospatial relation tracking

We developed a computational cognitive theory of visuospatial relation tracking to model how people perceive and monitor spatial relations such as left of and next to. Previous work suggests that they build mental simulations of their environment, or of imagined or described scenarios, to reason about space and spatial concepts – but no account explains how visual information is used to construct and update those simulations. Our model converts perceptual data (objects and locations detected by a convolutional neural network) into an integrated sparse iconic simulation of the scene – a perceptual model. Perceptual models are cognitively plausible representations that are more noise tolerant and stable than raw percepts. They allow for efficient latent encoding of high-level relations. We tested the cognitive model against a benchmark dataset for spatial relation recognition, and show a close fit between human and model-based perception of 2D spatial relations.