Human Subjective Concept Space: Experimental Framework,Benchmark, and Model

Research into concept formation is concerned with the fundamental question of how we categorise objects into a particular class. We offer a general framework for designing and testing human benchmarks of subjective judgments as well as their simulations. Our method accounts for subjectivity, ways to control for individual human’s prior knowledge, all while keeping to real-life complex categories. We apply the above framework to experimentally establish a benchmark of subjective judgements in the domain of English literature. We establish individual prior knowledge by conducting structured interviews with six English literature students which are then tasked with categorising unseen texts. We conclude the study by creating bespoke models for each of the students, with the models having the training and testing data of their respective human participants. The models produced good fit to the human data and were further evaluated by five progressively more stringent metrics that go beyond the traditional statistical significance measures. Each of the three parts of our approach – the framework, the benchmark, and the model – may be used separately from each other in future psychological applications that move away from modelling the average participant and towards capturing subjective concept space and individual differences in judgment.