Knowledge Spaces

Competence-based knowledge space theory and cognitive diagnostic models are two theoretical frameworks that allow for assessing the latent set of skills an individual has available (here referred to as the “skill profile”) from the observed responses to test items. Competence-based test development (CbTD) is a recent and novel approach for constructing, improving, and shortening tests for skills assessment, that may be of interest to practitioners in the two aforementioned frameworks. CbTD exploits concepts originally introduced in rough set theory to construct tests that are as informative as possible about individuals’ skill profiles (i.e., adding any item does not make the tests more informative) and minimal (i.e., no item can be eliminated without making the tests less informative). Let a competency be a set of skills such that an item requiring them is available or can be constructed. A fundamental concept that underlies the construction of the tests is that of a reduct, which is defined as a minimal collection of competencies that is as informative about the skill profiles as a larger set. The talk presents two procedures for constructing a reduct. One is a competency deletion procedure that starts with a full set of competencies and consecutively deletes one competency at a time until a reduct is obtained. The other is a competency addition procedure that starts with the empty set of competencies and consecutively adds one competency at a time until a reduct is obtained. Exemplary applications of the two procedures to test construction are presented and discussed.

This is an in-person presentation on July 20, 2023 (15:20 ~ 15:40 UTC).

In the past years, several theories for assessment have been developed within the overlapping fields of Psychometrics and Mathematical Psychology. The most notable are Item Response Theory (IRT), Cognitive Diagnostic Assessment (CDA), and Knowledge Structure Theory (KST). In spite of their common goals, these frameworks have been developed largely independently, focusing on slightly different aspects. Yet various connections between them can be found in literature (see, e.g., Junker & Sijtsma, 2001; von Davier, 2005; Stefanutti, 2006; Di Bello, Roussos, & Stout, 2007; Ünlü, 2007; Hong et al., 2015; Heller et al., 2015; Noventa et al., 2019, to name only a few). A unified perspective is suggested that uses two primitives (structure and process) and two operations (factorization and reparametrization) to derive IRT, CDA, and KST models. A Taxonomy of models is built using a two-processes sequential approach that captures the similarities between the conditional error parameters featured in these models and separates them into a first process modeling the effects of individual ability on item mastering, and a second process representing the effects of pure chance on item solving.

This is an in-person presentation on July 20, 2023 (15:40 ~ 16:00 UTC).

In procedural knowledge space theory (PKST), the family of solutions for a given problem is called a "problem space''. The knowledge of a problem solver is represented as a specific subset of a problem space that satisfies the "sub-path assumption". Different types of ``symmetries'' could be found in a problem space that make certain parts of it ``equivalent''. These equivalence relations are introduced here as a homomorphism of one problem space into another problem space. Two types of homomorphisms are examined, which are named the ``strong'' and the ``weak homomorphism''. The former corresponds to the usual notion of ``operation preserving mapping''. The latter preserves operations in only one direction. The practical application of the proposed approach is presented through an empirical application with the Tower of London (TOL) test. A problem of the TOL consists of matching an initial configuration (i.e., a spatial disposition of colored balls on the pegs) with a goal configuration using the minimum number of moves. Due to the physical features of the TOL, several "symmetries" can be hypothesized. The introduction of symmetries leads to the construction of a problem space that is homomorphic to the original one. In particular, the homomorphic problem space is usually simpler and more abstract than the original one. Different symmetries hypothesis lead to different problem spaces which were empirically validated and compared with one another. The results of the empirical study are presented and discussed.

This is an in-person presentation on July 20, 2023 (16:00 ~ 16:20 UTC).

In the last years, growing attention has been paid to the generalization of KST deterministic concepts to the case of polytomous items. As a consequence of this extension, a generalized version of the basic local independence model (BLIM) has been recently proposed, named polytomous local independence model (PoLIM). Some of the main features of this new model have been investigated, but, to date, nothing has been specifically stated about its identifiability. In this research we present the first theoretical results about the problem of identifiability of the PoLIM. Such results represent a generalization to this polytomous model of what has been proven about the identifiability of the BLIM, which is the most widely used probabilistic model in dichotomous knowledge space theory. The study of the identifiability of the BLIM produced several research articles in the last ten years, especially focusing on the relations between two particular kinds of gradation of the deterministic structure, called forward and backward gradedness, and the unidentifiability of the model when applied to such structures. Here we show that the same kind of gradedness happens to apply also to the case of polytomus structures, and further attention is paid to some properties of forward and backward gradedness in the case of polytomous structures. For instance, we show that in the polytomous case there is no need to distinguish between forward and backward gradedness, but it is possible to simply speak about gradedness. Moreover, we show how gradedness of the polytomous structure leads to the same kind of tradeoffs studied in the BLIM between the probability of knowledge states and the error parameters of the items in which the polytomous structure is graded. The tradeoff equations are displayed and further directions to study the identifiability of the PoLIM are discussed.

This is an in-person presentation on July 20, 2023 (16:20 ~ 16:40 UTC).

The dimension of a partial order P on a set A is a well-known and thoroughly studied concept. It is defined as the smallest number of linear orders on A whose intersection equals P. It can also be characterized as the smallest family F of mappings f from A to the reals such that a P b if and only if f(a) is less or equal to f(b) for all the mappings in F. The interest for this characterization stems from the fact that it qualifies the problem of determining the dimension of a partial order as a measurement theoretical question (Roberts, 1985). The states in a knowledge structure and, in particular, in a knowledge space are partially ordered by set inclusion. Therefore the dimension of a knowledge structure is a well-defined concept, and it corresponds to the dimension of the restriction of the set inclusion relation to the knowledge structure itself. We show that, under a rather general condition, named “join escape”, the dimension of a finite knowledge space equals the length d of a maximum antichain in its basis. If, for a given knowledge space, join escape does not hold, then d only provides an upper bound to the dimension of the knowledge space. It is further found that, under the stated condition, the dimension of a knowledge space coincides with the least number of maximal chains whose element-wise union reconstructs the entire knowledge space.

This is an in-person presentation on July 20, 2023 (16:40 ~ 17:00 UTC).