Procedures for constructing minimal, yet maximally informative tests for skill assessment

Competence-based knowledge space theory and cognitive diagnostic models are two theoretical frameworks that allow for assessing the latent set of skills an individual has available (here referred to as the “skill profile”) from the observed responses to test items. Competence-based test development (CbTD) is a recent and novel approach for constructing, improving, and shortening tests for skills assessment, that may be of interest to practitioners in the two aforementioned frameworks. CbTD exploits concepts originally introduced in rough set theory to construct tests that are as informative as possible about individuals’ skill profiles (i.e., adding any item does not make the tests more informative) and minimal (i.e., no item can be eliminated without making the tests less informative). Let a competency be a set of skills such that an item requiring them is available or can be constructed. A fundamental concept that underlies the construction of the tests is that of a reduct, which is defined as a minimal collection of competencies that is as informative about the skill profiles as a larger set. The talk presents two procedures for constructing a reduct. One is a competency deletion procedure that starts with a full set of competencies and consecutively deletes one competency at a time until a reduct is obtained. The other is a competency addition procedure that starts with the empty set of competencies and consecutively adds one competency at a time until a reduct is obtained. Exemplary applications of the two procedures to test construction are presented and discussed.