Do Discrete Choice Experiments and Rating Scales Elicit the Same Preference Judgments?

Discrete choice (DCE) and rating scale experiments (RSE) are commonly applied procedures for eliciting preference judgments in a plethora of applied settings such as consumer choices, health care, and transport economics. An almost universal assumption underlying their use is that the two procedures elicit reports generated from a common internal preference state; that is, actual “ground truth” preferences are not dependent on which procedure is used to elicit them. It is usually not possible to test this assumption, because typical studies using DCE and RSE methods have response options for which there is no objectively correct response, and no ground truth. To facilitate a comparison of DCE and RSE, we conducted a perceptual discrimination experiment where response options varied on a single attribute -- stimulus saturation level -- with a known objectively correct response. We had the same participants complete both a DCE and RSE version of the experiment, allowing a direct examination of the assumption that a common representation underpins responses in both. For this purpose, we developed a cognitive model with a response mechanism for both DCE and RSE based on latent Gaussian stimulus representations. This enabled us to compare a model version that featured one shared latent stimulus representation across DCE and RSE versus a model version which featured a separate latent representation for DCE and RSE. Our results support the assumption that a single internal state supports both DCE and RSE responses, and also suggest that the DCE method might provide more sensitive measurement of internal states than the RSE method.