Do people use all information when making decisions with an automated aid? An application of Systems Factorial Technology

Imperfect automation aids can lead to many negative consequences. To help mitigate those consequences, researchers have suggested that users be more vigilant, and particularly use multiple sources of information when making a decision with an automated aid. Prior research has suggested that people may still rely solely on the aid even when provided with other sources of information, but this research has tended to rely on strong assumptions and may have confounds. To test whether participants are using all or only one source of information when provided with an automated aid with more robust methods, we examined automation usage with Survivor Interaction Contrast from the Systems Factorial Technology framework. Additionally, we tested whether performance incentives and early experience with automation failures during training encourages more exhaustive processing. Participants completed a speeded length judgment task where they were provided with a reliable but imperfect aid to assist them in their decision. We found that across all conditions, participants used a serial, first-terminating process, supporting the view that participants use only one source of information. However, results from a logistic regression suggest that participants are likely using both the automated aid and the signal across all trials instead of relying solely on one. Implications of this research highlight a different strategy where participants may be alternating what source of information they use, which may be beneficial when using an imperfect aid in speeded decisions. This research can inform interface designs that support effective strategies for making speeded decisions with an automated aid.