Doing as AI says, not as AI does: Learning calibration from machine predictions
Well-calibrated probabilistic thinking is essential for effective human-AI collaboration, helping people use AI advice selectively and appropriately. We test whether calibration can be improved through trial-by-trial learning from AI-predicted probabilities in a medical decision making task. Novice participants (N=300) judged the probability that white blood cells were blast (cancer) cells across three phases: training with ground truth feedback, practice with AI exposure, and test without feedback. A single CNN was shown in three ways, holding accuracy fixed at ~94% while varying the certainty it expressed: its original probabilities, which were conservative and poorly calibrated for the model's accuracy; a recalibrated version, well calibrated for the model and correspondingly certain; or categorical predictions conveying no explicit certainty. Exposure to the original conservative AI substantially reduced characteristic over extremity at test, improving calibration relative to other conditions. At the same time, minimal condition-specific improvement in discrimination suggests that calibration improved through superficial mimicry of the original AI’s uncertainty, which, although miscalibrated relative to its own high accuracy, was better suited to the lower accuracy of novices. AI-assisted training of calibration and other decision skills may face an alignment problem: the certainty signal that most appropriately describes a model need not be the signal that best supports a learner.
Keywords
There is nothing here yet. Be the first to create a thread.
Cite this as: