1487: Molded by the Ghost: How Physicians Use–and Cease Using–Predictive AI to Learn a Mental Model (15509)

Scholarship on human-AI collaboration has demonstrated that using AI can stymie learning, induce dependence, and erode human decision making capabilities. Here, we make the case for an alternative path emerging from the use of AI for decision support. In evaluating the clinical deployment of a predictive AI tool in the early days of COVID-19, we discovered that AI use enabled physicians to incidentally learn a mental model. Our exploratory, longitudinal research featured a survey, an interview study, and a controlled experiment that revealed three discoveries. First, users derived from their use of the AI an understanding of the data that mattered, how to interpret it, and how to extrapolate from it to make decisions. They then used this mental model in the absence of the AI tool, itself. Second, owing to having learned this mental model, individuals who used the AI for decision support went on to make better decisions without it. And third, this incidental learning may have undermined decision support over time, as a result of negative feedback loops whereby learning reduced reliance. We contribute to the literature on learning from technology use and on human-AI teaming by illuminating a heretofore overlooked means by which predictive AI use can enable learning and improve autonomous human capabilities. We also shed light on the potential for incidental learning to undermine AI use, over time, with paradoxical implications for how users work with AI.

view paper (if available)