1925: AI-Human Sequential Evaluation: Enhancing Performance Assessments through AI-Human Collaboration (15090)

Designated as a “Best Paper” by HR
view abridged “best paper” (if available)

Organizations are increasingly incorporating AI alongside human evaluators to address biases and inconsistencies that can contaminate performance evaluations. We use information processing theories to develop the AI-Human Sequential Evaluation model and explain how AI and humans with differential access to data and distinct information processing capabilities collaborate to evaluate performance. We empirically test our model using machine learning techniques to analyze text and audio data from 94,776 call center interactions and conduct a field experiment. We found that when humans independently evaluate performance, they exhibit more bias against demographic minorities with perceived accents compared to AI. However, this bias diminishes in the AI-Human Sequential Evaluation model, where human evaluators use AI scores as a reference point, often aligning with AI decisions when they lack performance-related reasons to dispute them. Conversely, human evaluators diverge from AI evaluations if they identify performance-relevant signals in data, highlighting the continuing necessity of human expertise even in the age of intelligent machines. Taken together, our findings demonstrate that human evaluators can exhibit ‘selective’ trust in AI, and underscore the potential of our model to foster more fair and valid performance evaluations, contributing to the AI-Human collaboration and performance management literatures.

view paper (if available)