Search papers, labs, and topics across Lattice.
This paper introduces "offloading score," a simulation-based metric that quantifies AI reliance by estimating the fraction of cognitive effort offloaded to an AI tool through counterfactual workflow analysis. They validate this metric through a user study where developers used AI tools for programming tasks under varying time pressure, demonstrating that offloading score accurately captures increased reliance in time-constrained settings, unlike usage-based or self-reported measures. The authors further show how offloading score can be used to identify instances of inappropriate reliance by correlating it with task outcomes like code understanding.
Current reliance metrics miss the crucial shift in cognitive effort distribution between users and AI, but this new "offloading score" finally captures how much work is *actually* being delegated to the tool.
AI tools are increasingly integrated into real-world workflows. However, existing measures of reliance on these tools focus on AI output adoption or on self-reported indicators, rather than how task effort is distributed between users and tools. Here, we introduce offloading score, a measure of reliance that quantifies the fraction of cognitive effort offloaded to an AI tool. Offloading Score is simulation-based -- we construct a counterfactual workflow by estimating how the user would have completed the task without the tool, and then computing the fraction of steps saved by using the tool. We validate offloading score through intrinsic evaluations of metric validity, and a controlled user study ($n=40$) with developers performing programming tasks using AI tools. We vary time pressure to test whether reliance measures capture the known increase in reliance under time pressure. We show that offloading score detects significantly higher reliance in time-constrained settings ($+43\%$, $p=0.018$), while usage-based and self-reported baseline measures of reliance do not distinguish the conditions. We complement this with descriptive insights showing that higher reliance manifests as greater delegation of subtasks to the tool and more direct reuse of AI outputs. Finally, we demonstrate an approach of using offloading score in combination with target outcomes of a task (e.g., code understanding) to identify when reliance may be (in)appropriate. Our framework offers two contributions: an instrument users can apply to measure and reflect on their own reliance, and a quantitative signal that agent designers can utilize to mitigate overreliance.
