Search papers, labs, and topics across Lattice.
The paper introduces CoTrace, a framework for attributing contributions in human-AI collaboration by decomposing explicit goals into verifiable requirements and tracing direct/indirect influences across dialogue turns. Applying CoTrace to real-world collaboration logs reveals that LLMs contribute significantly to introducing lower-level requirements and exert various indirect influences on goal-shaping, despite accounting for a smaller percentage of overall goal-shaping contribution. User studies demonstrate that exposing participants to CoTrace's goal-level analyses significantly shifts their perception of AI's contribution, highlighting miscalibration in understanding AI-assisted work.
LLMs may only account for 11-26% of high-level goal-setting in collaborations, but they exert far more influence by shaping the micro-decisions and concrete requirements that define those goals.
As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.
CMU ML