Search papers, labs, and topics across Lattice.
This paper analyzes the evolution of Large Language Model (LLM) performance on open-ended creative tasks over three years, focusing on their responses to the Infinity-Chat100 dataset and the Alternate Uses Task. By employing sentence-embedding similarity to assess model outputs, the study reveals a statistically significant decrease in diversity among LLM responses, indicating a convergence in creative substance across different models. These findings raise concerns about the potential homogenization of LLM outputs, which could undermine human agency in collaborative creative endeavors with AI.
LLMs are becoming less diverse in their creative outputs, potentially stifling human agency in co-creative processes.
Many benchmarks track Large Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving on open-ended tasks, where creativity, originality and diversity may matter as much as quality. As LLMs increasingly support human ideation and creative work, understanding trends in LLM performance on open-ended tasks is critical. This paper presents a preliminary analysis of LLM creative outputs spanning three years of model releases, examining model responses to Infinity-Chat100, a real-world collection of open-ended user queries, and the Alternate Uses Task, an established psychometric creativity assessment. Using sentence-embedding similarity, we examine trends in LLM responses to these prompts. Our findings show a statistically significant decrease in model output diversity over time, suggesting that LLM outputs may be converging in creative substance across models. If this trend persists, LLM-driven homogenization may progressively diminish human agency in human-AI co-creative work, demanding careful consideration of LLMs'role in the human creative process.
