Search papers, labs, and topics across Lattice.
This study evaluates the diversity of analogy generation in ten state-of-the-art large language models (LLMs), revealing a significant issue of domain homogeneity that restricts the breadth of analogies produced. The research uncovers a trade-off where enhancing output diversity often compromises the quality of the generated analogies. Additionally, a causal analysis of information flow within the models indicates that different models exhibit varying sensitivities in regions that influence analogy diversity, providing insights into the underlying mechanisms at play.
LLMs exhibit a troubling tendency to generate analogies from a narrow set of domains, limiting their creative potential and cross-domain connections.
Large Language Models (LLMs) have demonstrated remarkable potential for analogy making, a core cognitive capability that drives novelty and creativity. While prior research has extensively investigated the applications and underlying mechanisms of LLM-based analogy making, its output diversity remains largely unexplored, despite being essential for broadening cross-domain connections and fostering scientific innovation. In this work, we present a comprehensive evaluation of analogy diversity across ten state-of-the-art open- and closed-source LLMs. Our findings highlight a concerning issue of domain homogeneity, a prevalent tendency for LLMs to generate analogies from a narrow set of target domains, limiting both inter-query and intra-model diversity. Furthermore, our analysis reveals a fundamental trade-off in existing LLM diversity-enhancement methods: increasing output diversity often comes at the expense of output quality. Finally, our causal analysis of LLM information flow reveals substantial differences in the model-sensitive regions governing analogy diversity across LLMs, suggesting a potential mechanism for the observed diversity-quality trade-off. To our knowledge, this is among the first studies to systematically investigate output diversity in LLM-based analogy making.
