Search papers, labs, and topics across Lattice.
This paper introduces a novel topic modeling approach using LLMs to generate topics from text data that are interpretable, specific, and polarity-consistent, addressing limitations of existing methods. They propose an evaluation framework incorporating topic specificity and polarity stance consistency as key criteria, and benchmark automated evaluation methods. Applied to leadership analysis using corporate reviews from OpenWork, the proposed method demonstrates improved topic quality and explanatory power for external outcomes like employee morale.
LLMs can generate higher-quality, more consistent topics from text data, leading to better insights about external outcomes like employee morale.
Analyzing topics extracted from text data in relation to external outcomes is important across fields such as computational social science and organizational research. However, existing topic modeling methods struggle to simultaneously achieve interpretability, topic specificity (alignment with concrete actions or characteristics), and polarity stance consistency (absence of mixed positive and negative evaluations within a topic). Focusing on leadership analysis using corporate review data, this study proposes a method leveraging large language models to generate topics that satisfy these properties, along with an evaluation framework tailored to external outcome analysis. The framework explicitly incorporates topic specificity and polarity stance consistency as evaluation criteria and examines automated evaluation methods based on existing metrics. Using employee reviews from OpenWork, a major corporate review platform in Japan, the proposed method achieves improved interpretability, specificity, and polarity consistency compared to existing approaches. In analyses of external outcomes such as employee morale, it also produces topics with higher explanatory power. These results suggest that the proposed method and evaluation framework provide a generalized approach for topic analysis in applications involving external outcomes.
