Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To determine whether syntactic dependency parsing meaningfully aids representation learning in topic modeling, this work evaluates a graph-augmented variational topic model across four distinct discourse registers against standard generative and dense embedding baselines. The authors find that while the variational autoencoder itself drives document-topic alignment gains over classical models, syntactic edges specifically boost topic diversity and effectively capture predicate-argument and stance patterns. Ultimately, the utility of syntax is shown to be strictly register-dependent: it enriches deliberative and argumentative texts, but acts as redundant noise in technical or administrative domains.
Explicit syntax in neural topic models does surprisingly little for document-topic alignment, but it selectively unlocks higher topic diversity and fine-grained stance representations exclusively in deliberative, argumentative registers.
Topic modeling is widely used in computational social sciences to identify latent themes in large text corpora. Traditional approaches rely on Bag-of-Words representations and generative models such as LDA, while recent methods like BERTopic operate on dense document embeddings. This paper introduces the Structural Contextual Probabilistic Topic Model (SCPTM), an architecture that incorporates syntactic dependency relations into topic inference. SCPTM represents a corpus as a heterogeneous graph of documents and words connected by lexical and syntactic edges, processed through a Graph Attention Network within a Variational Autoencoder to produce probabilistic, mixed-membership topic distributions. We evaluate seven topic modeling techniques (including four SCPTM ablations) across four corpora differing in register and discourse structure. Our framework combines coherence (C_V, C_NPMI), topic diversity, clustering-label alignment (NMI), and phrase-level diagnostics (complementarity and valence gap). Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines, but these gains are attributable to the variational encoder rather than to syntax. Syntax contributes to topic diversity, where graph-augmented variants outperform the no-graph baseline across all corpora, and to descriptor quality: dependency paths capture predicate-argument structures and stance in deliberative registers, while proving redundant in technical and institutional corpora. The valence gap is positive across all variants, but driven primarily by phrase grouping rather than syntactic filtering. We conclude that syntactic encoding matters conditionally: it benefits action-oriented, argumentative texts, but introduces noise in informational or administrative registers.