Search papers, labs, and topics across Lattice.
This paper explores the construction and evaluation of a Hypergraph of Text (HoT) to enhance navigability within arbitrary collections of documents. By introducing several methods for HoT construction and a novel quantitative metric called effort ratio, the authors demonstrate that straightforward TF-IDF approaches can achieve comparable structural quality to more complex LLM-based methods. These findings highlight the potential for simpler techniques to effectively organize and navigate textual data, challenging the assumption that advanced models are always necessary for optimal performance.
Even basic TF-IDF methods can rival LLM-based approaches in creating navigable text structures, as shown by a new metric for evaluating hypergraphs.
One reason the Web is more useful than a simple collection of documents is that the structure created by hyperlinks enables flexible navigation from one web page to another. However, hyperlinks are typically created manually and cannot fully capture a corpus'implicit semantic structures. Is there a general way to make an arbitrary collection navigable? Recent work has formalized this problem generally as constructing a Hypergraph of Text (HoT), which provides a formal mathematical structure for supporting navigation and browsing. However, how to construct and evaluate a Hypergraph of Text remains a challenge. In this paper, we propose and study several methods for constructing a HoT. We also propose a novel quantitative metric, effort ratio, for evaluating the structural quality of a constructed HoT. Experimental results show that even simple TF-IDF baselines can match LLM-based methods on our proposed effort ratio metric.
