Search papers, labs, and topics across Lattice.
This paper addresses the retrieval readiness gap in large-taxonomy retrieval by introducing Factorized Hypothesis Search (FHS), which leverages multiple partial interpretations across named semantic dimensions to enhance retrieval accuracy. The authors demonstrate that FHS significantly outperforms traditional methods on financial taxonomy tagging and clinical coding tasks, achieving superior Recall@1, MRR, and overall accuracy. Notably, the study reveals that using a free-text ensemble instead of FHS leads to a substantial decline in performance, underscoring the effectiveness of their approach.
Factorized Hypothesis Search reveals that maintaining multiple interpretations of evidence can dramatically improve taxonomy retrieval accuracy, outperforming conventional methods.
Large-taxonomy retrieval often assumes that the input already expresses the target concept. In many settings, however, the input is indirect evidence, such as a table cell whose meaning depends on its row, column, datatype, and context. We call this mismatch the retrieval readiness gap. Our analysis shows that the current index retrieves the target reliably when its semantics are explicit, while raw evidence often leaves it deep in the ranking. We propose Factorized Hypothesis Search (FHS), which maintains multiple partial interpretations over named semantic dimensions. These hypotheses support structured query rendering, multi-hypothesis retrieval, and dimension-level candidate verification. On both financial taxonomy tagging and CodiEsp clinical coding tasks, FHS achieves the best Recall@1, MRR, and final accuracy among the non-oracle methods. Replacing the factorized hypothesis path with a free-text ensemble causes the largest drop in head-ranking performance, while sequential refinement provides no additional gain over FHS's strong parallel first round.
