Search papers, labs, and topics across Lattice.
This study evaluates two systems for skill retrieval in agents using a corpus of 690 skills: a hybrid ranker that combines lexical and dense-embedding retrieval, and a typed knowledge graph that encodes workflow relations. The hybrid ranker successfully retrieves the correct skill within the top five results for 73.5% of queries, while the knowledge graph underperforms, failing to enhance retrieval reach due to its reliance on a pre-filter topology. The findings reveal that while the graph can provide richer semantic relations, it does not improve retrieval performance compared to the ranker, highlighting the limitations of structural interdependence in this context.
A hybrid ranker outperforms a knowledge graph in skill retrieval, achieving 73.5% accuracy while the graph fails to extend reach despite its structural advantages.
Agents backed by large skill libraries must decide which skills to load and in what order. Loading the entire library into context is expensive and provides no structure for autonomous sequencing. We study two systems for this problem over a corpus of 690 skills: a hybrid ranker combining lexical and dense-embedding retrieval for sparse, on-demand loading, and a typed knowledge graph encoding workflow relations such as prerequisites, data flow, and ordering. On a set of 117 realistic, non-echoing queries, the hybrid ranker retrieves the correct skill within the top five in 73.5% +/- 8.0 of cases, leaving roughly a quarter of queries unserved. When used as the design intended (substituting graph neighbours for additional ranked results at matched token budget), the graph is significantly worse (-11.2 points, p = 0.0007). Its LLM-generated edge layer adds nothing over neighbours obtained free from a local embedding pass, and 73% of the queries the ranker misses are not reachable through the graph at all. We attribute this to a pre-filter topology bound. Because the graph's candidate edges are drawn from the same embedding neighbourhood the ranker already searches, 98.6% of typed edges connect skills the ranker had already surfaced together. The graph can enrich relation semantics but cannot extend retrieval reach. We further show that evaluating on author-written queries overstates hit@5 by up to 44 points, which would have hidden these results entirely. Our contribution is a mechanistic account of why added structure does not improve retrieval over a strong ranker, and identify the conditions under which adding structural interdependence into the retrieval is optimal.
