Search papers, labs, and topics across Lattice.
This paper investigates the efficiency of maintaining an evolving semantic substrate for retrieval-augmented question-answering systems, arguing that the maintenance cost is determined by the amount of change rather than the corpus size. Through empirical evaluation, the authors demonstrate that incremental low-rank updates are significantly cheaper—33.7 times per update and 23.8 times cumulatively—compared to full re-SVD, while maintaining high fidelity to the original embeddings. The findings advocate for a maintenance approach that preserves semantic meaning over time, rather than reconstructing it with each update, achieving precision within floating-point limits.
Incremental updates to semantic substrates can be 33.7 times cheaper than full re-computation, challenging the assumption that corpus size dictates maintenance costs.
Retrieval-augmented and agentic question-answering systems increasingly re-derive the meaning of a corpus at query time. Put plainly, instead of re-deriving what a corpus means on every question, the work is done once when a document arrives and is thereafter merely consulted -- a compiler, not an interpreter, of meaning. An alternative is to compile that meaning once, at ingest time, into a compact, queryable semantic substrate and maintain it as the corpus evolves. The central objection is maintenance cost: rebuilding a truncated singular value decomposition (SVD) on every change appears prohibitive, and a change of embedding model seems to force a full re-embedding. We argue and show empirically that maintenance cost scales with the amount of change, not corpus size. On a controlled synthetic pilot (dimension 256, rank 32, a corpus grown from 3,000 to 9,000 documents over 50 update events), incremental low-rank updates were 33.7 times cheaper per update than full re-SVD and 23.8 times cheaper cumulatively, while the incremental subspace tracked the full recomputation to within floating-point precision (maximum principal-angle drift below 1e-11 degrees; recall@10 = 1.0). An orthogonal Procrustes virtual axis update recovered 0.95 mean cosine to truly re-embedded vectors by re-embedding only about 10 percent of the corpus. The results support maintaining, rather than repeatedly reconstructing, a semantic substrate.
