Search papers, labs, and topics across Lattice.
This paper introduces the concept of local-first information retrieval (IR), which allows sensitive data to be processed directly on user devices rather than relying on remote servers. The authors present a comprehensive framework that categorizes retrieval architectures based on privacy, capability, and accessibility, and validate their approach through experiments on consumer hardware, demonstrating that dense retrieval maintains high performance even as document volume increases. Key results indicate that a local language model can achieve answer quality comparable to cloud-based systems while preserving user privacy, highlighting that the tradeoff in local-first IR is primarily about the scope of searchable content rather than the quality of the search results.
Local-first information retrieval can achieve cloud-level answer quality while keeping sensitive data on user devices, redefining the tradeoff between search scope and quality.
The sensitive information in personal documents, legal files, and medical records is among the most valuable things to search, yet current retrieval-augmented generation systems still require sending content to remote servers. We propose local-first IR, a design philosophy where indexes, models, and inference reside on user devices, treating remote services as optional. This paper makes four contributions: (1) a framework organizing retrieval architectures along three dimensions: privacy and control, capability, and accessibility, (2) experiments on consumer hardware across five benchmarks, scaling from 1K to 1M documents with dense retrieval, BM25, and hybrid fusion. Dense retrieval keeps over 91% nDCG@10 up to 100K documents, with approximate HNSW indexes extending this to 1M with only 2% quality loss; a 7B local language model reaches within 4 points of a cloud baseline on answer quality, (3) competing perspectives for and against local-first IR, informed by experimental evidence, and (4) a research agenda identifying open problems. The real tradeoff is scope rather than quality: what matters is what you can search, not how well you can search it.
