Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces a benchmark for evaluating legal AI systems in the context of French immigration law, addressing a significant gap in existing benchmarks. By comparing a parametric LLM baseline with dense retrieval augmentation across two model scales (Qwen3.5-9B and -27B) on 52 annotated synthetic profiles, the authors demonstrate that retrieval significantly enhances the accuracy of permit-type recommendations and overall administrative guidance. The findings underscore the importance of retrieval grounding for improving the reliability of legal AI applications in complex regulatory environments.
Retrieval augmentation boosts permit-type accuracy in French immigration law AI applications, highlighting a critical leverage point for legal guidance systems.
International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks. We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage. Comparing a parametric LLM baseline against dense retrieval augmentation at two model scales (Qwen3.5-9B and -27B) on 52 annotated synthetic profiles, we find that retrieval improves administrative guidance at both scales, most notably permit-type accuracy. Our results confirm that retrieval grounding is important for more reliable administrative guidance in this domain, and motivate further investigation of hybrid retrieval strategies.