Search papers, labs, and topics across Lattice.
This study develops a two-stage Large Language Model (LLM) pipeline to extract structured causal evidence from humanitarian reports, addressing the challenges posed by their length and complexity. By implementing query-conditioned extraction and snippet grounding, the authors enhance the precision and auditability of intervention-outcome records, achieving a weighted F1 score of 94.15% with Llama-3.1-8B. The proposed context-preserving triangulation method quantifies evidence convergence, revealing strong positive outcomes for cash assistance interventions with a Level-of-Evidence score of 0.865.
Cash assistance interventions show a remarkable Level-of-Evidence score of 0.865, indicating strong convergence in humanitarian outcomes.
Humanitarian reports are long, noisy, and multi-topic, making it difficult to consolidate decision-relevant causal evidence. We present a ReliefWeb study (2000-2024) and a two-stage Large Language Model (LLM) pipeline that extracts structured intervention-outcome records with direction and strength attributes. Query-conditioned extraction restricts output to a specified intervention class, reducing retrieval-induced over-extraction, while snippet grounding links each relation to supporting text for auditability and classification. In an expert-annotated dataset of 100 reports, the best closed-source LLM achieved a weighted F1 score of 90.73% with strong cost-efficiency, while Llama-3.1-8B with supervised fine-tuning reached 94.15% weighted F1 score. We further propose context-preserving triangulation that aggregates strength-weighted evidence within disaster$\times$source cells, applies Laplace smoothing and equally weights cells to quantify cross-context convergence via a Level-of-Evidence score. Applied to cash assistance, food-related outcomes show strong positive convergence (LoE=0.865) and stable long-horizon trajectories.
