Search papers, labs, and topics across Lattice.
To address extreme token overhead and hallucinated reasoning in high-stakes structured domains, the authors design TEFM, a framework that compresses tabular and temporal records into compact "Behavioral Code" tokens. The architecture optimizes a dual-fidelity objective that aligns code-level input reconstruction with prediction-level fidelity to extract minimal sufficient feature subsets grounded in the raw data. Across clinical and cybersecurity benchmarks on Qwen3, Gemma-2, and Phi-4 backbones, TEFM matches baseline classification accuracy while slashing context token consumption by 98% to 99% and producing faithful rationales.
LLMs can parse complex clinical and security logs using just 1% to 2% of the original context length without sacrificing predictive performance or explanation faithfulness.
In this paper, we solve two fundamental obstacles in applying LLMs to critical domains: token efficiency and faithfulness. To address both constraints jointly, we present TEFM (Token-Efficient Faithful Modeling), a framework designed for structured data analysis in critical domains. TEFM achieves token efficiency by compressing lengthy structured observations into compact Behavioral Code tokens, dramatically reducing token consumption with minimal information loss. Moreover, TEFM enables faithful rationalization through a dual-fidelity objective that jointly optimizes code-level reconstruction and prediction-level fidelity, identifying minimal sufficient feature subsets grounded in input data. Comprehensive experiments across various domain datasets and model backbones (Qwen3, Gemma-2, Phi-4) show that TEFM achieves competitive classification accuracy with dramatic token reduction (approximately 1\% token retention in clinical and 2\% in security domains) while producing faithful rationales.
Tsinghua AI