Search papers, labs, and topics across Lattice.
This paper introduces SFAD, a speculative decoding framework designed to enhance contextual faithfulness in large language models while maintaining inference efficiency. By leveraging a preference dataset called ConFide and employing a novel mechanism called Epistemic Friction, SFAD detects potential hallucinations and refines model outputs through Asymmetric Logit Steering. The results show that SFAD significantly improves factual consistency and achieves a 2.48x speedup in inference time compared to traditional methods.
SFAD achieves a remarkable 2.48x speedup in inference while enhancing contextual faithfulness, addressing one of the most pressing challenges in large language models.
As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive applications. This task is particularly challenging as it requires balancing factual consistency with generation efficiency. Contrastive decoding methods require dual forward passes (with and without context) to compare model outputs, doubling inference computational overhead, while post-training alignment demands extensive reinforcement learning with substantial computational overhead. To address this challenge, we present \textbf{SFAD}, a speculative decoding framework that enhances contextual faithfulness without inference degradation. We first construct \textbf{ConFide}, a preference dataset with fine-grained atomic perturbations, to train a context-faithful draft model via Direct Preference Optimization. During inference, Epistemic Friction detects potential hallucinations by quantifying distributional tension weighted by specialist certainty. When friction exceeds the threshold, Asymmetric Logit Steering refines the target distribution through residual-based logit injection; otherwise, standard speculation proceeds. Extensive experiments demonstrate that SFAD substantially improves faithfulness while achieving $2.48\times$ speedup, offering a practical solution for efficient LLMs.
