Search papers, labs, and topics across Lattice.
The paper introduces PRISM, a risk-gated supervised fine-tuning (SFT) framework that mitigates factual hallucinations in language models by penalizing overconfident predictions on tokens deemed factually risky based on sentence-level factuality risk labels and inter-sentence dependency annotations. PRISM uses a model-aware probability reallocation objective guided by span-level risk weights and model-aware gating to selectively modify learning at fact-critical positions. Experiments on factual benchmarks demonstrate that PRISM improves factual accuracy while maintaining overall language capabilities, highlighting the importance of conservative risk application and the complementary roles of knowledge masking and model-aware reallocation.
Factually dubious LLM outputs can be tamed by strategically penalizing high-confidence predictions at "risky" tokens during fine-tuning, guided by sentence-level factuality labels.
Supervised fine-tuning (SFT) with token-level hard labels can amplify overconfident imitation of factually unsupported targets, causing hallucinations that propagate in multi-sentence generation. We study an augmented SFT setting in which training instances include coarse sentence-level factuality risk labels and inter-sentence dependency annotations, providing structured signals about where factual commitments are weakly supported. We propose \textbf{PRISM}, a differentiable risk-gated framework that modifies learning only at fact-critical positions. PRISM augments standard SFT with a lightweight, model-aware probability reallocation objective that penalizes high-confidence predictions on risky target tokens, with its scope controlled by span-level risk weights and model-aware gating. Experiments on hallucination-sensitive factual benchmarks and general evaluations show that PRISM improves factual aggregates across backbones while maintaining a competitive overall capability profile. Ablations further show that the auxiliary signal is most effective when used conservatively, and that knowledge masking and model-aware reallocation play complementary roles in balancing factual correction and capability preservation.
