Search papers, labs, and topics across Lattice.
ALIGNBEAM introduces a novel inference-time alignment transfer method that addresses the safety degradation of fine-tuned large language models by enabling cross-vocabulary logit mixing. This approach translates logits from a safe anchor model into the vocabulary of a target model on-the-fly, allowing for the selection of the safest candidate continuation without requiring any model retraining. The results demonstrate a significant increase in refusal rates on adversarial prompts while maintaining task accuracy and practical inference overhead, showcasing a new pathway for safety alignment across model families.
Safety alignment can now be effectively transferred between different model families at inference time, without any need for retraining.
Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor logits into the target model's vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the safety-utility trade-off can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on adversarial benchmarks while keeping task accuracy and inference overhead within practical bounds. The results show that safety alignment can be transferred between model families at inference time, without touching either model's weights.
