Search papers, labs, and topics across Lattice.
The paper introduces SafeBranch, a novel framework for aligning safety in embodied agents by utilizing branch pairs derived from the agent's own unsafe rollouts. This approach addresses the challenge of training agents to act safely while achieving task success, which has been difficult due to the distinct nature of safety and task objectives. Experimental results on IS-Bench and SafetyALFRED demonstrate that SafeBranch significantly improves safety performance, achieving approximately ten times more safe successes compared to untrained baselines, even in out-of-distribution scenarios with unseen tasks and objects.
SafeBranch enables embodied agents to achieve ten times more safe task completions without sacrificing performance, even in novel environments.
Vision-language-model-based embodied agents can complete instructed tasks but often violate safety constraints in the process, a problem recently framed as interactive safety. Training such agents to act safely is difficult, since safety and task success are distinct objectives, and safety arises only at a small number of safety-critical steps within a trajectory. Standard supervision is insufficient: imitating safe trajectories teaches behavior without explaining why it is safe, and contrasting arbitrary safe and unsafe trajectories mixes the safety signal with unrelated differences. We propose SafeBranch, a framework that aligns an embodied actor on safety through branch pairs constructed from the actor's own unsafe rollouts via environment rollback. SafeBranch rolls each unsafe rollout back to the safety-critical step that caused the violation, queries the actor for a safe alternative, and pairs the original action with the alternative so that the two branches differ only at that step. The trained actor acts safely at deployment with no critic in the loop. On IS-Bench, SafetyALFRED, and out-of-distribution variants with unseen tasks and objects, it handles safety reliably without sacrificing task success, achieving roughly ten times more safe successes than the untrained baseline on the unseen-object variant.
