Search papers, labs, and topics across Lattice.
To address the inability of scalar video quality metrics to diagnose physical failures, the authors introduce VeriPhy, an auditable verification framework that statically compiles text prompts into typed physical obligations before dispatching targeted perceptual experts. Unlike monolithic VLM critics that yield black-box judgments, the system evaluates dynamic scenes using explicit physical measurements (e.g., depth, tracking, audio-visual events) and resolves them into three-valued, provenance-backed verdicts. Evaluated on a benchmark of fine-grained physical flaws, VeriPhy captures 228 of 304 human-annotated failures鈥攐utperforming prior decomposition evaluators (164) and matching monolithic baselines while providing the localized physical evidence required for targeted model refinement.
Instead of trusting black-box scalar scores to evaluate world models, video verification can match end-to-end VLM recall while producing fully auditable, spatiotemporally grounded physical proofs of failure.
Visual fluency in generated video does not imply physical reliability, and a scalar quality score alone is incapable of indicating the obligation a clip violates or the moment it fails. We present VeriPhy, an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations and a statically validated execution plan before any frame is observed. During execution, observations gate and scope only declared calls to frozen low-level experts (e.g., segmentation and tracking, counting, eleven typed physical measurements over the resulting tracks, depth, OCR, and audio-event detection). Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state. Typed resolvers and fixed composition map usable records to a three-valued state (supported, contradicted, or unknown, surfaced as plausible, implausible, or abstain) with full provenance, so that every verdict is traceable to the evidence that produced it. We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time. On a 149-clip core carrying 304 such records, VeriPhy accounts for 228, against 164 for a published question-decomposition evaluator given the same clips and the same claims. Recall alone does not separate it from prompting the same backbone monolithically, which reaches 222; what separates them is that each decision retains its evidence record and provenance, making the traces auditable one verdict at a time and usable as the interface through which a critic verdict could be written back into generation.
