Search papers, labs, and topics across Lattice.
VGA-BenchV2 is an enhanced benchmark framework designed to evaluate and optimize video generation quality and aesthetic value, expanding upon its predecessor, VGA-Bench. It introduces a comprehensive taxonomy with 52 sub-dimensions and incorporates over 60,000 generated videos, significantly increasing human-labeled supervision with 36,000 new task-level annotations. The framework's hybrid evaluator architecture, which includes VAQA-Net and two Qwen-based models, effectively aligns with human judgments and facilitates an evaluation-to-optimization pipeline, allowing for improved generator fine-tuning based on aesthetic and generation quality feedback.
With 36,000 new human annotations, VGA-BenchV2 not only enhances evaluation but also transforms how video generators can be optimized for aesthetic quality and realism.
We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from benchmark construction and human supervision to automated evaluation and model optimization, enabling video generators to improve not only in realism but also in aesthetic quality and human preference alignment. Resources are available at https://huggingface.co/datasets/BestiVictoryLab/VGA-Bench.
