Search papers, labs, and topics across Lattice.
This paper introduces CHIEF, a human-AI co-creation framework for video generation that integrates creator-driven iterative refinement with automatic subjective feedback. By leveraging persona-conditioned multimodal LLMs, the system provides nuanced critiques that enhance narrative coherence and creative direction, addressing common shortcomings in AI-generated videos. The framework was tested with high school and college students, demonstrating significant improvements in the quality and complexity of videos produced, even by those with no prior filmmaking experience.
Creator-driven feedback loops in video generation can transform novice filmmakers into skilled storytellers, enhancing narrative depth and coherence.
Generative AI has made content creation increasingly accessible, but many AI-generated videos lack narrative coherence and creative direction, issues that become more substantial at longer durations. Unlike coding, where AI generation benefits from reliable feedback and techniques such as recurrent self-improvement, video generation requires subjective feedback about plot, scenes, and narrative, which naturally motivates approaches that incorporate human creative direction. We introduce CHIEF, a human-AI co-creation video generation framework that places the creator at the center of human-in-the-loop iterative video refinement, and supports them by providing automatic subjective feedback. The creator incorporates their creative direction by driving each iteration, while their revisions are incorporated by a specialized refiner agent. The feedback loop is generated by persona-conditioned multimodal LLMs that watch generated videos and produce subjective critique from the audience perspectives, providing feedback that self-evaluation alone cannot capture. To test the effectiveness of our proposed framework, we work with high school and college students with no prior filmmaking experience to create videos, from short 1-minute videos to a complete short 10-minute film with a complicated plot.
