Search papers, labs, and topics across Lattice.
This paper introduces SAGE, a framework that automates storyboarding by learning and evolving directing knowledge from expert demonstrations while addressing challenges in knowledge acquisition, refinement, and injection. By contrasting training screenplays with expert storyboards, SAGE derives adaptable rules that can be updated based on localized feedback, allowing for efficient retrieval of relevant directing knowledge during narrative generation. In practical deployment, SAGE achieved a score of 77.8 on a rubric validated by experts, outperforming professional directors, and significantly reduced authoring time for the production team by over 83%.
SAGE not only matches professional directors in storyboard quality but also slashes authoring time, revolutionizing automated narrative production.
Storyboards turn screenplays into visual shot plans for automated short drama production. Professional storyboarding relies on tacit directorial expertise and remains an industrial bottleneck. Large language models can automate this step, but methods for supplying directing knowledge face three challenges: (1) Knowledge acquisition: the craft remains implicit in exemplars or must be written manually. (2) Knowledge refinement: authored knowledge is not evaluated against execution outcomes, and opaque generation prevents feedback attribution to the knowledge behind each decision. (3) Knowledge injection: injecting all knowledge exceeds usable context, while manual selection for every narrative group does not scale. We present SAGE (Skill with Attribution-Guided Evolution), a deployed framework that learns, attributes, evolves, and routes directing knowledge from expert demonstrations. SAGE derives rules that are independent of episode content by contrasting each training screenplay with its expert storyboard. During generation, the model records each narrative group's adopted rules. Combining these records with localized feedback enables targeted updates to individual rules. Evolved rules form scenario packages with a routing index, so each group retrieves only a bounded set appropriate to its situation without expert intervention. On 18 test episodes across three genres, SAGE scored 77.8 on a rubric validated by experts, versus 77.1 for professional directors. Deployed for 14 days on Virtual Film Studio, SAGE produced 1,344 narrative group outputs; 87.2 percent were accepted without substantive edits, and the production team recorded over 83 percent less authoring time per episode. We release PROSE, the first public dataset pairing screenplays with storyboards by professional directors across 68 episodes: https://github.com/creDreams/PROSE.
