Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the capability of large language models (LLMs) to design video coding tools, specifically focusing on the Planar mode used in video coding standards. By employing a generation-and-evaluation loop, the LLM generated new predictors which were then evaluated for performance, resulting in a new mode that achieved a 0.18% bitrate savings with only a 0.4% increase in complexity compared to the conventional Planar mode. The findings indicate that LLMs can effectively contribute to the design of video coding tools, although challenges remain in their application and integration.
LLMs can outperform traditional video coding methods, achieving measurable efficiency gains in bitrate while maintaining complexity.
This paper explores whether large language models (LLMs) can design video coding tools, a highly challenging task due to the intricate algorithmic coupling of tool modifications. In particular, we present an empirical case study on the Planar mode, a long-standing intra prediction tool in video coding standards. Our experiments operate within a generation-and-evaluation loop, with the LLM generating new Planar predictors, encoder trials evaluating their coding performance, and the LLM re-generating refined implementations based on the evaluation feedback. We first examine directly replacing the default Planar mode in the Fraunhofer Versatile Video Encoder (VVenC) under its faster preset. Experimental results demonstrate that the LLM-generated mode can outperform the conventional Planar mode on this lightweight toolset, achieving 0.18% bitrate savings with 0.4% complexity overhead on the standard benchmark. We further extend our evaluation to the Enhanced Compression Model (ECM). Leveraging newly introduced directional Planar modes, we investigate two integration strategies: directly replacing them, and introducing the LLM-generated predictor as an additional prediction mode with new syntax elements. The empirical results suggest that both strategies can yield coding gains under a constrained low-resolution setting. Overall, this study offers preliminary evidence and practical insights, highlighting both the potential and open challenges of LLM-based coding tool design.