Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces FSFVE, a framework that enhances highly compressed face video during videocalls by training a model with as few as 10 frames, achieving significant improvements in video quality. The system operates in real-time on low-compute devices without requiring modifications to existing video conferencing applications. Experimental results demonstrate both quantitative and perceptual enhancements in video quality, making it a practical solution for users facing bandwidth limitations.
Achieving real-time enhancement of compressed video calls with just 10 frames could revolutionize the quality of remote communication for millions with bandwidth constraints.
Videocalling has become a popular form of communication in the world today, with many companies providing free services for it. However, there are still millions of people around the world that experience poor quality videocalls due to limitations in bandwidth. This despite, most people having the required hardware. In this paper we present a novel framework for enhancing highly compressed videocalls. We show, that with as little as 10 frames of the face, we can rapidly (in under 100 seconds) train a model to enhance that instance of the videocall. The model can be trained either prior to or during the call, enhancing the rest of the call by producing better quality video. The video conferencing application need not be modified - it can be off the shelf with our system as a layer on top that trains quickly then simply lets the video conferencing application (e.g. Zoom) run as usual, where our system intercepts and improves images before they are displayed. The model is designed to run in realtime on low-compute devices such as a typical laptop CPU. Experimentally, we show that the model significantly improves quality of compressed face video both quantitatively as well as perceptually. Code can be found at https://github.com/varun-jois/FSFVE.