Search papers, labs, and topics across Lattice.
This paper introduces ScalablePromptus, an enhanced version of the Promptus framework for prompt-based video streaming that addresses the vulnerability of existing methods to network fluctuations. By incorporating semantic and color-aware prompt inversion, spherical linear interpolation for intermediate frames, and a dropout training strategy for rank-ordered prompt representations, the system allows for effective video reconstruction even from truncated prompts. The results show that under lossy network conditions, ScalablePromptus significantly reduces performance degradation by 82%-95% compared to the baseline, making it a viable solution for real-world applications.
ScalablePromptus transforms prompt-based video streaming by enabling robust reconstruction from incomplete data, slashing performance loss by up to 95% in lossy conditions.
Prompt-based video streaming transmits compact semantic prompts instead of pixel-level content for generative reconstruction, enabling ultra-low-bitrate communication. However, the state-of-the-art Promptus framework is vulnerable to network fluctuation, where partially received prompts lead to catastrophic quality collapse. We propose ScalablePromptus, which enhances Promptus with semantic and color-aware prompt inversion, spherical linear interpolation for intermediate frames, and--most critically--a dropout training strategy that produces rank-ordered prompt representations. This allows the receiver to reconstruct meaningful video from arbitrarily truncated prompts without any adaptation. Under stable networks, ScalablePromptus achieves modest quality gains. Under lossy conditions, it reduces the performance degradation caused by truncation by 82%-95% compared to the baseline, making prompt-based streaming robust enough for real-world deployment.
