Search papers, labs, and topics across Lattice.
This paper introduces a Video Token Communication (Video TokenCom) framework that leverages textual intent to guide multi-rate video communication using unequal error protection (UEP). The framework identifies semantically important video tokens based on user-provided textual descriptions via vision-language modeling and optical flow, then employs a multi-rate bit allocation strategy that prioritizes these tokens with full codebook precision while using reduced precision for others. The system adapts bit allocation and channel coding based on available resources and link conditions, demonstrating improved perceptual and semantic quality compared to baselines.
Achieve higher video quality under bandwidth constraints by intelligently prioritizing tokens that align with the user's textual intent.
Token Communication (TokenCom) is a new paradigm, motivated by the recent success of Large AI Models (LAMs) and Multimodal Large Language Models (MLLMs), where tokens serve as unified units of communication and computation, enabling efficient semantic- and goal-oriented information exchange in future wireless networks. In this paper, we propose a novel Video TokenCom framework for textual intent-guided multi-rate video communication with Unequal Error Protection (UEP)-based source-channel coding adaptation. The proposed framework integrates user-intended textual descriptions with discrete video tokenization and unequal error protection to enhance semantic fidelity under restrictive bandwidth constraints. First, discrete video tokens are extracted through a pretrained video tokenizer, while text-conditioned vision-language modeling and optical-flow propagation are jointly used to identify tokens that correspond to user-intended semantics across space and time. Next, we introduce a semantic-aware multi-rate bit-allocation strategy, in which tokens highly related to the user intent are encoded using full codebook precision, whereas non-intended tokens are represented through reduced codebook precision differential encoding, enabling rate savings while preserving semantic quality. Finally, a source and channel coding adaptation scheme is developed to adapt bit allocation and channel coding to varying resources and link conditions. Experiments on various video datasets demonstrate that the proposed framework outperforms both conventional and semantic communication baselines, in perceptual and semantic quality on a wide SNR range.
