Search papers, labs, and topics across Lattice.
This paper introduces a rate-distortion-complexity (RDC) framework for semantic communication, extending classical rate-distortion theory to account for semantic distance (measured by bit-wise distortion and divergence) and model complexity (using minimum description length and information bottleneck principles). The authors derive closed-form expressions for the minimum achievable rate under constraints on semantic distance and complexity for Gaussian and binary semantic sources, revealing a fundamental three-way tradeoff. Empirical validation on image and video datasets confirms the theoretical tradeoff and demonstrates the correlation between the proposed information-theoretic complexity measure and practical computational costs.
Semantic communication doesn't have to be computationally expensive: this paper introduces a framework that explicitly trades off rate, semantic distortion, and model complexity, enabling efficient system design.
Semantic communication is a novel communication paradigm that focuses on conveying the user's intended meaning rather than the bit-wise transmission of source signals. One of the key challenges is to effectively represent and extract the semantic meaning of any given source signals. While deep learning (DL)-based solutions have shown promising results in extracting implicit semantic information from a wide range of sources, existing work often overlooks the high computational complexity inherent in both model training and inference for the DL-based encoder and decoder. To bridge this gap, this paper proposes a rate-distortion-complexity (RDC) framework which extends the classical rate-distortion theory by incorporating the constraints on semantic distance, including both the traditional bit-wise distortion metric and statistical difference-based divergence metric, and complexity measure, adopted from the theory of minimum description length and information bottleneck. We derive the closed-form theoretical results of the minimum achievable rate under given constraints on semantic distance and complexity for both Gaussian and binary semantic sources. Our theoretical results show a fundamental three-way tradeoff among achievable rate, semantic distance, and model complexity. Extensive experiments on real-world image and video datasets validate this tradeoff and further demonstrate that our information-theoretic complexity measure effectively correlates with practical computational costs, guiding efficient system design in resource-constrained scenarios.
