Search papers, labs, and topics across Lattice.
This paper introduces TISC, a novel text-driven image semantic communication framework designed to enhance the faithful reconstruction of images transmitted as text descriptions. By addressing critical bottlenecks in semantic extraction and reconstruction, TISC employs a Tree-Structured Attribute Semantic Extraction (TSASE) method to improve the granularity of image descriptions and an Initial Noise Optimization (INO) mechanism to ensure semantic consistency during image generation. Experimental results demonstrate that TISC significantly enhances both object-position recovery and the fidelity of semantic descriptions compared to existing methods.
TISC achieves unprecedented levels of semantic fidelity in image reconstruction by optimizing both the extraction of image attributes and the selection of noise seeds during generation.
Generative image semantic communication converts an image into a text description and then performs text-to-image reconstruction at the receiver via diffusion-based generative models. This paradigm has attracted broad attention due to its extremely low bandwidth cost. However, existing methods still face two critical bottlenecks across image-to-text (I2T) semantic extraction at the transmitter and text-to-image (T2I) semantic reconstruction at the receiver: (i) semantic loss and distortion in I2T, where holistic image descriptions may omit fine-grained object attributes and spatial-position information, causing the generated text to deviate from the original image semantics; and (ii) insufficient semantic faithfulness in T2I, where even with the same semantically faithful text description, different initial noise settings may lead diffusion-based reconstruction to produce images with different levels of semantic consistency with the original image. These issues jointly limit the semantic faithfulness of image reconstruction. To address them, we propose TISC, a text-driven image semantic communication framework tailored for faithful reconstruction. TISC incorporates two key designs: (1) Tree-Structured Attribute Semantic Extraction (TSASE), which decomposes semantic extraction into global scene, background, and object-level attribute descriptions, covering spatial position, shape/pose, color, material, and other physical attributes for each detected object; and (2) an Initial Noise Optimization (INO) mechanism, which selects an initial noise seed at the transmitter according to a comprehensive similarity score that jointly considers visual and semantic consistency. Experiments on multiple datasets show that TSASE improves object-position recovery and semantic description faithfulness, while the INO parameter study supports the adopted configuration for noise selection.
