Search papers, labs, and topics across Lattice.
This paper introduces MD-SigLIP, a margin-regularized structured semantic alignment framework that aligns brain embeddings with text embeddings in a shared semantic space to enhance brain-language decoding. By employing duplicate-aware sigmoid contrastive learning and a listwise margin-regularized term, the method effectively captures the structured ranking of semantic clusters, improving interpretability and correspondence between neural representations and language semantics. Experimental results show that MD-SigLIP achieves state-of-the-art retrieval performance, addressing critical ambiguities in current brain-language decoding approaches.
Aligning brain and language embeddings reveals the true semantic correspondence, overcoming the limitations of existing decoding methods.
With the rapid advancement of large language models, brain-language decoding has achieved remarkable progress. However, it remains unclear whether decoded content genuinely reflects neural representations or is largely reconstructed by the language model itself. This ambiguity limits interpretability and hinders the investigation of intrinsic brain-language correspondence. To address this challenge, we propose MD-SigLIP. This margin-regularized structured semantic alignment framework directly aligns brain embeddings with text embeddings in a shared semantic space, enabling retrieval-based decoding. This formulation enables explicit modeling of the correspondence between neural representations and language semantics. Building upon duplicate-aware sigmoid contrastive learning, we introduce a listwise margin-regularized term that enforces structured ranking constraints between positive semantic clusters and negative samples. By modeling multi-positive semantic structure and margin-based ordering simultaneously, the method captures the manifold organization of language embeddings reflected in neural signals. Experiments demonstrate state-of-the-art retrieval performance under both full-vocabulary and subset evaluation settings.
