Yu Li

D plane. For video or event sequences, a temporal position embedding, EtempE_{\text{temp}}, is additionally incorporated to capture the sequential order of the frames. To maintain modality differentiability within the unified feature space, tokens from auxiliary modalities, such as event streams, are marked with a unique modality type embedding, EmodE_{\text{mod}}. Furthermore, to efficiently handle temporal inputs with multiple frames, we introduce a lightweight Time Adapter. This adapter, composed of a multi-layer perceptron, can fuse and compress features from multiple frame tokens, significantly improving computational efficiency while preserving key dynamic information. Through this series of operations, any form of visual input is standardized into a visual token sequence rich in information, Fvis