Seung Ha Hwang

C𝐅i)\mathbf{f}_{\text{out}}=\text{BatchNorm}\left(\frac{1}{C}\sum_{i=1}^{C}\mathbf{F}_{i}\right) (19) Gated Linear Unit (GLU) Fusion: A gating mechanism that learns to suppress noise and control information flow. The gating mechanism can learn to suppress noisy or irrelevant feature dimensions while preserving informative ones, acting as a learned noise filter. 𝐜=𝐱s+𝐱t\mathbf{c}=\mathbf{x}_{s}+\mathbf{x}_{t} (20) 𝐠=σ​(𝐖g​[𝐱s∥𝐱t]+𝐛g)∈ℝB×D\mathbf{g}=\sigma\left(\mathbf{W}_{g}[\mathbf{x}_{s}\|\mathbf{x}_{t}]+\mathbf{b}_{g}\right)\in\mathbb{R}^{B\times D} (21) 𝐟out=BatchNorm​(𝐜⊙𝐠)\mathbf{f}_{\text{out}}=\text{BatchNorm}(\mathbf{c}\odot\mathbf{g}) (22) where σ\sigma is the sigmoid function, ⊙\odot denotes element-wise multiplication, and 𝐖g∈ℝD×2​D\mathbf{W}_{g}\in\mathbb{R}^{D\times, D}. Multiplicative Fusion: Element-wise product of projected features to capture second-order interactions. Multiplicative interactions can capture correlations between the two streams that additive fusion cannot, enabling the model to learn feature conjunctions. 𝐟=(𝐖s​𝐱s+𝐛s)⊙(𝐖t​𝐱t+𝐛t)\mathbf{f}=(\mathbf{W}_{s}\mathbf{x}_{s}+\mathbf{b}_{s})\odot(\mathbf{W}_{t}\mathbf{x}_{t}+\mathbf{b}_{t}) (23) 𝐟out=BatchNorm​(Dropout​(𝐟))\mathbf{f}_{\text{out}}=\text{BatchNorm}(\text{Dropout}(\mathbf{f})) (24) where 𝐖s,𝐖t∈ℝD×D\mathbf{W}_{s},\mathbf{W}_{t}\in\mathbb{R}^{D\times D}. Low-Rank Bilinear Fusion: Full bilinear interaction approximated with low-rank factorization for efficiency. Captures rich pairwise interactions between all feature dimensions while maintaining computational efficiency through the low-rank bottleneck. A full bilinear model would compute 𝐱s⊤​𝐖𝐱t\mathbf{x}_{s}^{\top}\mathbf{W}\mathbf{x}_{t} with 𝐖∈ℝD×D×D\mathbf{W}\in\mathbb{R}^{D\times D\times D}, which is computationally prohibitive. Instead, we use a low-rank approximation: 𝐳s=𝐔s​𝐱s∈ℝB×R\mathbf{z}_{s}=\mathbf{U}_{s}\mathbf{x}_{s}\in\mathbb{R}^{B\times R} (25) 𝐳t=𝐔t​𝐱t∈ℝB×R\mathbf{z}_{t}=\mathbf{U}_{t}\mathbf{x}_{t}\in\mathbb{R}^{B\times R} (26) 𝐳=𝐳s⊙𝐳t∈ℝB×R\mathbf{z}=\mathbf{z}_{s}\odot\mathbf{z}_{t}\in\mathbb{R}^{B\times R} (27) 𝐟out=BatchNorm​(Dropout​(𝐖out​𝐳+𝐛out))\mathbf{f}_{\text{out}}=\text{BatchNorm}(\text{Dropout}(\mathbf{W}_{\text{out}}\mathbf{z}+\mathbf{b}_{\text{out}})) (28) where 𝐔s,𝐔t∈ℝR×D\mathbf{U}_{s},\mathbf{U}_{t}\in\mathbb{R}^{R\times D} are low-rank projection matrices, R≪DR\ll D is the rank (typically 8-32), and 𝐖out∈ℝD×R\mathbf{W}_{\text{out}}\in\mathbb{R}^{D\times R}. Cross-Attention Fusion: Multi-head cross-attention allowing one stream to attend to the other, followed by global weighting. Cross-attention allows the spectral stream to selectively incorporate information from the temporal stream based on learned relevance, providing a more flexible fusion mechanism than simple weighting. Step 1: Multi-Head Cross-Attention For HH attention heads with head dimension dh=D/Hd_{h}=D/H: 𝐐=𝐖Q​𝐱s,𝐊=𝐖K​𝐱t,𝐕=𝐖V​𝐱t\mathbf{Q}=\mathbf{W}_{Q}\mathbf{x}_{s},\quad\mathbf{K}=\mathbf{W}_{K}\mathbf{x}_{t},\quad\mathbf{V}=\mathbf{W}_{V}\mathbf{x}_{t} (29) Attention​(𝐐,𝐊,𝐕)=softmax​(𝐐𝐊⊤τ​dh)​𝐕\text{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\text{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\tau\sqrt{d_{h}}}\right)\mathbf{V} (30) 𝐱attn=𝐖O⋅Concat​(head1,…,headH)\mathbf{x}_{\text{attn}}=\mathbf{W}_{O}\cdot\text{Concat}(\text{head}_{1},\ldots,\text{head}_{H}) (31) Step 2: Global Weighting 𝐰=softmax​(MLP​([𝐱s∥𝐱attn])τ)∈ℝB×2\mathbf{w}=\text{softmax}\left(\frac{\text{MLP}([\mathbf{x}_{s}\|\mathbf{x}_{\text{attn}}])}{\tau}\right)\in\mathbb{R}^{B\times 2} (32) 𝐟=w0⋅𝐱s+w1⋅𝐱attn\mathbf{f}=w_{0}\cdot\mathbf{x}_{s}+w_{1}\cdot\mathbf{x}_{\text{attn}} (33) 𝐟out=BatchNorm​(𝐟+σ​(α)⋅𝐱s+𝐱t2)\mathbf{f}_{\text{out}}=\text{BatchNorm}\left(\mathbf{f}+\sigma(\alpha)\cdot\frac{\mathbf{x}_{s}+\mathbf{x}_{t}}{2}\right) (34) C.2 Fusion Ablation Results Table 10: Fusion Strategy Ablation on Wang 2016 SSVEP STFT Config Strategy Val Acc Unseen Acc Unseen F1 Unseen Rec Unseen AUC nperseg=128 noverlap=64 nfft=256