Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study benchmarks fifteen machine learning and deep learning models for predicting familiarity from EEG data, focusing on two cognitive domains: faces and mathematical equations. By employing a rigorous trial-independent validation method, the authors reveal that previously reported high classification performances are misleading due to temporal leakage, with the best model achieving a peak F1-score of 0.6038. The findings underscore the importance of robust evaluation methods in assessing model generalizability and identify specific EEG features that are critical for familiarity prediction.
Trial-independent validation reveals that EEG-based familiarity prediction models may overestimate performance, with the best achieving only 0.6038 F1-score.
Objective assessment of learning remains a fundamental challenge in education. Electroencephalography (EEG) provides a direct, non-invasive window into the neural correlates of knowledge acquisition, including cognitive familiarity. This study benchmarks fifteen machine learning (ML) and deep learning (DL) models for EEG-based familiarity prediction across two cognitive domains: faces (factual knowledge) and mathematical equations (conceptual knowledge). Using continuous EEG data from 23 participants, we extract spectral features (Power Spectral Density) across six frequency bands. We show that while standard stratified cross-validation yields artificially high classification performance (up to 0.9853 F1-score using CNN) due to temporal leakage across neighboring epochs, a rigorous trial-independent validation (Group K-Fold) drops the peak performance to 0.6038 F1-score (using CNN), which is still statistically significant above the 25% chance level. This highlights the critical necessity of trial-independent evaluation to avoid overestimating model generalizability. Furthermore, feature importance and SHAP analysis reveal that temporal and frontal Gamma and Beta oscillations are the most critical biomarkers for familiarity. This work establishes a realistic benchmark for EEG-based cognitive monitoring in educational technologies.