Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates whether discrete regimes in temporally drifting data can be extracted from the weights of neural network models, employing a hidden Markov model (HMM) to analyze the trajectories of these weights. By applying this method to multimodal misinformation detection and sentiment analysis, the authors demonstrate that classifiers trained on data from specific temporal windows exhibit improved generalization when tested on data from the same latent state. The findings reveal that the estimated states align more closely with shifts in class distribution than with the underlying weight-space geometry, highlighting the relevance of these latent states for effective transfer learning across data distributions.
Classifiers trained on temporally coherent data windows show a significant boost in generalization, revealing hidden structures in model weights that enhance transfer learning.
A temporally drifting data stream may pass through discrete regimes rather than changing continuously. We ask whether such regimes are recoverable from the weights of models trained on the stream, using a hidden Markov model (HMM) fit to the chronologically ordered trajectory of those weights. We study this question in two domains known to drift over time: multimodal misinformation detection, using the Fakeddit dataset; and sentiment analysis, using the Yelp dataset. We train classifiers on consecutive temporal windows and fit an HMM to the trajectory of their aligned weights, recovering latent states that partition each timeline into coherent phases. On both datasets, classifiers generalize better to data from windows sharing the state of their training window than to windows across state boundaries. This within-state transfer advantage survives a control for temporal proximity and modestly exceeds the advantage recovered by a naive partition into contiguous states of equal size. Although the states are estimated solely from model weights, they correlate more strongly with shifts in the data's class distribution than with the weight-space geometry used to estimate them. After class divergence and lag are residualized out, the within-state advantage exceeds its permutation null on both tasks, indicating that the states recover structure relevant to transfer beyond the data distribution. Every effect replicates on both tasks but is attenuated on Yelp, whose label distribution is more temporally stable.