Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a knowledge distillation framework leveraging Koopman operator theory to extract linearized models from pre-trained neural networks for classification. The approach aims to create machine learning architectures suitable for emerging hardware optimized for linear operations. Experiments on MNIST and Fashion-MNIST show the distilled linearized models achieve higher classification accuracy and numerical stability compared to standard least-squares Koopman approximations.
Linearized neural networks, distilled from pre-trained models using Koopman theory, can outperform traditional linear approximations while retaining the benefits of efficient linear operations.
Recent developments in hardware, such as photonic integrated circuits and optical devices, are driving demand for research on constructing machine learning architectures tailored for linear operations. Hence, it is valuable to explore methods for constructing learning machines with only linear operations after simple nonlinear preprocessing. In this study, we propose a framework to extract a linearized model from a pre-trained neural network for classification tasks by integrating Koopman operator theory with knowledge distillation. Numerical demonstrations on the MNIST and the Fashion-MNIST datasets reveal that the proposed model consistently outperforms the conventional least-squares-based Koopman approximation in both classification accuracy and numerical stability.