Search papers, labs, and topics across Lattice.
This paper details the implementation of the k-nearest neighbors (kNN) method on CUDA GPUs within MATLAB, specifically targeting regression problems with large datasets that exceed GPU memory capacity. The authors developed MATLAB scripts to optimize hyperparameters and enhance computational efficiency, addressing challenges such as data transfer between RAM and GPU memory and parallelizing computations across multiple GPUs. Key findings indicate significant reductions in regression calculation time and the ability to handle larger training sets using the proposed methods, underscoring the practical benefits of GPU acceleration for kNN in regression tasks.
Achieving substantial speedups in kNN regression calculations on large datasets by leveraging CUDA GPUs could transform how we approach real-time data analysis.
This article describes the implementation of the k-nearest neighbors (kNN) method on NVIDIA graphics accelerators (CUDA GPUs) in MATLAB for solving regression problems with large training data sets that do not fit in GPU memory. The introduction lists the main features of the kNN method, describes the principles of its use for classification and regression, and indicates methods for summing the responses of nearest neighbors. The second part of the article examines the main computational blocks included in a typical kNN regression code. Prediction error metrics for individual predicted responses and averaged errors for a group of predicted responses are presented. The article describes MATLAB m.scripts developed during the author’s work, which implement various methods for calculating kNN regression on a central processing unit (CPU), a single GPU, and multiple GPUs. Various issues related to the implementation and acceleration of regression calculations that arose during the course of this work are discussed: Features of the knnsearch function when searching for nearest neighbors; Parallel implementation of the code for weighted summation of neighbor responses when predicting functions or sets of samples; Processing large arrays of input data; Transferring data between the computer’s RAM and GPU memory; Parallelizing computations across graphics accelerators. The dependence of regression calculation time on the training set array size using various variants of the developed m.-scripts on a workstation with two AMD EPYC 7K62 processors (Zen 2 (Rome) architecture) and three NVIDIA CUDA GeForce RTX3090 GPUs (Ampere generation) is presented. The relevance of accelerating regression calculations in MATLAB using the kNN method is demonstrated, especially for large training set sizes. The achieved gains in computation time and maximum training set size using the developed script, which cyclically utilizes a single GPU, are noted. The third section describes m.-scripts developed for finding optimal hyperparameter values: the number of nearest neighbors (kNN); the Parzen window kernel type (CoreType); the Parzen window width coefficient (kD0); the set of significant input features; the distance metric type (DistType); the input window length (Nwindow - the number of analyzed features); the history length (Ntrain - the training set size). The conclusion lists the obtained results.
