Search papers, labs, and topics across Lattice.
This paper investigates spline-based numerical encodings for tabular deep learning, comparing B-splines, M-splines, and I-splines with various knot placement strategies (uniform, quantile-based, target-aware, and learnable). They introduce a differentiable knot parameterization for stable end-to-end optimization of knot locations. Experiments across diverse datasets and backbones (MLP, ResNet, FT-Transformer) reveal that the effectiveness of numerical encodings is highly task-dependent, with piecewise-linear encoding proving robust for classification and performance varying significantly for regression based on spline family, knot placement, and output size.
Forget one-size-fits-all preprocessing: the best way to encode numerical features for tabular deep learning hinges on the task, backbone, and even the output size.
Numerical preprocessing remains an important component of tabular deep learning, where the representation of continuous features can strongly affect downstream performance. Although its importance is well established for classical statistical and machine learning models, the role of explicit numerical preprocessing in tabular deep learning remains less well understood. In this work, we study this question with a focus on spline-based numerical encodings. We investigate three spline families for encoding numerical features, namely B-splines, M-splines, and integrated splines (I-splines), under uniform, quantile-based, target-aware, and learnable-knot placement. For the learnable-knot variants, we use a differentiable knot parameterization that enables stable end-to-end optimization of knot locations jointly with the backbone. We evaluate these encodings on a diverse collection of public regression and classification datasets using MLP, ResNet, and FT-Transformer backbones, and compare them against common numerical preprocessing baselines. Our results show that the effect of numerical encodings depends strongly on the task, output size, and backbone. For classification, piecewise-linear encoding (PLE) is the most robust choice overall, while spline-based encodings remain competitive. For regression, no single encoding dominates uniformly. Instead, performance depends on the spline family, knot-placement strategy, and output size, with larger gains typically observed for MLP and ResNet than for FT-Transformer. We further find that learnable-knot variants can be optimized stably under the proposed parameterization, but may substantially increase training cost, especially for M-spline and I-spline expansions. Overall, the results show that numerical encodings should be assessed not only in terms of predictive performance, but also in terms of computational overhead.
