Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a hybrid QUBO framework for neural network pruning that combines gradient-aware filter importance estimation (Taylor and Fisher information) with data-driven activation similarity to capture both individual filter relevance and inter-filter redundancy. They also introduce a dynamic capacity-driven search to enforce target sparsity and a Tensor-Train (TT) Refinement stage to fine-tune the QUBO-derived solution. Experiments on image denoising demonstrate that this approach outperforms greedy Taylor pruning and traditional L1-based QUBO, suggesting that hybrid combinatorial formulations can improve neural network compression.
Ditch the greedy heuristics: QUBO optimization, when informed by gradients and refined with Tensor-Trains, can drastically outperform traditional pruning methods.
Neural network pruning can be formulated as a combinatorial optimization problem, yet most existing approaches rely on greedy heuristics that ignore complex interactions between filters. Formal optimization methods such as Quadratic Unconstrained Binary Optimization (QUBO) provide a principled alternative but have so far underperformed due to oversimplified objective formulations based on metrics like the L1-norm. In this work, we propose a unified Hybrid QUBO framework that bridges heuristic importance estimation with global combinatorial optimization. Our formulation integrates gradient-aware sensitivity metrics - specifically first-order Taylor and second-order Fisher information - into the linear term, while utilizing data-driven activation similarity in the quadratic term. This allows the QUBO objective to jointly capture individual filter relevance and inter-filter functional redundancy. We further introduce a dynamic capacity-driven search to strictly enforce target sparsity without distorting the optimization landscape. Finally, we employ a two-stage pipeline featuring a Tensor-Train (TT) Refinement stage - a gradient-free optimizer that fine-tunes the QUBO-derived solution directly against the true evaluation metric. Experiments on the SIDD image denoising dataset demonstrate that the proposed Hybrid QUBO significantly outperforms both greedy Taylor pruning and traditional L1-based QUBO, with TT Refinement providing further consistent gains at appropriate combinatorial scales. This highlights the potential of hybrid combinatorial formulations for robust, scalable, and interpretable neural network compression.