Search papers, labs, and topics across Lattice.
This paper introduces semantic sampling via learnable observation front ends, which generates finite-dimensional observations from learned signal responses rather than directly subsampled waveform points. By employing a semantic feature filterbank, a constrained semantic observation matrix, and a low-rate readout module, the method captures more relevant spectral-temporal structures for acoustic signals like speech. Experimental results demonstrate that this approach significantly enhances waveform fidelity, spectral consistency, and perceptual quality compared to conventional low-rate sampling methods, all while maintaining the same observation budget.
Learnable observation front ends can dramatically improve low-rate speech reconstruction by capturing richer semantic information than traditional sampling methods.
Sampling determines the form of information available to downstream reconstruction systems. Conventional lowrate sampling forms finite-dimensional observations directly from the raw waveform, with the sampling rule mainly guided by bandwidth, sparsity, or fixed signal-level structures. For acoustic signals such as speech, however, reconstruction-relevant information is often expressed through content-related spectral-temporal structures rather than waveform samples alone. This paper proposes semantic sampling via learnable observation front ends, where finite-dimensional observations are generated from learned signal responses instead of directly subsampled waveform points. The proposed front end consists of a semantic feature filterbank, a constrained semantic observation matrix, and a low-rate readout module. The filterbank maps the input waveform into multiple acoustic response channels, the observation matrix combines these responses into a small number of observation channels, and the readout module produces low-rate finite-dimensional samples. A reconstruction network is then used to recover the signal from the resulting observations. Experiments on low-rate speech reconstruction show that, under the same observation budget, the proposed semantic sampling front end provides more informative observations than fixed low-rate sampling and neural restoration methods based on predetermined low-rate waveforms. The improvements in waveform fidelity, spectral consistency, and perceptual quality show that learnable observation front ends preserve more useful information for acoustic signal reconstruction under the same observation budget.
