Search papers, labs, and topics across Lattice.
This study investigates the impact of positional encoding on memristor-based analog computation for automatic speech recognition, revealing that large output values significantly degrade performance during analog-to-digital conversion (ADC). By optimizing the weight and precision bits of the ADC in specific memristor layers, the authors achieve a ~50% reduction in execution degradation while maintaining stable energy consumption. Furthermore, they explore alternative strategies that can mitigate degradation by ~30% when ADC modifications are not feasible, enhancing the robustness of analog computations in NLP tasks.
Large output values from positional encodings can severely compromise the performance of memristor-based computations, but targeted adjustments can halve this degradation.
Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.
