Franz Nowak

Finite precision in transformers can drastically alter memory operations, revealing a surprising hierarchy of expressivity based on attention mechanisms.

Recurrent neural networks can exhibit drastically different expressivity based on the arithmetic model used, revealing hidden capabilities tied to quantization methods.