Yeachan Park

Floating-point quirks can actually limit what your neural network can compute, with some common activation function implementations provably incapable of representing all possible functions.

Grokking isn't magic: it's all about neural nets learning to exploit the hidden symmetries baked into algorithmic tasks.