Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We introduce Rondo, a stateful testing framework for deep learning compilers that addresses the challenge of latent bugs arising from cached compiler artifacts across invocations. By representing tests as model-invocation sequences and employing a novel three-way differential oracle, Rondo effectively identifies stateful compilation errors. Evaluation on four frameworks and 24 model architectures yielded 117 bugs, including 35 stateful ones, with a high alarm precision and significant reduction in test case lengths.
Deep learning compilers are silently introducing bugs through state retention, with 35 stateful issues found in just four frameworks.
Modern deep learning compilers compile models lazily and cache artifacts across invocations. Retained compiler state can make a later invocation incorrect even when it is correct in isolation. An analysis of 26 publicly reported issues in PyTorch, JAX, and TensorFlow shows that detecting such bugs requires sequences that first establish compiler state and then expose a failure. We present Rondo, a stateful testing method that represents each test case as a model paired with an invocation sequence. Rondo mutates invocation sequences and uses a Markov chain Monte Carlo sampler guided by compilation behavior to favor rarely observed behaviors. Its three-way differential oracle compares execution without compilation, compiled execution with compiler state reset, and compiled execution with state retained. It confirms stateful bugs by replaying the same invocation with state retained and reset while preserving inputs and program state, then shortens sequences while preserving the original bug. We evaluate Rondo on four deep learning frameworks using 24 common model architectures. Rondo detects 117 bugs, including 35 stateful bugs. We submit 39 bug reports, including 18 for stateful bugs; developers confirm 35 reports, including 16 for stateful bugs, and fix 18 bugs, including 11 stateful bugs. Rondo achieves the highest behavior coverage and compiler Python source-line coverage among the compared methods, with 99.26% alarm precision. It shortens 44 of 117 invocation sequences, reducing the lengths of these 44 sequences by 83.34% on average.