Search papers, labs, and topics across Lattice.
This study investigates whether large language models (LLMs) can identify and recover semantic optimization opportunities that traditional compilers overlook due to missing enabling semantics in program representations. Utilizing SeGaBench, a benchmark with 120 cases designed to test LLMs on their ability to generate validated, contract-preserving artifacts, the authors evaluate five LLMs and find that the strongest model produces correct artifacts in 94.8% of responses and achieves performance improvements in 83.3% of cases. These findings indicate that LLMs can serve as valuable tools for enhancing compiler analysis by proposing speculative semantics, although they may not fully close the performance gap with oracle artifacts.
LLMs can achieve a staggering 94.8% correctness in generating compiler optimizations that traditional methods miss, revealing a new frontier in program analysis.
Optimizing compilers miss profitable transformations when their enabling semantics are absent from the analyzed program representation. We ask whether large language models (LLMs) can recover such semantics from heterogeneous C/C++ context and realize them as validated, contract-preserving artifacts. We introduce SeGaBench, an executable benchmark containing 100 synthetic and 20 source-backed cases spanning low-level assumptions, data-structure invariants, and high-level semantic lifting. Each case includes hidden enabling semantics, an oracle artifact, correctness and semantic validators, and a reproducible performance protocol. We evaluate five LLMs using five independent responses per case. The strongest model produces correct artifacts in 94.8% of responses, achieves at least 1.05x speedup in 83.3%, and obtains a performance success on 93.3% of cases. Nevertheless, correct artifacts often close only part of the oracle gap. These results show that LLMs can complement compiler analysis as speculative semantic proposers, provided that their artifacts are validated and evaluated.
