Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Forge-UGC, a novel four-phase compiler, optimizes transformer deployment on heterogeneous hardware by addressing limitations in existing frameworks like OpenVINO and ONNX Runtime through improved graph optimization and buffer management. The compiler captures graphs at the ATen operator level, applies six optimization passes to reduce graph complexity, lowers the graph into a typed intermediate representation, and performs buffer allocation and device scheduling. Experimental results across various model families demonstrate that Forge-UGC achieves 6.9 to 9.2x faster compilation, 18.2 to 35.7% lower inference latency, and 30.2 to 40.9% lower energy per inference compared to existing frameworks.
Transformer deployment just got a whole lot faster: Forge-UGC slashes compilation time by up to 9x and inference latency by 35% compared to OpenVINO and ONNX Runtime.
We present Forge-UGC (FX Optimization and Register-Graph Engine for Universal Graph Compilation), a four-phase compiler for transformer deployment on heterogeneous accelerator hardware, validated on Intel AI Boost NPU. Existing frameworks such as OpenVINO and ONNX Runtime often use opaque compilation pipelines, limited pass-level visibility, and weak buffer management, which can lead to higher compilation cost and runtime overhead. Forge-UGC addresses this with a hardware-agnostic design that separates graph capture, optimization, intermediate representation lowering, and backend scheduling. Phase 1 captures graphs with torch.export at the ATen operator level, supporting modern transformer components such as rotary position embeddings, grouped-query attention, and SwiGLU without manual decomposition. Phase 2 applies six optimization passes: dead code elimination, common subexpression elimination, constant folding, attention fusion, operator fusion, and layout optimization, reducing graph node count by 14.2 to 21.9%. Phase 3 lowers the optimized graph into a typed intermediate representation with explicit virtual register assignments. Phase 4 performs liveness analysis, linear-scan buffer allocation, reducing peak buffer count by 30 to 48%, and device-affinity scheduling, reducing NPU-CPU transitions by 42 to 65%. Across six model families ranging from 125M to 8B parameters, evaluated on WikiText-103 and GLUE, Forge-UGC delivers 6.9 to 9.2x faster compilation than OpenVINO and ONNX Runtime, 18.2 to 35.7% lower inference latency, and 30.2 to 40.9% lower energy per inference. Fidelity is preserved, with max absolute logit differences below 2.1e-5 and KL divergence below 8.4e-9. We also introduce Fusion Gain Ratio, Compilation Efficiency Index, and per-pass execution profiling for systematic evaluation of NPU compilation pipelines.