Search papers, labs, and topics across Lattice.
This paper introduces fabric_ext, an eBPF middleware compiler and runtime designed to enable extensible OS policies across GPU-CXL fabrics by executing a single policy program across various hardware hooks. The authors leverage a semantic movement graph that captures complex data movement semantics, which is then compiled into device-specific eBPF programs to optimize dataflow in heterogeneous computing environments. The results demonstrate that fabric_ext effectively manages the intricate data dependencies and transformations required for large language model (LLM) prefill tasks, showcasing its potential to enhance performance in complex GPU workloads.
fabric_ext revolutionizes dataflow management in GPU-CXL fabrics, optimizing performance for large language model tasks by executing extensible policies across diverse hardware hooks.
We present fabric_ext, an eBPF middleware compiler and runtime for extensible OS policies over GPU--CXL fabrics. fabric_ext lets one policy program execute across GPU hooks, driver/runtime hooks, DPU/NIC hooks, and CXL switch or near-memory hooks. The key abstraction is a semantic movement graph: edges describe bytes, stride, reuse distance, read/write ratio, source and destination, ordering requirement, alias set, ownership, and transformations such as Move, Quantize, Compress, Checksum, Filter, Reduce, Scatter/Gather, Replicate, and Persist. The compiler lowers this graph into per-device eBPF programs, verifier obligations, consistency-classed BPF maps, and artifacts for bpftime and dputime. At the fabric edge, fabric_ext treats a near-Type-2 small core as a hardware-JIT and state manager: it specializes verified movement descriptors into local copy, placement, ordering, and transformation commands, while the surrounding Von Neumann island of memory, DMA, and compute engines performs the dataflow. Because this dataflow is data-driven, fabric_ext also places observation beside the island, where queues, DMA completions, memory placement, and ownership transitions are visible as they happen. The canonical stress case is LLM prefill: attention streams KV blocks and reductions while FFN streams activations, weights, and compressible intermediates, forcing one request to cross GPU tensor execution, DPU/NIC event execution, and CXL or switch-local dataflow islands.
