Search papers, labs, and topics across Lattice.
This paper conducts a mixed-methods analysis of llama.cpp, examining over 7,681 pull requests and various discussions to explore the dynamics of local AI inference infrastructure. The findings reveal that while local inference enhances user participation, it simultaneously shifts control to hardware vendors and core maintainers, effectively redistributing the costs of model execution. This research highlights the critical need for policy interventions that address the infrastructure supporting model deployment, rather than focusing solely on the models themselves.
Local AI inference may democratize access, but it also shifts power dynamics, placing control in the hands of hardware vendors and core maintainers.
Open AI scholarship has focused on model releases and cloud ecosystems, leaving the local inference infrastructure that makes open-weight models runnable on user-owned devices largely unexamined. We address this gap through a mixed-methods analysis of llama.cpp, combining 7,681 merged pull requests from March 2023 through March 2026 with repository discussions, corporate statements, and contributor blogs. We show that local inference broadens participation at execution while relocating capture into the infrastructure that makes execution possible. Through hardware backends, model integration labor, and Hugging Face's February 2026 absorption of the project, we document how control shifts to hardware vendors, model distributors, and core maintainers while model owners and individual contributors bear the cost of making models runnable. These dynamics suggest that preserving openness outside the cloud requires attention to the infrastructure that makes models runnable, not just to the models themselves. This calls for policy mechanisms---analysis of format dependencies and vendor influence, model compatibility requirements, and sustained public funding for inference tooling---that extend beyond model release conditions to the infrastructure layer.
