Search papers, labs, and topics across Lattice.
This paper addresses critical deployment issues in the Nanbeige4.2-3B model, a 3B-parameter agentic model utilizing a Looped Transformer architecture, by identifying and fixing five significant bugs that hinder its functionality on Apple Silicon. Despite these fixes, the model's layer-reuse strategy results in excessive peak attention memory usage, prompting the introduction of a chunked-prefill strategy that increases context width by 2.7 times. The final evaluation shows that the patched model successfully completes 30% of agentic tasks, a substantial improvement from its original performance of 0%, while also demonstrating near-perfect results on single tool calls in benchmark tests.
Fixing deployment bugs in Nanbeige4.2-3B transforms it from a non-functional model to one that can tackle real agentic tasks with a significant performance boost.
Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding effective depth without additional parameters. Evaluated on Apple Silicon (MPS), we identify five independent bugs which prevent the released checkpoint from running via Hugging Face transformers out of the box (including a silently-zeroed RoPE buffer and calls to removed transformers cache APIs). Furthermore, we show that fixing these bugs is still not sufficient for agentic tasks, due to the LT's layer-reuse strategy (which effectively doubles peak attention memory) used to achieve parameter efficiency. We thus introduce a chunked-prefill strategy which alleviates the incurred memory-capacity penalty, extending allowable context width by 2.7 times on 32~GiB shared memory. However, even with the reduced memory overhead, we show that patches are required to render Nanbeige4.2-3B usable; resolving both system prompt and MPS-native memory bugs finally allows reliable evaluation on standard MCP and tool-calling benchmarks. On a subset of MCPMark, the debugged model completes up to 30\% of real agentic tasks (up from the original's 0\%), while, on BFCL, it is near-perfect at single tool calls (yet fails the majority of multi-tool tests). We release the patched checkpoint, system prompt optimizer, and evaluation harnesses at https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix.
