Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To address reliability and reproducibility failures in enterprise data analytics, this work compares unconstrained runtime tool generation against a governed architecture where language models are restricted to intent interpretation while deterministic policy executes pre-approved analytical programs. Across 440 empirical runs on analytical workloads involving relational operations, aggregation, and ranking, runtime-planning 8B models failed completely (0/330) to fulfill a strict answer-and-evidence verification contract. In contrast, isolating Qwen3-8B to intent parsing while delegating execution to deterministic policy achieved a 100% (110/110) contract match with full result replayability.
Runtime tool-use agents failed 330 out of 330 enterprise analytics tasks, but stripping the LLM down to pure intent parsing paired with deterministic execution hit a perfect 110/110.
We study a governed approach to enterprise analytics: a language model interprets the question, while deterministic policy selects and runs a pre-approved analytical program that returns both results and evidence. We show that this restriction can remain expressive within a defined analytical class, using relational operations plus aggregation, comparison, windows, ranking, and similarity. Fixed meaning, policy, data, and execution rules also make results replayable. Across 440 runs, three 8B models generated SQL and selected tools at runtime, while Qwen3-8B interpreted intent only and policy executed the approved program. None of 330 runtime-planning episodes matched the full answer-and-evidence contract across all test datasets; the policy-executed analyzer matched 110 of 110. This is a configuration-specific result, not evidence that runtime agents cannot succeed under other designs.