Search papers, labs, and topics across Lattice.
This study addresses the inefficiencies of large language models (LLMs) in enterprise workflows, particularly focusing on automated expense itemization within Microsoft Dynamics 365. By evaluating four configurations of GPT-5, the authors demonstrate that selective context pruning and automated summarization can significantly enhance performance, achieving a completion rate of 91.6% with reduced token usage and runtime. The findings indicate that optimizing context management can lead to more reliable and efficient LLM agents in long-horizon tool-using tasks.
Selective context pruning combined with automated summarization can boost LLM performance to over 91% in complex enterprise tasks while slashing token usage and processing time.
Large language models deployed as autonomous agents for enterprise workflows face a key challenge: verbose tool responses from enterprise systems can cause context overflow, stale-state errors, and high inference cost. We study this problem in automated expense itemization in Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools. We evaluate four GPT-5 configurations on a 50-task hotel expense benchmark: no user model, full conversation history, context pruned to the last 5 tool call/response pairs, and pruning with automated summarization. Results are averaged across 5 independent runs, with the user model held constant for the context-engineering comparison. The no-user-model baseline achieves only 8.0% complete itemization. Full-context retention improves completion to 71.0%, but consumes 1,480,996 tokens and 14.56 hours per benchmark. Pruning to the last 5 tool calls improves completion to 79.0% while reducing token use to 535,274 and runtime to 5.39 hours. Adding summarization achieves the best result: 91.6% complete itemization and 99.64% average amount itemized, with 553,374 tokens and 5.79 hours. We further report confidence intervals, effect-size analysis, sensitivity over pruning and summary windows, failure analysis, results across five expense types grouped into three categories, and cross-model evidence with Claude Sonnet 4.5. These results show that, for this class of enterprise tool-use workflow, selective retention of recent tool interactions plus compact summarization can improve both reliability and efficiency compared with full-history retention.
Microsoft Research