Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper benchmarks four instruction-tuned LLMs (Claude 3.7 Sonnet, Claude 4 Sonnet, Mixtral 8x7B Instruct, and Mistral 7B Instruct) on receipt-item categorization using AWS Bedrock. The study evaluates accuracy, response stability, and token-level cost under zero-shot and few-shot prompting. Claude 3.7 Sonnet emerges as the most cost-effective model for this task, offering the best trade-off between accuracy and cost.
Claude 3.7 Sonnet punches above its weight, delivering the best accuracy-to-cost ratio for receipt categorization compared to larger models on AWS Bedrock.
This paper presents a systematic, cost-aware evaluation of large language models (LLMs) for receipt-item categorisation within a production-oriented classification framework. We compare four instruction-tuned models available through AWS Bedrock: Claude 3.7 Sonnet, Claude 4 Sonnet, Mixtral 8x7B Instruct, and Mistral 7B Instruct. The aim of the study was (1) to assess performance across accuracy, response stability, and token-level cost, and (2) to investigate what prompting methods, zero-shot or few-shot, are especially appropriate both in terms of accuracy and in terms of incurred costs. Results of our experiments demonstrated that Claude 3.7 Sonnet achieves the most favourable balance between classification accuracy and cost efficiency.