Search papers, labs, and topics across Lattice.
This paper introduces cascaded batch prompting, a two-stage method that enhances the efficiency of large language model inference by separating complex reasoning from symbol grounding. By applying this approach to multiple-choice question answering and natural language inference tasks, the authors show significant improvements in performance compared to traditional single prompting methods. The results indicate that cascaded batch prompting not only improves accuracy but also achieves speedups that scale with batch size, setting a new benchmark in the field.
Cascaded batch prompting resolves the unpredictability of conventional batch prompting, achieving both superior performance and efficiency in large language models.
Although batch prompting makes large language model inference more efficient by processing multiple instances simultaneously, it suffers from unpredictable downstream task performance. We propose cascaded batch prompting, a two-stage approach designed to resolve the unpredictability of conventional batch prompting by disentangling complex reasoning from symbol grounding. Experiments on multiple-choice question answering and natural language inference demonstrate that the proposed method outperforms the standard single prompting baseline while achieving a speedup proportional to batch size, establishing a new state of the art on the Pareto frontier.
