Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper tackles the limitations of in-context learning (ICL) in classifying complex, multi-party B2B conversations by introducing the Call Playbook dataset, which includes five classification tasks focused on core sales concepts. The authors propose innovative knowledge extraction methods that condense verbose examples into compact, interpretable representations, achieving a remarkable 99% reduction in token usage while enhancing macro-averaged AUC by up to 7%. Importantly, their approach maintains robustness as context length increases, addressing key challenges in transparency and efficiency for real-world NLP applications.
A novel framework reduces token usage by 99% while improving classification accuracy in complex B2B conversations, overcoming traditional ICL limitations.
In-context learning (ICL) is the standard method for low-resource classification, yet its efficacy in specialized domains remains largely unexplored. We address the challenge of classifying semantically complex, multi-party B2B conversations, where traditional ICL encounters significant limitations, especially as context length increases due to the concatenation of multiple few-shot examples. We introduce the Call Playbook dataset, featuring five classification tasks derived from real-world B2B conversations targeting core sales concepts. To bridge the gap between performance and practical utility, we propose novel knowledge extraction methods that distill verbose examples into compact, interpretable representations of structured classification criteria and precise task descriptions. Our approach achieves a 99\% reduction in token usage and improves macro-averaged AUC by up to 7\% over traditional ICL. Notably, it remains robust as context grows, unlike advanced token compression baselines which degrade by over 9 F1 points. Importantly, our framework enables direct refinement of classification logic, addressing critical needs for transparency, efficiency, and user interaction in real-world NLP applications.