Search papers, labs, and topics across Lattice.
This paper introduces an iterative definition refinement framework for zero-shot web content classification, addressing the sensitivity of embedding-based methods to definition quality. The framework uses LLMs to optimize category definitions through example-guided, confusion-aware, and history-aware refinement strategies, without updating model parameters. Experiments across 13 embedding models on a new 10-category, 10,000-sample URL benchmark show consistent performance gains, highlighting the importance of definition quality.
Zero-shot classification accuracy hinges more on the *definition* of a category than the model architecture itself.
Web filtering systems rely on accurate web content classification to block cyber threats, prevent data exfiltration, and ensure compliance. However, classification is increasingly difficult due to the dynamic and rapidly evolving nature of the modern web. Embedding-based zero-shot approaches map content and category descriptions into a shared semantic space, enabling label assignment without labeled training data, but remain highly sensitive to definition quality. Poorly specified or ambiguous definitions create semantic overlap in the embedding space, leading to systematic misclassification. In this paper, we propose a training-free, adaptive iterative definition refinement framework that improves zero-shot web content classification by progressively optimizing category definitions rather than updating model parameters. Using LLMs as feedback-driven definition optimizers, we investigate three refinement strategies namely example-guided, confusion-aware, and history-aware, each refining class descriptions using structured signals from misclassified instances. Furthermore, we introduce a human-labeled benchmark of 10 URL categories with 1,000 samples per class and evaluate across 13 state-of-the-art embedding foundation models. Results demonstrate that iterative definition refinement consistently improves classification performance across diverse architectures, establishing definition quality as a critical and underexplored factor in embedding-based systems. The dataset is available at https://github.com/naeemrehmat/B2MWT-10C.
