Search papers, labs, and topics across Lattice.
This paper introduces emb-diversity, a comprehensive tool designed to measure data diversity using embedding-based approaches, addressing the inconsistency in existing diversity measurement methods. By leveraging the flexibility of embedding models, emb-diversity can quantify various notions of diversity, including stylistic, semantic, language, and speaker diversity. The tool's applicability across different datasets and embedding models highlights its potential to enhance the fairness and robustness of NLP models through improved diversity assessment.
A unified tool for measuring data diversity reveals that existing methods are fragmented and inadequate for modern NLP needs.
There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools for measuring the lexical diversity of texts, researchers lack standardized tools for quantifying diversity based on embeddings. Embedding-based diversity measures are highly flexible: They work with any embedding model and any data that can be embedded, and are thus applicable to many notions of diversity. With emb-diversity, we provide a comprehensive embedding-based diversity measurement tool, spanning a broad range of measures. We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets. https://github.com/nlpsoc/emb-diversity/
