Search papers, labs, and topics across Lattice.
This paper introduces a scalable pipeline for labeling large text corpora using LLM teachers, addressing the challenges of label quality and GPU worker efficiency. The proposed work-stealing job scheduling and memory-aware concurrency approach significantly enhances throughput, achieving up to 3.4 times the efficiency of static sharding under skewed workloads while maintaining task integrity even in failure scenarios. Additionally, the methodology provides a framework for evaluating label quality through a relabeling benchmark that measures agreement against gold labels, yielding actionable insights on cost and performance.
Achieving 3.4x throughput over static sharding under skewed loads while ensuring zero task loss during worker failures could revolutionize large-scale data labeling.
Labeling large text corpora with LLM teachers has become a practical route to training data at scale. At millions of items, hand-labeling every batch is not feasible, and two questions dominate: what label quality a teacher buys per dollar, and how to keep a fleet of GPU workers busy under skewed, failure-prone workloads. We present a simple, reproducible pipeline that addresses both. First, a work-stealing ring pool: each worker owns a queue, drains it first, and then steals from ring successors, with exactly-once task claims via atomic conditional writes and crash tolerance via stale-claim sweeping. The claim protocol requires only a compare-and-set primitive from its storage layer; we implement it on a single SQLite file, which makes the reference implementation dependency-free and the experiments reproducible on one machine. Second, a memory-aware concurrency rule that sizes per-node parallelism by how many model copies fit on the GPU, so the same code runs safely across device sizes. Third, a relabeling benchmark methodology in which the teacher relabels a public dataset that already has gold labels, so quality reduces to an agreement measurement and cost follows from measured throughput. Under skewed load the pool sustains up to 3.4 times the throughput of static sharding while matching it at zero skew, loses 0 of 2,000 tasks when half the workers are killed mid-run (static sharding loses 953), and yields measured quality and cost points for an instruction-tuned teacher on irony and sentiment tasks. All experiments run on public data and commodity hardware; code, tests, and run logs are released.
