Search papers, labs, and topics across Lattice.
The paper introduces SRICL, a framework for extracting job skills from text using LLMs, addressing issues like malformed spans and hallucinations. SRICL combines semantic retrieval of relevant examples, in-context learning, and supervised fine-tuning, along with a deterministic verifier to ensure output validity. Experiments on six datasets show SRICL significantly improves STRICT-F1 scores compared to GPT-3.5, while also reducing invalid tags and hallucinations.
LLMs can now reliably extract job skills from text, even in low-resource settings, thanks to a novel framework that enforces output validity and reduces hallucinations.
Span-level skill extraction from job advertisements underpins candidate-job matching and labor-market analytics, yet generative large language models (LLMs) often yield malformed spans, boundary drift, and hallucinations, especially with long-tail terms and cross-domain shift. We present SRICL, an LLM-centric framework that combines semantic retrieval (SR), in-context learning (ICL), and supervised fine-tuning (SFT) with a deterministic verifier. SR pulls in-domain annotated sentences and definitions from ESCO to form format-constrained prompts that stabilize boundaries and handle coordination. SFT aligns output behavior, while the verifier enforces pairing, non-overlap, and BIO legality with minimal retries. On six public span-labeled corpora of job-ad sentences across sectors and languages, SRICL achieves substantial STRICT-F1 improvements over GPT-3.5 prompting baselines and sharply reduces invalid tags and hallucinated spans, enabling dependable sentence-level deployment in low-resource, multi-domain settings.
