Search papers, labs, and topics across Lattice.
This paper addresses the security threat of slopsquatting, where adversaries exploit fabricated Python package names generated by coding language models to compromise supply chains. The authors propose a two-layer detection system combining a deterministic PyPI existence check with a Random Forest classifier that analyzes ten features from package names and metadata. Their approach achieves a 76% success rate in generating hallucination-free code across 300 prompts, highlighting the effectiveness of model retries and fallback mechanisms in mitigating risks associated with hallucinated package names.
Slopsquatting risks are significantly mitigated with a two-layer detection system that achieves 76% hallucination-free code generation, even in adversarial contexts.
When a code generating language model fabricates a Python package name, an adversary who has pre-registered that name on PyPI can convert that hallucination into a supply chain compromise. This event has been termed as'slopsquatting'. We propose a two layer detector to counter this issue. The first layer performs a deterministic PyPI existence check. The second is a Random Forest classifier trained on ten features derived from the package name and its PyPI metadata. An import name reconciler bridges the two, resolving cases such as'import cv2'versus'pip install opencv-python'without a security bypass. The detector is embedded in a LangGraph state machine that retries at escalating temperatures and, on repeated failure, routes to a stronger fallback model. Across 300 curated prompts, the pipeline produces hallucination free code on 76% of runs. The primary exhausts its retry budget on 28.7%; intra model retries recover roughly a quarter of those, and cross model fallback recovers a further 16.5% of the remainder. Four findings have been observed. First, half of the flagged hallucinations are packages already registered on PyPI, as low quality lookalikes of well known projects, caught by the classifier rather than the deterministic layer (e.g., pil, faiss, tabula, haystack). Second, hallucination rate scales almost linearly with prompt adversariality, from 0 to 10% on routine coding to 40 to 73% on slopsquat baits. Third, the weaker primary refused 6 of 10 direct baits unaided, suggesting recent instruction tuning provides a baseline defense. Fourth, when primary and fallback share a model family, approximately 84% of primary failures recur on the fallback, motivating cross family pairing. A user study (n = 24) reports mean satisfaction 4.4 out of 5 and 21 of 24 stated adoption intent.
