Search papers, labs, and topics across Lattice.
This study analyzes the signaling value of credentials in the context of generative AI by auditing Kaggle competition data from 2010 to 2026. It finds that while competition medals initially predict subsequent performance well, their informativeness significantly declines over time, particularly for upload-competitions, which lost 82% of their signaling value due to institutional stranding. The findings highlight the need for adaptive credentialing systems that account for the rapid evolution of skills in the AI era, as older credentials may misrepresent an individual's current capabilities.
Credentialing systems risk becoming obsolete as generative AI alters the landscape of skill assessment, with old competition medals losing 82% of their predictive power.
Generative AI can now perform many tasks that credentialing institutions count on to assess skill. During the AI era, do credentials retain their signaling value for subsequent performance? Mostly, yes. We audit the 2010-2026 archive of Kaggle, the largest data science competition platform, which ran two evaluation formats concurrently: upload-competitions, which directly score entrants' predictions computed on published data, and code-competitions, which score predictions by executing entrants' code on hidden data. Across 444,698 participations, competition medals predict subsequent leaderboard performance almost entirely in the first year after being earned, in both formats. Fresh medals retained most of their signaling value through the AI transition; credential stocks are only as informative as their replenishment. Although upload-competition medal stocks lost 82% of their informativeness, institutional stranding explains half to three quarters of the loss: upload-competitions had exited for reasons predating AI, and their frozen medal stock aged out under the pre-existing decay pattern. Old upload-competition medals look more valuable only in isolation, by proxying for the rest of the holder's record (e.g., experience). The measured changes are institutional rather than personal: an AI-like working style predicts performance similarly in both formats. The platform's official credential tiers, based on lifetime medal counts, discard 13-16% of the medals' information; an index weighting recent medals more heavily, built on pre-AI-era data alone, outperforms the official tiers in predicting AI-era performance. In conclusion, credentials are informative, perishable, institution-bound, and interdependent; sustaining their value under AI is a high-stakes, socio-economic problem of institutional design.
