Search papers, labs, and topics across Lattice.
This study evaluates the ability of Large Language Models (LLMs), specifically GPT-3.5 and GPT-4, to obscure code stylometry while maintaining code functionality, which is crucial for authorship attribution and cybersecurity. By employing various prompt engineering strategies, the research reveals that multi-shot prompting significantly outperforms single-shot methods in obscuring stylistic signatures from a Random Forest classifier. The findings underscore the challenges of preserving code integrity post-modification, highlighting the evolving landscape of software development in the face of advanced AI capabilities.
Multi-shot prompting can drastically enhance the ability of LLMs to obscure code authorship, raising alarms for traditional stylometry techniques.
In the rapidly evolving field of software development, code stylometry analyzing unique stylistic signatures of programmers plays a crit-ical role in authorship attribution and cybersecurity. Recent advancements in artificial intelligence, particularly Large Language Models (LLMs) like GPT-3.5 and GPT-4, have introduced new dimensions to this field, challenging traditional stylometry techniques. This study investigates the effectiveness of LLMs in altering code stylometry while preserving functionality and evaluates the impact of various prompt engineering strategies. Through comprehensive experiments, we assess how well these models can obscure stylistic signatures to avoid detection by a Random Forest classifier trained for authorship attribution. The results reveal significant differences in effectiveness between single-shot and multi-shot methods and highlight the importance of detailed, structured prompts. Additionally, functionality preservation checks demonstrate the challenges in maintaining code integrity post-modification. This research provides critical insights into the robustness of authorship attribution techniques against advanced AI capabilities, informing future cybersecurity and software engineering developments
