Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Test-Oriented Programming (TOP), a paradigm where developers primarily focus on verifying LLM-generated test code derived from natural language specifications, rather than directly writing production code. The authors implemented a proof-of-concept TOP tool and used it to generate a command-line program, demonstrating the potential of this approach. Results show promise for TOP, while also highlighting challenges in scaling it for real-world software projects.
Imagine a world where you only write tests and LLMs handle the rest: this paper explores that tantalizing possibility.
Large language models (LLMs) have shown astonishing capability of generating software code, leading to its use to support developers in programming. Proposed tools have relied either on assistants for improved auto-complete or multi-agents, in which different model instances are orchestrated to perform parts of a problem to reach a complete solution. We argue that LLMs can enable a higher-level of abstraction, a new paradigm we called Test-Oriented Programming (TOP). Within this paradigm, developers only have to check test code generated based on natural language specifications, rather than focusing on production code, which could be delegated to the LLMs. To evaluate the feasibility of this proposal, we developed a proof-of-concept tool and used it to generate a small command-line program employing two different LLMs. We obtained promising results and identified challenges for the use of this paradigm for real projects.