Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Spec2Vision, a framework designed to generate and evaluate computer vision (CV) pipelines that adhere to explicit task contracts throughout their lifecycle. By conducting a comprehensive benchmark across 17 CV tasks, the authors demonstrate that maintaining an explicit task contract significantly improves the success rate of evaluator-test passes, achieving 81 out of 85 passes compared to lower rates when structural repairs and compatibility scaffolding are removed. The findings highlight the critical distinction between runnability and successful delivery, emphasizing that merely generating runnable code does not guarantee it meets the specified requirements.
Spec2Vision reveals that explicit task contracts during CV pipeline generation can boost evaluator-test success rates from 39% to 95%, underscoring the importance of specification adherence in AI-generated code.
Generated computer-vision code can be runnable without satisfying the task contract enforced by a downstream evaluator. We study that gap with Spec2Vision, an experimental framework for producing and evaluating specification-grounded CV pipeline bundles through a staged runtime that keeps the task contract explicit across synthesis, screening, testing, and bounded repair. The benchmark evaluates 17 CV tasks, 10 executable conditions, and 5 repeats per task-condition cell, for 850 primary runs. In the primary 850-run evaluation, Spec2Vision reaches 81/85 evaluator-test passes; removing structural repair drops to 55/85, compatibility scaffolding to 58/85, and generator preflight to 39/85. The executable single-agent baselines expose progressively richer task specifications to the model, culminating in direct source-spec exposure, yet remain much weaker overall, from 17/85 for lightweight task grounding to 35/85 evaluator-test passes. The lightweight baseline nevertheless remains core-runnable in 85/85 runs but reaches only 17/85 evaluator-test passes and 6/85 strict-delivery successes, showing that runnability is not equivalent to delivery. Across this benchmark, the strongest evidence comes from keeping the task contract explicit across staged generation, checking, and repair. Artifacts are provided to support audit of run bundles, model-visible inputs, and derived tables.