Search papers, labs, and topics across Lattice.
This study investigates the resilience of Qiskit homework assignments against generative AI solutions by testing three distinct quantum programming packages with ChatGPT. The findings reveal that all tested instances were successfully completed by ChatGPT without requiring any modifications to the quantum logic, indicating a lack of robustness in the assessment design. The results suggest that while generative AI can produce correct outputs, it does not ensure that students engage deeply with the material or demonstrate understanding, highlighting the need for supplementary assessment methods.
ChatGPT effortlessly solved all tested Qiskit homework assignments, revealing critical vulnerabilities in quantum software education assessments.
Generative AI creates an assessment challenge in quantum software education: a student can provide a homework notebook to ChatGPT and request a completed submission. This study examined whether introductory Qiskit homework could remain autogradable while requiring students to run, review, and discuss results rather than banning AI. Three packages were tested: seeded basis-state circuits with bit flips and customized measurement mappings; Quantum Fourier Transform followed by inverse-transform recovery; and seeded Deutsch-Jozsa with customized oracle masks. The designs used personalization, simulator execution, JSON submissions, hidden references, circuit metrics, reflections, and optional IBM Quantum execution. For each package, one student-visible instance was tested in 50 separate ChatGPT sessions, yielding 150 sessions overall. Every final artifact was executed and passed its grader. Nine sessions were fully archived; none required operator code changes or correction of quantum logic. Under the study's operational definition, each tested instance had zero observed ChatGPT-resiliency. Seeds changed parameters rather than task structure, expected results remained derivable from visible assignment logic, scaffolding exposed key solution steps, and hidden grading verified output consistency without establishing independent authorship or understanding. Because one instance was repeated for each package, the results do not establish solvability for every seed or possible Qiskit assessment. The tested personalized, execution-oriented take-home designs therefore did not prevent successful completion under a minimally engaged-student workflow. Correct artifacts should be complemented by direct assessment through supervised modification, oral defense, prediction, and transfer tasks.
