Search papers, labs, and topics across Lattice.
This study introduces the Graph-Retrieval Automated Scoring Pipeline (GRASP) to address the challenge of grading label-free multi-topic science essays, where student responses to multiple questions are combined without explicit segmentation. By encoding reference answers into a FAISS vector index and constructing a semantic similarity graph, GRASP employs a combination of sentence count heuristics and a large language model to identify distinct topics in student essays. The results demonstrate that GRAG, which enhances retrieval through graph traversal, significantly improves grading accuracy compared to traditional cosine similarity methods, particularly in complex essays.
Grading multi-topic essays without explicit labels can achieve unprecedented accuracy through graph-augmented retrieval techniques.
Automated short-answer grading research has historically focused on exams consisting solely of questions pertaining to a single topic. Automatic grading of exams containing questions about more than one topic remains less explored. In this work, a Graph-Retrieval Automated Scoring Pipeline (GRASP) is introduced for grading label-free multi-topic science exams. Label-free exams are short-answer exams in which a student's responses to several distinct topics are merged into a single paragraph, with no markup labels or segmentation indicating which span answers which question. Reference answers for each question are encoded into a FAISS vector index via Sentence-BERT, and a semantic similarity graph is constructed over this set of reference answers. At grading time, sentence count heuristics, with a large language model used to resolve ambiguous cases, are first applied to predict how many distinct topics were answered in the student essay. This process is performed without training data or domain-specific example essays. Candidate reference nodes, each storing one (question, reference answer, concatenation of both) from the reference index, are then retrieved through cosine similarity based Retrieval-Augmented Generation (RAG) and Graph Retrieval-Augmented Generation (GRAG). GRAG operates by taking the top cosine matches as seed nodes and then performing a graph traversal over strong edges to find additional reference nodes that may have been missed by RAG. The Hungarian algorithm is then used to optimally assign one reference node per question segment such that no reference is duplicated. Each segment is then graded against its assigned reference independently using GPT-4.1-mini. This experiment is performed to show the effect of retrieval quality on grading accuracy and the benefit of graph-augmented retrieval versus strict cosine similarity methods at various levels of essay complexity.
