Search papers, labs, and topics across Lattice.
This paper introduces proFILL, a novel method that enhances the hoBIT chatbot by enabling it to become a profile-aware retrieval-augmented generation (RAG) system for university academic advising. By progressively acquiring only the necessary profile attributes based on user queries and retrieved evidence, proFILL improves the relevance of responses while avoiding the pitfalls of profile-blind retrieval. Experimental results demonstrate that proFILL significantly outperforms existing RAG baselines and is favored by users, offering a cost-effective solution for on-premise deployment.
Tailoring academic advising responses to individual student profiles boosts relevance and user satisfaction, outperforming traditional methods.
In university academic advising, identical questions can require different answers depending on a student's department, admission cohort, and degree program, causing profile-blind retrievers to surface plausible but inapplicable evidence. We present proFILL, a method for transforming hoBIT, our college's current rule-based advising chatbot, into a profile-aware retrieval-augmented generation (RAG) system. Rather than requiring a complete user profile upfront, proFILL progressively acquires only the profile attributes needed for each query, guided by both the query intent and the initially retrieved evidence, and uses them to condition retrieval over a profile-aware index. Extensive experiments and a human preference study show that proFILL outperforms diverse RAG baselines, is preferred by target users, and remains effective with open-weight models for cost-effective on-premise deployment.
