Search papers, labs, and topics across Lattice.
This study evaluates six information retrieval methods for academic advisor discovery, focusing on ranking CS faculty by relevance to graduate applicants' research interests. Using a dataset of 768 faculty profiles and 162 graded relevance judgments, the Reranked method achieved the highest mean NDCG@10 score, outperforming other approaches such as Semantic and Hybrid retrieval. Notably, the analysis revealed that using biography alone yielded better results than the full model, highlighting the importance of specific data features in retrieval effectiveness.
Reranked retrieval methods significantly outperform traditional approaches in matching graduate applicants with relevant academic advisors, revealing critical insights into effective faculty profiling.
We present a comparative evaluation of six information retrieval methods for the task of academic advisor discovery: ranking CS faculty members by relevance to a graduate applicant's research interest statement. The methods span sparse lexical matching (Jaccard overlap, TF-IDF, BM25), dense semantic retrieval (all-MiniLM-L6-v2 sentence embeddings), hybrid score fusion, and learning-to-rank. Evaluation uses a new domain-specific collection: 768 faculty profiles scraped from 9 US CS departments, with 162 graded relevance judgments (grade 0/1/2) across 5 queries representing distinct graduate student research profiles. Across all five queries, Reranked achieves the highest mean NDCG@10 (0.477, std 0.138), followed by Semantic (0.450), Hybrid (0.421), BM25 (0.406), Jaccard (0.303), and TF-IDF (0.246). After Bonferroni correction across all 15 pairwise comparisons, TF-IDF is significantly worse than BM25, Semantic, Hybrid, and Reranked; no other pairwise difference survives correction at 5 queries. A field ablation reveals that biography alone (NDCG 0.634) outperforms the full model combining biography with research area tags (0.593). A controlled experiment shows that concatenating arXiv paper abstracts reduces NDCG@10 by 0.176, motivating a late-fusion architecture. All code, scrapers, and relevance labels are released openly.
