Search papers, labs, and topics across Lattice.
This paper introduces SynH-Rank, a novel framework for quality-aware code search that integrates LLM-driven diverse data synthesis with hierarchical ranking training to address the limitations of existing rerankers that focus solely on semantic relevance. By implementing a three-level labeling scheme that prioritizes high-quality relevant code over low-quality and irrelevant code, SynH-Rank effectively captures critical non-functional qualities such as execution speed and maintainability. Experimental results demonstrate that SynH-Rank achieves a 20.15% improvement in Quality Preference Accuracy and a 15.80% enhancement over traditional relevance-only contrastive training, while also improving generalizability under complex constraints.
Quality-aware code search can significantly boost developer productivity by prioritizing high-quality code that meets specific resource optimization needs.
Code search enhances developer productivity by enabling efficient code reuse. Current code search systems often use a retrieve-then-rerank pipeline, where rerankers focus on modeling semantic relevance between queries and code. However, these rerankers overlook critical non-functional qualities like execution speed, memory usage, and maintainability, which are essential for practical software development. Studies reveal developers expect results to maintain high coding standards and satisfy specific needs, such as resource optimization, highlighting the importance of quality-aware code search. Achieving quality-aware code search faces two major challenges: the scarcity of quality-annotated datasets for effective training and the limitations of standard contrastive learning objectives, which fail to capture the ordinal relationships among high-quality, low-quality, and irrelevant code. Although contrastive learning excels in distinguishing relevant from irrelevant code, its binary objective does not support nuanced quality distinctions.To address these challenges, we propose SynH-Rank, a quality-aware code reranking framework that combines LLM-driven diverse data synthesis with hierarchical ranking training. SynH-Rank employs a three-level labeling scheme to explicitly model the hierarchy: high-quality relevant > low-quality relevant > irrelevant. Additionally, we introduce a new benchmark with 4,209 pairs and two novel metrics: Quality Preference Accuracy (QPA) for assessing prioritization of high-quality code and Multi-Condition Accuracy (MCA) for evaluating performance under complex constraints.Experimental results show SynH-Rank improves QPA by 20.15\% over backbone models and outperforms standard relevance-only contrastive training by 15.80\%, while simultaneously enhancing traditional relevance metrics and multi-condition generalizability.
