Search papers, labs, and topics across Lattice.
This paper introduces HexEval, a novel hexagonal framework designed for multidimensional scholar assessment that integrates both intrinsic research quality and externally verifiable scholarly behavior. By organizing assessment into two evidence layers鈥攊ntrinsic and external鈥擧exEval evaluates scholars on multiple dimensions, including research rigor and academic impact, using diverse data sources like GitHub and OpenAlex. Experimental results demonstrate that HexEval achieves dimension-dependent agreement with human evaluations, highlighting its potential for creating interpretable and auditable scholar profiles while addressing the limitations of existing bibliometric methods.
HexEval reveals that a multidimensional approach to scholar assessment can significantly enhance transparency and accuracy by integrating diverse evidence sources.
Scholar assessment plays a fundamental role in faculty recruitment, funding allocation, academic promotion, and talent discovery. Existing scholar assessment methods predominantly rely on bibliometric indicators and reputation proxies, while recent large language model (LLM)-based approaches mainly focus on evaluating individual research papers rather than comprehensively assessing scholars. We argue that scholar assessment should be formulated as an evidence-driven reasoning problem that jointly considers intrinsic research quality and externally verifiable scholarly behavior. To this end, we propose HexEval, an evidence-driven hexagonal framework for multidimensional scholar assessment. HexEval explicitly organizes scholar assessment into two complementary evidence layers. The intrinsic layer evaluates anonymized representative works along three dimensions, namely research rigor, methodological innovation, and scientific contribution, whereas the external layer characterizes scholars through knowledge translation, research coherence, and academic impact using heterogeneous evidence collected from GitHub, Lens, OpenAlex, and other publicly verifiable sources. Instead of producing opaque aggregate scores, HexEval preserves intermediate evidence, dimension-specific rationales, and verification signals throughout the evaluation process, enabling interpretable and auditable scholar profiles. Experiments across all six dimensions show dimension-dependent agreement with human or external reference criteria: structured calibration improves absolute agreement for intrinsic quality, while the external modules recover broad trajectory and ordinal impact signals. These results support evidence-driven reasoning over heterogeneous scholarly evidence as a promising paradigm for auditable AI-assisted scholar assessment, while exposing the coverage and attribution limitations of public scholarly data.
