A Machine Learning and Large Language Model-Integrated Approach to Research Project Evaluation
Author:
Affiliation:
1. City University of Hong Kong, Hong Kong
2. National Natural Science Foundation of China, China
3. InnoCity.com Ltd., Hong Kong
Abstract
Research project evaluation upon completion is one of the important tasks for research management in government funding agencies and research institutions. Due to the increased number of funded projects, it is hard to find qualified reviewers in the same research disciplines. This paper proposes a machine learning and large language model integrated approach to provide decision support for research project evaluation. Machine learning algorithms are proposed to compute the weights of key performance indicators (KPIs) and scores of KPIs based on the evaluation results of completed projects, large language models are used to summarize research contributions or findings on project reports. Then domain experts are invited to consolidate the weights and scores for the KPIs and assess the novelty and impact of research contribution or findings. Experiments have been conducted in practical settings and the results have shown that the proposed method can greatly improve research management efficiency and provide more consistent evaluation results on funded research projects.
Publisher
IGI Global
Reference35 articles.
1. AlYahmady, H. H., & Alabri, S. S. (2013). Using NVivo for data analysis in qualitative research. International Interdisciplinary Journal of Education, 2(2), 181–186. https://www.semanticscholar.org/paper/Using-Nvivo-for-Data-Analysis-in-Qualitative-AlYahmady-Alabri/6a007b8b30daa0c55c775482792a9e6b019b1f9d
2. Uncertainty management in electricity demand forecasting with machine learning and ensemble learning: Case studies of COVID-19 in the US metropolitans
3. The Influence of Peer Review on the Research Assessment Exercise
4. Latent Dirichlet allocation.;D. M.Blei;Journal of Machine Learning Research,2003
5. Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgments of experts?;L.Bornmann;Journal of Informetrics,2014
1.学者识别学者识别
2.学术分析学术分析
3.人才评估人才评估
"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370
www.globalauthorid.com
TOP
Copyright © 2019-2024 北京同舟云网络信息技术有限公司 京公网安备11010802033243号 京ICP备18003416号-3