What is in the KGQA Benchmark Datasets? Survey on Challenges in Datasets for Question Answering on Knowledge Graphs-Reference-Cited by-同舟云学术

What is in the KGQA Benchmark Datasets? Survey on Challenges in Datasets for Question Answering on Knowledge Graphs

Published:2021-06-01 Issue:3-4 Volume:10 Page:241-265
ISSN:1861-2032
Container-title:Journal on Data Semantics
language:en
Short-container-title:J Data Semant

Author:

Steinmetz Nadine^ORCID,Sattler Kai-Uwe

Abstract

AbstractQuestion Answering based on Knowledge Graphs (KGQA) still faces difficult challenges when transforming natural language (NL) to SPARQL queries. Simple questions only referring to one triple are answerable by most QA systems, but more complex questions requiring complex queries containing subqueries or several functions are still a tough challenge within this field of research. Evaluation results of QA systems therefore also might depend on the benchmark dataset the system has been tested on. For the purpose to give an overview and reveal specific characteristics, we examined currently available KGQA datasets regarding several challenging aspects. This paper presents a detailed look into the datasets and compares them in terms of challenges a KGQA system is facing.

Funder

Deutsche Forschungsgemeinschaft

Publisher

Springer Science and Business Media LLC

Subject

Artificial Intelligence,Computer Networks and Communications,Information Systems

Link

https://link.springer.com/content/pdf/10.1007/s13740-021-00128-9.pdf

Reference13 articles.

1. Affolter K, Stockinger K, Bernstein A (2019) A comparative survey of recent natural language interfaces for databases. CoRR, abs/1906.08990,

2. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: A nucleus for a web of open data. In: Aberer K, Choi K-S, Noy N, Allemang D, Lee K-Il, Nixon L, Golbeck J, Mika P, Maynard D, Mizoguchi R, Schreiber G, and Cudré-Mauroux P (eds) The Semantic Web, pp 722–735, Berlin, Heidelberg, Springer Berlin Heidelberg. ISBN 978-3-540-76298-0

3. Azmy M, Shi P, Lin J, Ilyas I (2018) Farewell freebase: Migrating the simplequestions dataset to dbpedia. In: Proceedings of the 27th international conference on computational linguistics, pp 2093–2103

4. Bouziane A, Bouchiha D, Doumi N, Malki M (2015) Question answering systems: Survey and trends. Procedia Comput Sci, 73:366 – 375, 2015. ISSN 1877-0509. https://doi.org/10.1016/j.procs.2015.12.005. http://www.sciencedirect.com/science/article/pii/S1877050915034663. International Conference on Advanced Wireless Information and Communication Technologies (AWICT 2015)

5. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2020) Language models are few-shot learners

Cited by 6 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Chinese mineral question and answering system based on knowledge graph;Expert Systems with Applications;2023-11

2. KGrEaT: A Framework to Evaluate Knowledge Graphs via Downstream Tasks;Proceedings of the 32nd ACM International Conference on Information and Knowledge Management;2023-10-21

3. Entity Linking for KGQA Using AMR Graphs;The Semantic Web;2023

4. Knowledge Graph Question Answering Datasets and Their Generalizability;Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval;2022-07-06

5. A better entity detection of question for knowledge graph question answering through extracting position-based patterns;Journal of Big Data;2022-06-17