Leveraging LLMs in Scholarly Knowledge Graph Question Answering

Tilahun Abedissa Taffa, Ricardo Usbeck
11/16/2023
cs.CLcs.AIcs.DB

Official code: huntila/scholarly-kgqa (10 stars, MIT, last commit 2023-12-28).

Abstract

This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identifies the top-n similar training questions related to a given test question via a BERT-based sentence encoder and retrieves their corresponding SPARQL. Using the top-n similar question-SPARQL pairs as an example and the test question creates a prompt. Then pass the prompt to the LLM and generate a SPARQL. Finally, runs the SPARQL against the underlying KG - ORKG (Open Research KG) endpoint and returns an answer. Our system achieves an F1 score of 99.0%, on SciQA - one of the Scholarly-QALD-23 challenge benchmarks.

View on arXivView PDF

Code Implementations(1)

huntila/scholarly-kgqaOfficial100%
MIT

This repository contains our Scholarly KGQA system code, for the Scholarly-QALD-23 challenge.

100Python, Rich Text FormatOct 21, 20232 years agoMIT

Similar Papers

LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering

88% match1 repo302024

A Human-in-the-Loop, LLM-Centered Architecture for Knowledge-Graph Question Answering

88% match2026

Interpretable Question Answering with Knowledge Graphs

87% match2025

ARK-V1: An LLM-Agent for Knowledge Graph Question Answering Requiring Commonsense Reasoning

86% match2025

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

86% match1 repo252024

Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering

86% match1 repo22025

Cite this paper

@article{taffa2023leveraging,
  title  = {Leveraging LLMs in Scholarly Knowledge Graph Question Answering},
  author = {Tilahun Abedissa Taffa and Ricardo Usbeck},
  year   = {2023},
  eprint = {2311.09841},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.09841v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.09841.svg)](https://codeofpaper.com/paper/2311.09841)

Discussion