Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets

Satanu Ghosh, Neal R. Brodnik, Carolina Frey, Collin Holgate, Tresa M. Pollock +2 more
6/8/2024
cs.CLcond-mat.mtrl-scics.AI

No official code found. Best implementation: SatanuG/ad_hoc_information_extraction (1 star, MIT, last commit 2024-08-08).

Abstract

We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task.

View on arXivView PDF

Code Implementations(1)

SatanuG/ad_hoc_information_extractionCommunity100%
MIT

This is the official code repository for the paper titled "Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Dataset"

10Python, Jupyter NotebookMay 10, 20242 years agoMIT

Similar Papers

Mining experimental data from Materials Science literature with Large Language Models: an evaluation study

82% match1 repo102024

MaTableGPT: GPT-based Table Data Extractor from Materials Science Literature

81% match2024

Human-artificial intelligence teaming for scientific information extraction from data-driven additive manufacturing research using large language models

81% match2024

Reliable End-to-End Material Information Extraction from the Literature with Source-Tracked Multi-Stage Large Language Models

80% match2025

Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models

80% match2025

GPT-3 Powered Information Extraction for Building Robust Knowledge Bases

80% match2024

Cite this paper

@article{ghosh2024toward,
  title  = {Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets},
  author = {Satanu Ghosh and Neal R. Brodnik and Carolina Frey and Collin Holgate and Tresa M. Pollock and Samantha Daly and Samuel Carton},
  year   = {2024},
  eprint = {2406.05348},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.05348v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.05348.svg)](https://codeofpaper.com/paper/2406.05348)

Discussion