Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang +7 more
9/24/2018
cs.CLcs.AI

Official code: taoyds/spider (1,101 stars, Apache-2.0, last commit 2024-05-29).

Abstract

We present Spider, a large-scale, complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 college students. It consists of 10,181 questions and 5,693 unique complex SQL queries on 200 databases with multiple tables, covering 138 different domains. We define a new complex and cross-domain semantic parsing and text-to-SQL task where different complex SQL queries and databases appear in train and test sets. In this way, the task requires the model to generalize well to both new SQL queries and new database schemas. Spider is distinct from most of the previous semantic parsing tasks because they all use a single database and the exact same programs in the train set and the test set. We experiment with various state-of-the-art models and the best model achieves only 12.4% exact matching accuracy on a database split setting. This shows that Spider presents a strong challenge for future research. Our dataset and task are publicly available at https://yale-lily.github.io/spider

View on arXivView PDF

Code Implementations(1)

taoyds/spiderOfficial100%
Apache-2.0

scripts and baselines for Spider: Yale complex and cross-domain semantic parsing and text-to-SQL challenge

1,101227CSS, PerlSep 24, 20182 years agoApache-2.0

Similar Papers

Semantic Decomposition of Question and SQL for Text-to-SQL Parsing

85% match1 repo62023

Adapt and Decompose: Efficient Generalization of Text-to-SQL via Domain Adapted Least-To-Most Prompting

84% match2023

Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs

83% match2023

SParC: Cross-Domain Semantic Parsing in Context

83% match1 repo2072019

SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task

82% match1 repo1372018

CSS: A Large-scale Cross-schema Chinese Text-to-SQL Medical Dataset

82% match1 repo92023

Cite this paper

@article{yu2018spider,
  title  = {Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task},
  author = {Tao Yu and Rui Zhang and Kai Yang and Michihiro Yasunaga and Dongxu Wang and Zifan Li and James Ma and Irene Li and Qingning Yao and Shanelle Roman and Zilin Zhang and Dragomir Radev},
  year   = {2018},
  eprint = {1809.08887},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1809.08887}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1809.08887.svg)](https://codeofpaper.com/paper/1809.08887)

Discussion