Mapping Natural Language Commands to Web Elements

Panupong Pasupat, Tian-Shun Jiang, Evan Zheran Liu, Kelvin Guu, Percy Liang
8/28/2018
cs.CL

Official code: stanfordnlp/phrasenode (41 stars, last commit 2022-07-26).

Abstract

The web provides a rich, open-domain environment with textual, structural, and spatial properties. We propose a new task for grounding language in this environment: given a natural language command (e.g., "click on the second article"), choose the correct element on the web page (e.g., a hyperlink or text box). We collected a dataset of over 50,000 commands that capture various phenomena such as functional references (e.g. "find who made this site"), relational reasoning (e.g. "article by john"), and visual reasoning (e.g. "top-most article"). We also implemented and analyzed three baseline models that capture different phenomena present in the dataset.

View on arXivView PDF

Code Implementations(1)

stanfordnlp/phrasenodeOfficial100%

Mapping natural language commands to web elements

4112Shell, PythonAug 27, 20184 years agoNOASSERTION

Similar Papers

Anchor Prediction: Automatic Refinement of Internet Links

82% match1 repo1.8k2023

WebIE: Faithful and Robust Information Extraction on the Web

79% match2023

Link Prediction for Wikipedia Articles as a Natural Language Inference Task

79% match2023

Quantifying Organizational Environmental Action from Web Data and Large Language Models

79% match2026

DBLPLink: An Entity Linker for the DBLP Scholarly Knowledge Graph

78% match2023

DAGverse: Building Document-Grounded Semantic DAGs from Scientific Papers

78% match2026

Cite this paper

@article{pasupat2018mapping,
  title  = {Mapping Natural Language Commands to Web Elements},
  author = {Panupong Pasupat and Tian-Shun Jiang and Evan Zheran Liu and Kelvin Guu and Percy Liang},
  year   = {2018},
  eprint = {1808.09132},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1808.09132}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1808.09132.svg)](https://codeofpaper.com/paper/1808.09132)

Discussion