Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints

Chao Lou, Kewei Tu
6/5/2023
cs.CL

No official code found. Best implementation: LouChao98/seq2seq_with_qcfg (10 stars, last commit 2023-05-06).

Abstract

Neural QCFG is a grammar-based sequence-tosequence (seq2seq) model with strong inductive biases on hierarchical structures. It excels in interpretability and generalization but suffers from expensive inference. In this paper, we study two low-rank variants of Neural QCFG for faster inference with different trade-offs between efficiency and expressiveness. Furthermore, utilizing the symbolic interface provided by the grammar, we introduce two soft constraints over tree hierarchy and source coverage. We experiment with various datasets and find that our models outperform vanilla Neural QCFG in most settings.

View on arXivView PDF

Code Implementations(1)

LouChao98/seq2seq_with_qcfgCommunity100%
100TeX, PerlJul 5, 20223 years ago

Similar Papers

The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation

80% match1 repo2.9k2018

Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline

79% match1 repo932023

Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving

78% match2024

Learning Transductions and Alignments with RNN Seq2seq Models

78% match2023

A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization

78% match1 repo6462019

Smoothing Entailment Graphs with Language Models

77% match1 repo2022

Cite this paper

@article{lou2023improving,
  title  = {Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints},
  author = {Chao Lou and Kewei Tu},
  year   = {2023},
  eprint = {2306.02671},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.02671v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.02671.svg)](https://codeofpaper.com/paper/2306.02671)

Discussion