Context-Free Transductions with Neural Stacks

Yiding Hao, William Merrill, Dana Angluin, Robert Frank, Noah Amsel +2 more
9/8/2018
cs.NEcs.CLcs.LG

Official code: viking-sudo-rm/StackNN (145 stars, last commit 2019-10-07).

Abstract

This paper analyzes the behavior of stack-augmented recurrent neural network (RNN) models. Due to the architectural similarity between stack RNNs and pushdown transducers, we train stack RNN models on a number of tasks, including string reversal, context-free language modelling, and cumulative XOR evaluation. Examining the behavior of our networks, we show that stack-augmented RNNs can discover intuitive stack-based strategies for solving our tasks. However, stack RNNs are more difficult to train than classical architectures such as LSTMs. Rather than employ stack-based strategies, more complex networks often find approximate solutions by using the stack as unstructured memory.

View on arXivView PDF

Code Implementations(1)

viking-sudo-rm/StackNNOfficial100%

Experiments with differentiable stacks and queues in PyTorch

14524Python, Jupyter NotebookNov 17, 20177 years ago

Similar Papers

Nondeterministic Stacks in Neural Networks

86% match2023

End-To-End Memory Networks

84% match1 repo1.8k2015

On Efficiently Representing Regular Languages as RNNs

83% match2024

Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages

83% match1 repo122019

Universal Transformers

82% match1 repo17.5k2018

Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations

82% match1 repo122018

Cite this paper

@article{hao2018contextfree,
  title  = {Context-Free Transductions with Neural Stacks},
  author = {Yiding Hao and William Merrill and Dana Angluin and Robert Frank and Noah Amsel and Andrew Benz and Simon Mendelsohn},
  year   = {2018},
  eprint = {1809.02836},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1809.02836}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1809.02836.svg)](https://codeofpaper.com/paper/1809.02836)

Discussion