Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs

W. James Murdoch, Peter J. Liu, Bin Yu
1/16/2018
cs.CLcs.LGstat.ML

Official code: jamie-murdoch/ContextualDecomposition (55 stars, MIT, last commit 2020-07-23).

Abstract

The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships. Consequently, our inability to describe these relationships has led to LSTMs being characterized as black boxes. To this end, we introduce contextual decomposition (CD), an interpretation algorithm for analysing individual predictions made by standard LSTMs, without any changes to the underlying model. By decomposing the output of a LSTM, CD captures the contributions of combinations of words or variables to the final prediction of an LSTM. On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction. Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.

View on arXivView PDF

Code Implementations(1)

jamie-murdoch/ContextualDecompositionOfficial100%
MIT

Demo for method introduced in "Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs"

5510Python, Jupyter NotebookFeb 23, 20186 years agoMIT

Similar Papers

IndoBERT-Sentiment: Context-Conditioned Sentiment Classification for Indonesian Text

83% match2026

From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning

82% match2026

Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)

82% match2026

SOUL: Towards Sentiment and Opinion Understanding of Language

80% match1 repo72023

iACOS: Advancing Implicit Sentiment Extraction with Informative and Adaptive Negative Examples

80% match2023

Indeterminacy in Affective Computing: Considering Meaning and Context in Data Collection Practices

80% match2025

Cite this paper

@article{murdoch2018beyond,
  title  = {Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs},
  author = {W. James Murdoch and Peter J. Liu and Bin Yu},
  year   = {2018},
  eprint = {1801.05453},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1801.05453}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1801.05453.svg)](https://codeofpaper.com/paper/1801.05453)

Discussion