NeuroX Library for Neuron Analysis of Deep NLP Models

Fahim Dalvi, Hassan Sajjad, Nadir Durrani
5/26/2023
cs.CL

No official code found. Best implementation: fdalvi/NeuroX (109 stars, BSD-3-Clause, last commit 2023-10-04).

Abstract

Neuron analysis provides insights into how knowledge is structured in representations and discovers the role of neurons in the network. In addition to developing an understanding of our models, neuron analysis enables various applications such as debiasing, domain adaptation and architectural search. We present NeuroX, a comprehensive open-source toolkit to conduct neuron analysis of natural language processing models. It implements various interpretation methods under a unified API, and provides a framework for data processing and evaluation, thus making it easier for researchers and practitioners to perform neuron analysis. The Python toolkit is available at https://www.github.com/fdalvi/NeuroX. Demo Video available at https://youtu.be/mLhs2YMx4u8.

View on arXivView PDF

Code Implementations(1)

fdalvi/NeuroXCommunity100%
BSD-3-Clause

A Python library that encapsulates various methods for neuron interpretation and analysis in Deep NLP models.

10927Shell, PythonAug 16, 20183 years agoBSD-3-Clause
explainable-ainatural-language-processingneuronsnlpnlp-machine-learning

Similar Papers

NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks

89% match1 repo1092018

Evaluating Neuron Interpretation Methods of NLP Models

85% match1 repo32023

NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactions

83% match1 repo72025

Neuron to Graph: Interpreting Language Model Neurons at Scale

83% match1 repo112023

N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models

83% match2023

Automated Natural Language Explanation of Deep Visual Neurons with Large Models

83% match2023

Cite this paper

@article{dalvi2023neurox,
  title  = {NeuroX Library for Neuron Analysis of Deep NLP Models},
  author = {Fahim Dalvi and Hassan Sajjad and Nadir Durrani},
  year   = {2023},
  eprint = {2305.17073},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.17073v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.17073.svg)](https://codeofpaper.com/paper/2305.17073)

Discussion