SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases

Yanchen Liu, Jing Yan, Yan Chen, Jing Liu, Hua Wu
2/28/2023
cs.CL

No official code found. Best implementation: liuyanchen1015/MixtureOfAdapters (0 stars, last commit 2026-03-27).

Abstract

Recent studies reveal that various biases exist in different NLP tasks, and over-reliance on biases results in models' poor generalization ability and low adversarial robustness. To mitigate datasets biases, previous works propose lots of debiasing techniques to tackle specific biases, which perform well on respective adversarial sets but fail to mitigate other biases. In this paper, we propose a new debiasing method Sparse Mixture-of-Adapters (SMoA), which can mitigate multiple dataset biases effectively and efficiently. Experiments on Natural Language Inference and Paraphrase Identification tasks demonstrate that SMoA outperforms full-finetuning, adapter tuning baselines, and prior strong debiasing methods. Further analysis indicates the interpretability of SMoA that sub-adapter can capture specific pattern from the training data and specialize to handle specific bias.

View on arXivView PDF

Code Implementations(1)

liuyanchen1015/MixtureOfAdaptersCommunity67%

This is the oficial repository for "SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases".

00HTML, PythonJun 22, 20226 months ago

Similar Papers

Hybrid Sample Synthesis-based Debiasing of Classifier in Limited Data Setting

89% match2023

Improving Bias Mitigation through Bias Experts in Natural Language Understanding

88% match1 repo62023

Generative Bias for Robust Visual Question Answering

87% match1 repo282022

Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models

87% match1 repo22023

End-to-End Bias Mitigation by Modelling Biases in Corpora

86% match1 repo172019

BLADE: Bias-Linked Adaptive DEbiasing

85% match2025

Cite this paper

@article{liu2023smoa,
  title  = {SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases},
  author = {Yanchen Liu and Jing Yan and Yan Chen and Jing Liu and Hua Wu},
  year   = {2023},
  eprint = {2302.14413},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.14413v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.14413.svg)](https://codeofpaper.com/paper/2302.14413)

Discussion