Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference

Aby Mammen Mathew
4/21/2026
cs.CLcs.AI

No confident code implementation found for this paper.

Abstract

Neural NLI models overfit dataset artifacts instead of truly reasoning. A hypothesis-only model gets 57.7% in SNLI, showing strong spurious correlations, and 38.6% of the baseline errors are the result of these artifacts. We propose Product-of-Experts (PoE) training, which downweights examples where biased models are overconfident. PoE nearly preserves accuracy (89.10% vs. 89.30%) while cutting bias reliance by 4.71% (bias agreement 49.85% to 45%). An ablation finds lambda = 1.5 that best balances debiasing and accuracy. Behavioral tests still reveal issues with negation and numerical reasoning.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities

86% match2023

Hybrid Sample Synthesis-based Debiasing of Classifier in Limited Data Setting

85% match2023

When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA

83% match2026

Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models

83% match1 repo22023

Mitigating Spurious Correlations via Disagreement Probability

83% match2026

SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases

82% match1 repo2023

Cite this paper

@article{mathew2026productofexperts,
  title  = {Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference},
  author = {Aby Mammen Mathew},
  year   = {2026},
  eprint = {2604.19069},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2604.19069v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2604.19069.svg)](https://codeofpaper.com/paper/2604.19069)

Discussion