Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling

Suman Adhya, Avishek Lahiri, Debarshi Kumar Sanyal
3/28/2023
cs.CLcs.LG

No official code found. Best implementation: AdhyaSuman/NTMs_Dropout_Analysis (4 stars, last commit 2024-04-09).

Abstract

Dropout is a widely used regularization trick to resolve the overfitting issue in large feedforward neural networks trained on a small dataset, which performs poorly on the held-out test subset. Although the effectiveness of this regularization trick has been extensively studied for convolutional neural networks, there is a lack of analysis of it for unsupervised models and in particular, VAE-based neural topic models. In this paper, we have analyzed the consequences of dropout in the encoder as well as in the decoder of the VAE architecture in three widely used neural topic models, namely, contextualized topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly available datasets. We characterize the dropout effect on these models in terms of the quality and predictive performance of the generated topics.

View on arXivView PDF

Code Implementations(1)

AdhyaSuman/NTMs_Dropout_AnalysisCommunity100%

This repository is associated with the paper "Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling", accepted at EACL 2023.

41CSS, HTMLFeb 26, 20232 years ago
dropoutneural-topic-models

Similar Papers

Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training

82% match2024

NeuFair: Neural Network Fairness Repair with Dropout

80% match1 repo12024

Evaluating Negative Sampling Approaches for Neural Topic Models

80% match1 repo2025

Drop Dropout on Single-Epoch Language Model Pretraining

80% match2025

Dropout Reduces Underfitting

80% match1 repo3162026

STLM Engineering Report: Dropout

79% match1 repo312024

Cite this paper

@article{adhya2023neural,
  title  = {Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling},
  author = {Suman Adhya and Avishek Lahiri and Debarshi Kumar Sanyal},
  year   = {2023},
  eprint = {2303.15973},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.15973v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.15973.svg)](https://codeofpaper.com/paper/2303.15973)

Discussion