Reinforcement Learning for Topic Models

Jeremy Costello, Marek Z. Reformat
5/8/2023
cs.CLcs.LG

No official code found. Best implementation: jeremy-costello/rl-for-topic-models (7 stars, GPL-3.0, last commit 2023-08-17).

Abstract

We apply reinforcement learning techniques to topic modeling by replacing the variational autoencoder in ProdLDA with a continuous action space reinforcement learning policy. We train the system with a policy gradient algorithm REINFORCE. Additionally, we introduced several modifications: modernize the neural network architecture, weight the ELBO loss, use contextual embeddings, and monitor the learning process via computing topic diversity and coherence for each training step. Experiments are performed on 11 data sets. Our unsupervised model outperforms all other unsupervised models and performs on par with or better than most models using supervised labeling. Our model is outperformed on certain data sets by a model using supervised labeling and contrastive learning. We have also conducted an ablation study to provide empirical evidence of performance improvements from changes we made to ProdLDA and found that the reinforcement learning formulation boosts performance.

View on arXivView PDF

Code Implementations(1)

jeremy-costello/rl-for-topic-modelsCommunity100%
GPL-3.0

Reinforcement Learning for Topic Models

71PythonMay 2, 20233 years agoGPL-3.0
reinforcement-learningsentence-embeddingstopic-modeling

Similar Papers

Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization

83% match2025

Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features

82% match2026

REBEL: Reinforcement Learning via Regressing Relative Rewards

81% match2 repos582024

REBEL: Reinforcement Learning via Regressing Relative Rewards

81% match1 repo582026

Representation-Driven Reinforcement Learning

81% match2023

RL-finetuning LLMs from on- and off-policy data with a single algorithm

81% match2025

Cite this paper

@article{costello2023reinforcement,
  title  = {Reinforcement Learning for Topic Models},
  author = {Jeremy Costello and Marek Z. Reformat},
  year   = {2023},
  eprint = {2305.04843},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.04843v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.04843.svg)](https://codeofpaper.com/paper/2305.04843)

Discussion