A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification

Junichiro Niimi
4/26/2025
cs.CLcs.AI

No official code found. Best implementation: jniimi/ensemble_inference (0 stars, MIT, last commit 2025-04-29).

Abstract

With the advance of large language models (LLMs), LLMs have been utilized for the various tasks. However, the issues of variability and reproducibility of results from each trial of LLMs have been largely overlooked in existing literature while actual human annotation uses majority voting to resolve disagreements among annotators. Therefore, this study introduces the straightforward ensemble strategy to a sentiment analysis using LLMs. As the results, we demonstrate that the ensemble of multiple inference using medium-sized LLMs produces more robust and accurate results than using a large model with a single attempt with reducing RMSE by 18.6%.

View on arXivView PDF

Code Implementations(1)

jniimi/ensemble_inferenceCommunity100%
MIT

Python scripts for NLDB2025

00Python, Jupyter NotebookApr 26, 20251 years agoMIT

Similar Papers

Harnessing Multiple Large Language Models: A Survey on LLM Ensemble

89% match1 repo2482025

Not All Layers of LLMs Are Necessary During Inference

85% match2024

Text Classification in the LLM Era -- Where do we stand?

85% match2025

LLM Chain Ensembles for Scalable and Accurate Data Annotation

85% match1 repo32024

An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainability

84% match2025

Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applications

84% match2025

Cite this paper

@article{niimi2025simple,
  title  = {A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification},
  author = {Junichiro Niimi},
  year   = {2025},
  eprint = {2504.18884},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.18884v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.18884.svg)](https://codeofpaper.com/paper/2504.18884)

Discussion