DeliberationBench: When Do More Voices Hurt? A Controlled Study of Multi-LLM Deliberation Protocols

Vaarunay Kaushal, Taranveer Singh
12/14/2025
cs.CLcs.AI

No confident code implementation found for this paper.

Abstract

Multi-agent systems where Large Language Models (LLMs) deliberate to form consensus have gained significant attention, yet their practical value over simpler methods remains under-scrutinized. We introduce DELIBERATIONBENCH, a controlled benchmark evaluating three deliberation protocols against a strong baseline of selecting the best response from a pool of model outputs. Across 270 questions and three independent seeds (810 total evaluations), we find a striking negative result: the best-single baseline achieves an 82.5% +- 3.3% win rate, dramatically outperforming the best deliberation protocol(13.8% +- 2.6%). This 6.0x performance gap is statistically significant (p < 0.01) and comes at 1.5-2.5x higher computational cost. Our findings challenge assumptions that complexity enhances quality in multi-LLM systems.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

From Debate to Deliberation: Structured Collective Reasoning with Typed Epistemic Acts

85% match1 repo2026

Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMs

85% match2025

Limits of Large Language Models in Debating Humans

84% match2024

Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluation

84% match2024

Multi-Agent Large Language Models for Conversational Task-Solving

84% match2024

Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity

84% match2025

Cite this paper

@article{kaushal2025deliberationbench,
  title  = {DeliberationBench: When Do More Voices Hurt? A Controlled Study of Multi-LLM Deliberation Protocols},
  author = {Vaarunay Kaushal and Taranveer Singh},
  year   = {2025},
  eprint = {2601.08835},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.08835v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.08835.svg)](https://codeofpaper.com/paper/2601.08835)

Discussion