How Model Size, Temperature, and Prompt Style Affect LLM-Human Assessment Score Alignment

Julie Jung, Max Lu, Sina Chole Benker, Dogus Darici
9/14/2025
cs.CLstat.ME

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We examined how model size, temperature, and prompt style affect Large Language Models' (LLMs) alignment within itself, between models, and with human in assessing clinical reasoning skills. Model size emerged as a key factor in LLM-human score alignment. Study highlights the importance of checking alignments across multiple levels.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale

87% match2026

Exploring the Effects of Alignment on Numerical Bias in Large Language Models

86% match2026

Uncovering Factor Level Preferences to Improve Human-Model Alignment

85% match2024

On Evaluating LLM Alignment by Evaluating LLMs as Judges

85% match2025

Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs

84% match1 repo2025

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

84% match1 repo92024

Cite this paper

@article{jung2025model,
  title  = {How Model Size, Temperature, and Prompt Style Affect LLM-Human Assessment Score Alignment},
  author = {Julie Jung and Max Lu and Sina Chole Benker and Dogus Darici},
  year   = {2025},
  eprint = {2509.19329},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.19329v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.19329.svg)](https://codeofpaper.com/paper/2509.19329)

Discussion