Performance of Large Language Models in Answering Critical Care Medicine Questions

Mahmoud Alwakeel, Aditya Nagori, An-Kwok Ian Wong, Neal Chaisson, Vijay Krishnamoorthy +1 more
9/16/2025
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Large Language Models have been tested on medical student-level questions, but their performance in specialized fields like Critical Care Medicine (CCM) is less explored. This study evaluated Meta-Llama 3.1 models (8B and 70B parameters) on 871 CCM questions. Llama3.1:70B outperformed 8B by 30%, with 60% average accuracy. Performance varied across domains, highest in Research (68.4%) and lowest in Renal (47.9%), highlighting the need for broader future work to improve models across various subspecialty domains.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Enhancing Healthcare through Large Language Models: A Study on Medical Question Answering

88% match2024

An Empirical Evaluation of Large Language Models on Consumer Health Questions

88% match2024

MedLM: Exploring Language Models for Medical Question Answering Systems

87% match1 repo152024

Towards Evaluating and Building Versatile Large Language Models for Medicine

87% match1 repo792024

Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes

86% match2024

A Comprehensive Study on Fine-Tuning Large Language Models for Medical Question Answering Using Classification Models and Comparative Analysis

86% match2025

Cite this paper

@article{alwakeel2025performance,
  title  = {Performance of Large Language Models in Answering Critical Care Medicine Questions},
  author = {Mahmoud Alwakeel and Aditya Nagori and An-Kwok Ian Wong and Neal Chaisson and Vijay Krishnamoorthy and Rishikesan Kamaleswaran},
  year   = {2025},
  eprint = {2509.19344},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.19344v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.19344.svg)](https://codeofpaper.com/paper/2509.19344)

Discussion