Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering

Saeel Sandeep Nachane, Ojas Gramopadhye, Prateek Chanda, Ganesh Ramakrishnan, Kshitij Sharad Jadhav +3 more
3/7/2024
cs.CL

No official code found. Best implementation: ColdSeal/ClinicR (4 stars, last commit 2025-01-30).

Abstract

In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Liévin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE.

View on arXivView PDF

Code Implementations(1)

ColdSeal/ClinicRCommunity100%

This is the official Github Repository for the following paper : Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering Contains the code for all experiments, demo, and training

40Shell, PythonOct 2, 20241 years ago

Similar Papers

To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering

85% match2026

Structured Outputs Enable General-Purpose LLMs to be Medical Experts

83% match2025

Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs

83% match2025

MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning

82% match1 repo452026

LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models

81% match2024

MedLogic-AQA: Enhancing Medical Question Answering with Abstractive Models Focusing on Logical Structures

81% match1 repo2024

Cite this paper

@article{nachane2024shot,
  title  = {Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering},
  author = {Saeel Sandeep Nachane and Ojas Gramopadhye and Prateek Chanda and Ganesh Ramakrishnan and Kshitij Sharad Jadhav and Yatin Nandwani and Dinesh Raghu and Sachindra Joshi},
  year   = {2024},
  eprint = {2403.04890},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.04890v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.04890.svg)](https://codeofpaper.com/paper/2403.04890)

Discussion