Do Physicians Know How to Prompt? The Need for Automatic Prompt Optimization Help in Clinical Note Generation

Zonghai Yao, Ahmed Jaafar, Beining Wang, Zhichao Yang, Hong Yu
11/16/2023
cs.CLcs.AI

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

This study examines the effect of prompt engineering on the performance of Large Language Models (LLMs) in clinical note generation. We introduce an Automatic Prompt Optimization (APO) framework to refine initial prompts and compare the outputs of medical experts, non-medical experts, and APO-enhanced GPT3.5 and GPT4. Results highlight GPT4 APO's superior performance in standardizing prompt quality across clinical note sections. A human-in-the-loop approach shows that experts maintain content quality post-APO, with a preference for their own modifications, suggesting the value of expert customization. We recommend a two-phase optimization process, leveraging APO-GPT4 for consistency and expert input for personalization.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices

85% match2024

An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing

84% match2023

MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records

84% match1 repo1022023

To Write or to Automate Linguistic Prompts, That Is the Question

84% match2026

DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task

83% match1 repo8332023

Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs

83% match2026

Cite this paper

@article{yao2023physicians,
  title  = {Do Physicians Know How to Prompt? The Need for Automatic Prompt Optimization Help in Clinical Note Generation},
  author = {Zonghai Yao and Ahmed Jaafar and Beining Wang and Zhichao Yang and Hong Yu},
  year   = {2023},
  eprint = {2311.09684},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.09684v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.09684.svg)](https://codeofpaper.com/paper/2311.09684)

Discussion