Overview of the PromptCBLUE Shared Task in CHIP2023

Wei Zhu, Xiaoling Wang, Mosha Chen, Buzhou Tang
12/29/2023
cs.CL

No official code found. Best implementation: michael-wzhu/PromptCBLUE (394 stars, last commit 2024-01-23).

Abstract

This paper presents an overview of the PromptCBLUE shared task (http://cips-chip.org.cn/2023/eval1) held in the CHIP-2023 Conference. This shared task reformualtes the CBLUE benchmark, and provide a good testbed for Chinese open-domain or medical-domain large language models (LLMs) in general medical natural language processing. Two different tracks are held: (a) prompt tuning track, investigating the multitask prompt tuning of LLMs, (b) probing the in-context learning capabilities of open-sourced LLMs. Many teams from both the industry and academia participated in the shared tasks, and the top teams achieved amazing test results. This paper describes the tasks, the datasets, evaluation metrics, and the top systems for both tasks. Finally, the paper summarizes the techniques and results of the evaluation of the various approaches explored by the participating teams.

View on arXivView PDF

Code Implementations(1)

michael-wzhu/PromptCBLUECommunity100%

PromptCBLUE: a large-scale instruction-tuning dataset for multi-task and few-shot learning in the medical domain in Chinese

39443Shell, PythonApr 4, 20232 years ago

Similar Papers

MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models

88% match2023

PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain

88% match1 repo3942023

Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding

88% match1 repo542023

BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processing

88% match1 repo62023

Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations

87% match1 repo512023

Efficient Finetuning Large Language Models For Vietnamese Chatbot

86% match2023

Cite this paper

@article{zhu2023overview,
  title  = {Overview of the PromptCBLUE Shared Task in CHIP2023},
  author = {Wei Zhu and Xiaoling Wang and Mosha Chen and Buzhou Tang},
  year   = {2023},
  eprint = {2312.17522},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.17522v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.17522.svg)](https://codeofpaper.com/paper/2312.17522)

Discussion