Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models

Louis Abraham, Charles Arnal, Antoine Marie
7/15/2024
cs.CLcs.AIcs.CY

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/ .

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting

86% match2024

iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop

86% match2024

Large Language Model Prompt Datasets: An In-depth Analysis and Insights

85% match2025

Best Practices for Text Annotation with Large Language Models

85% match2024

Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models

85% match1 repo9722025

Magic Words or Methodical Work? Challenging Conventional Wisdom in LLM-Based Political Text Annotation

85% match2026

Cite this paper

@article{abraham2024prompt,
  title  = {Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models},
  author = {Louis Abraham and Charles Arnal and Antoine Marie},
  year   = {2024},
  eprint = {2407.10645},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.10645v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.10645.svg)](https://codeofpaper.com/paper/2407.10645)

Discussion