Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets

Ramya Tekumalla, Juan M. Banda
9/12/2023
cs.CLcs.LGcs.SI

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

The COVID-19 pandemic has presented significant challenges to the healthcare industry and society as a whole. With the rapid development of COVID-19 vaccines, social media platforms have become a popular medium for discussions on vaccine-related topics. Identifying vaccine-related tweets and analyzing them can provide valuable insights for public health research-ers and policymakers. However, manual annotation of a large number of tweets is time-consuming and expensive. In this study, we evaluate the usage of Large Language Models, in this case GPT-4 (March 23 version), and weak supervision, to identify COVID-19 vaccine-related tweets, with the purpose of comparing performance against human annotators. We leveraged a manu-ally curated gold-standard dataset and used GPT-4 to provide labels without any additional fine-tuning or instructing, in a single-shot mode (no additional prompting).

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models

85% match2024

Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy

84% match1 repo2025

Hierarchical Multi-Label Classification of Online Vaccine Concerns

83% match2024

Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks

83% match2023

Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI

83% match2024

ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

82% match2023

Cite this paper

@article{tekumalla2023leveraging,
  title  = {Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets},
  author = {Ramya Tekumalla and Juan M. Banda},
  year   = {2023},
  eprint = {2309.06503},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.06503v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.06503.svg)](https://codeofpaper.com/paper/2309.06503)

Discussion