Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wording

Aisha Khatun, Daniel G. Brown

6/9/2023

cs.CLcs.LG

Official code: tanny411/GPT3-Reliability-Check (4 stars, last commit 2024-01-08).

Abstract

Large language models (LLMs) have become mainstream technology with their versatile use cases and impressive performance. Despite the countless out-of-the-box applications, LLMs are still not reliable. A lot of work is being done to improve the factual accuracy, consistency, and ethical standards of these models through fine-tuning, prompting, and Reinforcement Learning with Human Feedback (RLHF), but no systematic analysis of the responses of these models to different categories of statements, or on their potential vulnerabilities to simple prompting changes is available. In this work, we analyze what confuses GPT-3: how the model responds to certain sensitive topics and what effects the prompt wording has on the model response. We find that GPT-3 correctly disagrees with obvious Conspiracies and Stereotypes but makes mistakes with common Misconceptions and Controversies. The model responses are inconsistent across prompts and settings, highlighting GPT-3's unreliability. Dataset and code of our analysis is available in https://github.com/tanny411/GPT3-Reliability-Check.

View on arXiv View PDF

Code Implementations(1)

tanny411/GPT3-Reliability-CheckOfficial100%

Systematic analysis of the responses of GPT-3 to different categories of statements and the potential vulnerabilities to simple prompting changes. We analyze what confuses GPT-3: how the model responds to certain sensitive topics and what affects the prompt wording has on the model response.

41HTML, Jupyter NotebookMay 28, 20232 years ago

Similar Papers

A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions

85% match1 repo2024

Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness

85% match2024

Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs

83% match2025

Removing RLHF Protections in GPT-4 via Fine-Tuning

83% match2023

Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias

83% match1 repo12023

How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?

83% match2024

Cite this paper

@article{khatun2023reliability,
  title  = {Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wording},
  author = {Aisha Khatun and Daniel G. Brown},
  year   = {2023},
  eprint = {2306.06199},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.06199v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

[![Code of Paper](https://codeofpaper.com/badge/2306.06199.svg)](https://codeofpaper.com/paper/2306.06199)