AutoEval Done Right: Using Synthetic Data for Model Evaluation

Pierre Boyeau, Anastasios N. Angelopoulos, Nir Yosef, Jitendra Malik, Michael I. Jordan
3/9/2024
cs.LGcs.AIcs.CL

No confident code implementation found for this paper.

Abstract

The evaluation of machine learning models using human-labeled validation data can be expensive and time-consuming. AI-labeled synthetic data can be used to decrease the number of human annotations required for this purpose in a process called autoevaluation. We suggest efficient and statistically principled algorithms for this purpose that improve sample efficiency while remaining unbiased. These algorithms increase the effective human-labeled sample size by up to 50% on experiments with GPT-4.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{boyeau2024autoeval,
  title  = {AutoEval Done Right: Using Synthetic Data for Model Evaluation},
  author = {Pierre Boyeau and Anastasios N. Angelopoulos and Nir Yosef and Jitendra Malik and Michael I. Jordan},
  year   = {2024},
  eprint = {2403.07008},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.07008v2},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.07008.svg)](https://codeofpaper.com/paper/2403.07008)

Discussion