DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit

Jessica Huynh, Ting-Rui Chiang, Jeffrey Bigham, Maxine Eskenazi
7/25/2022
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Dialog system developers need high-quality data to train, fine-tune and assess their systems. They often use crowdsourcing for this since it provides large quantities of data from many workers. However, the data may not be of sufficiently good quality. This can be due to the way that the requester presents a task and how they interact with the workers. This paper introduces DialCrowd 2.0 to help requesters obtain higher quality data by, for example, presenting tasks more clearly and facilitating effective communication with workers. DialCrowd 2.0 guides developers in creating improved Human Intelligence Tasks (HITs) and is directly applicable to the workflows used currently by developers and researchers.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Fanar 2.0: Arabic Generative AI Stack

80% match2026

Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions

78% match2023

A Note on Code Quality Score: LLMs for Maintainable Large Codebases

78% match2025

Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models

78% match1 repo7.1k2024

Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap

77% match2024

Comparing Software Developers with ChatGPT: An Empirical Investigation

77% match2023

Cite this paper

@article{huynh2022dialcrowd,
  title  = {DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit},
  author = {Jessica Huynh and Ting-Rui Chiang and Jeffrey Bigham and Maxine Eskenazi},
  year   = {2022},
  eprint = {2207.12551},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2207.12551v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2207.12551.svg)](https://codeofpaper.com/paper/2207.12551)

Discussion