Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench

Anton Cheshkov, Pavel Zadorozhny, Rodion Levichev, Evgeny Maslov, Ronaldo Franco Jaldin
10/6/2024
cs.SEcs.AIcs.MA

No confident code implementation found for this paper.

Abstract

Automatic program repair at project level may open yet to be seen opportunities in various fields of human activity. Since the SWE-Bench challenge was presented, we have seen numerous of solutions. Patch generation is a part of program repair, and test suite-based conversational patch generation has proven its effectiveness. However, the potential of conversational patch generation has not yet specifically estimated on SWE-Bench. This study reports experimental results aimed at evaluating the individual effectiveness of conversational patch generation on problems from SWE-Bench. The experiments show that a simple conversational pipeline based on LLaMA 3.1 70B can generate valid patches in 47\% of cases, which is comparable to the state-of-the-art in program repair on SWE-Bench.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Conversational Automated Program Repair

83% match2023

Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT

81% match2023

Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs

80% match1 repo62024

SpecRover: Code Intent Extraction via LLMs

79% match2024

Repair-R1: Better Test Before Repair

79% match1 repo102025

Evaluating Agent-based Program Repair at Google

79% match2025

Cite this paper

@article{cheshkov2024exploring,
  title  = {Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench},
  author = {Anton Cheshkov and Pavel Zadorozhny and Rodion Levichev and Evgeny Maslov and Ronaldo Franco Jaldin},
  year   = {2024},
  eprint = {2410.04485},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.04485v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.04485.svg)](https://codeofpaper.com/paper/2410.04485)

Discussion