Constructing Multilingual Code Search Dataset Using Neural Machine Translation

Ryo Sekizawa, Nan Duan, Shuai Lu, Hitomi Yanaka
6/27/2023
cs.CLcs.SE

No official code found. Best implementation: ynklab/XCodeSearchNet (3 stars, MIT, last commit 2023-07-20).

Abstract

Code search is a task to find programming codes that semantically match the given natural language queries. Even though some of the existing datasets for this task are multilingual on the programming language side, their query data are only in English. In this research, we create a multilingual code search dataset in four natural and four programming languages using a neural machine translation model. Using our dataset, we pre-train and fine-tune the Transformer-based models and then evaluate them on multiple code search test sets. Our results show that the model pre-trained with all natural and programming language data has performed best in most cases. By applying back-translation data filtering to our dataset, we demonstrate that the translation quality affects the model's performance to a certain extent, but the data size matters more.

View on arXivView PDF

Code Implementations(1)

ynklab/XCodeSearchNetCommunity100%
MIT
30Jun 15, 20233 years agoMIT

Similar Papers

CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation

86% match1 repo582023

Contrastive Prompt Learning-based Code Search based on Interaction Matrix

84% match2023

CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

84% match1 repo2.4k2019

REINFOREST: Reinforcing Semantic Code Similarity for Cross-Lingual Code Search Models

83% match1 repo62023

CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval

83% match2024

GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding

82% match1 repo42023

Cite this paper

@article{sekizawa2023constructing,
  title  = {Constructing Multilingual Code Search Dataset Using Neural Machine Translation},
  author = {Ryo Sekizawa and Nan Duan and Shuai Lu and Hitomi Yanaka},
  year   = {2023},
  eprint = {2306.15604},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.15604v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.15604.svg)](https://codeofpaper.com/paper/2306.15604)

Discussion