Can Programming Languages Boost Each Other via Instruction Tuning?

Daoguang Zan, Ailun Yu, Bo Shen, Jiaxin Zhang, Taihong Chen +6 more
8/31/2023
cs.CLcs.AIcs.PL

Official code: NL2Code/CodeM (44 stars, Apache-2.0, last commit 2024-06-02).

Abstract

When human programmers have mastered a programming language, it would be easier when they learn a new programming language. In this report, we focus on exploring whether programming languages can boost each other during the instruction fine-tuning phase of code large language models. We conduct extensive experiments of 8 popular programming languages (Python, JavaScript, TypeScript, C, C++, Java, Go, HTML) on StarCoder. Results demonstrate that programming languages can significantly improve each other. For example, CodeM-Python 15B trained on Python is able to increase Java by an absolute 17.95% pass@1 on HumanEval-X. More surprisingly, we found that CodeM-HTML 7B trained on the HTML corpus can improve Java by an absolute 15.24% pass@1. Our training data is released at https://github.com/NL2Code/CodeM.

View on arXivView PDF

Code Implementations(1)

NL2Code/CodeMOfficial100%
Apache-2.0
442Aug 29, 20232 years agoApache-2.0

Similar Papers

Analysis of AdvFusion: Adapter-based Multilingual Learning for Code Large Language Models

86% match2025

MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks

86% match2 repos7.5k2023

AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models

86% match2026

OctoPack: Instruction Tuning Code Large Language Models

85% match2 repos1.1k2026

The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

85% match1 repo1052023

Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback

85% match1 repo962023

Cite this paper

@article{zan2023programming,
  title  = {Can Programming Languages Boost Each Other via Instruction Tuning?},
  author = {Daoguang Zan and Ailun Yu and Bo Shen and Jiaxin Zhang and Taihong Chen and Bing Geng and Bei Chen and Jichuan Ji and Yafen Yao and Yongji Wang and Qianxiang Wang},
  year   = {2023},
  eprint = {2308.16824},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.16824v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.16824.svg)](https://codeofpaper.com/paper/2308.16824)

Discussion