Nemotron-4 15B Technical Report

Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian +22 more
2/26/2024
cs.CLcs.AIcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates strong performance when assessed on English, multilingual, and coding tasks: it outperforms all existing similarly-sized open models on 4 out of 7 downstream evaluation areas and achieves competitive performance to the leading open models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best multilingual capabilities of all similarly-sized models, even outperforming models over four times larger and those explicitly specialized for multilingual tasks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Orion-14B: Open-source Multilingual Large Language Models

86% match1 repo8112024

PersianLLaMA: Towards Building First Persian Large Language Model

85% match2023

LOLA -- An Open-Source Massively Multilingual Large Language Model

84% match1 repo112024

PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing

84% match2023

FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data

84% match1 repo102024

Baichuan 2: Open Large-scale Language Models

84% match1 repo4.1k2023

Cite this paper

@article{parmar2024nemotron,
  title  = {Nemotron-4 15B Technical Report},
  author = {Jupinder Parmar and Shrimai Prabhumoye and Joseph Jennings and Mostofa Patwary and Sandeep Subramanian and Dan Su and Chen Zhu and Deepak Narayanan and Aastha Jhunjhunwala and Ayush Dattagupta and Vibhu Jawa and Jiwei Liu and Ameya Mahabaleshwarkar and Osvald Nitski and Annika Brundyn and James Maki and Miguel Martinez and Jiaxuan You and John Kamalu and Patrick LeGresley and Denys Fridman and Jared Casper and Ashwath Aithal and Oleksii Kuchaiev and Mohammad Shoeybi and Jonathan Cohen and Bryan Catanzaro},
  year   = {2024},
  eprint = {2402.16819},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.16819v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.16819.svg)](https://codeofpaper.com/paper/2402.16819)

Discussion