LLaMA: Open and Efficient Foundation Language Models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux +9 more
2/27/2023
cs.CL

No official code found. Best implementation: meta-llama/llama (59,520 stars, last commit 2025-01-26). 1 more below.

Abstract

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

View on arXivView PDF

Code Implementations(2)

meta-llama/llamaCommunity100%

Inference code for Llama models

59,5209,798Shell, PythonFeb 14, 20231 years agoNOASSERTION
henrywoo/pyllamaCommunity69%
GPL-3.0

LLaMA: Open and Efficient Foundation Language Models

2,781299Shell, PythonFeb 28, 20232 years agoGPL-3.0

Similar Papers

CodeShell Technical Report

89% match2024

Danish Foundation Models

89% match2023

Code Llama: Open Foundation Models for Code

87% match1 repo16.3k2023

AutoML-GPT: Large Language Model for AutoML

87% match2023

Training and Serving System of Foundation Models: A Comprehensive Survey

87% match2024

The Falcon Series of Open Language Models

86% match2023

Cite this paper

@article{touvron2023llama,
  title  = {LLaMA: Open and Efficient Foundation Language Models},
  author = {Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and Baptiste Rozière and Naman Goyal and Eric Hambro and Faisal Azhar and Aurelien Rodriguez and Armand Joulin and Edouard Grave and Guillaume Lample},
  year   = {2023},
  eprint = {2302.13971},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.13971v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.13971.svg)](https://codeofpaper.com/paper/2302.13971)

Discussion