The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only

Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra-Aimée Cojocaru, Hamza Alobeidli +4 more
2/12/2026

No confident code implementation found for this paper. 2 possible matches can be shown below.

Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Cite this paper

@article{penedo2026refinedweb,
  title  = {The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only},
  author = {Guilherme Penedo and Quentin Malartic and Daniel Hesslow and Ruxandra-Aimée Cojocaru and Hamza Alobeidli and Alessandro Cappelli and B. Pannier and Ebtesam Almazrouei and Julien Launay},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:05aa63683f7d027c18f560d4472ac87c1ea754fe},
  journal = {NEURIPS 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/05aa63683f7d027c18f560d4472ac87c1ea754fe.svg)](https://codeofpaper.com/paper/05aa63683f7d027c18f560d4472ac87c1ea754fe)

Discussion