SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection

Richard Harang, Ethan M. Rudd
12/14/2020
cs.CR

Official code: sophos/SOREL-20M (709 stars, Apache-2.0, last commit 2021-04-07).

Abstract

In this paper we describe the SOREL-20M (Sophos/ReversingLabs-20 Million) dataset: a large-scale dataset consisting of nearly 20 million files with pre-extracted features and metadata, high-quality labels derived from multiple sources, information about vendor detections of the malware samples at the time of collection, and additional ``tags'' related to each malware sample to serve as additional targets. In addition to features and metadata, we also provide approximately 10 million ``disarmed'' malware samples -- samples with both the optional\_headers.subsystem and file\_header.machine flags set to zero -- that may be used for further exploration of features and detection strategies. We also provide Python code to interact with the data and features, as well as baseline neural network and gradient boosted decision tree models and their results, with full training and evaluation code, to serve as a starting point for further experimentation.

View on arXivView PDF

Code Implementations(1)

sophos/SOREL-20MOfficial100%
Apache-2.0

Sophos-ReversingLabs 20 million sample dataset

709150PythonNov 17, 20205 years agoApache-2.0

Similar Papers

EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models

89% match1 repo1.2k2018

Leveraging LSTM and GAN for Modern Malware Detection

86% match2024

Evaluating Ensemble and Deep Learning Models for Static Malware Detection with Dimensionality Reduction Using the EMBER Dataset

85% match2025

Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performance

85% match1 repo182023

Novel Feature Extraction, Selection and Fusion for Effective Malware Family Classification

84% match1 repo502015

Building an Adversarial Malware Dataset by Family and Type: Generation, Evasion, and Poisoning Evaluation

83% match2026

Cite this paper

@article{harang2020sorelm,
  title  = {SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection},
  author = {Richard Harang and Ethan M. Rudd},
  year   = {2020},
  eprint = {2012.07634},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2012.07634}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2012.07634.svg)](https://codeofpaper.com/paper/2012.07634)

Discussion