SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection

by   Richard Harang, et al.

In this paper we describe the SOREL-20M (Sophos/ReversingLabs-20 Million) dataset: a large-scale dataset consisting of nearly 20 million files with pre-extracted features and metadata, high-quality labels derived from multiple sources, information about vendor detections of the malware samples at the time of collection, and additional “tags” related to each malware sample to serve as additional targets. In addition to features and metadata, we also provide approximately 10 million “disarmed” malware samples – samples with both the optional_headers.subsystem and file_header.machine flags set to zero – that may be used for further exploration of features and detection strategies. We also provide Python code to interact with the data and features, as well as baseline neural network and gradient boosted decision tree models and their results, with full training and evaluation code, to serve as a starting point for further experimentation.


page 1

page 2

page 3

page 4


EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models

This paper describes EMBER: a labeled benchmark dataset for training mac...

"Influence Sketching": Finding Influential Samples In Large-Scale Regressions

There is an especially strong need in modern large-scale data analysis t...

ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentation

Malware detection is a popular application of Machine Learning for Infor...

KiloGrams: Very Large N-Grams for Malware Classification

N-grams have been a common tool for information retrieval and machine le...

R2-D2: ColoR-inspired Convolutional NeuRal Network (CNN)-based AndroiD Malware Detections

Machine Learning (ML) has found it particularly useful in malware detect...

Applications of PageRank to Function Comparison and Malware Classification

We classify .NET files as either benign or malicious by examining certai...

A First Look at the Crypto-Mining Malware Ecosystem: A Decade of Unrestricted Wealth

Illicit crypto-mining leverages resources stolen from victims to mine cr...

Code Repositories


Sophos-ReversingLabs 20 million sample dataset

view repo

Please sign up or login with your details

Forgot password? Click here to reset