MLPerf Inference Benchmark

by   Vijay Janapa Reddi, et al.

Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and four orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. Driven by more than 30 organizations as well as more than 200 ML engineers and practitioners, MLPerf implements a set of rules and practices to ensure comparability across systems with wildly differing architectures. In this paper, we present the method and design principles of the initial MLPerf Inference release. The first call for submissions garnered more than 600 inference-performance measurements from 14 organizations, representing over 30 systems that show a range of capabilities.


Adversarial Machine Learning – Industry Perspectives

Based on interviews with 28 organizations, we found that industry practi...

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress

Pushing the boundaries of machine learning often requires exploring diff...

CLAASIC: a Cortex-Inspired Hardware Accelerator

This work explores the feasibility of specialized hardware implementing ...

Pick the Right Edge Device: Towards Power and Performance Estimation of CUDA-based CNNs on GPGPUs

The emergence of Machine Learning (ML) as a powerful technique has been ...

Desiderata for next generation of ML model serving

Inference is a significant part of ML software infrastructure. Despite t...

Neural Network Inference on Mobile SoCs

The ever-increasing demand from mobile Machine Learning (ML) application...

Towards a multi-stakeholder value-based assessment framework for algorithmic systems

In an effort to regulate Machine Learning-driven (ML) systems, current a...

Code Repositories


Reference implementations of MLPerf™ inference benchmarks

view repo


Reference implementations of inference benchmarks

view repo


Reference implementations of inference benchmarks

view repo

Please sign up or login with your details

Forgot password? Click here to reset