AI Tax: The Hidden Cost of AI Data Center Applications

by   Daniel Richins, et al.

Artificial intelligence and machine learning are experiencing widespread adoption in industry and academia. This has been driven by rapid advances in the applications and accuracy of AI through increasingly complex algorithms and models; this, in turn, has spurred research into specialized hardware AI accelerators. Given the rapid pace of advances, it is easy to forget that they are often developed and evaluated in a vacuum without considering the full application environment. This paper emphasizes the need for a holistic, end-to-end analysis of AI workloads and reveals the "AI tax." We deploy and characterize Face Recognition in an edge data center. The application is an AI-centric edge video analytics application built using popular open source infrastructure and ML tools. Despite using state-of-the-art AI and ML algorithms, the application relies heavily on pre-and post-processing code. As AI-centric applications benefit from the acceleration promised by accelerators, we find they impose stresses on the hardware and software infrastructure: storage and network bandwidth become major bottlenecks with increasing AI acceleration. By specializing for AI applications, we show that a purpose-built edge data center can be designed for the stresses of accelerated AI at 15 lower TCO than one derived from homogeneous servers and infrastructure.


page 4

page 16

page 19

page 23

page 25


AI on the Edge: Rethinking AI-based IoT Applications Using Specialized Edge Architectures

Edge computing has emerged as a popular paradigm for supporting mobile a...

Continuous Subject-in-the-Loop Integration: Centering AI on Marginalized Communities

Despite its utopian promises as a disruptive equalizer, AI - like most t...

Artificial Intelligence at the Edge

The Internet of Things (IoT) and edge computing applications aim to supp...

Datamorphic Testing: A Methodology for Testing AI Applications

With the rapid growth of the applications of machine learning (ML) and o...

Traffic Analytics Development Kits (TADK): Enable Real-Time AI Inference in Networking Apps

Sophisticated traffic analytics, such as the encrypted traffic analytics...

PipeSim: Trace-driven Simulation of Large-Scale AI Operations Platforms

Operationalizing AI has become a major endeavor in both research and ind...

Artificial Intelligence in Electric Machine Drives: Advances and Trends

This review paper systematically summarizes the existing literature on a...

Please sign up or login with your details

Forgot password? Click here to reset