DeepAI AI Chat
Log In Sign Up

The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design

by   Jeffrey Dean, et al.

The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore's Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today.


page 4

page 7

page 8

page 12


An Overview of Multi-Task Learning in Deep Neural Networks

Multi-task learning (MTL) has led to successes in many applications of m...

Why Robust Natural Language Understanding is a Challenge

With the proliferation of Deep Machine Learning into real-life applicati...

An affective computational model for machine consciousness

In the past, several models of consciousness have become popular and hav...

A Machine Learning Tutorial for Operational Meteorology, Part II: Neural Networks and Deep Learning

Over the past decade the use of machine learning in meteorology has grow...

A Review of Sparse Expert Models in Deep Learning

Sparse expert models are a thirty-year old concept re-emerging as a popu...

The Rise of AI-Driven Simulators: Building a New Crystal Ball

The use of computational simulation is by now so pervasive in society th...

Learning Memory Access Patterns

The explosion in workload complexity and the recent slow-down in Moore's...