Myths and Legends in High-Performance Computing

01/06/2023
by   Satoshi Matsuoka, et al.
0

In this humorous and thought provoking article, we discuss certain myths and legends that are folklore among members of the high-performance computing community. We collected those myths from conversations at conferences and meetings, product advertisements, papers, and other communications such as tweets, blogs, and news articles within (and beyond) our community. We believe they represent the zeitgeist of the current era of massive change, driven by the end of many scaling laws such as Dennard scaling and Moore's law. While some laws end, new directions open up, such as algorithmic scaling or novel architecture research. However, these myths are rarely based on scientific facts but often on some evidence or argumentation. In fact, we believe that this is the very reason for the existence of many myths and why they cannot be answered clearly. While it feels like there should be clear answers for each, some may remain endless philosophical debates such as the question whether Beethoven was better than Mozart. We would like to see our collection of myths as a discussion of possible new directions for research and industry investment.

READ FULL TEXT

page 2

page 5

page 7

page 8

research
08/17/2022

Understanding Scaling Laws for Recommendation Models

Scale has been a major driving force in improving machine learning perfo...
research
02/13/2022

Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments

Neural scaling laws define a predictable relationship between a model's ...
research
10/20/2022

CONSISTENT: Open-Ended Question Generation From News Articles

Recent work on question generation has largely focused on factoid questi...
research
01/11/2017

Allometric Scaling in Scientific Fields

Allometric scaling can reflect underlying mechanisms, dynamics and struc...
research
02/17/2020

Re-evaluating scaling methods for distributed parallel systems

The paper explains why Amdahl's Law shall be interpreted specifically fo...
research
09/18/2022

On the existence of Scaling laws across Indian districts: A new prospect for urban scaling

Urban scaling analysis in generally performed based on cities. In this s...
research
11/26/2019

Summarizing CPU and GPU Design Trends with Product Data

Moore's Law and Dennard Scaling have guided the semiconductor industry f...

Please sign up or login with your details

Forgot password? Click here to reset