Which scaling rule applies to Artificial Neural Networks
share
Although Artificial Neural Networks are biology-mimicking systems, they are built from components designed/fabricated for use in conventional computing, created by experts trained in conventional computing. Building strongly cooperating and communicating computing systems using segregated single processors, however, has severe performance limitations. The achievable payload computing performance sensitively depends on workload type, and this effect is only poorly known. The workload type, the Artificial Intelligence systems require, has an especially low payload computational performance for Artificial Neural Network applications. Unfortunately, the initial successes of demo systems that comprise only a few "neurons" and solve simple tasks are misleading: the scaling of computing-based ANN systems is strongly nonlinear. The paper discusses some major limiting factors that affect performance and points out that for building biology-mimicking large systems, it is inevitable to perform drastic changes in the present computing paradigm and architectures.
READ FULL TEXT