Constant-Depth and Subcubic-Size Threshold Circuits for Matrix Multiplication
share
Boolean circuits of McCulloch-Pitts threshold gates are a classic model of neural computation studied heavily in the late 20th century as a model of general computation. Recent advances in large-scale neural computing hardware has made their practical implementation a near-term possibility. We describe a theoretical approach for multiplying two N by N matrices that integrates threshold gate logic with conventional fast matrix multiplication algorithms, that perform O(N^ω) arithmetic operations for a positive constant ω < 3. Our approach converts such a fast matrix multiplication algorithm into a constant-depth threshold circuit with approximately O(N^ω) gates. Prior to our work, it was not known whether the Θ(N^3)-gate barrier for matrix multiplication was surmountable by constant-depth threshold circuits. Dense matrix multiplication is a core operation in convolutional neural network training. Performing this work on a neural architecture instead of off-loading it to a GPU may be an appealing option.
READ FULL TEXT