Array BP-XOR Codes for Parallel Matrix Multiplication using Hierarchical Computing
share
This study presents a novel coded computation technique for parallel matrix-matrix product computation using hierarchical compute architectures that outperforms well known previous strategies in terms of total end-to-end execution time. The proposed method uses array codes to achieve this performance by distributing the encoding operation over the cluster (slave) nodes at the expense of increased master-slave communication. The matrix multiplication is performed using MDS array Belief Propagation (BP)-decodable codes based on pure XOR operations. The proposed scheme is shown to be configurable and suited for modern hierarchical compute architectures equipped with multiple nodes, each having multiple, independent and less capable processing units. In addition, to address scaling number of strugglers, asymptotic versions of the code is used and latency analysis is conducted. We shall demonstrate that the proposed scheme achieves order-optimal computation in both the sub-linear as well as the linear regimes in the size of the computed product from an end-to-end delay perspective while ensuring acceptable communication requirements that can be addressed by today's high speed network link infrastructures.
READ FULL TEXT