A Survey on Distributed Machine Learning

by   Joost Verbraeken, et al.

The demand for artificial intelligence has grown significantly over the last decade and this growth has been fueled by advances in machine learning techniques and the ability to leverage hardware acceleration. However, in order to increase the quality of predictions and render machine learning solutions feasible for more complex applications, a substantial amount of training data is required. Although small machine learning models can be trained with modest amounts of data, the input for training larger models such as neural networks grows exponentially with the number of parameters. Since the demand for processing training data has outpaced the increase in computation power of computing machinery, there is a need for distributing the machine learning workload across multiple machines, and turning the centralized into a distributed system. These distributed systems present new challenges, first and foremost the efficient parallelization of the training process and the creation of a coherent model. This article provides an extensive overview of the current state-of-the-art in the field by outlining the challenges and opportunities of distributed machine learning over conventional (centralized) machine learning, discussing the techniques used for distributed machine learning, and providing an overview of the systems that are available.


page 1

page 2

page 3

page 4


A Survey From Distributed Machine Learning to Distributed Deep Learning

Artificial intelligence has achieved significant success in handling com...

Creativity and Machine Learning: A Survey

There is a growing interest in the area of machine learning and creativi...

Uncertainty Estimation in Machine Learning

Most machine learning techniques are based upon statistical learning the...

State of the Art Review for Applying Computational Intelligence and Machine Learning Techniques to Portfolio Optimisation

Computational techniques have shown much promise in the field of Finance...

Challenging Images For Minds and Machines

There is no denying the tremendous leap in the performance of machine le...

Efficient and Robust Machine Learning for Real-World Systems

While machine learning is traditionally a resource intensive task, embed...

Asymptotic Network Independence in Distributed Optimization for Machine Learning

We provide a discussion of several recent results which have overcome a ...

Please sign up or login with your details

Forgot password? Click here to reset