Asset Management in Machine Learning: A Survey

by   Samuel Idowu, et al.

Machine Learning (ML) techniques are becoming essential components of many software systems today, causing an increasing need to adapt traditional software engineering practices and tools to the development of ML-based software systems. This need is especially pronounced due to the challenges associated with the large-scale development and deployment of ML systems. Among the most commonly reported challenges during the development, production, and operation of ML-based systems are experiment management, dependency management, monitoring, and logging of ML assets. In recent years, we have seen several efforts to address these challenges as witnessed by an increasing number of tools for tracking and managing ML experiments and their assets. To facilitate research and practice on engineering intelligent systems, it is essential to understand the nature of the current tool support for managing ML assets. What kind of support is provided? What asset types are tracked? What operations are offered to users for managing those assets? We discuss and position ML asset management as an important discipline that provides methods and tools for ML assets as structures and the ML development activities as their operations. We present a feature-based survey of 17 tools with ML asset management support identified in a systematic search. We overview these tools' features for managing the different types of assets used for engineering ML-based systems and performing experiments. We found that most of the asset management support depends on traditional version control systems, while only a few tools support an asset granularity level that differentiates between important ML assets, such as datasets and models.


Software Engineering Practices for Machine Learning

In the last couple of years we have witnessed an enormous increase of ma...

Management of Machine Learning Lifecycle Artifacts: A Survey

The explorative and iterative nature of developing and operating machine...

Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems

The industrial machine learning pipeline requires iterating on model fea...

Towards a safe MLOps Process for the Continuous Development and Safety Assurance of ML-based Systems in the Railway Domain

Traditional automation technologies alone are not sufficient to enable d...

Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?

Following continuous software engineering practices, there has been an i...

Tractography and machine learning: Current state and open challenges

Supervised machine learning (ML) algorithms have recently been proposed ...

Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report

Artificial Intelligence (AI) / Machine Learning (ML)-based systems are w...

Please sign up or login with your details

Forgot password? Click here to reset