How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

by   Florian Tambon, et al.

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called 'safety-critical' systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches. Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question 'How to Certify Machine Learning Based Safety-critical Systems?'. Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted. Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in terms of datasets and type of models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated the necessity to build connections between the above mention main pillars that are for now mainly studied separately. Conclusion: We highlighted current efforts deployed to enable the certification of ML based software systems, and discuss some future research directions.


page 15

page 16


Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey

Machine learning (ML) is finding its way into safety-critical systems (S...

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

Context: Neural Network (NN) algorithms have been successfully adopted i...

An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications

Context and motivation: The development and operation of critical softwa...

Machine Vision in the Context of Robotics: A Systematic Literature Review

Machine vision is critical to robotics due to a wide range of applicatio...

On Testing Machine Learning Programs

Nowadays, we are witnessing a wide adoption of Machine learning (ML) mod...

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

High-accurate machine learning (ML) image classifiers cannot guarantee t...

An Interpretable Graph-based Mapping of Trustworthy Machine Learning Research

There is an increasing interest in ensuring machine learning (ML) framew...

Please sign up or login with your details

Forgot password? Click here to reset