Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs

by   Syed Qutub, et al.

Object detection neural network models need to perform reliably in highly dynamic and safety-critical environments like automated driving or robotics. Therefore, it is paramount to verify the robustness of the detection under unexpected hardware faults like soft errors that can impact a systems perception module. Standard metrics based on average precision produce model vulnerability estimates at the object level rather than at an image level. As we show in this paper, this does not provide an intuitive or representative indicator of the safety-related impact of silent data corruption caused by bit flips in the underlying memory but can lead to an over- or underestimation of typical fault-induced hazards. With an eye towards safety-related real-time applications, we propose a new metric IVMOD (Image-wise Vulnerability Metric for Object Detection) to quantify vulnerability based on an incorrect image-wise object detection due to false positive (FPs) or false negative (FNs) objects, combined with a severity analysis. The evaluation of several representative object detection models shows that even a single bit flip can lead to a severe silent data corruption event with potentially critical safety implications, with e.g., up to (much greater than) 100 FPs generated, or up to approx. 90 single stuck-at-1 fault, an entire sequence of images can be affected, causing temporally persistent ghost detections that can be mistaken for actual objects (covering up to approx. 83 scene are continuously missed (up to approx. 64 establishes a detailed understanding of the safety-related vulnerability of such critical workloads against hardware faults.


page 2

page 10

page 11


Fault Injectors for TensorFlow: Evaluation of the Impact of Random Hardware Faults on Deep CNNs

Today, Deep Learning (DL) enhances almost every industrial sector, inclu...

Memory Vulnerability: A Case for Delaying Error Reporting

To face future reliability challenges, it is necessary to quantify the r...

Optimizing Lead Time in Fall Detection for a Planar Bipedal Robot

For legged robots to operate in complex terrains, they must be robust to...

Learning to Evaluate Perception Models Using Planner-Centric Metrics

Variants of accuracy and precision are the gold-standard by which the co...

An Automotive Case Study on the Limits of Approximation for Object Detection

The accuracy of camera-based object detection (CBOD) built upon deep lea...

Entropy-Based Modeling for Estimating Soft Errors Impact on Binarized Neural Network Inference

Over past years, the easy accessibility to the large scale datasets has ...

Performance Monitoring of Object Detection During Deployment

Performance monitoring of object detection is crucial for safety-critica...

Please sign up or login with your details

Forgot password? Click here to reset