Methodology for Testing and Evaluation of Safety Analytics Approaches

by   Antonio R. Paiva, et al.

There has been a significant increase in the development of data-driven safety analytics approaches in recent years. In light of these advances it has become imperative to evaluate such approaches in a principled way to determine their merits and limitations. To that end, we propose an evaluation methodology underpinned by a simulated environment that allows for a comprehensive assessment of safety analytics approaches. While assessing those approaches with historical field data is undoubtedly important, such an assessment has limited statistical power because it corresponds to only one realization. The proposed methodology enables validation over a large number of realizations, thereby circumventing the statistical limitations of evaluation on historical data. Moreover, by using a simulated environment one is able to clearly distinguish between the variability in the observed data and differences in performance between safety approaches. A simulated environment does this by comparing the approaches under controlled circumstances, resulting in a fair and systematic evaluation of the potential long-term benefits. We demonstrate the utility of the proposed methodology via a case study that compares a few candidate safety analytics approaches. These approaches differ in how they assimilate field safety data to assess safety risk and suggest mitigative actions. We show that the proposed methodology indeed reveals useful insights and quantifies the relative merits and drawbacks of the different approaches, which would be otherwise difficult to objectively determine in a real-world scenario.


page 1

page 2

page 3

page 4


Scenario-Based Safety Assessment Framework for Automated Vehicles

Automated vehicles (AVs) are expected to increase traffic safety and tra...

A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness

Autonomous Vehicles (AVs) have the potential to provide numerous societa...

Application of the Cox Regression Model for Analysis of Railway Safety Performance

The assessment of in-service safety performance is an important task, no...

Safe Adaptation Using Energy Functions

Adaptation has long been considered to be an important capability for au...

A Methodology for the Development of RL-Based Adaptive Traffic Signal Controllers

This article proposes a methodology for the development of adaptive traf...

Please sign up or login with your details

Forgot password? Click here to reset