An Efficient Method for Uncertainty Propagation in Robust Software Performance Estimation

by   Aldeida Aleti, et al.

Software engineers often have to estimate the performance of a software system before having full knowledge of the system parameters, such as workload and operational profile. These uncertain parameters inevitably affect the accuracy of quality evaluations, and the ability to judge if the system can continue to fulfil performance requirements if parameter results are different from expected. Previous work has addressed this problem by modelling the potential values of uncertain parameters as probability distribution functions, and estimating the robustness of the system using Monte Carlo-based methods. These approaches require a large number of samples, which results in high computational cost and long waiting times. To address the computational inefficiency of existing approaches, we employ Polynomial Chaos Expansion (PCE) as a rigorous method for uncertainty propagation and further extend its use to robust performance estimation. The aim is to assess if the software system is robust, i.e., it can withstand possible changes in parameter values, and continue to meet performance requirements. PCE is a very efficient technique, and requires significantly less computations to accurately estimate the distribution of performance indices. Through three very different case studies from different phases of software development and heterogeneous application domains, we show that PCE can accurately (>97%) estimate the robustness of various performance indices, and saves up to 225 hours of performance evaluation time when compared to Monte Carlo Simulation.


page 1

page 2

page 3

page 4


A probabilistic peridynamic framework with an application to the study of the statistical size effect

Mathematical models are essential for understanding and making predictio...

Robustness of the Sobol' indices to distributional uncertainty

Global sensitivity analysis (GSA) is used to quantify the influence of u...

Robustness of the Sobol' indices to marginal distribution uncertainty

Global sensitivity analysis (GSA) quantifies the influence of uncertain ...

Enhanced Multi-Index Monte Carlo by means of Multiple Semi-Coarsened Multigrid for Anisotropic Diffusion Problems

In many models used in engineering and science, material properties are ...

Shapley Effect Estimation using Polynomial Chaos

This paper presents an approach for estimating Shapley effects for use a...

Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation

Despite recent advances in the accuracy of brain tumor segmentation, the...

Uncertain Quality-Diversity: Evaluation methodology and new methods for Quality-Diversity in Uncertain Domains

Quality-Diversity optimisation (QD) has proven to yield promising result...

Please sign up or login with your details

Forgot password? Click here to reset