Sequential Model Optimization for Software Process Control
share
Many methods have been proposed to estimate how much effort is required to build and maintain software. Much of that research assumes a "classic" waterfall-based approach rather than contemporary agile projects. Also, much of that work tries to recommend a single method– an approach that makes the dubious assumption that one method can handle the diversity of software project data. To address these drawbacks, we apply a configuration technique called "ROME" (Rapid Optimizing Methods for Estimation), which uses sequential model-based optimization to find what combination of techniques works best for a particular data set. In this paper, we test this method using data from 1161 classic waterfall projects and 446 contemporary agile projects (from Github). We find that ROME achieves better performance (in terms of magnitude of relative error and standardized accuracy) than existing state-of-the-art methods for both classic and contemporary problems. From this work, we conclude that it is not best to recommend method for estimation. Rather, it is better to search through a wide range of different methods to find what works best for local data. To the best of our knowledge, this is the largest effort estimation experiment yet attempted and the only one to test its methods on classic and contemporary projects.
READ FULL TEXT