Making ordinary least squares linear classfiers more robust
share
In the field of statistics and machine learning, the sums-of-squares, commonly referred to as ordinary least squares, can be used as a convenient choice of cost function because of its many nice analytical properties, though not always the best choice. However, it has been long known that ordinary least squares is not robust to outliers. Several attempts to resolve this problem led to the creation of alternative methods that, either did not fully resolved the outlier problem or were computationally difficult. In this paper, we provide a very simple solution that can make ordinary least squares less sensitive to outliers in data classification, by scaling the augmented input vector by its length. We show some mathematical expositions of the outlier problem using some approximations and geometrical techniques. We present numerical results to support the efficacy of our method.
READ FULL TEXT
