Orthogonal and Linear Regressions and Pencils of Confocal Quadrics
share
We develop further and enhance bridges between three disciplines: statistics, mechanics, and geometry. We explore and employ links between pencils of quadrics, moments of inertia, and linear and orthogonal regressions. For a given system of points in ℝ^k representing a sample of a full rank, we construct an explicit pencil of confocal quadrics with the properties: (i) All the hyperplanes for which the hyperplanar moments of inertia for the given system of points are equal, are tangent to one of the quadrics from the pencil of quadrics. We develop regularization procedures for the orthogonal least square method, analogues of lasso and ridge methods from linear regression. (ii) For any given point P among all the hyperplanes that contain it, the best fit is the tangent hyperplane to the quadric from the confocal pencil corresponding to the maximal Jacobi coordinate of the point P. The confocal pencil of quadrics provides a universal tool to solve the restricted principal component analysis restricted at any given point. Results (i) and (ii) can be seen as generalizations of the classical result of Pearson on orthogonal regression. They have natural and important applications in the statistics of the measurement error models. For the classical linear regressions, we provide a geometric characterization of hyperplanes of least squares in a given direction among all hyperplanes which contain a given point. The obtained results have applications in restricted regressions, both ordinary and orthogonal ones. For the latter, a new formula for test statistic is derived. The developed methods and results are illustrated in natural statistics examples. As a byproduct, a novel, alternative approach to the tread construction of confocal pencils of quadrics in ℝ^3 is developed. We also indicate relationships of the obtained results with integrable billiards within quadrics.
READ FULL TEXT
