Subspace Embedding and Linear Regression with Orlicz Norm

by   Alexandr Andoni, et al.

We consider a generalization of the classic linear regression problem to the case when the loss is an Orlicz norm. An Orlicz norm is parameterized by a non-negative convex function G:R_+→R_+ with G(0)=0: the Orlicz norm of a vector x∈R^n is defined as x_G={α>0|∑_i=1^n G(|x_i|/α)≤ 1}. We consider the cases where the function G(·) grows subquadratically. Our main result is based on a new oblivious embedding which embeds the column space of a given matrix A∈R^n× d with Orlicz norm into a lower dimensional space with ℓ_2 norm. Specifically, we show how to efficiently find an embedding matrix S∈R^m× n,m<n such that ∀ x∈R^d,Ω(1/(d n)) ·Ax_G≤SAx_2≤ O(d^2 n) ·Ax_G. By applying this subspace embedding technique, we show an approximation algorithm for the regression problem _x∈R^dAx-b_G, up to a O(d^2 n) factor. As a further application of our techniques, we show how to also use them to improve on the algorithm for the ℓ_p low rank matrix approximation problem for 1≤ p<2.


page 1

page 2

page 3

page 4


Low-rank binary matrix approximation in column-sum norm

We consider ℓ_1-Rank-r Approximation over GF(2), where for a binary m× n...

Average Case Column Subset Selection for Entrywise ℓ_1-Norm Loss

We study the column subset selection problem with respect to the entrywi...

Sketching for Kronecker Product Regression and P-splines

TensorSketch is an oblivious linear sketch introduced in Pagh'13 and lat...

An homotopy method for ℓ_p regression provably beyond self-concordance and in input-sparsity time

We consider the problem of linear regression where the ℓ_2^n norm loss (...

Online Supervised Subspace Tracking

We present a framework for supervised subspace tracking, when there are ...

Efficient Symmetric Norm Regression via Linear Sketching

We provide efficient algorithms for overconstrained linear regression pr...

Covering vectors by spaces: Regular matroids

Seymour's decomposition theorem for regular matroids is a fundamental re...

Please sign up or login with your details

Forgot password? Click here to reset