Sharper dimension-free bounds on the Frobenius distance between sample covariance and its expectation

08/28/2023

∙

by   Nikita Puchkin, et al.

∙

0

∙

share

We study properties of a sample covariance estimate Σ= (𝐗_1 𝐗_1^⊤ + … + 𝐗_n 𝐗_n^⊤) / n, where 𝐗_1, …, 𝐗_n are i.i.d. random elements in ℝ^d with 𝔼𝐗_1 = 0, 𝔼𝐗_1 𝐗_1^⊤ = Σ. We derive dimension-free bounds on the squared Frobenius norm of (Σ- Σ) under reasonable assumptions. For instance, we show that | Σ- Σ_ F^2 - 𝔼Σ- Σ_ F^2| = 𝒪(Tr(Σ^2) / n) with overwhelming probability, which is a significant improvement over the existing results. This leads to a bound the ratio Σ- Σ_ F^2 / 𝔼Σ- Σ_ F^2 with a sharp leading constant when the effective rank 𝚛(Σ) = Tr(Σ) / Σ and n / 𝚛(Σ)^6 tend to infinity: Σ- Σ_ F^2 / 𝔼Σ- Σ_ F^2 = 1 + 𝒪(1 / 𝚛(Σ)).