Improved Kernel Alignment Regret Bound for Online Kernel Learning

by   Junfan Li, et al.
Tianjin University

In this paper, we improve the kernel alignment regret bound for online kernel learning in the regime of the Hinge loss function. Previous algorithm achieves a regret of O((𝒜_TTlnT)^1/4) at a computational complexity (space and per-round time) of O(√(𝒜_TTlnT)), where 𝒜_T is called kernel alignment. We propose an algorithm whose regret bound and computational complexity are better than previous results. Our results depend on the decay rate of eigenvalues of the kernel matrix. If the eigenvalues of the kernel matrix decay exponentially, then our algorithm enjoys a regret of O(√(𝒜_T)) at a computational complexity of O(ln^2T). Otherwise, our algorithm enjoys a regret of O((𝒜_TT)^1/4) at a computational complexity of O(√(𝒜_TT)). We extend our algorithm to batch learning and obtain a O(1/T√(𝔼[𝒜_T])) excess risk bound which improves the previous O(1/√(T)) bound.


page 1

page 2

page 3

page 4

∙ 06/14/2023

Nearly Optimal Algorithms with Sublinear Computational Complexity for Online Kernel Regression

The trade-off between regret and computational cost is a fundamental pro...
∙ 03/09/2023

Improved Regret Bounds for Online Kernel Selection under Bandit Feedback

In this paper, we improve the regret bound for online kernel selection u...
∙ 03/20/2021

Projection-free Distributed Online Learning with Strongly Convex Losses

To efficiently solve distributed online learning problems with complicat...
∙ 06/15/2017

Second-Order Kernel Online Convex Optimization with Adaptive Sketching

Kernel online convex optimization (KOCO) is a framework combining the ex...
∙ 05/25/2023

Fast Online Node Labeling for Very Large Graphs

This paper studies the online node classification problem under a transd...
∙ 02/27/2018

Online learning with kernel losses

We present a generalization of the adversarial linear bandits framework,...
∙ 02/09/2021

Graph-Aided Online Multi-Kernel Learning

Multi-kernel learning (MKL) has been widely used in function approximati...

Please sign up or login with your details

Forgot password? Click here to reset