Adaptive learning of density ratios in RKHS

by   Werner Zellinger, et al.
Johannes Kepler University
Austrian Academy of Sciences

Estimating the ratio of two probability densities from finitely many observations of the densities is a central problem in machine learning and statistics with applications in two-sample testing, divergence estimation, generative modeling, covariate shift adaptation, conditional density estimation, and novelty detection. In this work, we analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space (RKHS). We derive new finite-sample error bounds, and we propose a Lepskii type parameter choice principle that minimizes the bounds without knowledge of the regularity of the density ratio. In the special case of quadratic loss, our method adaptively achieves a minimax optimal error rate. A numerical illustration is provided.


page 1

page 2

page 3

page 4


f-divergence estimation and two-sample homogeneity test under semiparametric density-ratio models

A density ratio is defined by the ratio of two probability densities. We...

Kernel Conditional Density Operators

We introduce a conditional density estimation model termed the condition...

Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation

The estimation of the ratio of two probability densities has garnered at...

Density Ratio Estimation and Neyman Pearson Classification with Missing Data

Density Ratio Estimation (DRE) is an important machine learning techniqu...

Density-Difference Estimation

We address the problem of estimating the difference between two probabil...

Wald-Kernel: Learning to Aggregate Information for Sequential Inference

Sequential hypothesis testing is a desirable decision making strategy in...

Inverse Density as an Inverse Problem: The Fredholm Equation Approach

In this paper we address the problem of estimating the ratio q/p where p...

Please sign up or login with your details

Forgot password? Click here to reset