Nonparametric Finite Mixture Models with Possible Shape Constraints: A Cubic Newton Approach

by   Haoyue Wang, et al.

We explore computational aspects of maximum likelihood estimation of the mixture proportions of a nonparametric finite mixture model – a convex optimization problem with old roots in statistics and a key member of the modern data analysis toolkit. Motivated by problems in shape constrained inference, we consider structured variants of this problem with additional convex polyhedral constraints. We propose a new cubic regularized Newton method for this problem and present novel worst-case and local computational guarantees for our algorithm. We extend earlier work by Nesterov and Polyak to the case of a self-concordant objective with polyhedral constraints, such as the ones considered herein. We propose a Frank-Wolfe method to solve the cubic regularized Newton subproblem; and derive efficient solutions for the linear optimization oracles that may be of independent interest. In the particular case of Gaussian mixtures without shape constraints, we derive bounds on how well the finite mixture problem approximates the infinite-dimensional Kiefer-Wolfowitz maximum likelihood estimator. Experiments on synthetic and real datasets suggest that our proposed algorithms exhibit improved runtimes and scalability features over existing benchmarks.


page 1

page 2

page 3

page 4


Cubic-Regularized Newton for Spectral Constrained Matrix Optimization and its Application to Fairness

Matrix functions are utilized to rewrite smooth spectral constrained mat...

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

In this paper we study the computation of the nonparametric maximum like...

A Distributed Cubic-Regularized Newton Method for Smooth Convex Optimization over Networks

We propose a distributed, cubic-regularized Newton method for large-scal...

Partial Gaussian Graphical Model Estimation

This paper studies the partial estimation of Gaussian graphical models f...

Nonparametric estimation of continuous DPPs with kernel methods

Determinantal Point Process (DPPs) are statistical models for repulsive ...

A Fast Algorithm for Maximum Likelihood Estimation of Mixture Proportions Using Sequential Quadratic Programming

Maximum likelihood estimation of mixture proportions has a long history ...

Fusing Multiple Multiband Images

We consider the problem of fusing an arbitrary number of multiband, i.e....

Please sign up or login with your details

Forgot password? Click here to reset