Understanding Self-Paced Learning under Concave Conjugacy Theory

by   Shiqi Liu, et al.

By simulating the easy-to-hard learning manners of humans/animals, the learning regimes called curriculum learning (CL) and self-paced learning (SPL) have been recently investigated and invoked broad interests. However, the intrinsic mechanism for analyzing why such learning regimes can work has not been comprehensively investigated. To this issue, this paper proposes a concave conjugacy theory for looking into the insight of CL/SPL. Specifically, by using this theory, we prove the equivalence of the SPL regime and a latent concave objective, which is closely related to the known non-convex regularized penalty widely used in statistics and machine learning. Beyond the previous theory for explaining CL/SPL insights, this new theoretical framework on one hand facilitates two direct approaches for designing new SPL models for certain tasks, and on the other hand can help conduct the latent objective of self-paced curriculum learning, which is the advanced version of both CL/SPL and possess advantages of both learning regimes to a certain extent. This further facilitates a theoretical understanding for SPCL, instead of only CL/SPL as conventional. Under this theory, we attempt to attain intrinsic latent objectives of two curriculum forms, the partial order and group curriculums, which easily follow the theoretical understanding of the corresponding SPCL regimes.


page 1

page 2

page 3

page 4


An Analytical Theory of Curriculum Learning in Teacher-Student Networks

In humans and animals, curriculum learning – presenting data in a curate...

Curriculum Learning: A Survey

Training machine learning models in a meaningful order, from the easy sa...

Solving Non-Convex Non-Concave Min-Max Games Under Polyak-Łojasiewicz Condition

In this short note, we consider the problem of solving a min-max zero-su...

Theory of Curriculum Learning, with Convex Loss Functions

Curriculum Learning - the idea of teaching by gradually exposing the lea...

AlphaGomoku: An AlphaGo-based Gomoku Artificial Intelligence using Curriculum Learning

In this project, we combine AlphaGo algorithm with Curriculum Learning t...

On Convergence Property of Implicit Self-paced Objective

Self-paced learning (SPL) is a new methodology that simulates the learni...

Please sign up or login with your details

Forgot password? Click here to reset