research
∙
09/01/2023
Structure and Gradient Dynamics Near Global Minima of Two-layer Neural Networks
Under mild assumptions, we investigate the structure of loss landscape o...
share
research
∙
07/18/2023
Optimistic Estimate Uncovers the Potential of Nonlinear Models
We propose an optimistic estimate to evaluate the best possible fitting ...
research
∙
11/21/2022
Linear Stability Hypothesis and Rank Stratification for Nonlinear Models
Models with nonlinear architectures/parameterizations such as deep neura...
research
∙
05/26/2022
Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks
Unraveling the general structure underlying the loss landscapes of deep ...
research
∙
05/24/2022
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width
Substantial work indicates that the dynamics of neural networks (NNs) is...
research
∙
01/28/2022
Limitation of characterizing implicit regularization by data-independent functions
In recent years, understanding the implicit regularization of neural net...
research
∙
01/19/2022
Overview frequency principle/spectral bias in deep learning
Understanding deep learning is increasingly emergent as it penetrates mo...
research
∙
01/09/2022
A multi-scale sampling method for accurate and robust deep neural network to predict combustion chemical kinetics
Machine learning has long been considered as a black box for predicting ...
research
∙
01/06/2022
A deep learning-based model reduction (DeePMR) method for simplifying chemical kinetics
A deep learning-based model reduction (DeePMR) method for simplifying ch...
research
∙
11/30/2021
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks
We prove a general Embedding Principle of loss landscape of deep neural ...
research
∙
07/17/2021
Data-informed Deep Optimization
Complex design problems are common in the scientific and industrial fiel...
research
∙
07/08/2021
MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
In this paper, we propose a model-operator-data network (MOD-Net) for so...
research
∙
05/30/2021
Embedding Principle of Loss Landscape of Deep Neural Networks
Understanding the structure of loss landscape of deep neural networks (D...
research
∙
05/25/2021
Towards Understanding the Condensation of Two-layer Neural Networks at Initial Training
It is important to study what implicit regularization is imposed on the ...
research
∙
05/25/2021
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network
Deep neural network (DNN) usually learns the target function from low to...
research
∙
01/30/2021
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Why heavily parameterized neural networks (NNs) do not overfit the data ...
research
∙
12/06/2020
Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning
A supervised learning problem is to find a function in a hypothesis func...
research
∙
10/15/2020
On the exact computation of linear frequency principle dynamics and its generalization
Recent works show an intriguing phenomenon of Frequency Principle (F-Pri...
research
∙
07/15/2020
Phase diagram for two-layer ReLU neural networks at infinite-width limit
How neural network behaves during the training over different choices of...
research
∙
02/19/2020
Implicit bias with Ritz-Galerkin method in understanding deep learning for solving PDEs
This paper aims at studying the difference between Ritz-Galerkin (R-G) m...
research
∙
12/06/2019
A priori generalization error for two-layer ReLU neural network through minimum norm solution
We focus on estimating a priori generalization error of two-layer ReLU n...
research
∙
06/21/2019
Theory of the Frequency Principle for General Deep Neural Networks
Along with fruitful applications of Deep Neural Networks (DNNs) to reali...
research
∙
05/24/2019
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
It remains a puzzle that why deep neural networks (DNNs), with more para...
research
∙
05/19/2019
A type of generalization error induced by initialization in deep neural networks
How different initializations and loss functions affect the learning of ...
research
∙
01/19/2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
We study the training process of Deep Neural Networks (DNNs) from the Fo...
research
∙
07/03/2018