Dynamic Connected Neural Decision Classifier and Regressor with Dynamic Softing Pruning
share
In the regression problem, L1, L2 are the most commonly-used loss functions, which produce mean predictions with different biases. However, the predictions are neither robust nor adequate enough since they only capture a few conditional distributions instead of the whole distribution, especially for small datasets. To address this problem, we utilized arbitrary quantile modeling to regulate the prediction, which achieved better performance compared to traditional loss functions. More specifically, a new distribution regression method, Deep Distribution Regression (DDR), is proposed to estimate arbitrary quantiles of the response variable. The DDR consists of a Q model, which predicts the corresponding value for arbitrary quantile, and an F model, which predicts the corresponding quantile for arbitrary value. Experiments demonstrate that joint training of these two models outperforms previous methods like AdaBoost, random forest, LightGBM and neural networks both in terms of mean and quantile prediction.
READ FULL TEXT