Reformulating van Rijsbergen's F_β metric for weighted binary cross-entropy
share
The separation of performance metrics from gradient based loss functions may not always give optimal results and may miss vital aggregate information. This paper investigates incorporating a performance metric alongside differentiable loss functions to inform training outcomes. The goal is to guide model performance and interpretation by assuming statistical distributions on this performance metric for dynamic weighting. The focus is on van Rijsbergens F_β metric – a popular choice for gauging classification performance. Through distributional assumptions on the F_β, an intermediary link can be established to the standard binary cross-entropy via dynamic penalty weights. First, the F_β metric is reformulated to facilitate assuming statistical distributions with accompanying proofs for the cumulative density function. These probabilities are used within a knee curve algorithm to find an optimal β or β_opt. This β_opt is used as a weight or penalty in the proposed weighted binary cross-entropy. Experimentation on publicly available data with imbalanced classes mostly yields better and interpretable results as compared to the baseline. For example, for the IMDB text data with known labeling errors, a 14 can accelerate training and provide better interpretation.
READ FULL TEXT