In Defense of the Unitary Scalarization for Deep Multi-Task Learning

by   Vitaly Kurin, et al.

Recent multi-task learning research argues against unitary scalarization, where training simply minimizes the sum of the task losses. Several ad-hoc multi-task optimization algorithms have instead been proposed, inspired by various hypotheses about what makes multi-task settings difficult. The majority of these optimizers require per-task gradients, and introduce significant memory, runtime, and implementation overhead. We present a theoretical analysis suggesting that many specialized multi-task optimizers can be interpreted as forms of regularization. Moreover, we show that, when coupled with standard regularization and stabilization techniques from single-task learning, unitary scalarization matches or improves upon the performance of complex multi-task optimizers in both supervised and reinforcement learning settings. We believe our results call for a critical reevaluation of recent research in the area.


Multi-task Learning for Continuous Control

Reliable and effective multi-task learning is a prerequisite for the dev...

Do Current Multi-Task Optimization Methods in Deep Learning Even Help?

Recent research has proposed a series of specialized optimization algori...

Frustratingly Simple Entity Tracking with Effective Use of Multi-Task Learning Models

We present SET, a frustratingly Simple-yet-effective approach for Entity...

Distributed Stochastic Multi-Task Learning with Graph Regularization

We propose methods for distributed graph-based multi-task learning that ...

Multi-Task Learning as a Bargaining Game

In Multi-task learning (MTL), a joint model is trained to simultaneously...

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

Deep neural networks are a promising approach towards multi-task learnin...

Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise

We propose an end-to-end unified 3D mesh recovery of humans and quadrupe...

Please sign up or login with your details

Forgot password? Click here to reset