Unlocking Slot Attention by Changing Optimal Transport Costs

by   Yan Zhang, et al.

Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to handle videos with a dynamic number of objects because it cannot break ties. To overcome this limitation, we first establish a connection between slot attention and optimal transport. Based on this new perspective we propose MESH (Minimize Entropy of Sinkhorn): a cross-attention module that combines the tiebreaking properties of unregularized optimal transport with the speed of regularized optimal transport. We evaluate slot attention using MESH on multiple object-centric learning benchmarks and find significant improvements over slot attention in every setting.


page 7

page 15

page 16

page 17

page 18

page 19


Entropy Regularized Optimal Transport Independence Criterion

Optimal transport (OT) and its entropy regularized offspring have recent...

Weak Limits for Empirical Entropic Optimal Transport: Beyond Smooth Costs

We establish weak limits for the empirical entropy regularized optimal t...

Efficient and Accurate Optimal Transport with Mirror Descent and Conjugate Gradients

We design a novel algorithm for optimal transport by drawing from the en...

When Optimal Transport Meets Information Geometry

Information geometry and optimal transport are two distinct geometric fr...

MT-Net Submission to the Waymo 3D Detection Leaderboard

In this technical report, we introduce our submission to the Waymo 3D De...

Remote measurement of sea ice dynamics with regularized optimal transport

As Arctic conditions rapidly change, human activity in the Arctic will c...

Egalitarian and Congestion Aware Truthful Airport Slot Allocation Mechanism

We propose a mechanism to allocate slots fairly at congested airports. T...

Please sign up or login with your details

Forgot password? Click here to reset