Rectifying Noisy Labels with Sequential Prior: Multi-Scale Temporal Feature Affinity Learning for Robust Video Segmentation

by   Beilei Cui, et al.
National University of Singapore
The Chinese University of Hong Kong

Noisy label problems are inevitably in existence within medical image segmentation causing severe performance degradation. Previous segmentation methods for noisy label problems only utilize a single image while the potential of leveraging the correlation between images has been overlooked. Especially for video segmentation, adjacent frames contain rich contextual information beneficial in cognizing noisy labels. Based on two insights, we propose a Multi-Scale Temporal Feature Affinity Learning (MS-TFAL) framework to resolve noisy-labeled medical video segmentation issues. First, we argue the sequential prior of videos is an effective reference, i.e., pixel-level features from adjacent frames are close in distance for the same class and far in distance otherwise. Therefore, Temporal Feature Affinity Learning (TFAL) is devised to indicate possible noisy labels by evaluating the affinity between pixels in two adjacent frames. We also notice that the noise distribution exhibits considerable variations across video, image, and pixel levels. In this way, we introduce Multi-Scale Supervision (MSS) to supervise the network from three different perspectives by re-weighting and refining the samples. This design enables the network to concentrate on clean samples in a coarse-to-fine manner. Experiments with both synthetic and real-world label noise demonstrate that our method outperforms recent state-of-the-art robust segmentation approaches. Code is available at


page 8

page 12

page 13


Joint Class-Affinity Loss Correction for Robust Medical Image Segmentation with Noisy Labels

Noisy labels collected with limited annotation cost prevent medical imag...

Superpixel-guided Iterative Learning from Noisy Labels for Medical Image Segmentation

Learning segmentation from noisy labels is an important task for medical...

Model and Data Agreement for Learning with Noisy Labels

Learning with noisy labels is a vital topic for practical deep learning ...

Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction

This paper presents a unified training strategy that enables a novel mul...

STRAP: Structured Object Affordance Segmentation with Point Supervision

With significant annotation savings, point supervision has been proven e...

Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation

The essence of video semantic segmentation (VSS) is how to leverage temp...

Adaptive Fusion Affinity Graph with Noise-free Online Low-rank Representation for Natural Image Segmentation

Affinity graph-based segmentation methods have become a major trend in c...

Please sign up or login with your details

Forgot password? Click here to reset