Self-supervised Secondary Landmark Detection via 3D Representation Learning

by   Praneet C. Bala, et al.

Recent technological developments have spurred great advances in the computerized tracking of joints and other landmarks in moving animals, including humans. Such tracking promises important advances in biology and biomedicine. Modern tracking models depend critically on labor-intensive annotated datasets of primary landmarks by non-expert humans. However, such annotation approaches can be costly and impractical for secondary landmarks, that is, ones that reflect fine-grained geometry of animals, and that are often specific to customized behavioral tasks. Due to visual and geometric ambiguity, nonexperts are often not qualified for secondary landmark annotation, which can require anatomical and zoological knowledge. These barriers significantly impede downstream behavioral studies because the learned tracking models exhibit limited generalizability. We hypothesize that there exists a shared representation between the primary and secondary landmarks because the range of motion of the secondary landmarks can be approximately spanned by that of the primary landmarks. We present a method to learn this spatial relationship of the primary and secondary landmarks in three dimensional space, which can, in turn, self-supervise the secondary landmark detector. This 3D representation learning is generic, and can therefore be applied to various multiview settings across diverse organisms, including macaques, flies, and humans.


page 1

page 2

page 3

page 4

page 5

page 6

page 8


Improved ASR for Under-Resourced Languages Through Multi-Task Learning with Acoustic Landmarks

Furui first demonstrated that the identity of both consonant and vowel c...

Discovering Salient Anatomical Landmarks by Predicting Human Gaze

Anatomical landmarks are a crucial prerequisite for many medical imaging...

LEAD: Self-Supervised Landmark Estimation by Aligning Distributions of Feature Similarity

In this work, we introduce LEAD, an approach to discover landmarks from ...

Coordinated Transformer with Position & Sample-aware Central Loss for Anatomical Landmark Detection

Heatmap-based anatomical landmark detection is still facing two unresolv...

Cross-Task Representation Learning for Anatomical Landmark Detection

Recently, there is an increasing demand for automatically detecting anat...

Algorithms to automatically quantify the geometric similarity of anatomical surfaces

We describe new approaches for distances between pairs of 2-dimensional ...

Self-supervised learning: When is fusion of the primary and secondary sensor cue useful?

Self-supervised learning (SSL) is a reliable learning mechanism in which...

Please sign up or login with your details

Forgot password? Click here to reset