Transfer of View-manifold Learning to Similarity Perception of Novel Objects

by   Xingyu Lin, et al.

We develop a model of perceptual similarity judgment based on re-training a deep convolution neural network (DCNN) that learns to associate different views of each 3D object to capture the notion of object persistence and continuity in our visual experience. The re-training process effectively performs distance metric learning under the object persistency constraints, to modify the view-manifold of object representations. It reduces the effective distance between the representations of different views of the same object without compromising the distance between those of the views of different objects, resulting in the untangling of the view-manifolds between individual objects within the same category and across categories. This untangling enables the model to discriminate and recognize objects within the same category, independent of viewpoints. We found that this ability is not limited to the trained objects, but transfers to novel objects in both trained and untrained categories, as well as to a variety of completely novel artificial synthetic objects. This transfer in learning suggests the modification of distance metrics in view- manifolds is more general and abstract, likely at the levels of parts, and independent of the specific objects or categories experienced during training. Interestingly, the resulting transformation of feature representation in the deep networks is found to significantly better match human perceptual similarity judgment than AlexNet, suggesting that object persistence could be an important constraint in the development of perceptual similarity judgment in biological neural networks.


Statistical Mechanics of Neural Processing of Object Manifolds

Invariant object recognition is one of the most fundamental cognitive ta...

Embodied vision for learning object representations

Recent time-contrastive learning approaches manage to learn invariant ob...

Jointly Learning Multiple Measures of Similarities from Triplet Comparisons

Similarity between objects is multi-faceted and it can be easier for hum...

3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation

Text-guided 3D object generation aims to generate 3D objects described b...

Classification and Geometry of General Perceptual Manifolds

Perceptual manifolds arise when a neural population responds to an ensem...

Revealing interpretable object representations from human behavior

To study how mental object representations are related to behavior, we e...

What does it mean to understand a neural network?

We can define a neural network that can learn to recognize objects in le...

Please sign up or login with your details

Forgot password? Click here to reset