A Systematic Performance Analysis of Deep Perceptual Loss Networks Breaks Transfer Learning Conventions

by   Gustav Grund Pihlgren, et al.

Deep perceptual loss is a type of loss function in computer vision that aims to mimic human perception by using the deep features extracted from neural networks. In recent years the method has been applied to great effect on a host of interesting computer vision tasks, especially for tasks with image or image-like outputs. Many applications of the method use pretrained networks, often convolutional networks, for loss calculation. Despite the increased interest and broader use, more effort is needed toward exploring which networks to use for calculating deep perceptual loss and from which layers to extract the features. This work aims to rectify this by systematically evaluating a host of commonly used and readily available, pretrained networks for a number of different feature extraction points on four existing use cases of deep perceptual loss. The four use cases are implementations of previous works where the selected networks and extraction points are evaluated instead of the networks and extraction points used in the original work. The experimental tasks are dimensionality reduction, image segmentation, super-resolution, and perceptual similarity. The performance on these four tasks, attributes of the networks, and extraction points are then used as a basis for an in-depth analysis. This analysis uncovers essential information regarding which architectures provide superior performance for deep perceptual loss and how to choose an appropriate extraction point for a particular task and dataset. Furthermore, the work discusses the implications of the results for deep perceptual loss and the broader field of transfer learning. The results break commonly held assumptions in transfer learning, which imply that deep perceptual loss deviates from most transfer learning settings or that these assumptions need a thorough re-evaluation.


Using Human Perception to Regularize Transfer Learning

Recent trends in the machine learning community show that models with fi...

Enhancing Perceptual Loss with Adversarial Feature Matching for Super-Resolution

Single image super-resolution (SISR) is an ill-posed problem with an ind...

Improving Image Autoencoder Embeddings with Perceptual Loss

Autoencoders are commonly trained using element-wise loss. However, elem...

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

We consider image transformation problems, where an input image is trans...

A residual dense vision transformer for medical image super-resolution with segmentation-based perceptual loss fine-tuning

Super-resolution plays an essential role in medical imaging because it p...

Learning Implicit Generative Models by Matching Perceptual Features

Perceptual features (PFs) have been used with great success in tasks suc...

Trainable Loss Weights in Super-Resolution

In recent years, research on super-resolution has primarily focused on t...

Please sign up or login with your details

Forgot password? Click here to reset