Adaptive Transform Domain Image Super-resolution Via Orthogonally Regularized Deep Networks

by   Tiantong Guo, et al.

Deep learning methods, in particular, trained Convolutional Neural Networks (CNN) have recently been shown to produce compelling results for single image Super-Resolution (SR). Invariably, a CNN is learned to map the Low Resolution (LR) image to its corresponding High Resolution (HR) version in the spatial domain. We propose a novel network structure for learning the SR mapping function in an image transform domain, specifically the Discrete Cosine Transform (DCT). As the first contribution, we show that DCT can be integrated into the network structure as a Convolutional DCT (CDCT) layer. With the CDCT layer, we construct the DCT Deep SR (DCT-DSR) network. We further extend the DCT-DSR to allow the CDCT layer to become trainable (i.e., optimizable). Because this layer represents an image transform, we enforce pairwise orthogonality constraints and newly formulated complexity order constraints on the individual basis functions/filters. This Orthogonally Regularized Deep SR network (ORDSR) simplifies the SR task by taking advantage of image transform domain while adapting the design of transform basis to the training image set. Experimental results show ORDSR achieves state-of-the-art SR image quality with fewer parameters than most of the deep CNN methods. A particular success of ORDSR is in overcoming the artifacts introduced by bicubic interpolation. A key burden of deep SR has been identified as the requirement of generous training LR and HR image pairs; ORSDR exhibits a much more graceful degradation as training size is reduced with significant benefits in the regime of limited training. Analysis of memory and computation requirements confirms that ORDSR can allow for a more efficient network with faster inference.


page 1

page 3

page 5

page 8

page 9

page 12

page 13


Orthogonally Regularized Deep Networks For Image Super-resolution

Deep learning methods, in particular trained Convolutional Neural Networ...

Tchebichef Transform Domain-based Deep Learning Architecture for Image Super-resolution

The recent outbreak of COVID-19 has motivated researchers to contribute ...

CT-SRCNN: Cascade Trained and Trimmed Deep Convolutional Neural Networks for Image Super Resolution

We propose methodologies to train highly accurate and efficient deep con...

Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform

Despite that convolutional neural networks (CNN) have recently demonstra...

Can Maxout Units Downsize Restoration Networks? - Single Image Super-Resolution Using Lightweight CNN with Maxout Units

Rectified linear units (ReLU) are well-known to be helpful in obtaining ...

Local- and Holistic- Structure Preserving Image Super Resolution via Deep Joint Component Learning

Recently, machine learning based single image super resolution (SR) appr...

End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks

One impressive advantage of convolutional neural networks (CNNs) is thei...

Please sign up or login with your details

Forgot password? Click here to reset