Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses

01/31/2017
by   Eric Risser, et al.
0

Recently, methods have been proposed that perform texture synthesis and style transfer by using convolutional neural networks (e.g. Gatys et al. [2015,2016]). These methods are exciting because they can in some cases create results with state-of-the-art quality. However, in this paper, we show these methods also have limitations in texture quality, stability, requisite parameter tuning, and lack of user controls. This paper presents a multiscale synthesis pipeline based on convolutional neural networks that ameliorates these issues. We first give a mathematical explanation of the source of instabilities in many previous approaches. We then improve these instabilities by using histogram losses to synthesize textures that better statistically match the exemplar. We also show how to integrate localized style losses in our multiscale framework. These losses can improve the quality of large features, improve the separation of content and style, and offer artistic controls such as paint by numbers. We demonstrate that our approach offers improved quality, convergence in fewer iterations, and more stability over the optimization.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 7

page 10

page 13

research
09/10/2016

Style-Transfer via Texture-Synthesis

Style-transfer is a process of migrating a style from a given image to t...
research
03/24/2022

Neural Neighbor Style Transfer

We propose Neural Neighbor Style Transfer (NNST), a pipeline that offers...
research
10/28/2020

Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport

This paper presents a light-weight, high-quality texture synthesis algor...
research
06/12/2020

Pitfalls of the Gram Loss for Neural Texture Synthesis in Light of Deep Feature Histograms

Neural texture synthesis and style transfer are both powered by the Gram...
research
11/17/2016

Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer

Transferring artistic styles onto everyday photographs has become an ext...
research
12/03/2019

Maximum entropy methods for texture synthesis: theory and practice

Recent years have seen the rise of convolutional neural network techniqu...
research
06/21/2017

Two-Stream Convolutional Networks for Dynamic Texture Synthesis

We introduce a two-stream model for dynamic texture synthesis. Our model...

Please sign up or login with your details

Forgot password? Click here to reset