SAR-to-Optical Image Translation via Thermodynamics-inspired Network

by   Mingjin Zhang, et al.

Synthetic aperture radar (SAR) is prevalent in the remote sensing field but is difficult to interpret in human visual perception. Recently, SAR-to-optical (S2O) image conversion methods have provided a prospective solution for interpretation. However, since there is a huge domain difference between optical and SAR images, they suffer from low image quality and geometric distortion in the produced optical images. Motivated by the analogy between pixels during the S2O image translation and molecules in a heat field, Thermodynamics-inspired Network for SAR-to-Optical Image Translation (S2O-TDN) is proposed in this paper. Specifically, we design a Third-order Finite Difference (TFD) residual structure in light of the TFD equation of thermodynamics, which allows us to efficiently extract inter-domain invariant features and facilitate the learning of the nonlinear translation mapping. In addition, we exploit the first law of thermodynamics (FLT) to devise an FLT-guided branch that promotes the state transition of the feature values from the unstable diffusion state to the stable one, aiming to regularize the feature diffusion and preserve image structures during S2O image translation. S2O-TDN follows an explicit design principle derived from thermodynamic theory and enjoys the advantage of explainability. Experiments on the public SEN1-2 dataset show the advantages of the proposed S2O-TDN over the current methods with more delicate textures and higher quantitative results.


page 1

page 3

page 7


Reciprocal Translation between SAR and Optical Remote Sensing Images with Cascaded-Residual Adversarial Networks

Despite the advantages of all-weather and all-day high-resolution imagin...

Improved Flood Insights: Diffusion-Based SAR to EO Image Translation

Driven by rapid climate change, the frequency and intensity of flood eve...

SAR2EO: A High-resolution Image Translation Framework with Denoising Enhancement

Synthetic Aperture Radar (SAR) to electro-optical (EO) image translation...

A Strategy Optimized Pix2pix Approach for SAR-to-Optical Image Translation Task

This technical report summarizes the analysis and approach on the image-...

SAR-to-EO Image Translation with Multi-Conditional Adversarial Networks

This paper explores the use of multi-conditional adversarial networks fo...

1st Place Solution to MultiEarth 2023 Challenge on Multimodal SAR-to-EO Image Translation

The Multimodal Learning for Earth and Environment Workshop (MultiEarth 2...

Image-to-image Translation as a Unique Source of Knowledge

Image-to-image (I2I) translation is an established way of translating da...

Please sign up or login with your details

Forgot password? Click here to reset