DiffMatch: Diffusion Model for Dense Matching

by   Jisu Nam, et al.

The objective for establishing dense correspondence between paired images consists of two terms: a data term and a prior term. While conventional techniques focused on defining hand-designed prior terms, which are difficult to formulate, recent approaches have focused on learning the data term with deep neural networks without explicitly modeling the prior, assuming that the model itself has the capacity to learn an optimal prior from a large-scale dataset. The performance improvement was obvious, however, they often fail to address inherent ambiguities of matching, such as textureless regions, repetitive patterns, and large displacements. To address this, we propose DiffMatch, a novel conditional diffusion-based framework designed to explicitly model both the data and prior terms. Unlike previous approaches, this is accomplished by leveraging a conditional denoising diffusion model. DiffMatch consists of two main components: conditional denoising diffusion module and cost injection module. We stabilize the training process and reduce memory usage with a stage-wise training strategy. Furthermore, to boost performance, we introduce an inference technique that finds a better path to the accurate matching field. Our experimental results demonstrate significant performance improvements of our method over existing approaches, and the ablation studies validate our design choices along with the effectiveness of each component. Project page is available at https://ku-cvlab.github.io/DiffMatch/.


page 1

page 5

page 8

page 16

page 17

page 18

page 19

page 20


Diffusion Model for Camouflaged Object Detection

Camouflaged object detection is a challenging task that aims to identify...

Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Conventional techniques to establish dense correspondences across visual...

Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches

Deep learning methods have shown remarkable performance in image denoisi...

On the Design Fundamentals of Diffusion Models: A Survey

Diffusion models are generative models, which gradually add and remove n...

PMatch: Paired Masked Image Modeling for Dense Geometric Matching

Dense geometric matching determines the dense pixel-wise correspondence ...

NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement

The goal of speech enhancement (SE) is to eliminate the background inter...

Hierarchical Integration Diffusion Model for Realistic Image Deblurring

Diffusion models (DMs) have recently been introduced in image deblurring...

Please sign up or login with your details

Forgot password? Click here to reset