b'Herv\xc3\xa9 J\xc3\xa9gou'

research

∙ 06/01/2023

Birth of a Transformer: A Memory Viewpoint

Large language models based on transformers have achieved great empirica...

0 Alberto Bietti, et al. ∙

research

∙ 04/14/2023

DINOv2: Learning Robust Visual Features without Supervision

The recent breakthroughs in natural language processing for model pretra...

1 Maxime Oquab, et al. ∙

research

∙ 03/27/2023

The Stable Signature: Rooting Watermarks in Latent Diffusion Models

Generative image modeling enables a wide range of applications but raise...

0 Pierre Fernandez, et al. ∙

research

∙ 01/26/2023

Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models

Lossy image compression aims to represent images in as few bits as possi...

0 Matthew J. Muckley, et al. ∙

research

∙ 12/14/2022

Image Compression with Product Quantized Masked Image Modeling

Recent neural compression methods have been based on the popular hyperpr...

0 Alaaeldin El-Nouby, et al. ∙

research

∙ 12/09/2022

Co-training 2^L Submodels for Visual Recognition

We introduce submodel co-training, a regularization method related to co...

4 Hugo Touvron, et al. ∙

research

∙ 10/05/2022

Active Image Indexing

Image copy detection and retrieval from large databases leverage two com...

0 Pierre Fernandez, et al. ∙

research

∙ 04/14/2022

DeiT III: Revenge of the ViT

A Vision Transformer (ViT) is a simple neural architecture amenable to s...

21 Hugo Touvron, et al. ∙

research

∙ 03/18/2022

Three things everyone should know about Vision Transformers

After their initial success in natural language processing, transformer ...

21 Hugo Touvron, et al. ∙

research

∙ 12/27/2021

Augmenting Convolutional networks with attention-based aggregation

We show how to augment any convolutional network with an attention-based...

24 Hugo Touvron, et al. ∙

research

∙ 12/20/2021

Are Large-scale Datasets Necessary for Self-Supervised Pre-training?

Pre-training models on large scale datasets, like ImageNet, is a standar...

25 Alaaeldin El-Nouby, et al. ∙

research

∙ 12/17/2021

Watermarking Images in Self-Supervised Latent Spaces

We revisit watermarking techniques based on pre-trained deep networks, i...

0 Pierre Fernandez, et al. ∙

research

∙ 12/17/2021

Nearest neighbor search with compact codes: A decoder perspective

Modern approaches for fast retrieval of similar vectors on billion-scale...

0 Kenza Amara, et al. ∙

research

∙ 10/01/2021

ResNet strikes back: An improved training procedure in timm

The influential Residual Networks designed by He et al. remain the gold-...

0 Ross Wightman, et al. ∙

research

∙ 06/17/2021

XCiT: Cross-Covariance Image Transformers

Following their success in natural language processing, transformers hav...

0 Alaaeldin El-Nouby, et al. ∙

research

∙ 05/07/2021

ResMLP: Feedforward networks for image classification with data-efficient training

We present ResMLP, an architecture built entirely upon multi-layer perce...

43 Hugo Touvron, et al. ∙

research

∙ 04/29/2021

Emerging Properties in Self-Supervised Vision Transformers

In this paper, we question if self-supervised learning provides new prop...

12 Mathilde Caron, et al. ∙

research

∙ 04/15/2021

Gradient-based Adversarial Attacks against Text Transformers

We propose the first general-purpose gradient-based attack against trans...

0 Chuan Guo, et al. ∙

research

∙ 04/02/2021

LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

We design a family of image classification architectures that optimize t...

0 Ben Graham, et al. ∙

research

∙ 03/31/2021

Going deeper with Image Transformers

Transformers have been recently adapted for large scale image classifica...

0 Hugo Touvron, et al. ∙

research

∙ 02/10/2021

Training Vision Transformers for Image Retrieval

Transformers have shown outstanding results for natural language underst...

0 Alaaeldin El-Nouby, et al. ∙

research

∙ 12/23/2020

Training data-efficient image transformers distillation through attention

Recently, neural networks purely based on attention were shown to addres...

15 Hugo Touvron, et al. ∙

research

∙ 11/25/2020

Grafit: Learning fine-grained image representations with coarse labels

This paper tackles the problem of learning a finer representation than t...

0 Hugo Touvron, et al. ∙

research

∙ 08/13/2020

Powers of layers for image-to-image translation

We propose a simple architecture to address unpaired image-to-image tran...

10 Hugo Touvron, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Model Compression

We tackle the problem of producing compact models, maximizing their accu...

30 Angela Fan, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Fixed-Point Compression

We tackle the problem of producing compact models, maximizing their accu...

3 Angela Fan, et al. ∙

research

∙ 03/18/2020

Fixing the train-test resolution discrepancy: FixEfficientNet

This note complements the paper "Fixing the train-test resolution discre...

16 Hugo Touvron, et al. ∙

research

∙ 02/03/2020

Radioactive data: tracing through training

We want to detect whether a particular image dataset has been used to tr...

2 Alexandre Sablayrolles, et al. ∙

research

∙ 08/29/2019

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

Membership inference determines, given a sample and trained parameters o...

12 Alexandre Sablayrolles, et al. ∙

research

∙ 07/12/2019

And the Bit Goes Down: Revisiting the Quantization of Neural Networks

In this paper, we address the problem of reducing the memory footprint o...

1 Pierre Stock, et al. ∙

research

∙ 07/10/2019

Large Memory Layers with Product Keys

This paper introduces a structured memory which can be easily integrated...

0 Guillaume Lample, et al. ∙

research

∙ 07/02/2019

Augmenting Self-attention with Persistent Memory

Transformer networks have lead to important progress in language modelin...

6 Sainbayar Sukhbaatar, et al. ∙

research

∙ 06/14/2019

Fixing the train-test resolution discrepancy

Data-augmentation is key to the training of neural networks for image cl...

0 Hugo Touvron, et al. ∙

research

∙ 05/02/2019

Billion-scale semi-supervised learning for image classification

This paper presents a study of semi-supervised learning with large convo...

0 I. Zeki Yalniz, et al. ∙

research

∙ 02/27/2019

Equi-normalization of Neural Networks

Modern neural networks are over-parametrized. In particular, each rectif...

4 Pierre Stock, et al. ∙

research

∙ 02/14/2019

MultiGrain: a unified image embedding for classes and instances

MultiGrain is a network architecture producing compact vector representa...

0 Maxim Berman, et al. ∙

research

∙ 11/27/2018

Understanding and Improving Kernel Local Descriptors

We propose a multiple-kernel local-patch descriptor based on efficient m...

0 Arun Mukundan, et al. ∙

research

∙ 09/17/2018

Déjà Vu: an empirical evaluation of the memorization properties of ConvNets

Convolutional neural networks memorize part of their training data, whic...

0 Alexandre Sablayrolles, et al. ∙

research

∙ 06/08/2018

A neural network catalyzer for multi-dimensional similarity search

This paper aims at learning a function mapping input vectors to an outpu...

0 Alexandre Sablayrolles, et al. ∙

research

∙ 04/26/2018

Link and code: Fast indexing with graphs and compact regression codes

Similarity search approaches based on graph walks have recently attained...

0 MMatthijs Douze, et al. ∙

research

∙ 10/11/2017

Word Translation Without Parallel Data

State-of-the-art methods for learning cross-lingual word embeddings have...

0 Alexis Conneau, et al. ∙

research

∙ 08/09/2017

An evaluation of large-scale methods for image instance and class discovery

This paper aims at discovering meaningful subsets of related images from...

0 Matthijs Douze, et al. ∙

research

∙ 06/07/2017

Low-shot learning with large-scale diffusion

This paper considers the problem of inferring image labels for which onl...

0 Matthijs Douze, et al. ∙

research

∙ 02/28/2017

Billion-scale similarity search with GPUs

Similarity search finds application in specialized database systems hand...

0 Jeff Johnson, et al. ∙

research

∙ 12/12/2016

FastText.zip: Compressing text classification models

We consider the problem of producing compact architectures for text clas...

0 Armand Joulin, et al. ∙

research

∙ 11/24/2016

Interferences in match kernels

We consider the design of an image representation that embeds and aggreg...

0 Naila Murray, et al. ∙

research

∙ 09/21/2016

How should we evaluate supervised hashing?

Hashing produces compact representations for documents, to perform tasks...

0 Alexandre Sablayrolles, et al. ∙

research

∙ 09/14/2016

Efficient softmax approximation for GPUs

We propose an approximate strategy to efficiently train neural network b...

0 Edouard Grave, et al. ∙

research

∙ 09/07/2016

Polysemous codes

This paper considers the problem of approximate nearest neighbor search ...

0 Matthijs Douze, et al. ∙

research

∙ 08/10/2016

Approximate search with quantized sparse representations

This paper tackles the task of storing a large collection of vectors, su...

0 Himalaya Jain, et al. ∙

Hervé Jégou

Featured Co-authors