Ad-Net: Audio-Visual Convolutional Neural Network for Advertisement Detection In Videos

by   Shervin Minaee, et al.

Personalized advertisement is a crucial task for many of the online businesses and video broadcasters. Many of today's broadcasters use the same commercial for all customers, but as one can imagine different viewers have different interests and it seems reasonable to have customized commercial for different group of people, chosen based on their demographic features, and history. In this project, we propose a framework, which gets the broadcast videos, analyzes them, detects the commercial and replaces it with a more suitable commercial. We propose a two-stream audio-visual convolutional neural network, that one branch analyzes the visual information and the other one analyzes the audio information, and then the audio and visual embedding are fused together, and are used for commercial detection, and content categorization. We show that using both the visual and audio content of the videos significantly improves the model performance for video analysis. This network is trained on a dataset of more than 50k regular video and commercial shots, and achieved much better performance compared to the models based on hand-crafted features.


page 1

page 2

page 3


Role of Audio in Audio-Visual Video Summarization

Video summarization attracts attention for efficient video representatio...

An audio-only method for advertisement detection in broadcast television content

We address the task of advertisement detection in broadcast television c...

Multimodal Content Analysis for Effective Advertisements on YouTube

The rapid advances in e-commerce and Web 2.0 technologies have greatly i...

Detection of Audio-Video Synchronization Errors Via Event Detection

We present a new method and a large-scale database to detect audio-video...

VPN: Video Provenance Network for Robust Content Attribution

We present VPN - a content attribution method for recovering provenance ...

Audio Spectrogram Representations for Processing with Convolutional Neural Networks

One of the decisions that arise when designing a neural network for any ...

Large Scale Audio-Visual Video Analytics Platform for Forensic Investigations of Terroristic Attacks

The forensic investigation of a terrorist attack poses a huge challenge ...

Please sign up or login with your details

Forgot password? Click here to reset