Why is the video analytics accuracy fluctuating, and what can we do about it?

by   Sibendu Paul, et al.

It is a common practice to think of a video as a sequence of images (frames), and re-use deep neural network models that are trained only on images for similar analytics tasks on videos. In this paper, we show that this leap of faith that deep learning models that work well on images will also work well on videos is actually flawed. We show that even when a video camera is viewing a scene that is not changing in any human-perceptible way, and we control for external factors like video compression and environment (lighting), the accuracy of video analytics application fluctuates noticeably. These fluctuations occur because successive frames produced by the video camera may look similar visually, but these frames are perceived quite differently by the video analytics applications. We observed that the root cause for these fluctuations is the dynamic camera parameter changes that a video camera automatically makes in order to capture and produce a visually pleasing video. The camera inadvertently acts as an unintentional adversary because these slight changes in the image pixel values in consecutive frames, as we show, have a noticeably adverse impact on the accuracy of insights from video analytics tasks that re-use image-trained deep learning models. To address this inadvertent adversarial effect from the camera, we explore the use of transfer learning techniques to improve learning in video analytics tasks through the transfer of knowledge from learning on image analytics tasks. In particular, we show that our newly trained Yolov5 model reduces fluctuation in object detection across frames, which leads to better tracking of objects(40 mistakes in tracking). Our paper also provides new directions and techniques to mitigate the camera's adversarial effect on deep learning models used for video analytics applications.


page 3

page 7


Why Accuracy Is Not Enough: The Need for Consistency in Object Detection

Object detectors are vital to many modern computer vision applications. ...

Deployment of Customized Deep Learning based Video Analytics On Surveillance Cameras

This paper demonstrates the effectiveness of our customized deep learnin...

Understanding Patch-Based Learning by Explaining Predictions

Deep networks are able to learn highly predictive models of video data. ...

Visual Privacy: Current and Emerging Regulations Around Unconsented Video Analytics in Retail

Video analytics is the practice of combining digital video data with mac...

CamTuner: Reinforcement-Learning based System for Camera Parameter Tuning to enhance Analytics

Complex sensors like video cameras include tens of configurable paramete...

AQuA: Analytical Quality Assessment for Optimizing Video Analytics Systems

Millions of cameras at edge are being deployed to power a variety of dif...

Digital Image Forensics using Deep Learning

During the investigation of criminal activity when evidence is available...

Please sign up or login with your details

Forgot password? Click here to reset