Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach

by   Aidean Sharghi, et al.

Recent years have witnessed a resurgence of interest in video summarization. However, one of the main obstacles to the research on video summarization is the user subjectivity - users have various preferences over the summaries. The subjectiveness causes at least two problems. First, no single video summarizer fits all users unless it interacts with and adapts to the individual users. Second, it is very challenging to evaluate the performance of a video summarizer. To tackle the first problem, we explore the recently proposed query-focused video summarization which introduces user preferences in the form of text queries about the video into the summarization process. We propose a memory network parameterized sequential determinantal point process in order to attend the user query onto different video frames and shots. To address the second challenge, we contend that a good evaluation metric for video summarization should focus on the semantic information that humans can perceive rather than the visual features or temporal overlaps. To this end, we collect dense per-video-shot concept annotations, compile a new dataset, and suggest an efficient evaluation method defined upon the concept annotations. We conduct extensive experiments contrasting our video summarizer to existing ones and present detailed analyses about the dataset and the new evaluation method.


page 2

page 4


FrameRank: A Text Processing Approach to Video Summarization

Video summarization has been extensively studied in the past decades. Ho...

CLIP-It! Language-Guided Video Summarization

A generic video summary is an abridged version of a video that conveys t...

Image Conditioned Keyframe-Based Video Summarization Using Object Detection

Video summarization plays an important role in selecting keyframe for un...

IntentVizor: Towards Generic Query Guided Interactive Video Summarization Using Slow-Fast Graph Convolutional Networks

The target of automatic Video summarization is to create a short skim of...

Eliciting User Preferences for Personalized Explanations for Video Summaries

Video summaries or highlights are a compelling alternative for exploring...

Improving Sequential Determinantal Point Processes for Supervised Video Summarization

It is now much easier than ever before to produce videos. While the ubiq...

A Memory Network Approach for Story-based Temporal Summarization of 360° Videos

We address the problem of story-based temporal summarization of long 360...

Please sign up or login with your details

Forgot password? Click here to reset