VidCEP: Complex Event Processing Framework to Detect Spatiotemporal Patterns in Video Streams

by   piyush-yadav, et al.

Video data is highly expressive and has traditionally been very difficult for a machine to interpret. Querying event patterns from video streams is challenging due to its unstructured representation. Middleware systems such as Complex Event Processing (CEP) mine patterns from data streams and send notifications to users in a timely fashion. Current CEP systems have inherent limitations to query video streams due to their unstructured data model and lack of expressive query language. In this work, we focus on a CEP framework where users can define high-level expressive queries over videos to detect a range of spatiotemporal event patterns. In this context, we propose: i) VidCEP, an in-memory, on the fly, near real-time complex event matching framework for video streams. The system uses a graph-based event representation for video streams which enables the detection of high-level semantic concepts from video using cascades of Deep Neural Network models, ii) a Video Event Query language (VEQL) to express high-level user queries for video streams in CEP, iii) a complex event matcher to detect spatiotemporal video event patterns by matching expressive user queries over video data. The proposed approach detects spatiotemporal video event patterns with an F-score ranging from 0.66 to 0.89. VidCEP maintains near real-time performance with an average throughput of 70 frames per second for 5 parallel videos with sub-second matching latency.


page 1

page 2

page 3

page 4

page 6

page 8

page 9


Knowledge Graph Driven Approach to Represent Video Streams for Spatiotemporal Event Pattern Matching in Complex Event Processing

Complex Event Processing (CEP) is an event processing paradigm to perfor...

Visual Semantic Multimedia Event Model for Complex Event Detection in Video Streams

Multimedia data is highly expressive and has traditionally been very dif...

Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels

Many real-world video analysis applications require the ability to ident...

Video Event Recognition for Surveillance Applications (VERSA)

VERSA provides a general-purpose framework for defining and recognizing ...

Join Query Optimization Techniques for Complex Event Processing Applications

Complex event processing (CEP) is a prominent technology used in many mo...

Accelerating Aggregation Queries on Unstructured Streams of Data

Analysts and scientists are interested in querying streams of video, aud...

To Share, or not to Share Online Event Trend Aggregation Over Bursty Event Streams

Complex event processing (CEP) systems continuously evaluate large workl...

Please sign up or login with your details

Forgot password? Click here to reset