ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view

by   Mai Nishimura, et al.

We introduce a novel learning-based method for view birdification, the task of recovering ground-plane trajectories of pedestrians of a crowd and their observer in the same crowd just from the observed ego-centric video. View birdification becomes essential for mobile robot navigation and localization in dense crowds where the static background is hard to see and reliably track. It is challenging mainly for two reasons; i) absolute trajectories of pedestrians are entangled with the movement of the observer which needs to be decoupled from their observed relative movements in the ego-centric video, and ii) a crowd motion model describing the pedestrian movement interactions is specific to the scene yet unknown a priori. For this, we introduce a Transformer-based network referred to as ViewBirdiformer which implicitly models the crowd motion through self-attention and decomposes relative 2D movement observations onto the ground-plane trajectories of the crowd and the camera through cross-attention between views. Most important, ViewBirdiformer achieves view birdification in a single forward pass which opens the door to accurate real-time, always-on situational awareness. Extensive experimental results demonstrate that ViewBirdiformer achieves accuracy similar to or better than state-of-the-art with three orders of magnitude reduction in execution time.


View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements

We introduce view birdification, the problem of recovering ground-plane ...

InCrowdFormer: On-Ground Pedestrian World Model From Egocentric Views

We introduce an on-ground Pedestrian World Model, a computational model ...

Exemplar-AMMs: Recognizing Crowd Movements from Pedestrian Trajectories

In this paper, we present a novel method to recognize the types of crowd...

Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus

Predicting the collective motion of a group of pedestrians (a crowd) und...

Constrained Linear Movement Model (CALM): Simulation of passenger movement in airplanes

Pedestrian dynamics models the walking movement of individuals in a crow...

Collective behavior recognition using compact descriptors

This paper presents a novel hierarchical approach for collective behavio...

Crowd-Driven Mapping, Localization and Planning

Navigation in dense crowds is a well-known open problem in robotics with...

Please sign up or login with your details

Forgot password? Click here to reset