PresSim: An End-to-end Framework for Dynamic Ground Pressure Profile Generation from Monocular Videos Using Physics-based 3D Simulation

by   Lala Shakti Swarup Ray, et al.

Ground pressure exerted by the human body is a valuable source of information for human activity recognition (HAR) in unobtrusive pervasive sensing. While data collection from pressure sensors to develop HAR solutions requires significant resources and effort, we present a novel end-to-end framework, PresSim, to synthesize sensor data from videos of human activities to reduce such effort significantly. PresSim adopts a 3-stage process: first, extract the 3D activity information from videos with computer vision architectures; then simulate the floor mesh deformation profiles based on the 3D activity information and gravity-included physics simulation; lastly, generate the simulated pressure sensor data with deep learning models. We explored two approaches for the 3D activity information: inverse kinematics with mesh re-targeting, and volumetric pose and shape estimation. We validated PresSim with an experimental setup with a monocular camera to provide input and a pressure-sensing fitness mat (80x28 spatial resolution) to provide the sensor ground truth, where nine participants performed a set of predefined yoga sequences.


page 1

page 2

page 4

page 5


PressureTransferNet: Human Attribute Guided Dynamic Ground Pressure Profile Transfer using 3D simulated Pressure Maps

We propose PressureTransferNet, a novel method for Human Activity Recogn...

Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Data

People spend a substantial part of their lives at rest in bed. 3D human ...

Smart Pressure e-Mat for Human Sleeping Posture and Dynamic Activity Recognition

With the emphasis on healthcare, early childhood education, and fitness,...

Are Accelerometers for Activity Recognition a Dead-end?

Accelerometer-based (and by extension other inertial sensors) research f...

Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors

Unlike images or videos data which can be easily labeled by human being,...

Generative Sensing: Transforming Unreliable Sensor Data for Reliable Recognition

This paper introduces a deep learning enabled generative sensing framewo...

Please sign up or login with your details

Forgot password? Click here to reset