Sim-to-Real Learning for Casualty Detection from Ground Projected Point Cloud Data

by   Roni Permana Saputra, et al.

This paper addresses the problem of human body detection---particularly a human body lying on the ground (a.k.a. casualty)---using point cloud data. This ability to detect a casualty is one of the most important features of mobile rescue robots, in order for them to be able to operate autonomously. We propose a deep-learning-based casualty detection method using a deep convolutional neural network (CNN). This network is trained to be able to detect a casualty using a point-cloud data input. In the method we propose, the point cloud input is pre-processed to generate a depth image-like ground-projected heightmap. This heightmap is generated based on the projected distance of each point onto the detected ground plane within the point cloud data. The generated heightmap -- in image form -- is then used as an input for the CNN to detect a human body lying on the ground. To train the neural network, we propose a novel sim-to-real approach, in which the network model is trained using synthetic data obtained in simulation and then tested on real sensor data. To make the model transferable to real data implementations, during the training we adopt specific data augmentation strategies with the synthetic training data. The experimental results show that data augmentation introduced during the training process is essential for improving the performance of the trained model on real data. More specifically, the results demonstrate that the data augmentations on raw point-cloud data have contributed to a considerable improvement of the trained model performance.


page 1

page 2

page 3

page 5

page 7

page 8


Casualty Detection from 3D Point Cloud Data for Autonomous Ground Mobile Rescue Robots

One of the most important features of mobile rescue robots is the abilit...

On Automatic Data Augmentation for 3D Point Cloud Classification

Data augmentation is an important technique to reduce overfitting and im...

Pixel2point: 3D Object Reconstruction From a Single Image Using CNN and Initial Sphere

3D reconstruction from a single image has many useful applications. Howe...

GPR-based Model Reconstruction System for Underground Utilities Using GPRNet

Ground Penetrating Radar (GPR) is one of the most important non-destruct...

HUG model: an interaction point process for Bayesian detection of multiple sources in groundwaters from hydrochemical data

This paper presents a new interaction point process that integrates geol...

Point Cloud Data Simulation and Modelling with Aize Workspace

This work takes a look at data models often used in digital twins and pr...

Deep Reinforcement Learning with Mixed Convolutional Network

Recent research has shown that map raw pixels from a single front-facing...

Please sign up or login with your details

Forgot password? Click here to reset