DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentation

by   Yue Zhang, et al.

In this technical report, we present our findings from the research conducted on the Human-Object Interaction 4D (HOI4D) dataset for egocentric action segmentation task. As a relatively novel research area, point cloud video methods might not be good at temporal modeling, especially for long point cloud videos (, 150 frames). In contrast, traditional video understanding methods have been well developed. Their effectiveness on temporal modeling has been widely verified on many large scale video datasets. Therefore, we convert point cloud videos into depth videos and employ traditional video modeling methods to improve 4D action segmentation. By ensembling depth and point cloud video methods, the accuracy is significantly improved. The proposed method, named Mixture of Depth and Point cloud video experts (DPMix), achieved the first place in the 4D Action Segmentation Track of the HOI4D Challenge 2023.


page 1

page 2

page 3

page 4


Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos

Recently, the community has made tremendous progress in developing effec...

Marking anything: application of point cloud in extracting video target features

Extracting retrievable features from video is of great significance for ...

3D point cloud segmentation using GIS

In this paper we propose an approach to perform semantic segmentation of...

DVI: Depth Guided Video Inpainting for Autonomous Driving

To get clear street-view and photo-realistic simulation in autonomous dr...

Action Keypoint Network for Efficient Video Recognition

Reducing redundancy is crucial for improving the efficiency of video rec...

Solving Large-Scale 0-1 Knapsack Problems and its Application to Point Cloud Resampling

0-1 knapsack is of fundamental importance in computer science, business,...

3D Modeling and WebVR Implementation using Azure Kinect, Open3D, and Three.js

This paper proposes a method of extracting an RGB-D image usingAzure Kin...

Please sign up or login with your details

Forgot password? Click here to reset