Human Pose Driven Object Effects Recommendation

by   Zhaoxin Fan, et al.
Renmin University of China
Tsinghua University

In this paper, we research the new topic of object effects recommendation in micro-video platforms, which is a challenging but important task for many practical applications such as advertisement insertion. To avoid the problem of introducing background bias caused by directly learning video content from image frames, we propose to utilize the meaningful body language hidden in 3D human pose for recommendation. To this end, in this work, a novel human pose driven object effects recommendation network termed PoseRec is introduced. PoseRec leverages the advantages of 3D human pose detection and learns information from multi-frame 3D human pose for video-item registration, resulting in high quality object effects recommendation performance. Moreover, to solve the inherent ambiguity and sparsity issues that exist in object effects recommendation, we further propose a novel item-aware implicit prototype learning module and a novel pose-aware transductive hard-negative mining module to better learn pose-item relationships. What's more, to benchmark methods for the new research topic, we build a new dataset for object effects recommendation named Pose-OBE. Extensive experiments on Pose-OBE demonstrate that our method can achieve superior performance than strong baselines.


A Data-driven Approach for Human Pose Tracking Based on Spatio-temporal Pictorial Structure

In this paper, we present a data-driven approach for human pose tracking...

Lifting 2D Human Pose to 3D with Domain Adapted 3D Body Concept

Lifting the 2D human pose to the 3D pose is an important yet challenging...

ActionPrompt: Action-Guided 3D Human Pose Estimation With Text and Pose Prompting

Recent 2D-to-3D human pose estimation (HPE) utilizes temporal consistenc...

SPCNet:Spatial Preserve and Content-aware Network for Human Pose Estimation

Human pose estimation is a fundamental yet challenging task in computer ...

DH-AUG: DH Forward Kinematics Model Driven Augmentation for 3D Human Pose Estimation

Due to the lack of diversity of datasets, the generalization ability of ...

ProtoRes: Proto-Residual Architecture for Deep Modeling of Human Pose

Our work focuses on the development of a learnable neural representation...

Application-Driven AI Paradigm for Hand-Held Action Detection

In practical applications especially with safety requirement, some hand-...

Please sign up or login with your details

Forgot password? Click here to reset