Multimodal Hierarchical Dirichlet Process-based Active Perception

by   Tadahiro Taniguchi, et al.

In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing actions on an object. However, performing many actions on a target object requires a long time. In a real-time scenario, i.e., when the time is limited, the robot has to determine the set of actions that is most effective for recognizing a target object. We propose an MHDP-based active perception method that uses the information gain (IG) maximization criterion and lazy greedy algorithm. We show that the IG maximization criterion is optimal in the sense that the criterion is equivalent to a minimization of the expected Kullback--Leibler divergence between a final recognition state and the recognition state after the next set of actions. However, a straightforward calculation of IG is practically impossible. Therefore, we derive an efficient Monte Carlo approximation method for IG by making use of a property of the MHDP. We also show that the IG has submodular and non-decreasing properties as a set function because of the structure of the graphical model of the MHDP. Therefore, the IG maximization problem is reduced to a submodular maximization problem. This means that greedy and lazy greedy algorithms are effective and have a theoretical justification for their performance. We conducted an experiment using an upper-torso humanoid robot and a second one using synthetic data. The experimental results show that the method enables the robot to select a set of actions that allow it to recognize target objects quickly and accurately. The results support our theoretical outcomes.


page 18

page 19

page 21

page 24


How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learning

Selection of appropriate tools and use of them when performing daily tas...

Robot self/other distinction: active inference meets neural networks learning in a mirror

Self/other distinction and self-recognition are important skills for int...

Communication-Aware Multi-robot Coordination with Submodular Maximization

Submodular maximization has been widely used in many multi-robot task pl...

Exploiting Submodular Value Functions For Scaling Up Active Perception

In active perception tasks, an agent aims to select sensory actions that...

Submodular Maximization via Taylor Series Approximation

We study submodular maximization problems with matroid constraints, in p...

Online Submodular Coordination with Bounded Tracking Regret: Theory, Algorithm, and Applications to Multi-Robot Coordination

We enable efficient and effective coordination in unpredictable environm...

Plane and Sample: Maximizing Information about Autonomous Vehicle Performance using Submodular Optimization

As autonomous vehicles (AVs) take on growing Operational Design Domains ...

Please sign up or login with your details

Forgot password? Click here to reset