Towards Context-aware Interaction Recognition

by   Bohan Zhuang, et al.

Recognizing how objects interact with each other is a crucial task in visual recognition. If we define the context of the interaction to be the objects involved, then most current methods can be categorized as either: (i) training a single classifier on the combination of the interaction and its context; or (ii) aiming to recognize the interaction independently of its explicit context. Both methods suffer limitations: the former scales poorly with the number of combinations and fails to generalize to unseen combinations, while the latter often leads to poor interaction recognition performance due to the difficulty of designing a context-independent interaction classifier. To mitigate those drawbacks, this paper proposes an alternative, context-aware interaction recognition framework. The key to our method is to explicitly construct an interaction classifier which combines the context, and the interaction. The context is encoded via word2vec into a semantic space, and is used to derive a classification result for the interaction. The proposed method still builds one classifier for one interaction (as per type (ii) above), but the classifier built is adaptive to context via weights which are context dependent. The benefit of using the semantic space is that it naturally leads to zero-shot generalizations in which semantically similar contexts (subjectobject pairs) can be recognized as suitable contexts for an interaction, even if they were not observed in the training set.


CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context

Learning accurate models of the physical world is required for a lot of ...

A Contextual Framework for Adaptive User Interfaces: Modelling the Interaction Environment

The interaction context (or environment) is key to any HCI task and espe...

Context-Aware Zero-Shot Learning for Object Recognition

Zero-Shot Learning (ZSL) aims at classifying unlabeled objects by levera...

Zero-Shot Human-Object Interaction Recognition via Affordance Graphs

We propose a new approach for Zero-Shot Human-Object Interaction Recogni...

GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot Action Recognition

Zero-shot action recognition can recognize samples of unseen classes tha...

A context-aware knowledge transferring strategy for CTC-based ASR

Non-autoregressive automatic speech recognition (ASR) modeling has recei...

Classifying All Interacting Pairs in a Single Shot

In this paper, we introduce a novel human interaction detection approach...

Please sign up or login with your details

Forgot password? Click here to reset