A User Interface for Sense-making of the Reasoning Process while Interacting with Robots

by   Chao Wang, et al.

As knowledge graph has the potential to bridge the gap between commonsense knowledge and reasoning over actionable capabilities of mobile robotic platforms, incorporating knowledge graph into robotic system attracted increasing attention in recent years. Previously, graph visualization has been used wildly by developers to make sense of knowledge representations. However, due to lacking the link between abstract knowledge of the real-world environment and the robot's actions, transitional visualization tools are incompatible for expert-user to understand, test, supervise and modify the graph-based reasoning system with the embodiment of the robots. Therefore, we developed an interface which enables robotic experts to send commands to the robot in natural language, then interface visualizes the procedures of the robot mapping the command to the functions for querying in the commonsense knowledge database, links the result to the real world instances in a 3D map and demonstrate the execution of the robot from the first-person perspective of the robot. After 3 weeks of usage of the system by robotic experts in their daily development, some feedback was collected, which provides insight for designing such systems.


page 3

page 5

page 6

page 7

page 8

page 9


Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

Generative commonsense reasoning (GCR) in natural language is to reason ...

A Development Cycle for Automated Self-Exploration of Robot Behaviors

In this paper we introduce Q-Rock, a development cycle for the automated...

A Decision Support Design Framework for Selecting a Robotic Interface

The design and development of robots involve the essential step of selec...

Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning

Enabling robots to understand instructions provided via spoken natural l...

A Road-map to Robot Task Execution with the Functional Object-Oriented Network

Following work on joint object-action representations, the functional ob...

Explainable Knowledge Graph Embedding: Inference Reconciliation for Knowledge Inferences Supporting Robot Actions

Learned knowledge graph representations supporting robots contain a weal...

Commonsense Spatial Reasoning for Visually Intelligent Agents

Service robots are expected to reliably make sense of complex, fast-chan...

Please sign up or login with your details

Forgot password? Click here to reset