Learning how to Interact with a Complex Interface using Hierarchical Reinforcement Learning

by   Gheorghe Comanici, et al.

Hierarchical Reinforcement Learning (HRL) allows interactive agents to decompose complex problems into a hierarchy of sub-tasks. Higher-level tasks can invoke the solutions of lower-level tasks as if they were primitive actions. In this work, we study the utility of hierarchical decompositions for learning an appropriate way to interact with a complex interface. Specifically, we train HRL agents that can interface with applications in a simulated Android device. We introduce a Hierarchical Distributed Deep Reinforcement Learning architecture that learns (1) subtasks corresponding to simple finger gestures, and (2) how to combine these gestures to solve several Android tasks. Our approach relies on goal conditioning and can be used more generally to convert any base RL agent into an HRL agent. We use the AndroidEnv environment to evaluate our approach. For the experiments, the HRL agent uses a distributed version of the popular DQN algorithm to train different components of the hierarchy. While the native action space is completely intractable for simple DQN agents, our architecture can be used to establish an effective way to interact with different tasks, significantly improving the performance of the same DQN agent over different levels of abstraction.


AndroidEnv: A Reinforcement Learning Platform for Android

We introduce AndroidEnv, an open-source platform for Reinforcement Learn...

Hierarchical Program-Triggered Reinforcement Learning Agents For Automated Driving

Recent advances in Reinforcement Learning (RL) combined with Deep Learni...

Possibility Before Utility: Learning And Using Hierarchical Affordances

Reinforcement learning algorithms struggle on tasks with complex hierarc...

Deep Reinforcement Learning for Dexterous Manipulation with Concept Networks

Deep reinforcement learning yields great results for a large array of pr...

MARLUI: Multi-Agent Reinforcement Learning for Goal-Agnostic Adaptive UIs

The goal of Adaptive UIs is to automatically change an interface so that...

The Agent Web Model – Modelling web hacking for reinforcement learning

Website hacking is a frequent attack type used by malicious actors to ob...

Please sign up or login with your details

Forgot password? Click here to reset