A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld

02/10/2023
by   Laetitia Teodorescu, et al.
0

Building open-ended agents that can autonomously discover a diversity of behaviours is one of the long-standing goals of artificial intelligence. This challenge can be studied in the framework of autotelic RL agents, i.e. agents that learn by selecting and pursuing their own goals, self-organizing a learning curriculum. Recent work identified language has a key dimension of autotelic learning, in particular because it enables abstract goal sampling and guidance from social peers for hindsight relabelling. Within this perspective, we study the following open scientific questions: What is the impact of hindsight feedback from a social peer (e.g. selective vs. exhaustive)? How can the agent learn from very rare language goal examples in its experience replay? How can multiple forms of exploration be combined, and take advantage of easier goals as stepping stones to reach harder ones? To address these questions, we use ScienceWorld, a textual environment with rich abstract and combinatorial physics. We show the importance of selectivity from the social peer's feedback; that experience replay needs to over-sample examples of rare goals; and that following self-generated goal sequences where the agent's competence is intermediate leads to significant improvements in final performance.

READ FULL TEXT
research
05/21/2023

Augmenting Autotelic Agents with Large Language Models

Humans learn to master open-ended repertoires of skills by imagining and...
research
05/21/2019

Maximum Entropy-Regularized Multi-Goal Reinforcement Learning

In Multi-Goal Reinforcement Learning, an agent learns to achieve multipl...
research
02/10/2022

Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agents

In the quest for autonomous agents learning open-ended repertoires of sk...
research
06/17/2020

Automatic Curriculum Learning through Value Disagreement

Continually solving new, unsolved tasks is the key to learning diverse b...
research
10/07/2021

Situated Dialogue Learning through Procedural Environment Generation

We teach goal-driven agents to interactively act and speak in situated e...
research
02/18/2020

Generating Automatic Curricula via Self-Supervised Active Domain Randomization

Goal-directed Reinforcement Learning (RL) traditionally considers an age...
research
05/25/2021

Towards Teachable Autonomous Agents

Autonomous discovery and direct instruction are two extreme sources of l...

Please sign up or login with your details

Forgot password? Click here to reset