Residual Reinforcement Learning from Demonstrations - Apprentissage de modèles visuels à partir de données massives Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2021

Residual Reinforcement Learning from Demonstrations

Résumé

Residual reinforcement learning (RL) has been proposed as a way to solve challenging robotic tasks by adapting control actions from a conventional feedback controller to maximize a reward signal. We extend the residual formulation to learn from visual inputs and sparse rewards using demonstrations. Learning from images, proprioceptive inputs and a sparse task-completion reward relaxes the requirement of accessing full state features, such as object and target positions. In addition, replacing the base controller with a policy learned from demonstrations removes the dependency on a hand-engineered controller in favour of a dataset of demonstrations, which can be provided by non-experts. Our experimental evaluation on simulated manipulation tasks on a 6-DoF UR5 arm and a 28-DoF dexterous hand demonstrates that residual RL from demonstrations is able to generalize to unseen environment conditions more flexibly than either behavioral cloning or RL fine-tuning, and is capable of solving high-dimensional, sparse-reward tasks out of reach for RL from scratch.
Fichier principal
Vignette du fichier
RRLfD.pdf (2.28 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03260683 , version 1 (15-06-2021)

Identifiants

Citer

Minttu Alakuijala, Gabriel Dulac-Arnold, Julien Mairal, Jean Ponce, Cordelia Schmid. Residual Reinforcement Learning from Demonstrations. 2021. ⟨hal-03260683⟩
129 Consultations
270 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More