Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning

NeurIPS 2020

Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning

Dec 06, 2020
|
35 views
|
Details
Accurately predicting the dynamics of robotic systems is crucial for model-based control and reinforcement learning. The most common way to estimate dynamics is by fitting a one-step ahead prediction model and using it to recursively propagate the predicted state distribution over long horizons. Unfortunately, this approach is known to compound even small prediction errors, making long-term predictions inaccurate. In this paper, we propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons -- that we call a trajectory-based model. This trajectory-based model takes an initial state, a future time index, and control parameters as inputs, and predicts the state at the future time. Our results in simulated and experimental robotic tasks show that our trajectory-based models yield significantly more accurate long term predictions, improved sample efficiency, and ability to predict task reward. Speakers: Nathan Lambert, Albert Wilcox, Howard Zhang, Kristofer Pister, Roberto Calandra

Comments
loading...