Identification of partially observable environment based on on-line variational Bayes method and its application to reinforcement learning

摘要

In this report, we present an on-line variational Bayes (VB) method for system identification based on linear state space models. The learning algorithm is implemented as the maximization of an on-line free energy, which can be used for determining the dimension of the internal state. We also propose a reinforcement learning (RL) method using this system identification method. Our RL method is applied to a simple automatic control problem with hidden state variables. The result shows that our method is able to determine correctly the dimension of the internal state and to acquire a good control, even in a partially observable environment.