XCSG在多机器人强化学习中的应用

摘要

XCS classifier system has been shown to solve machine-learning problems in a competitive way.However,in multi-robot problems,XCS is restricted to solve very small problems modeled by a Markov decision process.In this paper a new leaming technique XCSG that combines XCS and gradient descent methods was proposed to solve multi-robot machine-learning problems.XCSG builds low-dimensional approximation of the function,and gradient descent techniques use on-line knowledge to establish a stable approximation of functions,so that the Q-form has been maintained at a low-dimensional stable state.Approximate of the function not only requires smaller storage space,but also allows the robot online knowledge is summarized on the generalization.Simulation results show that XCSG algorithm solves the multi-robot reinforcement learning in a large space,slow learning,learning uncertainty and other issues.%XCS分类器在解决机器人强化学习方面已显示出较强的能力,但在多机器人领域仅局限于MDP环境,只能解决环境空间较小的学习问题.提出了XCSG来解决多机器人的强化学习问题.XCSG建立低维的逼近函数,梯度下降技术利用在线知识建立稳定的逼近函数,使Q表格一直保持在稳定低维状态.逼近函数Q不仅所需的存储空间更小,而且允许机器人在线对已获得的知识进行归纳一般化.仿真实验表明,XCSG算法很好地解决了多机器人学习空间大、学习速度慢、学习效果不确定等问题.