Preparation of PLearning How to Soar: Steady State Autonomous Dynamic Soaring Through Reinforcement Learning AIAA Technical Conferences

摘要

Dynamic soaring is a cyclic climbing and diving maneuver that enables birds and gilding aircraft to perpetually stay aloft if it is performed in the presence of a horizontal wind gradient. An autonomous aircraft capable of performing a dynamic soaring maneuver could do the same. unlocking the potential for expansive travel beyond the scale of what seabirds like the albatross can achieve. This work formulates dynamic soaring of a small autonomous gliding aircraft in the context of a shear wind gradient as an optimal control problem. We describe an online reinforcement learning controller that can execute the PS maneuver to a steady state. The learning controller is taught by a tracking controller that has been shown to achieve steady state dynamic soaring control in simulation under stable and known (to the UAV) environmental conditions. Our experiments in simulation show that after fully training the learning controller it outperforms the teaching controller in terms of energy gain per cycle, and number of cycles to reach a steady state DS orbit (4 versus 6 for the teaching controller). In a final comparison, we test both controllers in a scenario where there is a mismatch between what the measured windspecd error is high. We show the where the teaching controller destabilizes and crashes alter a few orbits, the learning controller adapts quickly and converges to a successful steady state orbit.