In this paper, a car-like Unmanned Ground Vehicle (UGV) is simulated and trained as an intelligent agent to navigate and exit unknown obstacle filled environments given no prior knowledge of environment characteristics, using a Reinforcement Learning (RL) algorithm tailored for continuous action spaces. This is achieved using Deep Deterministic Policy Gradient (DDPG), an Actor-Critic network that combines multiple cutting-edge Artificial Intelligence methods including continuous Deep-Q learning, policy gradient methods and actor-critic networks. A combination of two feedforward neural networks with Rectified Linear Units (ReLU) is used for the critic and actor representations which combine both policy and value based methods to learn continuous action space policies via approximation functions. The role of the actor network in this architecture is to decide linear and angular velocity outputs from a continuous action space given current state inputs, to then be evaluated by the critic network to learn and estimate Q-values by minimizing a loss function. The proposed DDPG RL network is trained and evaluated in two obstacle filled environments for a car-like UGV with wheelbase, l of 0.3 m. During the 10,000 episode training period, the agent converges to a maximum reward value of 180 after 1100 training episodes in the first environment, and a maximum reward value of 80 after 7500 training episodes in the second, more complex environment. The agent is shown to exhibit intelligent human-like learning behavior to learn optimal policies and adapt to new environments at the end of each training period with no changes to network architecture.
展开▼
