Genetic Network Programming-Sarsa with Subroutines for Trading Rules on Stock Markets

摘要

The purpose of this paper is to propose a new approach to generate effective subroutines, which are automatically discovered by the GNP-Sarsa programs combining evolution and reinforcement learning. We name it as GNP-Sarsa with subroutines (GNP_(sb)-Sarsa) and apply it to the trading rules on stock markets. In the proposal method, GNP_(sb)-Sarsa offers an alternative population, where individuals are represented by subroutines. Each main program of GNP_(sb)-Sarsa can refer to an individual in the subroutine population, after adding a new kind of node, namely subroutine node, in the graph network structure of GNP-Sarsa. GNP_(sb)-Sarsa containing a main program and a subroutine evolves by natural selection and genetic operations, where the gene of GNP_(sb)-Sarsa is the pair of the main GNP and its subroutine. That is, the genetic operations on GNP_(sb)-Sarsa are constrained by the gene structure on which they can operate. In the simulations, the stock prices of different brands from 2001 to 2004 are used to test the effectiveness of the GNP_(sb)-Sarsa. The results show that the proposed approach can provide reasonable opportunities for evolving complex solutions.