The recent breakthrough finding of macroscopic fundamental diagram (MFD) establishes the foundation of macroscopic analysis in urban transportation studies. However, the implementation of MFD for traffic signal control remains challenging. This is because the compact network-wide information provided by MFD is insufficient for searching for the optimal microscopic control policy. In this paper, rather than implementing only MFD, we integrate MFD into our microscopic urban traffic flow model to constrain the searching space of control policies. This approach is able to maximize the contribution of MFD, without losing microscopic information in the control model. Specifically, we first build a traffic flow model and introduce the stochastic driver behaviors by a turning matrix. We then implement the approximate Q-learning with restricted control to reduce the computational cost of the large-scale stochastic control problem. Here, the information of MFD is used to design both the heuristic regularization term in the stage cost and the statebased feature vector of the approximate Q-function. By this approximate Q-learning algorithm, the traffic density distribution of the network tends to become homogenous, with the mean value around the optimal density of the MFD. The numerical experiments demonstrate that compared to a fixed policy, our policy could efficiently make a heterogeneous network more homogeneous, and thus guarantee a more robust shape of the MFD. Furthermore, our policy has a better performance on trip completion flow maximization compared to either a fixed or a greedy policy, since it can achieve the optimal density in the MFD.
展开▼
