Deep Reinforcement Learning-Based Spectrum Allocation in Integrated Access and Backhaul Networks

摘要

We develop a framework based on deep reinforcement learning (DRL) to solve the spectrum allocation problem in the emerging integrated access and backhaul (IAB) architecture with large scale deployment and dynamic environment. The available spectrum is divided into several orthogonal sub-channels, and the donor base station (DBS) and all IAB nodes have the same spectrum resource for allocation, where a DBS utilizes those sub-channels for access links of associated user equipment (UE) as well as for backhaul links of associated IAB nodes, and an IAB node can utilize all for its associated UEs. This is one of key features in which 5G differs from traditional settings where the backhaul networks are designed independently from the access networks. With the goal of maximizing the sum log-rate of all UE groups, we formulate the spectrum allocation problem into a mix-integer and non-linear programming. However, it is intractable to find an optimal solution especially when the IAB network is large and time-varying. To tackle this problem, we propose to use the latest DRL method by integrating an actor-critic spectrum allocation (ACSA) scheme and deep neural network (DNN) to achieve real-time spectrum allocation in different scenarios. The proposed methods are evaluated through numerical simulations and show promising results compared with some baseline allocation policies.