基于Q-learning的HTTP自适应流码率控制方法研究

摘要

HTTP adaptive streaming (HAS) has become the standard for adaptive video streaming service. In changing network environments, current hardcoded-based rate adaptation algorithm was less flexible, and it is insufficient to con-sider the quality of experience (QoE). To optimize the QoE of users, a rate control approach based on Q-learning strategy was proposed. the client environments of HTTP adaptive video streaming was modeled and the state transition rule was defined. Three parameters related to QoE were quantified and a novel reward function was constructed. The experiments were employed by the Q-learning rate control approach in two typical HAS algorithms. The experiments show the rate control approach can enhance the stability of rate switching in HAS clients.%基于HTTP的自适应流HAS已经成为自适应视频流服务的标准.在HAS客户端网络状态多变的情况下,硬编码形式的码率决策方法灵活性偏低,对用户体验考虑不足.为了优化用户体验质量(QoE),提出一种基于Q-Learning的码率控制算法,结合HTTP自适应视频流客户端环境进行建模并定义状态转移规则;量化与用户QoE相关的参数,构建新的回报函数;实验表明引入Q-Learning进行码率调整的自适应算法在码率切换的稳定性方面表现较好.