Intrusion Detection Based on Dynamic Behavior Modeling: Reinforcement Learning versus Hidden Markov Models

摘要

As an important active defense technique for computer systems, intrusion detection has received many research interests in recent years. However, the performance of current intrusion detection systems (IDSs) is unsatisfactory due to the increasing amount of complex behaviors in cyber attacks. In this paper, a novel dynamic behavior modeling approach for intrusion detection is studied, which is based on Markov reward process models and reinforcement learning (RL) algorithms. The RL-based approach is compared with the previous dynamic behavior modeling method based on Hidden Markov Models (HMMs). The main difference between Hidden Markov Models (HMMs) and RL is that in RL, the probabilistic structure and parameters of the underlying Markov process are not explicitly estimated. Instead, learning prediction of value functions is used to estimate the anomaly probabilities of sequential behaviors. Therefore, the RL-based method has the advantage of lower computational costs and simpler implementations. Experimental results on the behavior modeling and intrusion detection of computer programs demonstrate that the RL approach to intrusion detection not only has high computational efficiency but also has comparable or even better performance than previous HMM-based approaches.