Developing Cyberspace Data Understanding: Using CRISP-DM for Host-based IDS Feature Mining

摘要

Current intrusion detection systems (IDS) generate a large number of specific alerts, but typically do not provide actionable information. Compounding this problem is the fact that many alerts are false positive alerts. This paper applies the Cross Industry Standard Process for Data Mining (CRISP-DM) to develop an understanding of a host environment under attack. Data is generated by launching scans and exploits at a machine outfitted with a set of host-based forensic data collectors. Through knowledge discovery, features are selected to project human understanding of the attack process into the IDS model. By discovering relationships between the data collected and controlled events, false positive alerts were reduced by over 91% when compared to a leading open source IDS. This method of searching for hid- den forensic evidence relationships enhances understanding of novel attacks and vulnerabilities, bolstering ones ability to defend the cyberspace domain. The methodology presented can be used to further host-based intrusion detection research.