一种基于文本信息的三层过滤系统的设计

摘要

In order to improve the efficiency of text information filtering,a system of three-layer filtration based on text message is put forward. The system is divided into horizontal two parts and vertical three-tier structure,the first layer of information filtering is based on IP and URL address filtering,the second layer is based on the statistics of keyword frequency and weights,including information title, keywords and text content three parts to calculate the statistical value. The third layer is based on analysis of filter content features,while the split words,keywords weighting,VSM and theme tendency analysis is led in the system,to ensure the efficiency and accuracy of the bad information to identify. The experiments are shown that the system has a better filtering effect of the recall and precision significantly than the KNN method,timely to prevent the spread of bad information in real time information filtering.%　　为了提高文本信息过滤的效率，提出一种基于文本信息的三层过滤系统。系统分为横向二部分、纵向三层次的结构，在信息过滤时第一层采用基于IP、URL地址的过滤方式；第二层为关键词频与权重的统计，对信息标题、关键词及正文内容三部分分别计算统计值；第三层为内容特征分析过滤，同时引入分词、关键词权重计算、VSM与主题倾向分析技术，保证不良信息识别的高效与准确。实验表明系统具有较好的过滤效果，查全率和查准率明显优于KNN方法，在实时信息过滤时能及时阻止不良信息的传播。