一种基于噪音过滤包装器的Web新闻抽取方法

摘要

Extracting high-purity news from large Web pages,and stored in a structured form is the research foundation of the public opinion monitoring and topical updating.This paper proposes a Web news extraction method based on filtering noise wrapper.When inducing the wrapper,if the two strings don't match each other,according to the threshold,we calculate the string tag path ratio of the strings to distinguish purity news from noise.At the same time,we propose two naive Bayes classifiers to extract the title and time of the news.Experimental results show that compared with other extraction technologies,the method in this paper has significant improvement in terms of accuracy and robustness.So it has greatly utility value.%从海量Web新闻网页中抽取高纯度新闻,并以结构化的形式存储,是舆情监测、话题更新等研究的基础.本文提出一种基于噪音过滤包装器的方法抽取Web新闻,归纳包装器时,若2页面字符串对比失配,计算其字符串标签路径比,根据阈值α,标记不同符号区分新闻内容和噪音.同时提出了朴素贝叶斯Web新闻标题分类器和时间分类器抽取新闻题目和时间.实验结果表明,本文方法与其他抽取技术相比准确率和鲁棒性有显著的提高,具有重要的实际应用价值.