基于网络爬虫软件建设主题网络信息资源库的研究--以高铁网络信息资源库建设为例

摘要

As network information keeps increasing, the number of web pages has exceeded one trillion. It is necessary to build the themed network information resource database and permanently save the theme-related information. Based on the construction of the high-speed rail network information resource database, this paper puts forward the objectives and contents of the themed information resource database, studies the use of web crawler software for the construction, provides a process model of the system and points out the key points of the system construction.%网络信息与日俱增，网页数量已经超过万亿，建立主题网络信息资源库，永久保存主题相关信息资料十分必要。本文以建设高铁网络信息资源库为例，提出了主题信息资源库的建设目标和内容，研究了用网络爬虫软件建设网络信息资源库，提出了系统的流程模型，指出了系统建设的关键点。