Research and Implementation of Algorithm Based on Data Fusion Technology

摘要

As the growing amount of data stored on the Internet, the work of searching for information becomes complicated. The traditional collection method cannot achieve a certain effect, it is cumbersome and time-consuming. Using natural language processing technology and web crawler technology to collect and analyze data about student evaluation, the purpose is to obtain the key factors affecting teachers' comprehensive evaluation results and propose the methods to solve the problems. For the traditional Web crawler technology, there is a lack of certain intelligence, initiative, etc. the design of the best priority crawler framework has improved and optimized its structure. And the improved PageRank value, user demand correlation degree, and NDC algorithm denoising are added, it can effectively solve a series of problems such as long retrieval time, overlapping information, incomplete information, and improve the accuracy of information collection.