Search system for providing web crawling query prioritization based on classification operation performance

摘要

Various methods and systems for processing web crawling queries using a web crawling prioritization model based on classification operation performance. A classification operation for organizing products in a product listing platform is accessed. A web crawling engine is accessed for the classification operation. The web crawling engine operates based on a web crawling query prioritization model that supports determining web crawling priority scores that indicate a predicted performance improvement for classification operations executed with known data and web crawled data to be retrieved from executing a web crawling query operation. Using the web crawling prioritization model, a web crawling priority score is determined for a web crawling query for the corresponding classification operation. The classification operation is associated with a product in a product listing platform and known data for the product. Based on the web crawling priority score, the web crawling query is executed to identify web crawled data.