Method for Fast and Accurate Extraction of Key Information from Webpages

摘要

As the World Wide Web continues to grow unbounded, users expect intelligent processing and accurate coverage of all its domains. To allow for the same, we present a novel approach to identify and extract key information from web pages with commendable accuracy. We extract important information such as the Title, Main Image, Description, Keywords and FavIcon from a webpage where available, using only the HTML responses without any explicit webpage rendering. The algorithm was modelled to be fast without compromising on its accuracy, is fully automatic, language independent and runs without any human supervision or training. We test our algorithm extensively on over one hundred thousand webpages and successfully extract the key information for 97% of them with an impressive average extraction time of less than 500 milliseconds per webpage.