Retrieving dynamically-generated and database-driven web pages using a search engine robot

摘要

The present invention in one embodiment includes a computer implemented method for performing a crawl of a web-site that contains linked web pages. The invention includes retrieving a URL with variable that identifies said web page and utilizing said variable to gain access to said web page.