Saturday, 23 June 2007

Crawling policies

There are three important characteristics of the Web that generate a scenario in which Web crawling is very difficult:

Source: http://en.wikipedia.org/wiki/Web_crawler

No comments: