|Top Previous Next|
Maximum number of crawler threads to use. The maximum number cannot be greater than 50 threads.
Important Notice: Usage of too many crawler threads can do more harm than good. The remote server administrator can threat your requests as a Denial-Of-Service attack and take appropriate actions against you. The reasonable number - 3 to 8 crawlers per host name.
Why 50 Threads Then And Not Just 8?
Please read this topic for an explanation.