Click or drag to resize

WebScraperSetSiteSpecificCrawlRateLimit Method

Set a throttle limit for a specific domain

Namespace:  IronWebScraper
Assembly:  IronWebScraper (in IronWebScraper.dll) Version: 4.0.4.25470 (4.0.4.3)
Syntax
public void SetSiteSpecificCrawlRateLimit(
	string hostName,
	TimeSpan crawlRate
)

Parameters

hostName
Type: SystemString
The http host (domain name).
crawlRate
Type: SystemTimeSpan
The maximum frequency of http requests for the given hostName.
See Also