Class WebScraper.Throttle
Throttle remote clients by their host name or by their public IP address.
Inheritance
Namespace: IronWebScraper
Assembly: IronWebScraper.dll
Syntax
public sealed class Throttle : Enum
Throttle selects how a WebScraper groups its rate limiting, assigned to WebScraper.ThrottleMode. ByDomainHostName paces requests separately for each host name, which suits crawling several independent sites at once. ByIpAddress paces by the resolved server IP, so multiple host names served from one machine share a single rate limit, which is the polite choice when many subdomains sit behind one server. Pair the mode with RateLimitPerHost to set the actual pace. The advanced web scraping tutorial configures throttling on a live crawl.
scraper.ThrottleMode = Throttle.ByIpAddress;Fields
ByDomainHostName
Identify unique hosts by their host name
Declaration
public const WebScraper.Throttle ByDomainHostName
Field Value
| Type | Description |
|---|---|
| WebScraper.Throttle |
ByIpAddress
Identify unique hosts by resolving the IP address for each host in-case multiple host names share an IP address.
Declaration
public const WebScraper.Throttle ByIpAddress
Field Value
| Type | Description |
|---|---|
| WebScraper.Throttle |
value__
Declaration
public int value__
Field Value
| Type | Description |
|---|---|
| System.Int32 |