Search Results for

    Show / Hide Table of Contents

    Class WebScraper.Throttle

    Throttle remote clients by their host name or by their public IP address.

    Inheritance
    System.Object
    WebScraper.Throttle
    Namespace: IronWebScraper
    Assembly: IronWebScraper.dll
    Syntax
    public sealed class Throttle : Enum

    Throttle selects how a WebScraper groups its rate limiting, assigned to WebScraper.ThrottleMode. ByDomainHostName paces requests separately for each host name, which suits crawling several independent sites at once. ByIpAddress paces by the resolved server IP, so multiple host names served from one machine share a single rate limit, which is the polite choice when many subdomains sit behind one server. Pair the mode with RateLimitPerHost to set the actual pace. The advanced web scraping tutorial configures throttling on a live crawl.

    scraper.ThrottleMode = Throttle.ByIpAddress;

    Fields

    ByDomainHostName

    Identify unique hosts by their host name

    Declaration
    public const WebScraper.Throttle ByDomainHostName
    Field Value
    Type Description
    WebScraper.Throttle

    ByIpAddress

    Identify unique hosts by resolving the IP address for each host in-case multiple host names share an IP address.

    Declaration
    public const WebScraper.Throttle ByIpAddress
    Field Value
    Type Description
    WebScraper.Throttle

    value__

    Declaration
    public int value__
    Field Value
    Type Description
    System.Int32
    ☀
    ☾
    Downloads
    • Download with Nuget
    • Start for Free
    In This Article
    Back to top
    Install with Nuget