Search Results for

    Show / Hide Table of Contents

    Class CommonUserAgents

    Static helper class which lists common web-browser user-agent strings.

    Inheritance
    System.Object
    CommonUserAgents
    Namespace: IronWebScraper
    Assembly: IronWebScraper.dll
    Syntax
    public static class CommonUserAgents : Object

    Ready-made browser user-agent strings for a scraper to send live in CommonUserAgents. Presenting a real browser's user agent, rather than a default library string, helps a crawler blend in with ordinary traffic. The static arrays are grouped by client: ChromeDesktopUserAgents, FireFoxDesktopUserAgents, IE11DesktopUserAgents, and SafariDesktopUserAgents for desktops, and IPhoneUserAgents, IPadUserAgents, and WindowsTabletUserAgents for mobile devices. The All property combines every list into one.

    Assign a value to HttpIdentity.UserAgent, or pick from All at random across several identities so requests vary. Because each property is a plain string[], indexing or random selection works directly. Real sites see a spread of browsers and devices, so drawing user agents from more than one of these lists across a pool of identities makes a crawl's traffic look ordinary rather than like a single robotic client. Each array holds several current strings, so even one category offers variety.

    identity.UserAgent = CommonUserAgents.ChromeDesktopUserAgents[0];

    The advanced web scraping tutorial varies user agents across identities, and the movie-site how-to applies them to a real crawl.

    Fields

    ChromeDesktopUserAgents

    Common desktop Chrome web-browser user-agent strings, most popular first.

    Declaration
    public static string[] ChromeDesktopUserAgents
    Field Value
    Type Description
    System.String[]

    FireFoxDesktopUserAgents

    Common desktop FireFox web-browser user-agent strings, most popular first.

    Declaration
    public static string[] FireFoxDesktopUserAgents
    Field Value
    Type Description
    System.String[]

    IE11DesktopUserAgents

    Common IE11 desktop web-browser user-agent strings, most popular first. Only IE11 is listed for IE to give the best chance of consistent HTML being served.

    Declaration
    public static string[] IE11DesktopUserAgents
    Field Value
    Type Description
    System.String[]

    IPadUserAgents

    Common Apple iApd user agent strings.

    Declaration
    public static string[] IPadUserAgents
    Field Value
    Type Description
    System.String[]

    IPhoneUserAgents

    Common Apple iPhone user agent strings.

    Declaration
    public static string[] IPhoneUserAgents
    Field Value
    Type Description
    System.String[]

    SafariDesktopUserAgents

    Common OS X Safari web-browser user-agent strings, most popular first.

    Declaration
    public static string[] SafariDesktopUserAgents
    Field Value
    Type Description
    System.String[]

    WindowsTabletUserAgents

    Common Windows Tablet user-agent strings.

    Declaration
    public static string[] WindowsTabletUserAgents
    Field Value
    Type Description
    System.String[]

    Properties

    All

    Common desktop, tablet and mobile user-agent strings.

    Declaration
    public static string[] All { get; }
    Property Value
    Type Description
    System.String[]

    DesktopUserAgents

    Common desktop web-browser user-agent strings. Chrome, IE11 and Safari.

    Declaration
    public static string[] DesktopUserAgents { get; }
    Property Value
    Type Description
    System.String[]

    MobileUserAgents

    Common mobile web-browser user-agent strings. iPad, iPhone and Windows Tablet.

    Declaration
    public static string[] MobileUserAgents { get; }
    Property Value
    Type Description
    System.String[]
    ☀
    ☾
    Downloads
    • Download with Nuget
    • Start for Free
    In This Article
    Back to top
    Install with Nuget