Class CommonUserAgents
Static helper class which lists common web-browser user-agent strings.
Inheritance
Namespace: IronWebScraper
Assembly: IronWebScraper.dll
Syntax
public static class CommonUserAgents : Object
Ready-made browser user-agent strings for a scraper to send live in CommonUserAgents. Presenting a real browser's user agent, rather than a default library string, helps a crawler blend in with ordinary traffic. The static arrays are grouped by client: ChromeDesktopUserAgents, FireFoxDesktopUserAgents, IE11DesktopUserAgents, and SafariDesktopUserAgents for desktops, and IPhoneUserAgents, IPadUserAgents, and WindowsTabletUserAgents for mobile devices. The All property combines every list into one.
Assign a value to HttpIdentity.UserAgent, or pick from All at random across several identities so requests vary. Because each property is a plain string[], indexing or random selection works directly. Real sites see a spread of browsers and devices, so drawing user agents from more than one of these lists across a pool of identities makes a crawl's traffic look ordinary rather than like a single robotic client. Each array holds several current strings, so even one category offers variety.
identity.UserAgent = CommonUserAgents.ChromeDesktopUserAgents[0];The advanced web scraping tutorial varies user agents across identities, and the movie-site how-to applies them to a real crawl.
Fields
ChromeDesktopUserAgents
Common desktop Chrome web-browser user-agent strings, most popular first.
Declaration
public static string[] ChromeDesktopUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
FireFoxDesktopUserAgents
Common desktop FireFox web-browser user-agent strings, most popular first.
Declaration
public static string[] FireFoxDesktopUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
IE11DesktopUserAgents
Common IE11 desktop web-browser user-agent strings, most popular first. Only IE11 is listed for IE to give the best chance of consistent HTML being served.
Declaration
public static string[] IE11DesktopUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
IPadUserAgents
Common Apple iApd user agent strings.
Declaration
public static string[] IPadUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
IPhoneUserAgents
Common Apple iPhone user agent strings.
Declaration
public static string[] IPhoneUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
SafariDesktopUserAgents
Common OS X Safari web-browser user-agent strings, most popular first.
Declaration
public static string[] SafariDesktopUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
WindowsTabletUserAgents
Common Windows Tablet user-agent strings.
Declaration
public static string[] WindowsTabletUserAgents
Field Value
| Type | Description |
|---|---|
| System.String[] |
Properties
All
Common desktop, tablet and mobile user-agent strings.
Declaration
public static string[] All { get; }
Property Value
| Type | Description |
|---|---|
| System.String[] |
DesktopUserAgents
Common desktop web-browser user-agent strings. Chrome, IE11 and Safari.
Declaration
public static string[] DesktopUserAgents { get; }
Property Value
| Type | Description |
|---|---|
| System.String[] |
MobileUserAgents
Common mobile web-browser user-agent strings. iPad, iPhone and Windows Tablet.
Declaration
public static string[] MobileUserAgents { get; }
Property Value
| Type | Description |
|---|---|
| System.String[] |