Class MetaData
A flexible dictionary of object values which can be used to attach your own additional data or objects to any Request. Meta can contain objects of any Type including instances of classes, List and Dictionaries. This meta-data can then be accessed while Paring the Response and even passed forwards to the next Request.
Metadata send might include pagination page numbers, referrer Urls, User Ids etc.
E.g:Request["page-number"] = 2;
int pageNumber = Response.Request.Meta.Get<int>("page-number");Inheritance
Namespace: IronWebScraper
Assembly: IronWebScraper.dll
Syntax
public class MetaData : Dictionary<string, object>
MetaData is the per-request bag a scraper uses to carry its own objects from a Request through to the matching Response. It extends Dictionary<string, object>, so any value can be stored under a string key and read back when the page is parsed, which is how context such as a category, a parent record, or a page number reaches the parser.
Store values with the dictionary indexer and read them with the typed Get<T>(key) helper, which casts the stored object for you. Attach a MetaData to a WebScraper.Request call; the resulting Response.MetaData exposes the same instance during parsing. Because the bag holds object values, a whole record or domain model can ride along rather than just strings, which keeps a multi-step crawl stateful without external storage, and the typed Get<T> removes the cast at the read site.
var meta = new MetaData();
meta["pageType"] = "listing";
Request(url, Parse, meta);
// later, in Parse:
string type = response.MetaData.Get<string>("pageType");The blog web scraper tutorial threads context through MetaData, and the advanced tutorial uses it across multi-step crawls.
Constructors
MetaData()
Declaration
public MetaData()
Methods
Get<T>(String)
Returns the specified meta object cast as the appropriate type.
Get<int>("index")Declaration
public T Get<T>(string key)
Parameters
| Type | Name | Description |
|---|---|---|
| System.String | key | Key |
Returns
| Type | Description |
|---|---|
| T |
Type Parameters
| Name | Description |
|---|---|
| T | Type |
Exceptions
| Type | Condition |
|---|---|
| System.Collections.Generic.KeyNotFoundException |