Get started: C# code examples

C# + VB.Net: C# Web Scraper C# Web Scraper
using IronWebScraper;

namespace WebScrapingProject
{
    class MainClass
    {
        public static void Main(string[] args)
        {
            var scraper = new BlogScraper();
            scraper.Start();
        }
    }

    class BlogScraper : WebScraper
    {
        public override void Init()
        {
            this.LoggingLevel = WebScraper.LogLevel.All;
            this.Request("https://blog.scrapinghub.com", Parse);
        }

        public override void Parse(Response response)
        {
            foreach (var title_link in response.Css("h2.entry-title a"))
            {
                string strTitle = title_link.TextContentClean;
                Scrape(new ScrapedData() { { "Title", strTitle } });
            }

            if (response.CssExists("div.prev-post > a[href]"))
            {
                var next_page = response.Css("div.prev-post > a[href]")[0].Attributes["href"];
                this.Request(next_page, Parse);
            }
        }
    }
}
Imports IronWebScraper

Namespace WebScrapingProject
    
    Class MainClass
        
        Public Shared Sub Main(ByVal args() As String)
            Dim scraper = New BlogScraper
            scraper.Start
        End Sub
    End Class
    
    Class BlogScraper
        Inherits WebScraper
        
        Public Overrides Sub Init()
            Me.LoggingLevel = WebScraper.LogLevel.All
            Me.Request("https://blog.scrapinghub.com", Parse)
        End Sub
        
        Public Overrides Sub Parse(ByVal response As Response)
            For Each title_link In response.Css("h2.entry-title a")
                Dim strTitle As String = title_link.TextContentClean
                Scrape(New ScrapedData)
            Next
            If response.CssExists("div.prev-post > a[href]") Then
                Dim next_page = response.Css("div.prev-post > a[href]")(0).Attributes("href")
                Me.Request(next_page, Parse)
            End If
            
        End Sub
    End Class
End Namespace

Iron WebScraper provides a powerful framework to extract data and files from websites using C# code.

  1. Install IronWebScraper to your Project using Nuget
  2. Create a Class Extending WebScraper
  3. Create an Init method that uses the Request method to parse at least one URL.
  4. Create a Parse method to process the requests, and indeed Request more pages. Use response.Css to work with HTML elements using jQuery style CSS selectors
  5. In your application please create and instance of your web scraping class and call the Start(); method
  6. Read our C# webscraping tutorials to learn who to create advanced web crawlers using IronWebScraper

Human Support Directly From Our Development Team

Whether its product, integration or licensing queries, the Iron product development team are on hand to support all of your questions. Get in touch and start a dialog with Iron to make the most of our library in your project.

Ask a Question

Powerful Scraping Engine Under Your Control

Just write a single C# web-scraper class to scrape thousands or even millions of web pages into C# Class Instances, JSON or Downloaded Files. IronWebScraper allows you to code concise, linear workflows simulating human browsing behavior. IronWebScraper will run your code as a swarm of virtual web browsers, massively paralleled, yet polite and fault tolerant.

Get Started with Documentation

Simple, Flexible Logic

IronWebScraper must be programmed to know how to handle each “type” of page it encounters. This is achieved in a very concise manner using CSS Selectors or XPath expressions and can be fully customized in C#. This freedom allows you to decide which pages to scrape within a website, and what to do with the data extracted. Each method can be debugged and watched neatly in Visual Studio.

Follow a Tutorial

Fast and Polite Behavior

IronWebScraper deals with multithreading and web-requests to allow for hundreds of concurrent threads without the developer needing to manage them. Politeness can be set to throttle requests, so reducing risk of excessive load on target web servers.

Up and Running with WebScraper

Create virtual user Identities

IronWebScraper can use one or multiple “identities” - sessions that simulate real world human requests. Each request may programmatically or randomly assign its own Identity, User Agent, Cookies, Logins and even IP addresses. Requests are set as auto-unique with a combination of URL, parse method and post variables.

See Object Reference

Action Replay

IronWebScraper uses advanced caching to allow developers to change their code “on the fly” and replay every previous request without contacting the internet. Every scrape job is autosaved and can be resumed in the event of an exception or power outage.

WebScraper Setup Instructions
Visual Studio Library for PDF Creation and Content Editing.

Rapid Installation with Microsoft Visual Studio

IronWebScraper puts Web Scraping tools in your own hands quickly with a Visual Studio installer. Whether installing directly from Nuget within visual studio or downloading the DLL, you’ll be setup in no time. Just one DLL and no dependancies.

PM > Install-Package IronWebScraper Download DLL
Supports:
  • .Net Framework 4.0 and above support C#, VB, F#
  • Microsoft Visual Studio. .Net Development IDE Icon
  • Nuget Installer Support for Visual Studio
  • JetBrains ReSharper C# language assistant compatible
  • Microsoft Azure C# .NET  hosting platform compatible

Licensing & Pricing

Free community development licenses. Commercial licenses from $399.

Project C# + VB.NET Library Licensing

Project

Developer C# + VB.NET Library Licensing

Developer

Organization C# + VB.NET Library Licensing

Organization

Agency C# + VB.NET Library Licensing

Agency

SaaS C# + VB.NET Library Licensing

SaaS

OEM C# + VB.NET Library Licensing

OEM

View Full License Options  

.Net Webscraping Community Tutorials

Tutorial + Code Examples Webscraping in .Net | VB.Net & ASP.Net PDF

VB C# ASP.NET

Web Scraping in C# and VB.Net Projects

See how Ahmed uses IronWebScraper in his projects to migrate content from one site to another. Sample Projects and Code provided for scraping ecommerce and blog websites

Ahmed Aboelmagd .Net Software Engineer View Ahmed's WebScraping Tutorial
Our customers use IronWebScraper with...

Accounting and Finance Systems

  • # Recepits
  • # Reporting
  • # Invoice Printing
Add PDF Support to ASP.Net Accounting and Finance Systems

Business Digitization

  • # Documentation
  • # Ordering & Labelling
  • # Paper Replacement
C# Business Digitization Use Cases

Enterprise Content Management

  • # Content Production
  • # Document Management
  • # Content Distribution
.Net CMS PDF Support

Data and Reporting Applications

  • # Performance Tracking
  • # Trend Mapping
  • # Reports
C# PDF Reports
Join Them Today
Iron Software Enterprise .Net Component Developers

Thousands of corporations, governments, SME's and developers alike trust Iron software products.

Iron's team have over 10 years experience in the .Net software component market.

Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon
Iron Software Customer Icon