# IronWebscraper > The C# Webscraping Library. C# framework for extracting clean, structured data from html web applications. Useful for system migrations, populating search engines, competitive analysis and data mining. A single-file summary of the IronWebscraper documentation: 13 pages, each as its own abstract, so a model can read the documentation in one pass rather than fetching every URL. Blog posts are not included — see https://ironsoftware.com/webscraper/csharp/llms.txt for the link index, and https://ironsoftware.com/webscraper/csharp/llms-full-catalog.txt for one that covers the blog too. ## Getting Started ### Applying .NET 10 WebScraper License Keys | IronWebScraper https://ironsoftware.com/webscraper/csharp/get-started/license-keys/ IronWebScraper provides a streamlined process for implementing and managing license keys in C# applications. Upon purchasing a license, users immediately receive their key, which is displayed during checkout and sent via email. To integrate the license key, it can be set programmatically at the startup of an application using a simple line of C# code or configured globally in Web.Config or App.Config files by adding the key in the appSettings section. However, there is a known issue affecting ASP.NET projects and .NET Framework versions 4.6.2 and higher, where the key in Web.Config is not recognized. Users can refer to the troubleshooting article for solutions. To ensure the key is correctly installed, developers can run a test using a provided code snippet that checks the validity of the license key. The guide encourages users to explore further resources on getting started with IronWebScraper for efficient web scraping projects. ## Tutorials ### C# Web Scraping Tutorial for Wordpress and Blogs| Iron WebScraper https://ironsoftware.com/webscraper/csharp/tutorials/c-sharp-blog-web-scraper/ In the article "C# Web Scraping Tutorial for WordPress and Blogs" by Iron WebScraper, readers are guided through the process of creating a web scraper for blogs using C# and the IronWebScraper library. The tutorial specifically focuses on scraping content from WordPress blogs or similar platforms using .NET languages like C# or VB.NET. The process begins by defining a class, BlogScraper, which inherits from IronWebScraper's WebScraper class. The tutorial explains how to initialize the scraper by setting a license key, enabling logging, and configuring caching to store requested pages. A key feature of the process is the Parse method, which handles HTTP responses and extracts links from navigation sections to categorize pages such as Reviews and Science. The article provides a detailed explanation on setting up an object model, ScienceModel, to represent the structure of data found on Science pages. It then demonstrates how to parse individual posts to extract details like title, author, date, image, and text, and save this data in JSONL format. The tutorial aims to simplify web scraping in C# by leveraging IronWebScraper, providing a practical example for users to adapt to their blogging needs. ### Advanced C# Webscraping Settings, Configurations & Options https://ironsoftware.com/webscraper/csharp/tutorials/webscraping-in-c-sharp-advanced/ The article titled "Advanced Settings, Configurations, and Options" on IronWebScraper's website delves into sophisticated features of the IronWebScraper C# tool, focusing on enhancing web scraping efficiency and avoiding detection. A key feature discussed is `HttpIdentity`, which enables multiple login sessions using unique user credentials and browser engines. This functionality allows for authenticating users on sites requiring login, and configuring properties such as `NetworkDomain`, `NetworkUsername`, `NetworkPassword`, `Proxy`, `UserAgent`, and `UseCookies`. The article also highlights the web cache feature, which caches requested pages for reuse, conducive for development and testing phases, allowing execution on cached pages without reconnecting to live websites. Additionally, the tutorial explains how to resume scraping after a restart using `Start(CrawlID)`, which saves execution requests and responses for continuity. Throttling is another advanced feature, enabling control over connection numbers and speed per domain to avoid overloading servers. The article provides sample code snippets to demonstrate these functionalities, enhancing the reader's ability to implement advanced scraping techniques effectively. Overall, the guide is a comprehensive resource for utilizing IronWebScraper's advanced capabilities for efficient and ethical web scraping. ### C# Web Scraping Tutorial for .NET| Iron WebScraper https://ironsoftware.com/webscraper/csharp/tutorials/webscraping-in-c-sharp/ The article from Iron Software provides a comprehensive guide on web scraping using the IronWebScraper, a .NET library designed for web data extraction and content parsing in C#. The tutorial is aimed at software developers with basic to advanced programming skills, focusing on building robust web scraping solutions. It covers the installation of IronWebScraper via NuGet or manual methods, and guides users through creating a simple console application, "HelloScraper," to demonstrate basic scraping operations. The tutorial details the necessary programming skills and tools, including Microsoft Visual Studio and browser developer extensions. It also explains the importance of web scraping for data extraction and content comparison across websites. Further, the article elaborates on the library's functionalities, such as URL and domain management, request throttling, and caching. It provides step-by-step instructions for creating a more complex "MovieScraper" application, which includes scraping detailed information from a movie website. The guide discusses extending scraping logic to handle detailed pages, implementing user authentication for restricted content access, and optimizing requests with throttling and caching features. Additionally, the tutorial includes appendices on setting up Windows Form and ASP.NET Web Form applications, enhancing the reader's ability to integrate web scraping into various C# projects. ## How-To Guides ### C# Shopping Website Scraper Tutorial | IronWebScraper https://ironsoftware.com/webscraper/csharp/how-to/scraping-from-a-shopping-website/ This tutorial demonstrates how to build a C# shopping website scraper using the IronWebScraper library to extract product data from e-commerce sites. The guide begins with a quickstart example showing how to create a basic scraper that inherits from the WebScraper class, sets up licensing, and implements Parse() methods to extract product information like names, prices, and images. The tutorial emphasizes the importance of analyzing HTML structure before coding, using browser developer tools to understand how product categories and items are organized on the target site. The example focuses on scraping a fashion website with hierarchical categories (Men, Women, Kids) and their subcategories. Key steps include creating a Console App project, defining custom models for Category and Product data, overriding Init() and Parse() methods, and saving extracted data to JSONL files. The tutorial highlights that understanding the HTML structure is crucial for designing appropriate data models and scraping logic, especially when dealing with complex e-commerce sites that have nested categories and varied product layouts. This comprehensive approach helps developers build robust scrapers capable of extracting structured data from shopping websites efficiently. ### C# Web Scraper for Movie Websites | IronWebScraper Guide https://ironsoftware.com/webscraper/csharp/how-to/scraping-from-an-online-movie-website/ This article provides a comprehensive guide on using IronWebScraper, a C# library, to extract movie data from websites. The tutorial demonstrates how to parse HTML elements, create structured datasets, and navigate between pages to build movie information databases. It begins with a quickstart example showing the basic setup: installing IronWebScraper via NuGet, creating a class that inherits from WebScraper, and overriding Init() and Parse() methods to configure settings and extract data using CSS selectors. The guide includes practical code examples demonstrating how to scrape movie titles and URLs, save data in JSON format, and handle real-world website structures. Key features covered include setting license keys, configuring logging levels, defining working directories, and using the Scrape() method for data storage. The article emphasizes analyzing HTML structure to identify correct CSS selectors and follows object-oriented principles by creating dedicated scraper classes. This approach transforms unstructured web content into organized, analyzable data suitable for various applications. The tutorial is particularly useful for developers looking to automate movie data collection from websites using C# and provides a foundation for building more complex web scraping solutions. ## Code Examples ### Code Examples : C# Web Scraper | Iron WebScraper https://ironsoftware.com/webscraper/csharp/examples/c-sharp-web-scraper/ The article titled "Code Examples: C# Web Scraper | Iron WebScraper" from Iron Software provides a comprehensive guide on building a web scraper using C#. The IronWebScraper framework is highlighted as a robust tool for extracting data and files from websites. The guide begins with instructions on installing IronWebScraper via NuGet and creating a class that extends the WebScraper. It outlines the development of an Init method to initiate URL parsing through the Request method. Additionally, the guide explains the creation of a Parse method to handle requests and further explore additional pages. The use of response.Css is recommended for working with HTML elements using CSS selectors similar to jQuery. The article suggests creating an instance of the web scraping class and invoking the Start() method to begin the scraping process. For those seeking to build advanced web crawlers, the article references additional C# web scraping tutorials available through IronWebScraper. The document also provides links to related documentation, how-to guides, and class documentation, as well as options to download the IronWebScraper DLL and report issues. This resource serves as an efficient starting point for developers interested in web scraping using C#. ## Troubleshooting ### Apply a license key in IronWebScraper https://ironsoftware.com/webscraper/csharp/troubleshooting/apply-a-license-key-in-ironwebscraper/ The article titled "Apply a License Key in IronWebScraper" provides a detailed guide on how to correctly apply a license key to the IronWebScraper software to ensure proper licensing. It emphasizes the importance of copying the license key accurately, which is a long string ending with an expiry date, such as "IRONWEBSCRAPER-MYLICENSE-KEY-1EF01-EXPIRES.01.JAN.2050". The most reliable method to integrate the license key is by setting it directly in the code before invoking the library. This is crucial to utilize the library without restrictions. The article includes a sample C# code snippet illustrating how to assign the license key within the software environment. Additionally, it mentions alternative methods for applying the license key using web.config or appsettings.json files. For more in-depth options and troubleshooting, the article directs readers to a specific FAQ page on IronWebScraper's official website. This step-by-step guidance ensures users can effectively manage their web scraping software's licensing needs. ### IronWebScraper & JavaScript https://ironsoftware.com/webscraper/csharp/troubleshooting/ironwebscraper-and-javascript/ The article titled "IronWebScraper & JavaScript" on the Iron Software website provides a troubleshooting guide for developers using IronWebScraper, particularly in handling JavaScript content. The key takeaway is that IronWebScraper does not process JavaScript, making it well-suited for scraping large, structured content-based portals and standard HTML websites using implemented parse methods. This approach allows for quicker processing as it does not require a full browser model like Chrome, making it significantly faster. The article emphasizes the importance of targeting non-Ajax alternative pages for scraping, particularly on standards-adherent Ajax websites, as these pages can offer all the necessary content without the complications of JavaScript. This strategy enhances web scraping performance and efficiency by focusing on HTML content as written, avoiding the complexities and time consumption associated with processing JavaScript. The article serves as a practical guide for developers to navigate the limitations of IronWebScraper while optimizing their web scraping projects. ### IronWebScraper - Security CVE https://ironsoftware.com/webscraper/csharp/troubleshooting/ironwebscraper-security-cve/ The article on IronWebScraper security highlights the robust measures in place to protect users from vulnerabilities. IronWebScraper, a product of Iron Software, is DigiCert certified, ensuring a high level of trust and security. The platform does not utilize web services or transmit data over the internet, significantly reducing exposure to potential threats. It is designed without exposing COM or COM+ interfaces, and its C# foundation offers a natural defense against various common attack vectors. The software limits API entry points, thereby minimizing exploitation risks, and employs strong naming and advanced tamper protection techniques. Regular scans using advanced anti-virus and anti-malware tools are conducted to identify potential threats. Moreover, each line of code is reviewed by senior engineers to detect security vulnerabilities. Importantly, IronWebScraper avoids unmanaged code, further reducing vulnerability risks. It integrates several .NET dependencies, such as CsQuery, HTMLAgilityPack, and Newtonsoft.Json, none of which have known security vulnerabilities due to internalized static linking. This comprehensive security approach ensures IronWebScraper remains a safe tool for web scraping, with continuous vigilance against potential threats, safeguarding user projects effectively. ### Troubleshoot IronWebScraper License | IronWebScraper https://ironsoftware.com/webscraper/csharp/troubleshooting/license-key-web.config/ The article titled "Troubleshoot IronWebScraper License" on IronWebScraper's website addresses a licensing issue encountered with IronWebScraper in ASP.NET projects using the .NET Framework version 4.6.2 or higher. The problem arises when the license key stored in a `Web.config` file is not recognized by the product in versions between 2023.4.13 and 2024.3.6. Although this issue has been resolved in version 2024.3.6, users of older versions are provided with a workaround. The suggested solution involves manually retrieving the license key from the `Web.config` file using the `ConfigurationManager` in the code and then applying it to the `IronWebScraper.License.LicenseKey` property. The article provides a code example demonstrating how to implement this workaround. By utilizing the `ConfigurationManager` to access the license key from the `appSettings` section of the `Web.config` file, developers can ensure seamless functionality of IronWebScraper. This guidance allows for uninterrupted web scraping by addressing licensing exceptions that previously hindered development. ### Thread Safety https://ironsoftware.com/webscraper/csharp/troubleshooting/thread-safety/ The article titled "Thread Safety" on the Iron Software website provides a comprehensive overview of ensuring thread safety while using IronWebScraper. It affirms that IronWebScraper is highly thread-safe, making it a reliable tool for handling multithreading in web scraping projects. The tool automatically manages thread pools, and its built-in concurrency and rate-limiting controls (`MaxHttpConnectionLimit`, `RateLimitPerHost`, and `Throttle`) manage concurrency to avoid placing unfair load on remote servers, ensuring that web scraping operations are both effective and considerate. By following the guidance in the article, users can ensure their web scraping projects are safe and efficient, leveraging IronWebScraper's capabilities to maintain optimal performance. The article serves as a valuable resource for developers seeking to maximize the efficiency and safety of their web scraping tasks through proper multithreading practices. ## Product Updates ### IronWebScraper Product Updates | IronWebScraper https://ironsoftware.com/webscraper/csharp/product-updates/changelog/ The IronWebScraper changelog provides detailed updates on the C# web scraping library's version releases, bug fixes, and performance enhancements. Notably, the June 2025 update introduced version 2025.6.1, which upgraded IronSoftware.System.Drawing to version 2025.6.2. In May 2025, version 2025.5.3 was released, updating IronSoftware.System.Drawing to 2025.5.3. The April 2025 update, version 2025.4.2, also focused on upgrading IronSoftware.System.Drawing to 2025.3.6. The changelog documents consistent improvements and updates spanning back to December 2023, with each release focusing on dependency updates and bug fixes. For example, the February 2024 release fixed a bug with the CSS method, while the December 2023 release addressed issues with web scraper functionality and license key recognition. Major updates include the January 2023 release, which removed support for .NET Framework 4.5.2 and added support for .NET 6 and 7. It also introduced changes such as the removal of the Installation class, improvements in Intellisense documentation, and the removal of dependency on System.Common.Drawing in favor of IronSoftware.System.Drawing. These updates highlight IronWebScraper's commitment to enhancing functionality and maintaining compatibility with the latest technology standards.