How to Scrape Data from Websites in C

필수 역량

1. C#이나 VB.NET과 같은 Microsoft 프로그래밍 언어 중 하나를 사용하여 프로그래밍하는 데 필요한 기본 지식과 기술
2. 웹 기술(HTML, JavaScript, jQuery, CSS 등) 및 그 작동 원리에 대한 기본적인 이해
3. DOM, XPath, HTML 및 CSS 선택자에 대한 기본 지식

도구

1. Microsoft Visual Studio 2010 이상
2. Chrome용 웹 인스펙터나 Firefox용 Firebug와 같은 브라우저용 웹 개발자 확장 프로그램

NuGet을 사용하여 설치

수동 설치

1. 다음으로 이동하십시오 </https:>
2. IronWebscraper를 클릭하거나 URL을 사용하여 해당 페이지를 직접 방문하십시오. https://ironsoftware.com/csharp/webscraper/
3. 'DLL 다운로드'를 클릭합니다.
4. 다운로드한 압축 파일을 추출합니다

5. Visual Studio에서 프로젝트를 마우스 오른쪽 버튼으로 클릭 -> 추가 -> 참조 -> 찾아보기

   

6. 추출된 폴더로 이동 -> netstandard2.0 -> 모든 .dll 파일을 선택합니다

   

7. 이제 끝났습니다!

IronWebscraper 샘플 생성 단계

1. 폴더를 생성하고 이름을 "HelloScraperSample"로 지정하십시오.

2. 그런 다음 새 클래스를 추가하고 이름을 HelloScraper로 지정합니다.

3. 이 코드 스니펫을 HelloScraper에 추가하십시오.

   public class HelloScraper : WebScraper
   {
       /// <summary>
       /// Override this method to initialize your web scraper.
       /// Important tasks will be to request at least one start URL and set allowed/banned domain or URL patterns.
       /// </summary>
       public override void Init()
       {
           License.LicenseKey = "LicenseKey"; // Write License Key
           this.LoggingLevel = WebScraper.LogLevel.All; // Log all events
           this.Request("https://blog.scrapinghub.com", Parse); // Initialize a web request to the given URL
       }
   
       /// <summary>
       /// Override this method to create the default Response handler for your web scraper.
       /// If you have multiple page types, you can add additional similar methods.
       /// </summary>
       /// <param name="response">The HTTP Response object to parse</param>
       public override void Parse(Response response)
       {
           // Set working directory for the project
           this.WorkingDirectory = AppSetting.GetAppRoot() + @"\HelloScraperSample\Output\";
           // Loop on all links
           foreach (var titleLink in response.Css("h2.entry-title a"))
           {
               // Read link text
               string title = titleLink.TextContentClean;
               // Save result to file
               Scrape(new ScrapedData() { { "Title", title } }, "HelloScraper.json");
           }
   
           // Loop on all links for pagination
           if (response.CssExists("div.prev-post > a[href]"))
           {
               // Get next page URL
               var nextPage = response.Css("div.prev-post > a[href]")[0].Attributes["href"];
               // Scrape next URL
               this.Request(nextPage, Parse);
           }
       }
   }

   public class HelloScraper : WebScraper
   {
       /// <summary>
       /// Override this method to initialize your web scraper.
       /// Important tasks will be to request at least one start URL and set allowed/banned domain or URL patterns.
       /// </summary>
       public override void Init()
       {
           License.LicenseKey = "LicenseKey"; // Write License Key
           this.LoggingLevel = WebScraper.LogLevel.All; // Log all events
           this.Request("https://blog.scrapinghub.com", Parse); // Initialize a web request to the given URL
       }
   
       /// <summary>
       /// Override this method to create the default Response handler for your web scraper.
       /// If you have multiple page types, you can add additional similar methods.
       /// </summary>
       /// <param name="response">The HTTP Response object to parse</param>
       public override void Parse(Response response)
       {
           // Set working directory for the project
           this.WorkingDirectory = AppSetting.GetAppRoot() + @"\HelloScraperSample\Output\";
           // Loop on all links
           foreach (var titleLink in response.Css("h2.entry-title a"))
           {
               // Read link text
               string title = titleLink.TextContentClean;
               // Save result to file
               Scrape(new ScrapedData() { { "Title", title } }, "HelloScraper.json");
           }
   
           // Loop on all links for pagination
           if (response.CssExists("div.prev-post > a[href]"))
           {
               // Get next page URL
               var nextPage = response.Css("div.prev-post > a[href]")[0].Attributes["href"];
               // Scrape next URL
               this.Request(nextPage, Parse);
           }
       }
   }

   Public Class HelloScraper
   	Inherits WebScraper
   
   	''' <summary>
   	''' Override this method to initialize your web scraper.
   	''' Important tasks will be to request at least one start URL and set allowed/banned domain or URL patterns.
   	''' </summary>
   	Public Overrides Sub Init()
   		License.LicenseKey = "LicenseKey" ' Write License Key
   		Me.LoggingLevel = WebScraper.LogLevel.All ' Log all events
   		Me.Request("https://blog.scrapinghub.com", AddressOf Parse) ' Initialize a web request to the given URL
   	End Sub
   
   	''' <summary>
   	''' Override this method to create the default Response handler for your web scraper.
   	''' If you have multiple page types, you can add additional similar methods.
   	''' </summary>
   	''' <param name="response">The HTTP Response object to parse</param>
   	Public Overrides Sub Parse(ByVal response As Response)
   		' Set working directory for the project
   		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\HelloScraperSample\Output\"
   		' Loop on all links
   		For Each titleLink In response.Css("h2.entry-title a")
   			' Read link text
   			Dim title As String = titleLink.TextContentClean
   			' Save result to file
   			Scrape(New ScrapedData() From {
   				{ "Title", title }
   			},
   			"HelloScraper.json")
   		Next titleLink
   
   		' Loop on all links for pagination
   		If response.CssExists("div.prev-post > a[href]") Then
   			' Get next page URL
   			Dim nextPage = response.Css("div.prev-post > a[href]")(0).Attributes("href")
   			' Scrape next URL
   			Me.Request(nextPage, AddressOf Parse)
   		End If
   	End Sub
   End Class

   $vbLabelText   $csharpLabel

4. 이제 스크래핑을 시작하려면 다음 코드 스니펫을 Main에 추가하십시오.

   static void Main(string[] args)
   {
       // Create Object From Hello Scrape class
       HelloScraperSample.HelloScraper scrape = new HelloScraperSample.HelloScraper();
       // Start Scraping
       scrape.Start();
   }

   static void Main(string[] args)
   {
       // Create Object From Hello Scrape class
       HelloScraperSample.HelloScraper scrape = new HelloScraperSample.HelloScraper();
       // Start Scraping
       scrape.Start();
   }

   Shared Sub Main(ByVal args() As String)
   	' Create Object From Hello Scrape class
   	Dim scrape As New HelloScraperSample.HelloScraper()
   	' Start Scraping
   	scrape.Start()
   End Sub

   $vbLabelText   $csharpLabel
5. 결과물은 WebScraper.WorkingDirectory/classname.Json 형식의 파일로 저장됩니다.

코드 개요

Scrape.Start()는 다음과 같이 스크래핑 로직을 실행합니다:

1. Init() 메서드를 호출하여 변수를 초기화하고, 속성 및 동작 속성을 스크래핑합니다.
2. Init()에서 Request("https://blog.scrapinghub.com", Parse)을 사용하여 시작 페이지 요청을 설정합니다.
3. 여러 HTTP 요청과 스레드를 병렬로 처리하여 코드의 동기화를 유지하고 디버깅을 용이하게 합니다.
4. Parse() 메서드는 Init() 이후에 호출되어 응답을 처리하며, CSS 선택자를 사용하여 데이터를 추출하고 JSON 형식으로 저장합니다.

온라인 영화 웹사이트 스크래핑

영화 웹사이트를 스크래핑하는 예제를 만들어 보겠습니다.

새 클래스를 추가하고 이름을 MovieScraper로 지정하십시오:

HTML 구조

다음은 웹사이트의 홈페이지 HTML 중 일부입니다:

<div id="movie-featured" class="movies-list movies-list-full tab-pane in fade active">
    <div data-movie-id="20746" class="ml-item">
        <a href="https://website.com/film/king-arthur-legend-of-the-sword-20746/">
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 class="lazy thumb mli-thumb" alt="King Arthur: Legend of the Sword"
                  src="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>King Arthur: Legend of the Sword</h2></span>
        </a>
    </div>
    <div data-movie-id="20724" class="ml-item">
        <a href="https://website.com/film/snatched-20724/" >
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 class="lazy thumb mli-thumb" alt="Snatched" 
                 src="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>Snatched</h2></span>
        </a>
    </div>
</div>

<div id="movie-featured" class="movies-list movies-list-full tab-pane in fade active">
    <div data-movie-id="20746" class="ml-item">
        <a href="https://website.com/film/king-arthur-legend-of-the-sword-20746/">
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 class="lazy thumb mli-thumb" alt="King Arthur: Legend of the Sword"
                  src="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>King Arthur: Legend of the Sword</h2></span>
        </a>
    </div>
    <div data-movie-id="20724" class="ml-item">
        <a href="https://website.com/film/snatched-20724/" >
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 class="lazy thumb mli-thumb" alt="Snatched" 
                 src="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>Snatched</h2></span>
        </a>
    </div>
</div>

보시다시피, 영화 ID, 제목, 상세 페이지 링크가 있습니다. 이제 이 데이터를 스크래핑해 보겠습니다:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("www.website.com", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movieId = div.GetAttribute("data-movie-id");
                var link = div.Css("a")[0];
                var movieTitle = link.TextContentClean;
                Scrape(new ScrapedData() { { "MovieId", movieId }, { "MovieTitle", movieTitle } }, "Movie.Jsonl");
            }
        }           
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("www.website.com", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movieId = div.GetAttribute("data-movie-id");
                var link = div.Css("a")[0];
                var movieTitle = link.TextContentClean;
                Scrape(new ScrapedData() { { "MovieId", movieId }, { "MovieTitle", movieTitle } }, "Movie.Jsonl");
            }
        }           
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("www.website.com", AddressOf Parse)
	End Sub

	Public Overrides Sub Parse(ByVal response As Response)
		For Each div In response.Css("#movie-featured > div")
			If div.GetAttribute("class") <> "clearfix" Then
				Dim movieId = div.GetAttribute("data-movie-id")
				Dim link = div.Css("a")(0)
				Dim movieTitle = link.TextContentClean
				Scrape(New ScrapedData() From {
					{ "MovieId", movieId },
					{ "MovieTitle", movieTitle }
				},
				"Movie.Jsonl")
			End If
		Next div
	End Sub
End Class

구조화된 Movie 클래스

형식화된 데이터를 저장하기 위해 movie 클래스를 구현해 보겠습니다:

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
}

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
}

Public Class Movie
    Public Property Id As Integer
    Public Property Title As String
    Public Property URL As String
End Class

이제 Movie 클래스를 사용하도록 코드를 업데이트해 보겠습니다:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://website.com/", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movie = new Movie
                {
                    Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
                    Title = div.Css("a")[0].TextContentClean,
                    URL = div.Css("a")[0].Attributes["href"]
                };
                Scrape(movie, "Movie.Jsonl");
            }
        }
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://website.com/", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movie = new Movie
                {
                    Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
                    Title = div.Css("a")[0].TextContentClean,
                    URL = div.Css("a")[0].Attributes["href"]
                };
                Scrape(movie, "Movie.Jsonl");
            }
        }
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("https://website.com/", AddressOf Parse)
	End Sub

	Public Overrides Sub Parse(ByVal response As Response)
		For Each div In response.Css("#movie-featured > div")
			If div.GetAttribute("class") <> "clearfix" Then
				Dim movie As New Movie With {
					.Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
					.Title = div.Css("a")(0).TextContentClean,
					.URL = div.Css("a")(0).Attributes("href")
				}
				Scrape(movie, "Movie.Jsonl")
			End If
		Next div
	End Sub
End Class

상세 페이지 스크래핑

자세한 정보를 위한 새로운 속성을 추가하도록 Movie 클래스를 확장해 봅시다:

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
    public string Description { get; set; }
    public List<string> Genre { get; set; }
    public List<string> Actor { get; set; }
}

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
    public string Description { get; set; }
    public List<string> Genre { get; set; }
    public List<string> Actor { get; set; }
}

Public Class Movie
    Public Property Id As Integer
    Public Property Title As String
    Public Property URL As String
    Public Property Description As String
    Public Property Genre As List(Of String)
    Public Property Actor As List(Of String)
End Class

그런 다음 상세 페이지로 이동하여 IronWebScraper의 확장 기능을 사용하여 데이터를 스크래핑합니다:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://domain/", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movie = new Movie
                {
                    Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
                    Title = div.Css("a")[0].TextContentClean,
                    URL = div.Css("a")[0].Attributes["href"]
                };
                this.Request(movie.URL, ParseDetails, new MetaData() { { "movie", movie } });
            }
        }           
    }

    public void ParseDetails(Response response)
    {
        var movie = response.MetaData.Get<Movie>("movie");
        var div = response.Css("div.mvic-desc")[0];
        movie.Description = div.Css("div.desc")[0].TextContentClean;
        movie.Genre = div.Css("div > p > a").Select(element => element.TextContentClean).ToList();
        movie.Actor = div.Css("div > p:nth-child(2) > a").Select(element => element.TextContentClean).ToList();

        Scrape(movie, "Movie.Jsonl");
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://domain/", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var div in response.Css("#movie-featured > div"))
        {
            if (div.GetAttribute("class") != "clearfix")
            {
                var movie = new Movie
                {
                    Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
                    Title = div.Css("a")[0].TextContentClean,
                    URL = div.Css("a")[0].Attributes["href"]
                };
                this.Request(movie.URL, ParseDetails, new MetaData() { { "movie", movie } });
            }
        }           
    }

    public void ParseDetails(Response response)
    {
        var movie = response.MetaData.Get<Movie>("movie");
        var div = response.Css("div.mvic-desc")[0];
        movie.Description = div.Css("div.desc")[0].TextContentClean;
        movie.Genre = div.Css("div > p > a").Select(element => element.TextContentClean).ToList();
        movie.Actor = div.Css("div > p:nth-child(2) > a").Select(element => element.TextContentClean).ToList();

        Scrape(movie, "Movie.Jsonl");
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("https://domain/", AddressOf Parse)
	End Sub

	Public Overrides Sub Parse(ByVal response As Response)
		For Each div In response.Css("#movie-featured > div")
			If div.GetAttribute("class") <> "clearfix" Then
				Dim movie As New Movie With {
					.Id = Convert.ToInt32(div.GetAttribute("data-movie-id")),
					.Title = div.Css("a")(0).TextContentClean,
					.URL = div.Css("a")(0).Attributes("href")
				}
				Me.Request(movie.URL, AddressOf ParseDetails, New MetaData() From {
					{ "movie", movie }
				})
			End If
		Next div
	End Sub

	Public Sub ParseDetails(ByVal response As Response)
		Dim movie = response.MetaData.Get(Of Movie)("movie")
		Dim div = response.Css("div.mvic-desc")(0)
		movie.Description = div.Css("div.desc")(0).TextContentClean
		movie.Genre = div.Css("div > p > a").Select(Function(element) element.TextContentClean).ToList()
		movie.Actor = div.Css("div > p:nth-child(2) > a").Select(Function(element) element.TextContentClean).ToList()

		Scrape(movie, "Movie.Jsonl")
	End Sub
End Class

HttpIdentity 기능

일부 시스템은 콘텐츠를 보려면 사용자가 로그인해야 합니다; 인증 정보에는 HttpIdentity를 사용하십시오:

HttpIdentity id = new HttpIdentity
{
    NetworkUsername = "username",
    NetworkPassword = "pwd"
};
Identities.Add(id);

HttpIdentity id = new HttpIdentity
{
    NetworkUsername = "username",
    NetworkPassword = "pwd"
};
Identities.Add(id);

Dim id As New HttpIdentity With {
	.NetworkUsername = "username",
	.NetworkPassword = "pwd"
}
Identities.Add(id)

웹 캐시 활성화

개발 과정에서 재사용할 수 있도록 요청한 페이지를 캐시합니다:

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    EnableWebCache();
    this.Request("http://www.WebSite.com", Parse);
}

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    EnableWebCache();
    this.Request("http://www.WebSite.com", Parse);
}

Public Overrides Sub Init()
	License.LicenseKey = "LicenseKey"
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	EnableWebCache()
	Me.Request("http://www.WebSite.com", Parse)
End Sub

스로틀링

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    this.MaxHttpConnectionLimit = 80;
    this.RateLimitPerHost = TimeSpan.FromMilliseconds(50);
    this.OpenConnectionLimitPerHost = 25;
    this.ObeyRobotsDotTxt = false;
    this.ThrottleMode = Throttle.ByDomainHostName;
    this.Request("https://www.Website.com", Parse);
}

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    this.MaxHttpConnectionLimit = 80;
    this.RateLimitPerHost = TimeSpan.FromMilliseconds(50);
    this.OpenConnectionLimitPerHost = 25;
    this.ObeyRobotsDotTxt = false;
    this.ThrottleMode = Throttle.ByDomainHostName;
    this.Request("https://www.Website.com", Parse);
}

Public Overrides Sub Init()
	License.LicenseKey = "LicenseKey"
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	Me.MaxHttpConnectionLimit = 80
	Me.RateLimitPerHost = TimeSpan.FromMilliseconds(50)
	Me.OpenConnectionLimitPerHost = 25
	Me.ObeyRobotsDotTxt = False
	Me.ThrottleMode = Throttle.ByDomainHostName
	Me.Request("https://www.Website.com", Parse)
End Sub

Windows Form 애플리케이션을 만드는 방법은?

Visual Studio 2013 이상을 사용하십시오.

1. Visual Studio를 엽니다.

2. 파일 -> 새로 만들기 -> 프로젝트

3. Visual C# 또는 VB -> Windows -> Windows Forms Application을 선택합니다.

프로젝트 이름: IronScraperSample
위치: 디스크에서 위치를 선택하십시오.

ASP.NET 웹 폼 애플리케이션을 만드는 방법은?

1. Visual Studio를 엽니다.

2. 파일 -> 새로 만들기 -> 프로젝트

3. Visual C# 또는 VB -> 웹 -> ASP.NET 웹 애플리케이션 (.NET Framework)을 선택합니다.

프로젝트 이름: IronScraperSample
위치: 디스크에서 위치를 선택하십시오.

1. ASP.NET 템플릿에서 빈 템플릿을 선택하고 Web Forms를 선택합니다.

2. 기본 ASP.NET 웹 폼 프로젝트가 생성되었습니다.

전체 튜토리얼 샘플 프로젝트 코드는 여기에서 다운로드할 수 있습니다.