1. IronWebScraper
2. Anleitungen
3. Web Scraping in C#

Wie man Daten von Websites in C# ausliest#

von Ahmed Aboelmagd

IronWebscraper ist eine .NET Bibliothek für Web Scraping, Web Daten Extraktion und Web Content Parsing. Es handelt sich um eine einfach zu verwendende Bibliothek, die zu Microsoft Visual Studio-Projekten hinzugefügt werden kann, um sie in der Entwicklung und Produktion einzusetzen.

IronWebscraper verfügt über viele einzigartige Funktionen und Möglichkeiten, wie z.B. die Kontrolle von erlaubten und verbotenen Seiten, Objekten, Medien, etc. Es ermöglicht auch die Verwaltung mehrerer Identitäten, Web-Cache und viele andere Funktionen, die wir in diesem Tutorial behandeln werden.

Erste Schritte mit IronWebscraper

Beginnen Sie noch heute mit der Verwendung von IronWebScraper in Ihrem Projekt mit einer kostenlosen Testversion.

Erster Schritt:

Kostenloser Start

Zielpublikum

Dieses Tutorial richtet sich an Softwareentwickler mit grundlegenden oder fortgeschrittenen Programmierkenntnissen, die Lösungen für erweiterte Scraping-Funktionen erstellen und implementieren möchten(scraping von Websites, Sammeln und Extrahieren von Website-Daten, Parsing von Website-Inhalten, Web Harvesting).

Erforderliche Fähigkeiten

1. Grundlegende Programmierkenntnisse mit Kenntnissen in einer der Microsoft-Programmiersprachen wie C# oder VB.NET

2. Grundlegendes Verständnis von Webtechnologien(HTML, JavaScript, JQuery, CSS, usw.) und wie sie funktionieren

3. Grundkenntnisse von DOM, XPath, HTML und CSS-Selektoren

Werkzeuge

1. Microsoft Visual Studio 2010 oder höher

2. Webentwickler-Erweiterungen für Browser wie der Web-Inspektor für Chrome oder Firebug für Firefox

Warum kratzen?

(Begründungen und Konzepte)

Wenn Sie ein Produkt oder eine Lösung entwickeln wollen, das/die dazu in der Lage ist:

1. Website-Daten extrahieren

2. Vergleichen Sie Inhalte, Preise, Funktionen usw. von mehreren Websites

3. Scannen und Zwischenspeichern von Website-Inhalten

   Wenn Sie einen oder mehrere der oben genannten Gründe haben, dann ist IronWebscraper eine großartige Bibliothek, die Ihren Bedürfnissen entspricht

Wie wird IronWebScraper installiert?

Nachdem Sie ein neues Projekt erstellt haben(Siehe Anhang A) können Sie die IronWebscraper-Bibliothek zu Ihrem Projekt hinzufügen, indem Sie die Bibliothek automatisch mit NuGet einfügen oder die DLL manuell installieren.

Installation mit NuGet

Um die IronWebScraper-Bibliothek mit NuGet zu unserem Projekt hinzuzufügen, können wir die visuelle Schnittstelle verwenden(NuGet-Paketmanager) oder per Befehl über die Paketmanager-Konsole.

NuGet-Paketmanager verwenden

1. Mit der Maus -> Rechtsklick auf den Projektnamen -> NuGet-Paket verwalten auswählen

   

2. Auf der Registerkarte Durchsuchen -> IronWebscraper suchen -> Installieren

   

3. Ok klicken

   

4. Und wir sind fertig

   

NuGet-Paketkonsole verwenden

1. Über Werkzeuge -> NuGet Package Manager -> Package Manager Console

   

2. Wählen Sie Klassenbibliotheksprojekt als Standardprojekt

3. Befehl ausführen -> Install-Package IronWebscraper

   

Manuell installieren

1. Gehe zuironsoftware.de

2. Klicken Sie auf IronWebscraper oder besuchen Sie seine Seite direkt über die URLhttps://ironsoftware.com/csharp/webscraper

3. Klicken Sie auf DLL herunterladen.

4. Heruntergeladene komprimierte Datei entpacken

5. Klicken Sie in Visual Studio mit der rechten Maustaste auf Projekt -> Hinzufügen -> Referenz -> Durchsuchen

   

6. Gehen Sie zum extrahierten Ordner -> "netstandard2.0" -> und wählen Sie alle ".dll"-Dateien aus

   

7. Und es ist vollbracht!

HelloScraper - Unser erstes IronWebScraper-Beispiel

Wie üblich beginnen wir mit der Implementierung der Hello Scraper App, um unseren ersten Schritt mit IronWebscraper zu machen.

• Wir haben eine neue Konsolenanwendung mit dem Namen "IronWebScraperSample" erstellt

  Schritte zur Erstellung eines IronWebscraper Beispiels

1. Erstellen Sie einen Ordner und nennen Sie ihn "HelloScraperSample"

2. Dann eine neue Klasse und nennen Sie sie "HelloScraper"

   

3. Fügen Sie diesen Codeschnipsel zu HelloScraper hinzu

public class HelloScraper : WebScraper
{
        /// <summary>
        /// Override this method initialize your web-scraper.
        /// Important tasks will be to Request at least one start url... and set allowed/banned domain or url patterns.
        /// </summary>
        public override void Init()
        {
            License.LicenseKey = "LicenseKey"; // Write License Key
            this.LoggingLevel = WebScraper.LogLevel.All; // All Events Are Logged
            this.Request("https://blog.scrapinghub.com", Parse);
        }

        /// <summary>
        /// Override this method to create the default Response handler for your web scraper.
        /// If you have multiple page types, you can add additional similar methods.
        /// </summary>
        /// <param name="response">The http Response object to parse</param>
        public override void Parse(Response response)
        {
            // set working directory for the project
            this.WorkingDirectory = AppSetting.GetAppRoot()+ @"\HelloScraperSample\Output\";
            // Loop on all Links
            foreach (var title_link in response.Css("h2.entry-title a"))
            {
                // Read Link Text
                string strTitle = title_link.TextContentClean;
                // Save Result to File
                Scrape(new ScrapedData() { { "Title", strTitle } }, "HelloScraper.json");
            }
            // Loop On All Links
            if (response.CssExists("div.prev-post > a [href]"))
            {
                // Get Link URL
                var next_page = response.Css("div.prev-post > a [href]")[0].Attributes ["href"];
                // Scrape Next URL
                this.Request(next_page, Parse);
            }
        }
}

public class HelloScraper : WebScraper
{
        /// <summary>
        /// Override this method initialize your web-scraper.
        /// Important tasks will be to Request at least one start url... and set allowed/banned domain or url patterns.
        /// </summary>
        public override void Init()
        {
            License.LicenseKey = "LicenseKey"; // Write License Key
            this.LoggingLevel = WebScraper.LogLevel.All; // All Events Are Logged
            this.Request("https://blog.scrapinghub.com", Parse);
        }

        /// <summary>
        /// Override this method to create the default Response handler for your web scraper.
        /// If you have multiple page types, you can add additional similar methods.
        /// </summary>
        /// <param name="response">The http Response object to parse</param>
        public override void Parse(Response response)
        {
            // set working directory for the project
            this.WorkingDirectory = AppSetting.GetAppRoot()+ @"\HelloScraperSample\Output\";
            // Loop on all Links
            foreach (var title_link in response.Css("h2.entry-title a"))
            {
                // Read Link Text
                string strTitle = title_link.TextContentClean;
                // Save Result to File
                Scrape(new ScrapedData() { { "Title", strTitle } }, "HelloScraper.json");
            }
            // Loop On All Links
            if (response.CssExists("div.prev-post > a [href]"))
            {
                // Get Link URL
                var next_page = response.Css("div.prev-post > a [href]")[0].Attributes ["href"];
                // Scrape Next URL
                this.Request(next_page, Parse);
            }
        }
}

IRON VB CONVERTER ERROR developers@ironsoftware.com

4. Jetzt zu starten Scrape Fügen Sie diesen Code Snippet zu Main

static void Main(string [] args)
{
        // Create Object From Hello Scrape class
        HelloScraperSample.HelloScraper scrape = new HelloScraperSample.HelloScraper();
            // Start Scraping
            scrape.Start();
}

static void Main(string [] args)
{
        // Create Object From Hello Scrape class
        HelloScraperSample.HelloScraper scrape = new HelloScraperSample.HelloScraper();
            // Start Scraping
            scrape.Start();
}

IRON VB CONVERTER ERROR developers@ironsoftware.com

5. Das Ergebnis wird in einer Datei mit dem Format WebSraper.WorkingDirecty/classname.Json gespeichert

   

Code-Übersicht

Scrape.Start() => feuert die Scrape-Logik wie folgt ab:

1. Aufruf der Init() Methode zunächst für die Initiierung der Variablen, scrapen Eigenschaften und Verhalten Attribute..,

2. Wie wir sehen können, wird die Startseite auf Request("https://blog.scrapinghub.com", Parsen) und Parsen(Antwort Antwort) definiert als der Prozess, der zur Analyse der Antwort verwendet wird.

3. Webscraper verwaltet parallel: http und Threads... damit Ihr Code einfach zu debuggen und synchron ist.

4. Start der Parse-Methode nach Init() um die Seite zu analysieren.

   1. Sie können Elemente mit Hilfe von (Css-Selektoren, Js DOM, XPath) finden

   2. Ausgewählte Elemente werden in den Typ ScrapedData-Klasse umgewandelt, Sie können sie in jede beliebige benutzerdefinierte Klasse umwandeln (z. B. Produkt, Mitarbeiter, Nachrichten usw.)

   3. Die Objekte werden in einer Datei im Json-Format im Verzeichnis ("bin/Scrape/") gespeichert. Oder Sie können den Pfad der Datei als Parameter angeben, wie wir später in anderen Beispielen sehen werden.

IronWebscraper Bibliothek Funktionen und Optionen

Die aktualisierte Dokumentation finden Sie in der Zip-Datei, die mit der manuellen Installationsmethode heruntergeladen wurde(IronWebscraper Dokumentation.chm Datei)

Sie können auch online nachsehen, ob die Bibliothek auf dem neuesten Stand isthttps://ironsoftware.com/csharp/webscraper/object-reference/

Um IronWebscraper in Ihrem Projekt zu verwenden, müssen Sie von(IronWebScraper.WebScraper) klasse, die Ihre Klassenbibliothek erweitert und ihr Scraping-Funktionen hinzufügt.

Außerdem müssen Sie implementieren{Init()parsen(Antwort Antwort)} methoden.

namespace IronWebScraperEngine
{
    public class NewsScraper : IronWebScraper.WebScraper
    {
        public override void Init()
        {
            throw new NotImplementedException();
        }

        public override void Parse(Response response)
        {
            throw new NotImplementedException();
        }
    }
}

namespace IronWebScraperEngine
{
    public class NewsScraper : IronWebScraper.WebScraper
    {
        public override void Init()
        {
            throw new NotImplementedException();
        }

        public override void Parse(Response response)
        {
            throw new NotImplementedException();
        }
    }
}

Namespace IronWebScraperEngine
	Public Class NewsScraper
		Inherits IronWebScraper.WebScraper

		Public Overrides Sub Init()
			Throw New NotImplementedException()
		End Sub

		Public Overrides Sub Parse(ByVal response As Response)
			Throw New NotImplementedException()
		End Sub
	End Class
End Namespace

Beispiele und Praxis aus der Praxis

Scraping einer Online-Film-Website

Beginnen wir mit einem weiteren Beispiel von einer echten Website. Wir werden eine Film-Website scrapen.

Fügen wir eine neue Klasse hinzu und nennen sie "MovieScraper":

Werfen wir nun einen Blick auf die Website, die wir scrapen werden:

Dies ist ein Teil der HTML-Homepage, die wir auf der Website sehen:

<div id="movie-featured" class="movies-list movies-list-full tab-pane in fade active">
    <div data-movie-id="20746" class="ml-item">
        <a href="https://website.com/film/king-arthur-legend-of-the-sword-20746/">
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 class="lazy thumb mli-thumb" alt="King Arthur: Legend of the Sword"
                  src="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>King Arthur: Legend of the Sword</h2></span>
        </a>
    </div>
    <div data-movie-id="20724" class="ml-item">
        <a href="https://website.com/film/snatched-20724/" >
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 class="lazy thumb mli-thumb" alt="Snatched" 
                 src="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>Snatched</h2></span>
        </a>
    </div>
</div>

<div id="movie-featured" class="movies-list movies-list-full tab-pane in fade active">
    <div data-movie-id="20746" class="ml-item">
        <a href="https://website.com/film/king-arthur-legend-of-the-sword-20746/">
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 class="lazy thumb mli-thumb" alt="King Arthur: Legend of the Sword"
                  src="https://img.gocdn.online/2017/05/16/poster/2116d6719c710eabe83b377463230fbe-king-arthur-legend-of-the-sword.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>King Arthur: Legend of the Sword</h2></span>
        </a>
    </div>
    <div data-movie-id="20724" class="ml-item">
        <a href="https://website.com/film/snatched-20724/" >
            <span class="mli-quality">CAM</span>
            <img data-original="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 class="lazy thumb mli-thumb" alt="Snatched" 
                 src="https://img.gocdn.online/2017/05/16/poster/5ef66403dc331009bdb5aa37cfe819ba-snatched.jpg" 
                 style="display: inline-block;">
            <span class="mli-info"><h2>Snatched</h2></span>
        </a>
    </div>
</div>

Wie wir sehen können, haben wir eine Film-ID, einen Titel und einen Link zur Detailseite.

Beginnen wir mit dem Scrapen dieses Datensatzes:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("www.website.com", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var MovieId = Divs.GetAttribute("data-movie-id");
                var link = Divs.Css("a")[0];
                var MovieTitle = link.TextContentClean;
                Scrape(new ScrapedData() { { "MovieId", MovieId }, { "MovieTitle", MovieTitle } }, "Movie.Jsonl");
            }
        }           
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("www.website.com", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var MovieId = Divs.GetAttribute("data-movie-id");
                var link = Divs.Css("a")[0];
                var MovieTitle = link.TextContentClean;
                Scrape(new ScrapedData() { { "MovieId", MovieId }, { "MovieTitle", MovieTitle } }, "Movie.Jsonl");
            }
        }           
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("www.website.com", AddressOf Parse)
	End Sub
	Public Overrides Sub Parse(ByVal response As Response)
		For Each Divs In response.Css("#movie-featured > div")
			If Divs.Attributes ("class") <> "clearfix" Then
				Dim MovieId = Divs.GetAttribute("data-movie-id")
				Dim link = Divs.Css("a")(0)
				Dim MovieTitle = link.TextContentClean
				Scrape(New ScrapedData() From {
					{ "MovieId", MovieId },
					{ "MovieTitle", MovieTitle }
				},
				"Movie.Jsonl")
			End If
		Next Divs
	End Sub
End Class

Was ist neu an diesem Code?

Die Eigenschaft Arbeitsverzeichnis wird verwendet, um das Hauptarbeitsverzeichnis für alle gescrapten Daten und die zugehörigen Dateien festzulegen.

Lasst uns mehr tun.

Was, wenn wir typisierte Objekte erstellen müssen, die gescrapte Daten in formatierten Objekten enthalten?

Implementieren wir eine Filmklasse, die unsere formatierten Daten aufnehmen wird:

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }

}

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }

}

Public Class Movie
	Public Property Id() As Integer
	Public Property Title() As String
	Public Property URL() As String

End Class

Jetzt werden wir unseren Code aktualisieren:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://website.com/", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var movie = new Movie();
                movie.Id = Convert.ToInt32( Divs.GetAttribute("data-movie-id"));
                var link = Divs.Css("a")[0];
                movie.Title = link.TextContentClean;
                movie.URL = link.Attributes ["href"];
                Scrape(movie, "Movie.Jsonl");
            }
        }
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://website.com/", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var movie = new Movie();
                movie.Id = Convert.ToInt32( Divs.GetAttribute("data-movie-id"));
                var link = Divs.Css("a")[0];
                movie.Title = link.TextContentClean;
                movie.URL = link.Attributes ["href"];
                Scrape(movie, "Movie.Jsonl");
            }
        }
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("https://website.com/", AddressOf Parse)
	End Sub
	Public Overrides Sub Parse(ByVal response As Response)
		For Each Divs In response.Css("#movie-featured > div")
			If Divs.Attributes ("class") <> "clearfix" Then
				Dim movie As New Movie()
				movie.Id = Convert.ToInt32(Divs.GetAttribute("data-movie-id"))
				Dim link = Divs.Css("a")(0)
				movie.Title = link.TextContentClean
				movie.URL = link.Attributes ("href")
				Scrape(movie, "Movie.Jsonl")
			End If
		Next Divs
	End Sub
End Class

Was gibt es Neues?

1. Wir implementieren Movie Class, um unsere gescrapten Daten zu speichern

2. Wir übergeben Filmobjekte an die Scrape-Methode und sie versteht unser Format und speichert in einem definierten Format, wie wir hier sehen können:

   

   Beginnen wir mit dem Scannen einer detaillierteren Seite.

Die Filmseite sieht wie folgt aus:

<div class="mvi-content">
    <div class="thumb mvic-thumb"
         style="background-image: url(https://img.gocdn.online/2017/04/28/poster/5a08e94ba02118f22dc30f298c603210-guardians-of-the-galaxy-vol-2.jpg);"></div>
    <div class="mvic-desc">
        <h3>Guardians of the Galaxy Vol. 2</h3>        
        <div class="desc">
            Set to the backdrop of Awesome Mixtape #2, Marvel's Guardians of the Galaxy Vol. 2 continues the team's adventures as they travel throughout the cosmos to help Peter Quill learn more about his true parentage.
        </div>
        <div class="mvic-info">
            <div class="mvici-left">
                <p>
                    <strong>Genre: </strong>
                    <a href="https://Domain/genre/action/" title="Action">Action</a>,
                    <a href="https://Domain/genre/adventure/" title="Adventure">Adventure</a>,
                    <a href="https://Domain/genre/sci-fi/" title="Sci-Fi">Sci-Fi</a>
                </p>
                <p>
                    <strong>Actor: </strong>
                    <a target="_blank" href="https://Domain/actor/chris-pratt" title="Chris Pratt">Chris Pratt</a>,
                    <a target="_blank" href="https://Domain/actor/-zoe-saldana" title="Zoe Saldana">Zoe Saldana</a>,
                    <a target="_blank" href="https://Domain/actor/-dave-bautista-" title="Dave Bautista">Dave Bautista</a>
                </p>
                <p>
                    <strong>Director: </strong>
                    <a href="#" title="James Gunn">James Gunn</a>
                </p>
                <p>
                    <strong>Country: </strong>
                    <a href="https://Domain/country/us" title="United States">United States</a>
                </p>
            </div>
            <div class="mvici-right">
                <p><strong>Duration:</strong> 136 min</p>
                <p><strong>Quality:</strong> <span class="quality">CAM</span></p>
                <p><strong>Release:</strong> 2017</p>
                <p><strong>IMDb:</strong> 8.3</p>
            </div>
            <div class="clearfix"></div>
        </div>
        <div class="clearfix"></div>
    </div>
    <div class="clearfix"></div>
</div>

<div class="mvi-content">
    <div class="thumb mvic-thumb"
         style="background-image: url(https://img.gocdn.online/2017/04/28/poster/5a08e94ba02118f22dc30f298c603210-guardians-of-the-galaxy-vol-2.jpg);"></div>
    <div class="mvic-desc">
        <h3>Guardians of the Galaxy Vol. 2</h3>        
        <div class="desc">
            Set to the backdrop of Awesome Mixtape #2, Marvel's Guardians of the Galaxy Vol. 2 continues the team's adventures as they travel throughout the cosmos to help Peter Quill learn more about his true parentage.
        </div>
        <div class="mvic-info">
            <div class="mvici-left">
                <p>
                    <strong>Genre: </strong>
                    <a href="https://Domain/genre/action/" title="Action">Action</a>,
                    <a href="https://Domain/genre/adventure/" title="Adventure">Adventure</a>,
                    <a href="https://Domain/genre/sci-fi/" title="Sci-Fi">Sci-Fi</a>
                </p>
                <p>
                    <strong>Actor: </strong>
                    <a target="_blank" href="https://Domain/actor/chris-pratt" title="Chris Pratt">Chris Pratt</a>,
                    <a target="_blank" href="https://Domain/actor/-zoe-saldana" title="Zoe Saldana">Zoe Saldana</a>,
                    <a target="_blank" href="https://Domain/actor/-dave-bautista-" title="Dave Bautista">Dave Bautista</a>
                </p>
                <p>
                    <strong>Director: </strong>
                    <a href="#" title="James Gunn">James Gunn</a>
                </p>
                <p>
                    <strong>Country: </strong>
                    <a href="https://Domain/country/us" title="United States">United States</a>
                </p>
            </div>
            <div class="mvici-right">
                <p><strong>Duration:</strong> 136 min</p>
                <p><strong>Quality:</strong> <span class="quality">CAM</span></p>
                <p><strong>Release:</strong> 2017</p>
                <p><strong>IMDb:</strong> 8.3</p>
            </div>
            <div class="clearfix"></div>
        </div>
        <div class="clearfix"></div>
    </div>
    <div class="clearfix"></div>
</div>

Wir können unsere Filmklasse um neue Eigenschaften erweitern(Beschreibung, Genre, Darsteller, Regisseur, Land, Dauer, IMDB Score) aber wir verwenden(Beschreibung, Genre, Akteur) nur für unsere Stichprobe.

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
    public string Description { get; set; }
    public List<string> Genre { get; set; }
    public List<string> Actor { get; set; }

}

public class Movie
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string URL { get; set; }
    public string Description { get; set; }
    public List<string> Genre { get; set; }
    public List<string> Actor { get; set; }

}

Public Class Movie
	Public Property Id() As Integer
	Public Property Title() As String
	Public Property URL() As String
	Public Property Description() As String
	Public Property Genre() As List(Of String)
	Public Property Actor() As List(Of String)

End Class

Jetzt navigieren wir zu der Seite Detailliert, um sie zu scrapen.

IronWebscraper ermöglicht es Ihnen, die Scrape-Funktion zu erweitern, um verschiedene Arten von Seitenformaten zu scrapen

Wie wir hier sehen können:

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://domain/", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var movie = new Movie();
                movie.Id = Convert.ToInt32( Divs.GetAttribute("data-movie-id"));
                var link = Divs.Css("a")[0];
                movie.Title = link.TextContentClean;
                movie.URL = link.Attributes ["href"];
                this.Request(movie.URL, ParseDetails, new MetaData() { { "movie", movie } });// to scrap Detailed Page
            }
        }           
    }
    public void ParseDetails(Response response)
    {
        var movie = response.MetaData.Get<Movie>("movie");
        var Div = response.Css("div.mvic-desc")[0];
        movie.Description = Div.Css("div.desc")[0].TextContentClean;
        foreach(var Genre in Div.Css("div > p > a"))
        {
            movie.Genre.Add(Genre.TextContentClean);
        }
        foreach (var Actor in Div.Css("div > p:nth-child(2) > a"))
        {
            movie.Actor.Add(Actor.TextContentClean);
        }
        Scrape(movie, "Movie.Jsonl");
    }
}

public class MovieScraper : WebScraper
{
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\MovieSample\Output\";
        this.Request("https://domain/", Parse);
    }
    public override void Parse(Response response)
    {
        foreach (var Divs in response.Css("#movie-featured > div"))
        {
            if (Divs.Attributes ["class"] != "clearfix")
            {
                var movie = new Movie();
                movie.Id = Convert.ToInt32( Divs.GetAttribute("data-movie-id"));
                var link = Divs.Css("a")[0];
                movie.Title = link.TextContentClean;
                movie.URL = link.Attributes ["href"];
                this.Request(movie.URL, ParseDetails, new MetaData() { { "movie", movie } });// to scrap Detailed Page
            }
        }           
    }
    public void ParseDetails(Response response)
    {
        var movie = response.MetaData.Get<Movie>("movie");
        var Div = response.Css("div.mvic-desc")[0];
        movie.Description = Div.Css("div.desc")[0].TextContentClean;
        foreach(var Genre in Div.Css("div > p > a"))
        {
            movie.Genre.Add(Genre.TextContentClean);
        }
        foreach (var Actor in Div.Css("div > p:nth-child(2) > a"))
        {
            movie.Actor.Add(Actor.TextContentClean);
        }
        Scrape(movie, "Movie.Jsonl");
    }
}

Public Class MovieScraper
	Inherits WebScraper

	Public Overrides Sub Init()
		License.LicenseKey = "LicenseKey"
		Me.LoggingLevel = WebScraper.LogLevel.All
		Me.WorkingDirectory = AppSetting.GetAppRoot() & "\MovieSample\Output\"
		Me.Request("https://domain/", AddressOf Parse)
	End Sub
	Public Overrides Sub Parse(ByVal response As Response)
		For Each Divs In response.Css("#movie-featured > div")
			If Divs.Attributes ("class") <> "clearfix" Then
				Dim movie As New Movie()
				movie.Id = Convert.ToInt32(Divs.GetAttribute("data-movie-id"))
				Dim link = Divs.Css("a")(0)
				movie.Title = link.TextContentClean
				movie.URL = link.Attributes ("href")
				Me.Request(movie.URL, AddressOf ParseDetails, New MetaData() From {
					{ "movie", movie }
				}) ' to scrap Detailed Page
			End If
		Next Divs
	End Sub
	Public Sub ParseDetails(ByVal response As Response)
		Dim movie = response.MetaData.Get(Of Movie)("movie")
		Dim Div = response.Css("div.mvic-desc")(0)
		movie.Description = Div.Css("div.desc")(0).TextContentClean
		For Each Genre In Div.Css("div > p > a")
			movie.Genre.Add(Genre.TextContentClean)
		Next Genre
		For Each Actor In Div.Css("div > p:nth-child(2) > a")
			movie.Actor.Add(Actor.TextContentClean)
		Next Actor
		Scrape(movie, "Movie.Jsonl")
	End Sub
End Class

Was gibt es Neues?

1. Wir können Scrape-Funktionen hinzufügen(ParseDetails) zum Scrapen von Detailseiten

2. Wir haben die Scrape-Funktion, die unsere Datei erzeugt, in die neue Funktion

3. Wir haben die IronWebScraper-Funktion(MetaDaten) um unser Filmobjekt an die neue Scrape-Funktion zu übergeben

4. Wir haben die Seite ausgewertet und unsere Filmobjektdaten in einer Datei gespeichert

   

Scrapen von Inhalten einer Shopping-Website

Wir wählen eine Einkaufsseite aus, um den Inhalt von ihr zu scrapen

Wie Sie auf dem Bild sehen können, haben wir eine linke Leiste, die Links zu den Produktkategorien der Website enthält

Unser erster Schritt besteht also darin, den HTML-Code der Website zu untersuchen und zu planen, wie wir sie scrapen wollen.

Die Kategorien der Modeseiten haben Unterkategorien(Männer, Frauen, Kinder)

<li class="menu-item" data-id="">
    <a href="https://domain.com/fashion-by-/" class="main-category">
        <i class="cat-icon osh-font-fashion"></i> <span class="nav-subTxt">FASHION </span> <i class="osh-font-light-arrow-left"></i><i class="osh-font-light-arrow-right"></i>
    </a> <div class="navLayerWrapper" style="width: 633px; display: none;"><div class="submenu"><div class="column"><div class="categories"><a class="category" href="https://domain.com/fashion-by-/?sort=newest&amp;dir=desc&amp;viewType=gridView3">New Arrivals !</a>  </div><div class="categories"><a class="category" href="https://domain.com/men-fashion/">Men</a>   <a class="subcategory" href="https://domain.com/mens-shoes/">Shoes</a>   <a class="subcategory" href="https://domain.com/mens-clothing/">Clothing</a>   <a class="subcategory" href="https://domain.com/mens-accessories/">Accessories</a>  </div><div class="categories"><a class="category" href="https://domain.com/women-fashion/">Women</a>   <a class="subcategory" href="https://domain.com/womens-shoes/">Shoes</a>   <a class="subcategory" href="https://domain.com/womens-clothing/">Clothing</a>   <a class="subcategory" href="https://domain.com/womens-accessories/">Accessories</a>  </div><div class="categories"><a class="category" href="https://domain.com/girls-boys-fashion/">Kids</a>   <a class="subcategory" href="https://domain.com/boys-fashion/">Boys</a>   <a class="subcategory" href="https://domain.com/girls/">Girls</a>  </div><div class="categories"><a class="category" href="https://domain.com/maternity-clothes/">Maternity Clothes</a>  </div></div><div class="column"><div class="categories"> <span class="category defaultCursor">Men Best Sellers</span>  <a class="subcategory" href="https://domain.com/mens-casual-shoes/">Casual Shoes</a>   <a class="subcategory" href="https://domain.com/mens-sneakers/">Sneakers</a>   <a class="subcategory" href="https://domain.com/mens-t-shirts/">T-shirts</a>   <a class="subcategory" href="https://domain.com/mens-polos/">Polos</a>  </div><div class="categories"> <span class="category defaultCursor">Women Best Sellers</span>  <a class="subcategory" href="https://domain.com/womens-sandals/">Sandals</a>   <a class="subcategory" href="https://domain.com/womens-sneakers/">Sneakers</a>   <a class="subcategory" href="https://domain.com/women-dresses/">Dresses</a>   <a class="subcategory" href="https://domain.com/women-tops/">Tops</a>  </div><div class="categories"><a class="category" href="https://domain.com/womens-curvy-clothing/">Women's Curvy Clothing</a>  </div><div class="categories"><a class="category" href="https://domain.com/fashion-bundles/v/">Fashion Bundles</a>  </div><div class="categories"><a class="category" href="https://domain.com/hijab-fashion/">Hijab Fashion</a>  </div></div><div class="column"><div class="categories"><a class="category" href="https://domain.com/brands/fashion-by-/">SEE ALL BRANDS</a>   <a class="subcategory" href="https://domain.com/adidas/">Adidas</a>   <a class="subcategory" href="https://domain.com/converse/">Converse</a>   <a class="subcategory" href="https://domain.com/ravin/">Ravin</a>   <a class="subcategory" href="https://domain.com/dejavu/">Dejavu</a>   <a class="subcategory" href="https://domain.com/agu/">Agu</a>   <a class="subcategory" href="https://domain.com/activ/">Activ</a>   <a class="subcategory" href="https://domain.com/oxford--bellini--tie-house--milano/">Tie House</a>   <a class="subcategory" href="https://domain.com/shoe-room/">Shoe Room</a>   <a class="subcategory" href="https://domain.com/town-team/">Town Team</a>  </div></div></div></div>
</li>

<li class="menu-item" data-id="">
    <a href="https://domain.com/fashion-by-/" class="main-category">
        <i class="cat-icon osh-font-fashion"></i> <span class="nav-subTxt">FASHION </span> <i class="osh-font-light-arrow-left"></i><i class="osh-font-light-arrow-right"></i>
    </a> <div class="navLayerWrapper" style="width: 633px; display: none;"><div class="submenu"><div class="column"><div class="categories"><a class="category" href="https://domain.com/fashion-by-/?sort=newest&amp;dir=desc&amp;viewType=gridView3">New Arrivals !</a>  </div><div class="categories"><a class="category" href="https://domain.com/men-fashion/">Men</a>   <a class="subcategory" href="https://domain.com/mens-shoes/">Shoes</a>   <a class="subcategory" href="https://domain.com/mens-clothing/">Clothing</a>   <a class="subcategory" href="https://domain.com/mens-accessories/">Accessories</a>  </div><div class="categories"><a class="category" href="https://domain.com/women-fashion/">Women</a>   <a class="subcategory" href="https://domain.com/womens-shoes/">Shoes</a>   <a class="subcategory" href="https://domain.com/womens-clothing/">Clothing</a>   <a class="subcategory" href="https://domain.com/womens-accessories/">Accessories</a>  </div><div class="categories"><a class="category" href="https://domain.com/girls-boys-fashion/">Kids</a>   <a class="subcategory" href="https://domain.com/boys-fashion/">Boys</a>   <a class="subcategory" href="https://domain.com/girls/">Girls</a>  </div><div class="categories"><a class="category" href="https://domain.com/maternity-clothes/">Maternity Clothes</a>  </div></div><div class="column"><div class="categories"> <span class="category defaultCursor">Men Best Sellers</span>  <a class="subcategory" href="https://domain.com/mens-casual-shoes/">Casual Shoes</a>   <a class="subcategory" href="https://domain.com/mens-sneakers/">Sneakers</a>   <a class="subcategory" href="https://domain.com/mens-t-shirts/">T-shirts</a>   <a class="subcategory" href="https://domain.com/mens-polos/">Polos</a>  </div><div class="categories"> <span class="category defaultCursor">Women Best Sellers</span>  <a class="subcategory" href="https://domain.com/womens-sandals/">Sandals</a>   <a class="subcategory" href="https://domain.com/womens-sneakers/">Sneakers</a>   <a class="subcategory" href="https://domain.com/women-dresses/">Dresses</a>   <a class="subcategory" href="https://domain.com/women-tops/">Tops</a>  </div><div class="categories"><a class="category" href="https://domain.com/womens-curvy-clothing/">Women's Curvy Clothing</a>  </div><div class="categories"><a class="category" href="https://domain.com/fashion-bundles/v/">Fashion Bundles</a>  </div><div class="categories"><a class="category" href="https://domain.com/hijab-fashion/">Hijab Fashion</a>  </div></div><div class="column"><div class="categories"><a class="category" href="https://domain.com/brands/fashion-by-/">SEE ALL BRANDS</a>   <a class="subcategory" href="https://domain.com/adidas/">Adidas</a>   <a class="subcategory" href="https://domain.com/converse/">Converse</a>   <a class="subcategory" href="https://domain.com/ravin/">Ravin</a>   <a class="subcategory" href="https://domain.com/dejavu/">Dejavu</a>   <a class="subcategory" href="https://domain.com/agu/">Agu</a>   <a class="subcategory" href="https://domain.com/activ/">Activ</a>   <a class="subcategory" href="https://domain.com/oxford--bellini--tie-house--milano/">Tie House</a>   <a class="subcategory" href="https://domain.com/shoe-room/">Shoe Room</a>   <a class="subcategory" href="https://domain.com/town-team/">Town Team</a>  </div></div></div></div>
</li>

Richten wir ein Projekt ein

1. Erstellen Sie eine neue Konsolenanwendung oder fügen Sie einen neuen Ordner für unser neues Beispiel mit dem Namen "ShoppingSiteSample" hinzu

2. Neue Klasse mit dem Namen "ShoppingScraper" hinzufügen

3. Der erste Schritt besteht darin, die Website-Kategorien und ihre Unterkategorien zu durchsuchen

   Lassen Sie uns ein Kategorienmodell erstellen:

public class Category
{
    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    /// <value>
    /// The name.
    /// </value>
    public string Name { get; set; }
    /// <summary>
    /// Gets or sets the URL.
    /// </summary>
    /// <value>
    /// The URL.
    /// </value>
    public string URL { get; set; }
    /// <summary>
    /// Gets or sets the sub categories.
    /// </summary>
    /// <value>
    /// The sub categories.
    /// </value>
    public List<Category> SubCategories { get; set; }
}

public class Category
{
    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    /// <value>
    /// The name.
    /// </value>
    public string Name { get; set; }
    /// <summary>
    /// Gets or sets the URL.
    /// </summary>
    /// <value>
    /// The URL.
    /// </value>
    public string URL { get; set; }
    /// <summary>
    /// Gets or sets the sub categories.
    /// </summary>
    /// <value>
    /// The sub categories.
    /// </value>
    public List<Category> SubCategories { get; set; }
}

Public Class Category
	''' <summary>
	''' Gets or sets the name.
	''' </summary>
	''' <value>
	''' The name.
	''' </value>
	Public Property Name() As String
	''' <summary>
	''' Gets or sets the URL.
	''' </summary>
	''' <value>
	''' The URL.
	''' </value>
	Public Property URL() As String
	''' <summary>
	''' Gets or sets the sub categories.
	''' </summary>
	''' <value>
	''' The sub categories.
	''' </value>
	Public Property SubCategories() As List(Of Category)
End Class

4. Bauen wir nun unsere Scrape-Logik auf

public class ShoppingScraper : WebScraper
{
    /// <summary>
    /// Override this method initialize your web-scraper.
    /// Important tasks will be to Request at least one start url... and set allowed/banned domain or url patterns.
    /// </summary>
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
        this.Request("www.webSite.com", Parse);
    }

    /// <summary>
    /// Override this method to create the default Response handler for your web scraper.
    /// If you have multiple page types, you can add additional similar methods.
    /// </summary>
    /// <param name="response">The http Response object to parse</param>
    public override void Parse(Response response)
    {
        var categoryList = new List<Category>();

        foreach (var Links in response.Css("#menuFixed > ul > li > a "))
        {
            var cat = new Category();
            cat.URL = Links.Attributes ["href"];
            cat.Name = Links.InnerText;
            categoryList.Add(cat);
        }
        Scrape(categoryList, "Shopping.Jsonl");
    }
}

public class ShoppingScraper : WebScraper
{
    /// <summary>
    /// Override this method initialize your web-scraper.
    /// Important tasks will be to Request at least one start url... and set allowed/banned domain or url patterns.
    /// </summary>
    public override void Init()
    {
        License.LicenseKey = "LicenseKey";
        this.LoggingLevel = WebScraper.LogLevel.All;
        this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
        this.Request("www.webSite.com", Parse);
    }

    /// <summary>
    /// Override this method to create the default Response handler for your web scraper.
    /// If you have multiple page types, you can add additional similar methods.
    /// </summary>
    /// <param name="response">The http Response object to parse</param>
    public override void Parse(Response response)
    {
        var categoryList = new List<Category>();

        foreach (var Links in response.Css("#menuFixed > ul > li > a "))
        {
            var cat = new Category();
            cat.URL = Links.Attributes ["href"];
            cat.Name = Links.InnerText;
            categoryList.Add(cat);
        }
        Scrape(categoryList, "Shopping.Jsonl");
    }
}

IRON VB CONVERTER ERROR developers@ironsoftware.com

Scraping von Links aus dem Menü

Aktualisieren wir unseren Code, um die Hauptkategorien und alle ihre Unterlinks zu scrapen

public override void Parse(Response response)
{
    // List of Categories Links (Root)
    var categoryList = new List<Category>();

    foreach (var li in response.Css("#menuFixed > ul > li"))
    {
        // List Of Main Links
        foreach (var Links in li.Css("a"))
        {
            var cat = new Category();
            cat.URL = Links.Attributes ["href"];
            cat.Name = Links.InnerText;
            cat.SubCategories = new List<Category>();
            // List of Sub Catgories Links
            foreach (var subCategory in li.Css("a [class=subcategory]"))
            {
                var subcat = new Category();
                subcat.URL = Links.Attributes ["href"];
                subcat.Name = Links.InnerText;
                // Check If Link Exist Before 
                if (cat.SubCategories.Find(c=>c.Name== subcat.Name && c.URL == subcat.URL) == null)
                {
                    // Add Sublinks
                    cat.SubCategories.Add(subcat);
                }
            }
            // Add Categories
            categoryList.Add(cat);
        }
    }
    Scrape(categoryList, "Shopping.Jsonl");
}

public override void Parse(Response response)
{
    // List of Categories Links (Root)
    var categoryList = new List<Category>();

    foreach (var li in response.Css("#menuFixed > ul > li"))
    {
        // List Of Main Links
        foreach (var Links in li.Css("a"))
        {
            var cat = new Category();
            cat.URL = Links.Attributes ["href"];
            cat.Name = Links.InnerText;
            cat.SubCategories = new List<Category>();
            // List of Sub Catgories Links
            foreach (var subCategory in li.Css("a [class=subcategory]"))
            {
                var subcat = new Category();
                subcat.URL = Links.Attributes ["href"];
                subcat.Name = Links.InnerText;
                // Check If Link Exist Before 
                if (cat.SubCategories.Find(c=>c.Name== subcat.Name && c.URL == subcat.URL) == null)
                {
                    // Add Sublinks
                    cat.SubCategories.Add(subcat);
                }
            }
            // Add Categories
            categoryList.Add(cat);
        }
    }
    Scrape(categoryList, "Shopping.Jsonl");
}

Public Overrides Sub Parse(ByVal response As Response)
	' List of Categories Links (Root)
	Dim categoryList = New List(Of Category)()

	For Each li In response.Css("#menuFixed > ul > li")
		' List Of Main Links
		For Each Links In li.Css("a")
			Dim cat = New Category()
			cat.URL = Links.Attributes ("href")
			cat.Name = Links.InnerText
			cat.SubCategories = New List(Of Category)()
			' List of Sub Catgories Links
			For Each subCategory In li.Css("a [class=subcategory]")
				Dim subcat = New Category()
				subcat.URL = Links.Attributes ("href")
				subcat.Name = Links.InnerText
				' Check If Link Exist Before 
				If cat.SubCategories.Find(Function(c) c.Name= subcat.Name AndAlso c.URL = subcat.URL) Is Nothing Then
					' Add Sublinks
					cat.SubCategories.Add(subcat)
				End If
			Next subCategory
			' Add Categories
			categoryList.Add(cat)
		Next Links
	Next li
	Scrape(categoryList, "Shopping.Jsonl")
End Sub

Da wir nun Links zu allen Website-Kategorien haben, können wir damit beginnen, die Produkte innerhalb jeder Kategorie zu durchsuchen

Navigieren wir zu einer beliebigen Kategorie und prüfen wir den Inhalt.

Schauen wir uns den Code an

<section class="products">
    <div class="sku -gallery -validate-size " data-sku="AG249FA0T2PSGNAFAMZ" ft-product-sizes="41,42,43,44,45" ft-product-color="Multicolour">
        <a class="link" href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html">
            <div class="image-wrapper default-state">
                <img class="lazy image -loaded" alt="Bundle Of 2 Sneakers - Black &amp;amp; Navy Blue" data-image-vertical="1" width="210" height="262" src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" data-sku="AG249FA0T2PSGNAFAMZ" data-src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" data-placeholder="placeholder_m_1.jpg"><noscript>&lt;img src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" width="210" height="262" class="image" /&gt;</noscript>
            </div> <h2 class="title">
                <span class="brand ">Agu&nbsp;</span>
                <span class="name" dir="ltr">Bundle Of 2 Sneakers - Black &amp; Navy Blue</span>
            </h2><div class="price-container clearfix">
                <span class="price-box">
                    <span class="price">
                        <span data-currency-iso="EGP">EGP</span>
                        <span dir="ltr" data-price="299">299</span>
                    </span>   <span class="price -old  -no-special"></span>
                </span>
            </div><div class="rating-stars"><div class="stars-container"><div class="stars" style="width: 62%"></div></div> <div class="total-ratings">(30)</div> </div>    <span class="shop-first-logo-container"><img src="http://www.WebSite.com/images/local/logos/shop_first/ShoppingSite/logo_normal.png" data-src="http://www.WebSite.com/images/local/logos/shop_first/ShoppingSite/logo_normal.png" class="lazy shop-first-logo-img -mbxs -loaded"> </span>
            <span class="osh-icon -ShoppingSite-local shop_local--logo -block -mbs -mts"></span>
            <div class="list -sizes" data-selected-sku="">
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=41">41</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=42">42</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=43">43</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=44">44</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=45">45</span>
            </div>
        </a>
    </div>
    <div class="sku -gallery -validate-size " data-sku="LE047FA01SRK4NAFAMZ" ft-product-sizes="110,115,120,125,130,135" ft-product-color="Black">
        <a class="link" href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html">
            <div class="image-wrapper default-state"><img class="lazy image -loaded" alt="Genuine Leather Belt - Black" data-image-vertical="1" width="210" height="262" src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" data-sku="LE047FA01SRK4NAFAMZ" data-src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" data-placeholder="placeholder_m_1.jpg"><noscript>&lt;img src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" width="210" height="262" class="image" /&gt;</noscript></div>
            <h2 class="title"><span class="brand ">Leather Shop&nbsp;</span> <span class="name" dir="ltr">Genuine Leather Belt - Black</span></h2><div class="price-container clearfix">
                <span class="sale-flag-percent">-29%</span>  <span class="price-box"> <span class="price"><span data-currency-iso="EGP">EGP</span> <span dir="ltr" data-price="96">96</span> </span>   <span class="price -old "><span data-currency-iso="EGP">EGP</span> <span dir="ltr" data-price="135">135</span> </span> </span>
            </div><div class="rating-stars"><div class="stars-container"><div class="stars" style="width: 100%"></div></div> <div class="total-ratings">(1)</div> </div>
            <span class="osh-icon -ShoppingSite-local shop_local--logo -block -mbs -mts"></span>    <div class="list -sizes" data-selected-sku="">
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=110">110</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=115">115</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=120">120</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=125">125</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=130">130</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=135">135</span>
            </div>
        </a>
    </div>
</section>

<section class="products">
    <div class="sku -gallery -validate-size " data-sku="AG249FA0T2PSGNAFAMZ" ft-product-sizes="41,42,43,44,45" ft-product-color="Multicolour">
        <a class="link" href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html">
            <div class="image-wrapper default-state">
                <img class="lazy image -loaded" alt="Bundle Of 2 Sneakers - Black &amp;amp; Navy Blue" data-image-vertical="1" width="210" height="262" src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" data-sku="AG249FA0T2PSGNAFAMZ" data-src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" data-placeholder="placeholder_m_1.jpg"><noscript>&lt;img src="https://static.WebSite.com/p/agu-6208-488356-1-catalog_grid_3.jpg" width="210" height="262" class="image" /&gt;</noscript>
            </div> <h2 class="title">
                <span class="brand ">Agu&nbsp;</span>
                <span class="name" dir="ltr">Bundle Of 2 Sneakers - Black &amp; Navy Blue</span>
            </h2><div class="price-container clearfix">
                <span class="price-box">
                    <span class="price">
                        <span data-currency-iso="EGP">EGP</span>
                        <span dir="ltr" data-price="299">299</span>
                    </span>   <span class="price -old  -no-special"></span>
                </span>
            </div><div class="rating-stars"><div class="stars-container"><div class="stars" style="width: 62%"></div></div> <div class="total-ratings">(30)</div> </div>    <span class="shop-first-logo-container"><img src="http://www.WebSite.com/images/local/logos/shop_first/ShoppingSite/logo_normal.png" data-src="http://www.WebSite.com/images/local/logos/shop_first/ShoppingSite/logo_normal.png" class="lazy shop-first-logo-img -mbxs -loaded"> </span>
            <span class="osh-icon -ShoppingSite-local shop_local--logo -block -mbs -mts"></span>
            <div class="list -sizes" data-selected-sku="">
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=41">41</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=42">42</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=43">43</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=44">44</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/agu-bundle-of-2-sneakers-black-navy-blue-653884.html?size=45">45</span>
            </div>
        </a>
    </div>
    <div class="sku -gallery -validate-size " data-sku="LE047FA01SRK4NAFAMZ" ft-product-sizes="110,115,120,125,130,135" ft-product-color="Black">
        <a class="link" href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html">
            <div class="image-wrapper default-state"><img class="lazy image -loaded" alt="Genuine Leather Belt - Black" data-image-vertical="1" width="210" height="262" src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" data-sku="LE047FA01SRK4NAFAMZ" data-src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" data-placeholder="placeholder_m_1.jpg"><noscript>&lt;img src="https://static.WebSite.com/p/leather-shop-1831-030217-1-catalog_grid_3.jpg" width="210" height="262" class="image" /&gt;</noscript></div>
            <h2 class="title"><span class="brand ">Leather Shop&nbsp;</span> <span class="name" dir="ltr">Genuine Leather Belt - Black</span></h2><div class="price-container clearfix">
                <span class="sale-flag-percent">-29%</span>  <span class="price-box"> <span class="price"><span data-currency-iso="EGP">EGP</span> <span dir="ltr" data-price="96">96</span> </span>   <span class="price -old "><span data-currency-iso="EGP">EGP</span> <span dir="ltr" data-price="135">135</span> </span> </span>
            </div><div class="rating-stars"><div class="stars-container"><div class="stars" style="width: 100%"></div></div> <div class="total-ratings">(1)</div> </div>
            <span class="osh-icon -ShoppingSite-local shop_local--logo -block -mbs -mts"></span>    <div class="list -sizes" data-selected-sku="">
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=110">110</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=115">115</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=120">120</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=125">125</span>     <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=130">130</span>
                <span class="js-link sku-size" data-href="http://www.WebSite.com/leather-shop-genuine-leather-belt-black-712030.html?size=135">135</span>
            </div>
        </a>
    </div>
</section>

Lassen Sie uns unser Produktmodell für diesen Inhalt erstellen.

public class Product
{
    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    /// <value>
    /// The name.
    /// </value>
    public string Name { get; set; }
    /// <summary>
    /// Gets or sets the price.
    /// </summary>
    /// <value>
    /// The price.
    /// </value>
    public string Price { get; set; }
    /// <summary>
    /// Gets or sets the image.
    /// </summary>
    /// <value>
    /// The image.
    /// </value>
    public string Image { get; set; }
}

public class Product
{
    /// <summary>
    /// Gets or sets the name.
    /// </summary>
    /// <value>
    /// The name.
    /// </value>
    public string Name { get; set; }
    /// <summary>
    /// Gets or sets the price.
    /// </summary>
    /// <value>
    /// The price.
    /// </value>
    public string Price { get; set; }
    /// <summary>
    /// Gets or sets the image.
    /// </summary>
    /// <value>
    /// The image.
    /// </value>
    public string Image { get; set; }
}

Public Class Product
	''' <summary>
	''' Gets or sets the name.
	''' </summary>
	''' <value>
	''' The name.
	''' </value>
	Public Property Name() As String
	''' <summary>
	''' Gets or sets the price.
	''' </summary>
	''' <value>
	''' The price.
	''' </value>
	Public Property Price() As String
	''' <summary>
	''' Gets or sets the image.
	''' </summary>
	''' <value>
	''' The image.
	''' </value>
	Public Property Image() As String
End Class

Um Kategorieseiten zu scrapen, fügen wir eine neue Scrape-Methode hinzu:

public void ParseCatgory(Response response)
{          
    // List of Products Links (Root)
    var productList = new List<Product>();

    foreach (var Links in response.Css("body > main > section.osh-content > section.products > div > a"))
    {
        var product = new Product();
        product.Name = Links.InnerText;
        product.Image = Links.Css("div.image-wrapper.default-state > img")[0].Attributes ["src"];                
        productList.Add(product);
    }

    Scrape(productList, "Products.Jsonl");
}

public void ParseCatgory(Response response)
{          
    // List of Products Links (Root)
    var productList = new List<Product>();

    foreach (var Links in response.Css("body > main > section.osh-content > section.products > div > a"))
    {
        var product = new Product();
        product.Name = Links.InnerText;
        product.Image = Links.Css("div.image-wrapper.default-state > img")[0].Attributes ["src"];                
        productList.Add(product);
    }

    Scrape(productList, "Products.Jsonl");
}

Public Sub ParseCatgory(ByVal response As Response)
	' List of Products Links (Root)
	Dim productList = New List(Of Product)()

	For Each Links In response.Css("body > main > section.osh-content > section.products > div > a")
		Dim product As New Product()
		product.Name = Links.InnerText
		product.Image = Links.Css("div.image-wrapper.default-state > img")(0).Attributes ("src")
		productList.Add(product)
	Next Links

	Scrape(productList, "Products.Jsonl")
End Sub

Erweiterte Webscraping-Funktionen

HttpIdentity-Merkmal:

Bei einigen Website-Systemen muss der Nutzer angemeldet sein, um den Inhalt zu sehen; in diesem Fall können wir eine HttpIdentity verwenden: -

HttpIdentity id = new HttpIdentity();
id.NetworkUsername = "username";
id.NetworkPassword = "pwd";
Identities.Add(id); 

HttpIdentity id = new HttpIdentity();
id.NetworkUsername = "username";
id.NetworkPassword = "pwd";
Identities.Add(id); 

Dim id As New HttpIdentity()
id.NetworkUsername = "username"
id.NetworkPassword = "pwd"
Identities.Add(id)

Eine der beeindruckendsten und leistungsfähigsten Funktionen von IronWebscraper ist die Möglichkeit, Tausende von einzigartigen(die Anmeldedaten des Nutzers und/oder die Browser-Engines) zum Spoofing oder Scrapen von Websites mit mehreren Anmeldesitzungen.

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    var proxies = "IP-Proxy1: 8080,IP-Proxy2: 8081".Split(',');
    foreach (var UA in IronWebScraper.CommonUserAgents.ChromeDesktopUserAgents)
    {
        foreach (var proxy in proxies)
        {
            Identities.Add(new HttpIdentity()
            {
                UserAgent = UA,
                UseCookies = true,
                Proxy = proxy
            });
        }
    }
    this.Request("http://www.Website.com", Parse);
}

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    var proxies = "IP-Proxy1: 8080,IP-Proxy2: 8081".Split(',');
    foreach (var UA in IronWebScraper.CommonUserAgents.ChromeDesktopUserAgents)
    {
        foreach (var proxy in proxies)
        {
            Identities.Add(new HttpIdentity()
            {
                UserAgent = UA,
                UseCookies = true,
                Proxy = proxy
            });
        }
    }
    this.Request("http://www.Website.com", Parse);
}

Public Overrides Sub Init()
	License.LicenseKey = " LicenseKey "
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	Dim proxies = "IP-Proxy1: 8080,IP-Proxy2: 8081".Split(","c)
	For Each UA In IronWebScraper.CommonUserAgents.ChromeDesktopUserAgents
		For Each proxy In proxies
			Identities.Add(New HttpIdentity() With {
				.UserAgent = UA,
				.UseCookies = True,
				.Proxy = proxy
			})
		Next proxy
	Next UA
	Me.Request("http://www.Website.com", Parse)
End Sub

Sie haben mehrere Eigenschaften, die Ihnen verschiedene Verhaltensweisen ermöglichen und so verhindern, dass Websites Sie blockieren.

Einige dieser Eigenschaften: -

• NetworkDomain : Die Netzwerkdomäne, die für die Benutzerauthentifizierung verwendet werden soll. Unterstützt Windows, NTLM, Keroberos, Linux, BSD und Mac OS X Netzwerke. Muss verwendet werden mit(NetzwerkBenutzername und NetzwerkPasswort)
• Netzwerk-Benutzername : Der Netzwerk-/Htp-Benutzername, der für die Benutzerauthentifizierung verwendet werden soll. Unterstützt Http, Windows-Netzwerke, NTLM, Kerberos, Linux-Netzwerke, BSD-Netzwerke und Mac OS.
• NetworkPassword : Das für die Benutzerauthentifizierung zu verwendende Netzwerk-/http-Passwort. Unterstützt Http , Windows-Netzwerke, NTLM , Keroberos , Linux-Netzwerke, BSD-Netzwerke und Mac OS.
• Proxy : zum Festlegen der Proxy-Einstellungen
• UserAgent : zum Einstellen der Browser-Engine(chrome Desktop , Chrome Mobile , Chrome Tablet , IE und Firefox , etc.)
• HttpRequestHeaders : für benutzerdefinierte Kopfzeilenwerte, die mit dieser Identität verwendet werden, und es akzeptiert Wörterbuchobjekte(Wörterbuch <String, String>)

• UseCookies : Verwendung von Cookies aktivieren/deaktivieren

  IronWebscraper führt den Scraper mit Zufallsidentitäten aus. Wenn wir die Verwendung einer bestimmten Identität zum Parsen einer Seite angeben müssen, können wir dies tun.

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    HttpIdentity identity = new HttpIdentity();
    identity.NetworkUsername = "username";
    identity.NetworkPassword = "pwd";
    Identities.Add(id);
    this.Request("http://www.Website.com", Parse, identity);
}

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    HttpIdentity identity = new HttpIdentity();
    identity.NetworkUsername = "username";
    identity.NetworkPassword = "pwd";
    Identities.Add(id);
    this.Request("http://www.Website.com", Parse, identity);
}

Public Overrides Sub Init()
	License.LicenseKey = " LicenseKey "
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	Dim identity As New HttpIdentity()
	identity.NetworkUsername = "username"
	identity.NetworkPassword = "pwd"
	Identities.Add(id)
	Me.Request("http://www.Website.com", Parse, identity)
End Sub

Aktivieren Sie die Funktion Web-Cache:

Diese Funktion dient dazu, angeforderte Seiten zwischenzuspeichern. Sie wird häufig in der Entwicklungs- und Testphase eingesetzt; die es Entwicklern ermöglicht, benötigte Seiten zwischenzuspeichern und nach einer Codeaktualisierung wiederzuverwenden. So können Sie Ihren Code auf zwischengespeicherten Seiten ausführen, nachdem Sie Ihren Web Scraper neu gestartet haben, und müssen nicht jedes Mal eine Verbindung zur Live-Website herstellen(aktionswiedergabe).

Sie können es in Init verwenden() Methode

EnableWebCache();

OR

EnableWebCache(Zeitspanne Verfall);

Es speichert Ihre zwischengespeicherten Daten im WebCache-Ordner unter dem Ordner des Arbeitsverzeichnisses

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    EnableWebCache(new TimeSpan(1,30,30));
    this.Request("http://www.WebSite.com", Parse);
}

public override void Init()
{
    License.LicenseKey = " LicenseKey ";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    EnableWebCache(new TimeSpan(1,30,30));
    this.Request("http://www.WebSite.com", Parse);
}

Public Overrides Sub Init()
	License.LicenseKey = " LicenseKey "
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	EnableWebCache(New TimeSpan(1,30,30))
	Me.Request("http://www.WebSite.com", Parse)
End Sub

IronWebscraper verfügt auch über Funktionen, die es Ihrer Engine ermöglichen, das Scraping nach einem Neustart des Codes fortzusetzen, indem Sie den Namen des Startprozesses der Engine mit Start(CrawlID)

static void Main(string [] args)
{
    // Create Object From Scraper class
    EngineScraper scrape = new EngineScraper();
    // Start Scraping
    scrape.Start("enginestate");
}

static void Main(string [] args)
{
    // Create Object From Scraper class
    EngineScraper scrape = new EngineScraper();
    // Start Scraping
    scrape.Start("enginestate");
}

Shared Sub Main(ByVal args() As String)
	' Create Object From Scraper class
	Dim scrape As New EngineScraper()
	' Start Scraping
	scrape.Start("enginestate")
End Sub

Die Ausführungsanforderung und die Antwort werden im Ordner "SavedState" im Arbeitsverzeichnis gespeichert.

Drosselung

Wir können die minimale und maximale Anzahl der Verbindungen und die Verbindungsgeschwindigkeit pro Domain steuern.

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    // Gets or sets the total number of allowed open HTTP requests (threads)
    this.MaxHttpConnectionLimit = 80;
    // Gets or sets minimum polite delay (pause)between request to a given domain or IP address.
    this.RateLimitPerHost = TimeSpan.FromMilliseconds(50);            
    //     Gets or sets the allowed number of concurrent HTTP requests (threads) per hostname
    //     or IP address. This helps protect hosts against too many requests.
    this.OpenConnectionLimitPerHost = 25;
    this.ObeyRobotsDotTxt = false;
    //     Makes the WebSraper intelligently throttle requests not only by hostname, but
    //     also by host servers' IP addresses. This is polite in-case multiple scraped domains
    //     are hosted on the same machine.
    this.ThrottleMode = Throttle.ByDomainHostName;

    this.Request("https://www.Website.com", Parse);
}

public override void Init()
{
    License.LicenseKey = "LicenseKey";
    this.LoggingLevel = WebScraper.LogLevel.All;
    this.WorkingDirectory = AppSetting.GetAppRoot() + @"\ShoppingSiteSample\Output\";
    // Gets or sets the total number of allowed open HTTP requests (threads)
    this.MaxHttpConnectionLimit = 80;
    // Gets or sets minimum polite delay (pause)between request to a given domain or IP address.
    this.RateLimitPerHost = TimeSpan.FromMilliseconds(50);            
    //     Gets or sets the allowed number of concurrent HTTP requests (threads) per hostname
    //     or IP address. This helps protect hosts against too many requests.
    this.OpenConnectionLimitPerHost = 25;
    this.ObeyRobotsDotTxt = false;
    //     Makes the WebSraper intelligently throttle requests not only by hostname, but
    //     also by host servers' IP addresses. This is polite in-case multiple scraped domains
    //     are hosted on the same machine.
    this.ThrottleMode = Throttle.ByDomainHostName;

    this.Request("https://www.Website.com", Parse);
}

Public Overrides Sub Init()
	License.LicenseKey = "LicenseKey"
	Me.LoggingLevel = WebScraper.LogLevel.All
	Me.WorkingDirectory = AppSetting.GetAppRoot() & "\ShoppingSiteSample\Output\"
	' Gets or sets the total number of allowed open HTTP requests (threads)
	Me.MaxHttpConnectionLimit = 80
	' Gets or sets minimum polite delay (pause)between request to a given domain or IP address.
	Me.RateLimitPerHost = TimeSpan.FromMilliseconds(50)
	'     Gets or sets the allowed number of concurrent HTTP requests (threads) per hostname
	'     or IP address. This helps protect hosts against too many requests.
	Me.OpenConnectionLimitPerHost = 25
	Me.ObeyRobotsDotTxt = False
	'     Makes the WebSraper intelligently throttle requests not only by hostname, but
	'     also by host servers' IP addresses. This is polite in-case multiple scraped domains
	'     are hosted on the same machine.
	Me.ThrottleMode = Throttle.ByDomainHostName

	Me.Request("https://www.Website.com", Parse)
End Sub

Drosselungseigenschaften

• MaxHttpConnectionLimit
  gesamtzahl der zulässigen offenen HTTP-Anfragen (gewinde)
• RateLimitPerHost
  minimale höfliche Verzögerung oder Unterbrechung (in Millisekunden) zwischen Anfragen an eine bestimmte Domain oder IP-Adresse
• OpenConnectionLimitPerHost
  zulässige Anzahl gleichzeitiger HTTP-Anfragen (gewinde)
• ThrottleMode
  Ermöglicht es dem WebSraper, Anfragen nicht nur nach Hostnamen, sondern auch nach IP-Adressen der Hostserver zu drosseln. Dies ist höflich für den Fall, dass mehrere gescrapte Domains auf demselben Rechner gehostet werden.

Anhang

Wie erstellt man eine Windows-Formularanwendung?

Wir sollten dafür Visual Studio 2013 oder höher verwenden.

Folgen Sie diesen Schritten, um ein neues Windows Forms-Projekt zu erstellen:

1. Visual Studio öffnen

   

2. Datei -> Neu -> Projekt

   

3. Wählen Sie unter Vorlage die Programmiersprache(Visuelles C# oder VB) -> Windows -> Windows Forms-Anwendung

   

   Name des Projekts: IronScraperSample
   

Speicherort: Wählen Sie einen Speicherort auf Ihrer Festplatte

Wie erstellt man eine Webformular-Anwendung?

Sie sollten dafür Visual Studio 2013 oder höher verwenden.

Folgen Sie den Schritten zum Erstellen eines neuen Asp.NET-Webforms-Projekts

1. Visual Studio öffnen

   

2. Datei -> Neu -> Projekt

   

3. Aus Vorlage Programmiersprache wählen(Visuelles C# oder VB) -> Web -> ASP.NET-Webanwendung(.NET-Framework).

   

   Name des Projekts: IronScraperSample
   

Ort: Wählen Sie einen Speicherort auf Ihrer Festplatte

4. Von Ihren ASP.NET-Vorlagen

   1. Leere Vorlage auswählen
   2. Webformulare prüfen

   

5. Ihr grundlegendes ASP.NET-Webformularprojekt ist nun erstellt

   

   Klicken Sie hier um das vollständige Beispielprojekt des Tutorials herunterzuladen.

Ahmed Aboelmagd

.NET Software Solution Architect bei einem multinationalen IT-Unternehmen

Ahmed ist ein erfahrener und zertifizierter Microsoft Technologiespezialist mit mehr als 10 Jahren Erfahrung in der IT- und Softwareentwicklung. Er hat in einer Reihe von Unternehmen gearbeitet und ist jetzt Country Manager für ein multinationales IT-Unternehmen.

Ahmed arbeitet seit über einem Jahr mit IronPDF und IronWebScraper und setzt sie in mehreren Projekten seines Unternehmens ein.