How to OCR Subtitles in C# (Tutorial)

In this tutorial, we will learn about extracting hardcoded subtitles from video files. We will take a sample video file and extract the hardcoded subtitles into a txt file. We will develop a C# .NET program that will extract the hardcoded subtitles using OCR Process. I will keep this tutorial simple and easy so that even a beginner C# Programmer can understand it.

We need an efficient Optical Character Recognition (OCR) engine that can process the video and get subtitle files irrespective of the subtitle language.

There are many libraries available that provide OCR results. Some of them are paid, some of them are difficult to use, and some of them are not efficient or accurate, so It is very difficult to find a library that is free, efficient, easy to use, and provide accurate result.

IronOCR, which is free for development, provides a one-month free trial for commercial purposes. It supports over 150 languages and provides better accuracy than most other OCR libraries available. It is also efficient and easy-to-use. We will use this library for our demonstration.

IronOCR

IronOCR is a library developed and maintained by Iron Software that helps C# Software Engineers perform OCR, Barcode Scanning, and Text Extraction in .NET projects.

The features of IronOCR include:

• Reading text from many formats such as images (JPEG, PNG, BMP), GIF, TIF/TIFF, Streams, and PDFs
• Correction of low-quality scans and photos with a plethora of filters such as Deskew, Denoise, Binarize, Enhance Resolution, Dilate, and many more
• Reading barcodes from over 20 different formats, along with QR Code Support
• Utilizing the latest build of Tesseract OCR, with its performance tuned above and beyond other libraries of its kind
• Exporting Searchable PDFs, hOCR / HTML Exporting, and image content text.

Let's develop a demo application to read license plate numbers.

Create a Visual Studio Project

The first step is to create a new project.

Open Visual Studio. Click on Create New Project, and select the Console Application project template.

Click on the Next button, and name the project (I have named it "OCR Subtitles", you can name it as per your choice).

Click on the Next button, and Select your target Framework. Finally, Click on Create button to create the project.

The project will be created as shown below.

Creating a New Project in Visual Studio

Now, we need to install the IronOCR library to use it in our project. The easiest way is to install it via NuGet Package Manager for Solution.

Install IronOCR NuGet Package

Click on Tools from the top menu bar, and select NuGet Package Manager > Manage NuGet Packages for Solution, as shown below.

Installing IronOCR within Visual Studio

The following window will appear.

Visual Studio NuGet Package Manager UI

Click on browse, and search for IronOCR. Select IronOCR Package and click on the Install button, as shown below.

Searching for IronOCR in the NuGet Package Manager UI

The IronOCR Library will be installed and ready to use.

Extract Hardcoded Subtitles

Let's write a program to extract hardcoded subtitles.

We are going to use the following screenshot for extracting subtitles.

Sample video screenshot from which text will be extracted

Add the following namespace:

using IronOcr;

using IronOcr;

Imports IronOcr

Write the following code below the namespace declaration.

var ocr = new IronTesseract();
using (var input = new OcrInput(@"D:\Liscence Plate\plate3.jpg"))
{
    var result = ocr.Read(input);
    Console.WriteLine(result.Text);
}

var ocr = new IronTesseract();
using (var input = new OcrInput(@"D:\Liscence Plate\plate3.jpg"))
{
    var result = ocr.Read(input);
    Console.WriteLine(result.Text);
}

Dim ocr = New IronTesseract()
Using input = New OcrInput("D:\Liscence Plate\plate3.jpg")
	Dim result = ocr.Read(input)
	Console.WriteLine(result.Text)
End Using

The code above works as follows:

1. Initialize IronTesseract object. It will create a default instance of IronTessearct.
2. Create a new OcrInput object populated with an input image file or PDF document. OcrInput is the preferred input type because it allows for OCR of multi-paged documents, and allows images to be enhanced before OCR to obtain faster, more accurate results.
3. Read a text from an OCR Input Object and return an OCR Result object. ocr.Read will extract subtitles from the given input screenshot.
4. result.Text will return the entire content extracted from the given input.

The sample program produces the console output below:

Console output generated from performing text extraction on the sample image using IronOCR

Let's suppose that you have a video frame which contains both the title of the video and the subtitles:

A single frame of a longer video containing text regions for the video title and the video subtitles

Our goal is to extract the hardcoded subtitles from the bottom region of the image. In this case, we need to specify the text region in which the subtitle is displayed.

Specify Subtitle Location in the Frame

We can use a System.Drawing.Rectangle to specify a region in which we will read a subtitle from the Video frame. The unit of measurement is always pixels.

We will use the following sample code to specify the text region.

var ocr = new IronTesseract();
using (var input = new OcrInput())
{
    // a 41% improvement on speed
    var contentArea = new CropRectangle(x: 189, y: 272, height: 252, width: 77);
    input.AddImage(@"D:\subtitle\image.png", contentArea);
    var result = ocr.Read(input);
    Console.WriteLine(result.Text);
}

var ocr = new IronTesseract();
using (var input = new OcrInput())
{
    // a 41% improvement on speed
    var contentArea = new CropRectangle(x: 189, y: 272, height: 252, width: 77);
    input.AddImage(@"D:\subtitle\image.png", contentArea);
    var result = ocr.Read(input);
    Console.WriteLine(result.Text);
}

Dim ocr = New IronTesseract()
Using input = New OcrInput()
	' a 41% improvement on speed
	Dim contentArea = New CropRectangle(x:= 189, y:= 272, height:= 252, width:= 77)
	input.AddImage("D:\subtitle\image.png", contentArea)
	Dim result = ocr.Read(input)
	Console.WriteLine(result.Text)
End Using

This yields a 41% speed increase - and allows us to be specific. In contentArea, we have specified the start point in x and y, and then the height and width of the required subtitle region.

Save Subtitle into a Subtitle text file

Let's save the extracted subtitles into a TXT file.

var ocr = new IronTesseract();
using (var input = new OcrInput(@"D:\subtitle\subtitle1.png"))
{
    var result = ocr.Read(input);
    result.SaveAsTextFile(@"D:\subtitle\subtitlefile.txt");
}

var ocr = new IronTesseract();
using (var input = new OcrInput(@"D:\subtitle\subtitle1.png"))
{
    var result = ocr.Read(input);
    result.SaveAsTextFile(@"D:\subtitle\subtitlefile.txt");
}

Dim ocr = New IronTesseract()
Using input = New OcrInput("D:\subtitle\subtitle1.png")
	Dim result = ocr.Read(input)
	result.SaveAsTextFile("D:\subtitle\subtitlefile.txt")
End Using

result.SaveAsTextFile will take the output path as an argument, and save the file in the given path.

A single frame of a longer video containing text regions for the video title and the video subtitles

Summary

In this tutorial, we have learned to use IronOCR and develop a very simple program to read subtitles from the video screenshot. We can also specify the region for which we want to extract the text.

IronOCR provides the features of OpenCV for Computer Vision. We have seen that IronOCR enables us to read text from blurred or low-resolution images. This library is efficient and provides accuracy. It supports 127+ languages with full accuracy. It's free for development and has no restriction on production.

In summary, IronOCR provides:

• The ability to scan and read images and scanned documents
• Support for 150+ global languages
• Output as text, structured data, or searchable PDFs
• Supports .NET 6, 5, Core, Standard, Framework

IronOCR is part of Iron Software's suite of libraries useful for reading and writing PDFs, manipulating Excel files, reading text from images, and scrapping content from websites. Purchase the complete Iron Suite for the price of two individual libraries.