OCR of TIFFs Over 2 GB with LibTiff.NET
IronOCR loads each image into one in-memory AnyBitmap buffer indexed by a 32-bit integer, so it caps at roughly 2 GB regardless of how much system memory is available. A TIFF larger than that fails to load. Split the oversized file into sub-2 GB chunks with BitMiracle.LibTiff.NET and OCR each chunk.
This is a fully managed alternative to the Magick.NET workaround: it copies pages at the raw strip or tile level, with no decode or re-encode step.
Solution
Before you start, make sure you have IronOCR (IronTesseract) in a .NET project. The LibTiff types used below live in the BitMiracle.LibTiff.Classic namespace.
1. Add the LibTiff.NET Package
dotnet add package BitMiracle.LibTiff.NET2. Add the TiffPageSplitter Helper
The helper copies each page at the raw strip or tile level, so pixel data and compression are preserved exactly. It streams pages one at a time, keeping peak memory to roughly one chunk rather than the whole file, and yields multi-page chunks that each stay under the size cap.
/// <summary>
/// Splits a multi-page TIFF into single-page TIFF byte streams without ever
/// holding the whole file in memory. Each page is copied at the raw
/// (still-encoded) strip/tile level, so pixel data and compression are
/// preserved exactly - there is no decode/re-encode step.
///
/// This is the chunking step only. It produces sub-2 GB single-page byte
/// arrays; feeding them to IronOCR (which is where the AnyBitmap 2 GB
/// single-buffer limit lives) is the consumer's job - see TiffOcrExample.
/// </summary>
public static class TiffPageSplitter
{
// Tags that describe how a page's raw strip/tile data is encoded.
// With a raw copy nothing is re-encoded, so every one of these must be
// carried over verbatim or the copied bytes become uninterpretable.
// Extend this list if your TIFFs carry tags not covered here
// (e.g. ICC profiles, EXTRASAMPLES for alpha channels).
private static readonly TiffTag[] ScalarIntTags =
{
TiffTag.IMAGEWIDTH,
TiffTag.IMAGELENGTH,
TiffTag.BITSPERSAMPLE,
TiffTag.SAMPLESPERPIXEL,
TiffTag.COMPRESSION,
TiffTag.PHOTOMETRIC,
TiffTag.FILLORDER,
TiffTag.PLANARCONFIG,
TiffTag.ORIENTATION,
TiffTag.RESOLUTIONUNIT,
TiffTag.PREDICTOR, // required for LZW / Deflate raw copies
TiffTag.SAMPLEFORMAT,
TiffTag.T4OPTIONS, // CCITT Group 3
TiffTag.T6OPTIONS, // CCITT Group 4
TiffTag.SUBFILETYPE,
};
private static readonly TiffTag[] ScalarDoubleTags =
{
TiffTag.XRESOLUTION, // DPI directly affects OCR accuracy
TiffTag.YRESOLUTION,
};
/// <summary>
/// Lazily yields each page of <paramref name="inputPath"/> as a standalone
/// single-page TIFF. The source file stays open for the lifetime of the
/// enumeration and only one page is materialised at a time, so peak memory
/// is roughly one page rather than the whole file.
/// </summary>
public static IEnumerable<byte[]> SplitTiffToPages(string inputPath)
{
using (Tiff input = Tiff.Open(inputPath, "r"))
{
if (input == null)
throw new InvalidOperationException($"Could not open TIFF: {inputPath}");
int pageCount = input.NumberOfDirectories();
for (int page = 0; page < pageCount; page++)
{
input.SetDirectory((short)page);
yield return ExtractCurrentPage(input);
}
}
}
private static byte[] ExtractCurrentPage(Tiff input)
{
using (var ms = new MemoryStream())
{
// Default TiffStream operates on the MemoryStream passed as clientData.
using (Tiff output = Tiff.ClientOpen("InMemory", "w", ms, new TiffStream()))
{
if (output == null)
throw new InvalidOperationException("Could not create in-memory TIFF.");
CopyTags(input, output);
if (input.IsTiled())
CopyRawTiles(input, output);
else
CopyRawStrips(input, output);
output.WriteDirectory();
}
return ms.ToArray();
}
}
private static void CopyTags(Tiff input, Tiff output)
{
foreach (TiffTag tag in ScalarIntTags)
{
FieldValue[] v = input.GetField(tag);
if (v != null && v.Length > 0)
output.SetField(tag, v[0].ToInt());
}
foreach (TiffTag tag in ScalarDoubleTags)
{
FieldValue[] v = input.GetField(tag);
if (v != null && v.Length > 0)
output.SetField(tag, v[0].ToDouble());
}
// Strip vs tile layout must match the raw data exactly, otherwise the
// raw bytes won't line up with the declared boundaries.
if (input.IsTiled())
{
output.SetField(TiffTag.TILEWIDTH, input.GetField(TiffTag.TILEWIDTH)[0].ToInt());
output.SetField(TiffTag.TILELENGTH, input.GetField(TiffTag.TILELENGTH)[0].ToInt());
}
else
{
FieldValue[] rps = input.GetField(TiffTag.ROWSPERSTRIP);
if (rps != null && rps.Length > 0)
output.SetField(TiffTag.ROWSPERSTRIP, rps[0].ToInt());
}
// Palette images: the colour map is required to interpret pixel indices.
FieldValue[] cmap = input.GetField(TiffTag.COLORMAP);
if (cmap != null && cmap.Length >= 3)
output.SetField(TiffTag.COLORMAP,
cmap[0].ToShortArray(), cmap[1].ToShortArray(), cmap[2].ToShortArray());
}
private static void CopyRawStrips(Tiff input, Tiff output)
{
int stripCount = input.NumberOfStrips();
int[] byteCounts = input.GetField(TiffTag.STRIPBYTECOUNTS)[0].ToIntArray();
for (int strip = 0; strip < stripCount; strip++)
{
byte[] buffer = new byte[byteCounts[strip]];
int read = input.ReadRawStrip(strip, buffer, 0, buffer.Length);
output.WriteRawStrip(strip, buffer, read);
}
}
private static void CopyRawTiles(Tiff input, Tiff output)
{
int tileCount = input.NumberOfTiles();
int[] byteCounts = input.GetField(TiffTag.TILEBYTECOUNTS)[0].ToIntArray();
for (int tile = 0; tile < tileCount; tile++)
{
byte[] buffer = new byte[byteCounts[tile]];
int read = input.ReadRawTile(tile, buffer, 0, buffer.Length);
output.WriteRawTile(tile, buffer, read);
}
}
/// <summary>
/// Lazily yields multi-page TIFF chunks (the equivalent of the old
/// Magick.NET 100-pages-per-chunk approach). A new chunk is started when
/// adding the next page would push the chunk past
/// <paramref name="maxChunkBytes"/>, or when <paramref name="maxPagesPerChunk"/>
/// is reached - whichever comes first.
///
/// Size is the real guard: page count alone can exceed the 2 GB AnyBitmap
/// limit on large pages. The byte total here is the encoded (compressed)
/// size, which is a cheap proxy - validate the cap against your actual
/// pages, since decoded size can be much larger than encoded.
/// </summary>
public static IEnumerable<byte[]> SplitTiffToChunks(
string inputPath,
int maxPagesPerChunk = 100,
long maxChunkBytes = 1_500_000_000L)
{
using (Tiff input = Tiff.Open(inputPath, "r"))
{
if (input == null)
throw new InvalidOperationException($"Could not open TIFF: {inputPath}");
int pageCount = input.NumberOfDirectories();
int page = 0;
while (page < pageCount)
{
using (var ms = new MemoryStream())
{
using (Tiff output = Tiff.ClientOpen("InMemory", "w", ms, new TiffStream()))
{
if (output == null)
throw new InvalidOperationException("Could not create in-memory TIFF.");
int pagesInChunk = 0;
long chunkBytes = 0;
while (page < pageCount && pagesInChunk < maxPagesPerChunk)
{
input.SetDirectory((short)page);
long pageBytes = RawPageByteSize(input);
// Stop before exceeding the cap, but always allow at
// least one page so a single large page still goes through.
if (pagesInChunk > 0 && chunkBytes + pageBytes > maxChunkBytes)
break;
CopyTags(input, output);
if (input.IsTiled())
CopyRawTiles(input, output);
else
CopyRawStrips(input, output);
output.WriteDirectory(); // finalise this page as one directory in the chunk
chunkBytes += pageBytes;
pagesInChunk++;
page++;
}
}
yield return ms.ToArray();
}
}
}
}
private static long RawPageByteSize(Tiff page)
{
TiffTag tag = page.IsTiled() ? TiffTag.TILEBYTECOUNTS : TiffTag.STRIPBYTECOUNTS;
int[] counts = page.GetField(tag)[0].ToIntArray();
long total = 0;
foreach (int c in counts)
total += c;
return total;
}
}SplitTiffToChunks starts a new chunk whenever the next page would push the total past maxChunkBytes or once maxPagesPerChunk is reached, whichever comes first. A single page larger than the cap is always allowed through on its own.
3. OCR Each Chunk
Iterate SplitTiffToChunks and load each byte array with OcrInput.LoadImage(byte[]) rather than the file path, so nothing larger than the limit ever reaches AnyBitmap.
var inputPath = "2gb_benchmark1200.tiff";
var ocr = new IronTesseract();
int chunk = 0;
foreach (byte[] chunkBytes in TiffPageSplitter.SplitTiffToChunks(inputPath, maxPagesPerChunk: 100))
{
using (var ocrInput = new OcrInput())
{
ocrInput.LoadImage(chunkBytes); // loads every page in the chunk
var result = ocr.Read(ocrInput);
Console.WriteLine($"Chunk {chunk}: {result.Text?.Length ?? 0} chars");
}
chunk++;
}Passing the byte array keeps each input under the cap. Note that OcrInput.LoadImage(filePath) currently returns zero loaded pages instead of throwing a clear error when the file is too large; this silent failure is a known issue, and chunking sidesteps it entirely.
4. Tune the Chunk Limits for Your Data
Adjust maxPagesPerChunk and maxChunkBytes to match your TIFFs. Lower them if a chunk approaches 2 GB once decoded or if memory is tight; raise the page count for smaller pages to cut overhead.
Notes and Limitations
- Tag coverage: the splitter carries over only the tags listed in
ScalarIntTagsandScalarDoubleTags. If your TIFFs use tags not covered there, such as ICC profiles orEXTRASAMPLESfor alpha channels, extend those lists or the raw-copied bytes may be misinterpreted. - Managed dependency: LibTiff.NET is fully managed with no native binaries, unlike Magick.NET, which ships ImageMagick native libraries that add to package size and deployment footprint.
- Exact preservation: the raw strip or tile copy avoids the decode and re-encode that the Magick.NET approach performs, keeping the original compression and pixel data intact.
For further reading, see BitMiracle.LibTiff.NET on NuGet.
Curtis Chau holds a Bachelor’s degree in Computer Science (Carleton University) and specializes in front-end development with expertise in Node.js, TypeScript, JavaScript, and React. Passionate about crafting intuitive and aesthetically pleasing user interfaces, Curtis enjoys working with modern frameworks and creating well-structured, visually appealing manuals.
