Compression and file size

document.Options, a PdfDocumentOptions, decides how a document is written when you save it. None of its settings changes how a page looks. They change only how many bytes it takes to say the same thing. The defaults already compress what matters, so most documents need no change here.

Read this page when a file is larger than you expect, when you need the smallest file you can get, or when you want to read what the library wrote. All the settings are in the core PdfPinata package, in the PdfPinata.Pdf namespace.

Measure before you change anything

The Compress demo saves the same page under each setting and compares the sizes. To do the same with your own content, save to a MemoryStream and read its length:

src/SampleApp/Demos/CompressDemo.cs

// Saved under one arrangement of the options and measured. Nothing is written to disk.
long Measure(Action<PdfDocumentOptions> configure)
{
    using var probe = new PdfDocument();
    configure(probe.Options);
    Representative(probe);

    using var buffer = new MemoryStream();
    probe.Save(buffer, false);
    return buffer.Length;
}

Set the options straight after you create the document, before you draw anything. Some of them are applied when an image is added, not when the document is saved.

The settings

Setting	Default	What it does
CompressContentStreams	true	Compresses the drawing operators of each page, and of forms.
NoCompression	false	When true, leaves embedded fonts, object streams and cross-reference streams uncompressed.
FlateEncodeMode	Default	How hard the compressor works: BestSpeed, Default or BestCompression.
UseFlateDecoderForJpegImages	Never	Whether to compress JPEG images a second time: Never, Automatic or Always.
ColorMode	Rgb	Writes every colour as RGB, as CMYK, or as it was given (Undefined).
CrossReferenceFormat	Classic	Stream gathers small objects into compressed object streams.
MaxObjectsPerObjectStream	200	How many objects go into one object stream, when CrossReferenceFormat is Stream.

Content streams

src/SampleApp/Demos/CompressDemo.cs

var compressed = Measure(options => options.CompressContentStreams = true);
var uncompressed = Measure(options => options.CompressContentStreams = false);
var noCompression = Measure(options =>
{
    options.CompressContentStreams = true;
    options.NoCompression = true;
});

Turning CompressContentStreams off writes each page's operators as plain text. The file is much larger, but you can open it in a text editor and read what the library drew. Do that once when you debug, and ship the compressed file.

NoCompression does not affect content streams. For a file with as little compressed as possible, set CompressContentStreams to false and NoCompression to true. Images are compressed either way.

How hard to compress

src/SampleApp/Demos/CompressDemo.cs

var flateBest = Measure(options =>
{
    options.CompressContentStreams = true;
    options.FlateEncodeMode = PdfFlateEncodeMode.BestCompression;
});
var flateFast = Measure(options =>
{
    options.CompressContentStreams = true;
    options.FlateEncodeMode = PdfFlateEncodeMode.BestSpeed;
});

BestCompression gives a slightly smaller file and takes longer to save. BestSpeed saves faster and gives a larger file. The difference is small on most documents.

JPEG images

src/SampleApp/Demos/CompressDemo.cs

var jpegFlate = Measure(options =>
{
    options.CompressContentStreams = true;
    options.UseFlateDecoderForJpegImages = PdfUseFlateDecoderForJpegImages.Always;
});
var jpegAuto = Measure(options =>
{
    options.CompressContentStreams = true;
    options.UseFlateDecoderForJpegImages = PdfUseFlateDecoderForJpegImages.Automatic;
});

A JPEG image is already compressed, and compressing it again rarely gains much. Automatic tries it and keeps the result only when it is smaller, so it can never make the file larger. Always keeps the result whatever it costs, and is meant for testing. Whether Automatic gains anything depends on the picture.

Colour mode

src/SampleApp/Demos/CompressDemo.cs

var cmyk = Measure(options =>
{
    options.CompressContentStreams = true;
    options.ColorMode = PdfColorMode.Cmyk;
});

ColorMode is not a compression setting, but it changes the size a little: CMYK writes four numbers for each colour where RGB writes three. It also converts the colours, so a document drawn in RGB and saved as CMYK shows the nearest CMYK colours, not the same ones. Choose it for your printer, not for size.

Cross-reference streams and object streams

A PDF is a set of numbered objects: one for each page, font, annotation, form field and, in a tagged document, each paragraph and table cell. By default each object is written out on its own, uncompressed, and listed in a plain-text table at the end of the file.

CrossReferenceFormat.Stream gathers up to MaxObjectsPerObjectStream small objects into one object stream and compresses them together, and writes the table as a compressed stream too:

src/SampleApp/Demos/CompressDemo.cs

var xrefStream = Measure(options =>
{
    options.CompressContentStreams = true;
    options.CrossReferenceFormat = PdfCrossReferenceFormat.Stream;
});

The saving depends on the shape of the document. A page of drawing is mostly one large content stream, which cannot go into an object stream, so the setting gains little there. A document made mostly of objects gains a lot: many short pages, many links or form fields, and above all tagged documents. The demo measures a hundred nearly empty pages both ways:

src/SampleApp/Demos/CompressDemo.cs

long ManyObjects(PdfCrossReferenceFormat format)
{
    using var probe = new PdfDocument();
    probe.Options.CompressContentStreams = true;
    probe.Options.CrossReferenceFormat = format;

    for (var number = 1; number <= 100; number++)
    {
        var page = probe.AddPage();
        using var gfx = XGraphics.FromPdfPage(page);
        gfx.DrawString($"Page {number}", body, XBrushes.Black, new XPoint(50, 60));
    }

    using var buffer = new MemoryStream();
    probe.Save(buffer, false);
    return buffer.Length;
}

var manyClassic = ManyObjects(PdfCrossReferenceFormat.Classic);
var manyStream = ManyObjects(PdfCrossReferenceFormat.Stream);

A larger MaxObjectsPerObjectStream gives a smaller file, but a reader must decompress a whole object stream to reach any one object in it, so the file opens a little more slowly.

Images decide most of the size

In most documents the images outweigh everything else, and no option above shrinks them much. What matters is how you load them:

• A PNG is stored without loss, compressed, with its transparency as a separate mask.
• Every other format is stored as a JPEG, encoded again at quality 75 by default. This includes JPEG files, so a JPEG you load is decoded and compressed a second time.

To choose the quality, load the image through ImageSource with a quality from 0 to 100:

using PdfPinata.Drawing;
using PinataLayout.DocumentObjectModel.Shapes;

XImage photo = XImage.FromImageSource(ImageSource.FromFile("photo.jpg", 60));

ImageSource is in the PinataLayout.DocumentObjectModel.Shapes namespace, although it ships in the core PdfPinata package.

PDF stores an image's pixels, not a resolution. A 4000-pixel photograph drawn 5 cm wide is stored with all 4000 pixels. Scale large pictures down before you load them. When a merged document holds the same image twice, or pages name fonts they do not use, see Merge, split and assemble for ConsolidateImages and PruneUnusedResources. The Images page covers loading images in full.

Things to know

• CompressContentStreams is true in every build. In PDFsharp it was false in debug builds, so the same code wrote a larger file from a debug build than from a release build. If you compare file sizes with older output, check how that output was built.
• Cross-reference streams need a PDF 1.5 reader. Every current reader is one. Saving with Stream raises the document's version to 1.5 if it was lower.
• PDF/A-1 refuses cross-reference streams. PDF/A-1 is based on PDF 1.4, so a document that claims it cannot be saved with CrossReferenceFormat.Stream. PDF/A-2 and later allow it. See PDF/A.
• A document you open and save does not keep its old format. A file that used cross-reference streams is saved with a classic table unless you set CrossReferenceFormat to Stream.
• MaxObjectsPerObjectStream does nothing on its own. It applies only when CrossReferenceFormat is Stream.
• An incremental save cannot make a file smaller. It only appends. See Incremental saving.

See it in action

The Compress demo saves one page of text, paths and a photograph under each setting and prints the size of each file, then compares the two cross-reference formats on a hundred-page document.

The full Compress demo

src/SampleApp/Demos/CompressDemo.cs

var heading = new XFont("Liberation Sans", 16, XFontStyle.Bold);
var label = new XFont("Liberation Sans", 9, XFontStyle.Bold);
var body = new XFont("Liberation Sans", 9);
var mono = new XFont("Source Code Pro", 8.5);

// One page of representative content - text, a long path, a photograph - built the same
// way every time so that the only thing that varies between the measurements below is the
// options the document was saved under.
void Representative(PdfDocument target)
{
    var page = target.AddPage();
    using var gfx = XGraphics.FromPdfPage(page);

    gfx.DrawString("Representative content", heading, XBrushes.Black, new XPoint(50, 60));

    var prose = new XTextFormatter(gfx);
    for (var block = 0; block < 4; block++)
    {
        prose.DrawString(
            "Compression acts on the content stream, which is the list of drawing "
            + "operators a page is made of. Text is cheap, paths are dear, and an image is "
            + "usually already compressed by the time it arrives - which is why the "
            + "settings below move the total by wildly different amounts.",
            body, XBrushes.Black, new XRect(50, 90 + block * 60, 495, 55));
    }

    // A path with a great many segments. This is what compression has something to work
    // on: a few hundred coordinates written out as text.
    var path = new XGraphicsPath();
    for (var step = 0; step < 400; step++)
    {
        var t = step / 400.0 * Math.PI * 8;
        var point = new XPoint(
            50 + step * 495.0 / 400,
            500 + Math.Sin(t) * 60 * (1 - step / 400.0));

        if (step == 0)
            path.AddLine(point, point);
        else
            path.AddLine(new XPoint(50 + (step - 1) * 495.0 / 400,
                500 + Math.Sin((step - 1) / 400.0 * Math.PI * 8) * 60 * (1 - (step - 1) / 400.0)), point);
    }

    gfx.DrawPath(new XPen(XColors.MidnightBlue, 0.8), path);

    using var photograph = XImage.FromStream(
        () => Assets.Open(Assets.ImagePrefix + "pdf-pinata.jpg"));
    gfx.DrawImage(photograph, 50, 580, 240, 180);
}

// Saved under one arrangement of the options and measured. Nothing is written to disk.
long Measure(Action<PdfDocumentOptions> configure)
{
    using var probe = new PdfDocument();
    configure(probe.Options);
    Representative(probe);

    using var buffer = new MemoryStream();
    probe.Save(buffer, false);
    return buffer.Length;
}

// Every row sets CompressContentStreams explicitly rather than leaving it alone. A table of
// "the defaults" would go quietly wrong the day a default changed, and this one did change:
// it used to be declared false under #if DEBUG and true otherwise, so the same code wrote a
// materially larger file from a debug build than from a release one.
var defaultCompression = new PdfDocument().Options.CompressContentStreams;

var compressed = Measure(options => options.CompressContentStreams = true);
var uncompressed = Measure(options => options.CompressContentStreams = false);
var noCompression = Measure(options =>
{
    options.CompressContentStreams = true;
    options.NoCompression = true;
});
var flateBest = Measure(options =>
{
    options.CompressContentStreams = true;
    options.FlateEncodeMode = PdfFlateEncodeMode.BestCompression;
});
var flateFast = Measure(options =>
{
    options.CompressContentStreams = true;
    options.FlateEncodeMode = PdfFlateEncodeMode.BestSpeed;
});
var jpegFlate = Measure(options =>
{
    options.CompressContentStreams = true;
    options.UseFlateDecoderForJpegImages = PdfUseFlateDecoderForJpegImages.Always;
});
var jpegAuto = Measure(options =>
{
    options.CompressContentStreams = true;
    options.UseFlateDecoderForJpegImages = PdfUseFlateDecoderForJpegImages.Automatic;
});
var cmyk = Measure(options =>
{
    options.CompressContentStreams = true;
    options.ColorMode = PdfColorMode.Cmyk;
});
var xrefStream = Measure(options =>
{
    options.CompressContentStreams = true;
    options.CrossReferenceFormat = PdfCrossReferenceFormat.Stream;
});

// A page of drawing is mostly one large content stream and a handful of objects, which is
// the shape a cross-reference stream has least to offer. Measured again over a document that
// is mostly objects - a hundred nearly empty pages - because that is where the setting is
// worth reaching for, and a table showing only the first number would teach the opposite of
// what is true.
long ManyObjects(PdfCrossReferenceFormat format)
{
    using var probe = new PdfDocument();
    probe.Options.CompressContentStreams = true;
    probe.Options.CrossReferenceFormat = format;

    for (var number = 1; number <= 100; number++)
    {
        var page = probe.AddPage();
        using var gfx = XGraphics.FromPdfPage(page);
        gfx.DrawString($"Page {number}", body, XBrushes.Black, new XPoint(50, 60));
    }

    using var buffer = new MemoryStream();
    probe.Save(buffer, false);
    return buffer.Length;
}

var manyClassic = ManyObjects(PdfCrossReferenceFormat.Classic);
var manyStream = ManyObjects(PdfCrossReferenceFormat.Stream);

// ----- the document the demo hands back -----

var document = new PdfDocument();
document.Info.Title = "Compress";

// Page one is the content itself, so the reader can see that every measurement above was
// taken over this and that none of the settings changed how it looks.
Representative(document);

var report = document.AddPage();
using (var gfx = XGraphics.FromPdfPage(report))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("What each setting costs", heading, XBrushes.Black, new XPoint(50, 60));

    prose.DrawString(
        "The page before this one, saved eight times under different options. Every one of "
        + "those files renders identically; the only difference between them is how many "
        + "bytes it takes to say the same thing. That is why this demo is a table - there "
        + "is nothing else to look at.",
        body, XBrushes.Black, new XRect(50, 80, 495, 50));

    (string Setting, long Bytes, string Note)[] rows =
    {
        ("CompressContentStreams = true", compressed, "The baseline every other row is measured against"),
        ("CompressContentStreams = false", uncompressed, "Operators written as readable text"),
        ("NoCompression = true", noCompression, "Nothing in the file is compressed at all"),
        ("FlateEncodeMode.BestCompression", flateBest, "Slower to save, smaller to keep"),
        ("FlateEncodeMode.BestSpeed", flateFast, "Faster to save, larger to keep"),
        ("UseFlateDecoderForJpegImages.Always", jpegFlate, "Flate over the already-compressed JPEG, whatever it costs"),
        ("UseFlateDecoderForJpegImages.Automatic", jpegAuto, "The same, kept only if it turned out smaller"),
        ("ColorMode.Cmyk", cmyk, "Four components per colour instead of three"),
        ("CrossReferenceFormat.Stream", xrefStream, "Barely moves a page that is mostly one content stream")
    };

    double y = 145;
    gfx.DrawString("setting", label, XBrushes.Black, new XPoint(50, y));
    gfx.DrawString("bytes", label, XBrushes.Black, new XPoint(280, y));
    gfx.DrawString("against the first row", label, XBrushes.Black, new XPoint(350, y));
    y += 18;

    foreach (var row in rows)
    {
        var delta = row.Bytes - compressed;
        gfx.DrawString(row.Setting, mono, XBrushes.Black, new XPoint(50, y));
        gfx.DrawString($"{row.Bytes:N0}", body, XBrushes.Black, new XPoint(280, y));
        gfx.DrawString(
            delta == 0 ? "-" : $"{(delta > 0 ? "+" : "")}{delta:N0}",
            body, delta > 0 ? XBrushes.Firebrick : XBrushes.SeaGreen, new XPoint(350, y));
        gfx.DrawString(row.Note, body, XBrushes.DimGray, new XPoint(50, y + 11));
        y += 28;
    }

    gfx.DrawString("The default used not to be a constant", label, XBrushes.Firebrick,
        new XPoint(50, y + 15));

    prose.DrawString(
        $"CompressContentStreams defaults to {defaultCompression.ToString().ToLowerInvariant()}, "
        + "and now does so in every build. It used to be declared false under #if DEBUG and "
        + "true otherwise, which made the size of the output a property of how the library "
        + "had been compiled rather than of the code calling it: the same program wrote a "
        + "materially larger PDF from a debug build, and two files could not be compared "
        + "without knowing which configuration each came from. Set it to false to get the "
        + "readable content stream back - that was the only thing the conditional bought.",
        body, XBrushes.Black, new XRect(50, y + 28, 495, 80));

    gfx.DrawString("What to make of the rest", label, XBrushes.Black, new XPoint(50, y + 105));

    prose.DrawString(
        "Turning compression off has a large and predictable cost, and it is worth doing "
        + "exactly once: an uncompressed PDF can be opened in a text editor and read, "
        + "which is the fastest way to find out what the library actually wrote. Ship the "
        + "compressed one.",
        body, XBrushes.Black, new XRect(50, y + 118, 495, 45));

    prose.DrawString(
        "UseFlateDecoderForJpegImages runs flate over a stream that is already compressed, "
        + "and whether that wins depends entirely on the picture - the number above is "
        + "this photograph's answer and not a general one. Always takes the flated form "
        + "whatever it costs, because some readers will not decode a bare DCT stream; "
        + "Automatic keeps it only when it came out smaller, which is the setting to reach "
        + "for if size is the reason you are here.",
        body, XBrushes.Black, new XRect(50, y + 168, 495, 60));

    prose.DrawString(
        "ColorMode decides what is written for every colour in the file - three components "
        + "or four. Switching to CMYK is what a press wants and what a screen does not; "
        + "the colours are converted on the way out, so a document built in RGB and saved "
        + "as CMYK is not the same colours, only the nearest ones.",
        body, XBrushes.Black, new XRect(50, y + 235, 495, 45));

    gfx.DrawString("One of those rows is measured unfairly", label, XBrushes.Black,
        new XPoint(50, y + 290));

    prose.DrawString(
        "CrossReferenceFormat, which barely moves the table above and moves a great deal on "
        + "the right document. That comparison is the next page, because it needs a second "
        + "document to make it against.",
        body, XBrushes.Black, new XRect(50, y + 304, 495, 34));
}

// A page of its own rather than the foot of the one before it. The table above ends near the
// bottom margin already, and a block appended after it was drawn off the media box entirely -
// painted, and invisible, which is the one kind of layout mistake nothing complains about.
var objects = document.AddPage();
using (var gfx = XGraphics.FromPdfPage(objects))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("Where a cross-reference stream pays", heading, XBrushes.Black,
        new XPoint(50, 60));

    prose.DrawString(
        "CrossReferenceFormat.Stream gathers the objects that may be gathered into object "
        + "streams and compresses them together, and it has almost nothing to work on in the "
        + "measurement on the page before: one page of drawing is a single large content "
        + "stream and a dozen objects, and a content stream is not something an object stream "
        + "may hold. Measured over a document that is mostly objects instead - a hundred "
        + "nearly empty pages - the same setting reads differently.",
        body, XBrushes.Black, new XRect(50, 80, 495, 72));

    gfx.DrawString("format", label, XBrushes.Black, new XPoint(50, 170));
    gfx.DrawString("bytes", label, XBrushes.Black, new XPoint(280, 170));
    gfx.DrawString("against classic", label, XBrushes.Black, new XPoint(350, 170));

    gfx.DrawString("Classic, 100 pages", mono, XBrushes.Black, new XPoint(50, 190));
    gfx.DrawString($"{manyClassic:N0}", body, XBrushes.Black, new XPoint(280, 190));
    gfx.DrawString("-", body, XBrushes.DimGray, new XPoint(350, 190));

    gfx.DrawString("Stream, 100 pages", mono, XBrushes.Black, new XPoint(50, 208));
    gfx.DrawString($"{manyStream:N0}", body, XBrushes.Black, new XPoint(280, 208));
    gfx.DrawString(
        $"{(manyStream > manyClassic ? "+" : "")}{manyStream - manyClassic:N0}",
        body, manyStream > manyClassic ? XBrushes.Firebrick : XBrushes.SeaGreen,
        new XPoint(350, 208));

    gfx.DrawString("The same page, measured twice", mono, XBrushes.Black, new XPoint(50, 226));
    gfx.DrawString($"{compressed:N0} / {xrefStream:N0}", body, XBrushes.Black, new XPoint(280, 226));
    gfx.DrawString($"{xrefStream - compressed:N0}", body,
        xrefStream > compressed ? XBrushes.Firebrick : XBrushes.SeaGreen, new XPoint(350, 226));

    gfx.DrawString("What it costs", label, XBrushes.Black, new XPoint(50, 265));

    prose.DrawString(
        "A reader that understands PDF 1.5, which by now is all of them - and it is the one "
        + "option here PDF/A-1 refuses outright, because a cross-reference stream is a PDF "
        + "1.5 construction and PDF/A-1 is defined against 1.4. The Archive demo shows that "
        + "refusal in the writer's own words.",
        body, XBrushes.Black, new XRect(50, 280, 495, 48));

    gfx.DrawString("MaxObjectsPerObjectStream trades the other way", label, XBrushes.Black,
        new XPoint(50, 345));

    prose.DrawString(
        "A reader has to decompress a whole object stream to reach any one object in it, so a "
        + "larger number is a smaller file and a slower open. Acrobat uses 200 and so does "
        + "this. It means nothing at all unless CrossReferenceFormat is Stream, which is the "
        + "sort of option that is easy to set and hard to notice has done nothing.",
        body, XBrushes.Black, new XRect(50, 360, 495, 58));
}