Incremental saving

Save writes a whole new file from the document in memory. SaveIncremental does not: it copies the bytes the document was read from, unchanged, and appends only what you changed after them, as a new revision. A PDF reader shows the latest revision. The earlier ones are still in the file.

You need an incremental save in two cases:

The document is signed. A signature covers the bytes of the file as they were when it was signed. Save rewrites those bytes and breaks every signature. An incremental save leaves them alone, so it is the only way to change a signed document and keep its signatures valid.
You must keep every earlier state of the document. Each revision stays in the file and can be recovered, which gives you an audit trail.

An incremental save does not make a file smaller. It only ever makes it larger.

SaveIncremental is a method of PdfDocument in the core PdfPinata package.

Open for appending, change, save

Open the document with PdfDocumentOpenMode.Append.
Change it as you would in Modify mode.
Call SaveIncremental with an empty stream.

The Revise demo saves a document, opens it again for appending and changes its subject line:

src/SampleApp/Demos/ReviseDemo.cs
var revisionOne = new MemoryStream();
original.Save(revisionOne, false);
var sizeOfOne = revisionOne.Length;

// ----- revision two: opened for appending, a page added ------------------------------------

revisionOne.Position = 0;
using var appended = PdfReader.Open(revisionOne, PdfDocumentOpenMode.Append);

// Changing something that was already there, as well as adding. The title is in the
// information dictionary, which is an object like any other: the appended revision carries a
// second definition of it and the reader takes that one.
appended.Info.Subject = "Amended by revision two";

It then adds a page, saves the second revision and opens that for a third:

src/SampleApp/Demos/ReviseDemo.cs
var revisionTwo = new MemoryStream();
appended.SaveIncremental(revisionTwo);
var afterTwo = revisionTwo.ToArray();

// ----- revision three: opened again, and this is the one written to the file ----------------

revisionTwo.Position = 0;
var document = PdfReader.Open(revisionTwo, PdfDocumentOpenMode.Append);

With files on disk, write the new revision to a new file:

using PdfPinata.Pdf;
using PdfPinata.Pdf.IO;

using (PdfDocument document = PdfReader.Open("contract.pdf", PdfDocumentOpenMode.Append))
{
    document.Info.Subject = "Countersigned";

    using FileStream output = new FileStream("contract-new.pdf", FileMode.Create);
    document.SaveIncremental(output);
}
File.Move("contract-new.pdf", "contract.pdf", overwrite: true);

Why the mode has to be `Append`

An incremental save replaces an object by writing a new version of it under the same object number. The reader reads the file from the end, finds the newest version of each object first, and ignores the older ones.

That only works if the object numbers are the ones in the file. Modify renumbers every object when it opens the file, and Import and ReadOnly do not permit changes. Append keeps the numbers and also keeps the original bytes, so that SaveIncremental can copy them. On any other document, SaveIncremental throws InvalidOperationException and says why. A document you created with new PdfDocument() has nothing to append to, so it cannot be saved incrementally either.

`Save` and `SaveIncremental` compared

	`Save`	`SaveIncremental`
Writes	The whole document, from memory.	The original bytes, then the changed and new objects.
Signatures	Broken.	Kept valid.
Earlier revisions	Discarded.	Kept in the file.
File size	Only what the document holds now.	Grows with every revision.
Open mode	`Modify` or `Append`.	`Append` only.

A document opened in Append mode can still be saved with Save. That writes a single-revision file, which is how you reclaim the space that many revisions take, and how you remove content for good. Do not do it to a signed document.

What a revision contains

A revision holds every object you changed, written out in full, and every new object, followed by a new cross-reference section. It holds more than you might expect:

Adding a page changes the page list, so the page tree node is written again as well.
Changing a document property, such as Info.Subject, writes the document information dictionary again.
Fonts are embedded again. A document opened for appending does not reuse the font objects already in the file. A new page drawn in a face the file already uses embeds that face a second time, and that is usually most of the size of the revision. The Revise demo shows the numbers.

The library tracks what you change. Changes you make through the document, its pages and the Elements of its objects mark those objects as changed. If you change an object some other way, call MarkAsChanged on it, or the revision will not contain it and the file will still show the old version.

Things to know

The stream must be empty. SaveIncremental writes the whole file, original bytes and all. Given a stream that already holds data, such as the file the document came from, it throws ArgumentException. Overwriting the source in place would leave the end of the old file behind the new revision, and a reader would find the old end first and ignore the revision.
Append mode holds the original file in memory for as long as the document is open. A 200 MB file takes 200 MB, and a file larger than 2 GB cannot be opened in this mode at all.
An earlier revision still holds everything it ever said. If you cover a name with a black rectangle and save incrementally, the name is still in the file, one revision back, and anyone can recover it. To remove content, change it and call Save, which also removes any signature.
Readers need nothing special. Any PDF reader opens an incrementally saved file and shows the latest revision. Only a tool that looks at the bytes sees the earlier ones.
Reading a page's content can mark it as changed. ContentReader.ReadContent(page) rearranges how the page stores its content streams, so the page goes into the next revision even if you change nothing else. See Reading content streams.
Signing uses the same mechanism. PdfSigner appends a revision that carries the signature, which is why signing an already-signed document keeps the first signature valid. See Digital signatures.

See it in action

The Revise demo writes one file with three revisions. Page one is revision one, page three was added by revision two, and page four by revision three. The pages report the size of each revision and count the markers a reader follows from one revision to the one before.

The full Revise demo

src/SampleApp/Demos/ReviseDemo.cs
var heading = new XFont(BundledFontResolver.SansFamily, 16, XFontStyle.Bold);
var label = new XFont(BundledFontResolver.SansFamily, 9.5, XFontStyle.Bold);
var body = new XFont(BundledFontResolver.SansFamily, 9);
var mono = new XFont(BundledFontResolver.MonoFamily, 7.5);
var stamp = new XFont(BundledFontResolver.SansFamily, 11, XFontStyle.Bold);

// ----- revision one: an ordinary document, saved the ordinary way --------------------------

using var original = new PdfDocument();
original.Info.Title = "Revise";
original.Info.Author = "PdfPinata sample app";

var first = original.AddPage();
using (var gfx = XGraphics.FromPdfPage(first))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("Revision one", heading, XBrushes.Black, 50, 60);

    prose.DrawString(
        "This page and the next were written by an ordinary Save. Nothing about them knows "
        + "anything is going to be added, and nothing about them has to: an incremental "
        + "update leaves the bytes it was given exactly as they were and writes after them.",
        body, XBrushes.Black, new XRect(50, 80, 495, 44));

    prose.DrawString(
        "A reader opening the finished file starts at the last startxref, follows /Prev "
        + "backwards through the cross-reference sections, and takes the first definition it "
        + "finds of each object - so a later revision shadows an earlier one without "
        + "erasing it. Everything on this page is still the first definition of itself; "
        + "nothing later contradicts it.",
        body, XBrushes.Black, new XRect(50, 136, 495, 62));

    gfx.DrawString("What an incremental update is for", label, XBrushes.Black, 50, 220);

    prose.DrawString(
        "Three things, and file size is not one of them. A signed document can only be "
        + "changed this way, because a signature covers a byte range of the file and "
        + "rewriting the file invalidates it. An audited document keeps every earlier state "
        + "recoverable. And a very large document can be annotated without being written out "
        + "again from end to end.",
        body, XBrushes.Black, new XRect(50, 235, 495, 62));

    gfx.DrawString("What it costs", label, XBrushes.Black, 50, 320);

    prose.DrawString(
        "The file only ever grows, and it grows by more than the change: every object "
        + "touched is written again in full, and so is a whole cross-reference section. A "
        + "document revised a hundred times carries a hundred copies of whatever kept "
        + "changing. Rewriting it with Save is how that is reclaimed, and is exactly what "
        + "must not be done to a signed one.",
        body, XBrushes.Black, new XRect(50, 335, 495, 62));
}

var second = original.AddPage();
using (var gfx = XGraphics.FromPdfPage(second))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("Append is a mode, not a flag", heading, XBrushes.Black, 50, 60);

    prose.DrawString(
        "PdfDocumentOpenMode.Append is the only mode a document can be appended to in, and "
        + "the reason is object numbers. Modify reads everything into memory and renumbers "
        + "it, so an appended definition of object 12 would shadow whatever happened to be "
        + "numbered 12 this time round - which is not what it was numbered before. Import "
        + "does not keep the bytes at all. Append keeps both, and SaveIncremental refuses "
        + "without them.",
        body, XBrushes.Black, new XRect(50, 80, 495, 62));

    gfx.DrawString("Only what changed is written", label, XBrushes.Black, 50, 165);

    prose.DrawString(
        "Which means something has to know what changed. An object modified through the "
        + "object model marks itself; a direct one - an array held inside a page dictionary "
        + "rather than indirectly in its own right - cannot, because changing it changes the "
        + "page and not the array. That is what PdfObject.MarkAsChanged is for, and "
        + "forgetting it on the page is how an appended annotation ends up in a file no "
        + "reader shows it in.",
        body, XBrushes.Black, new XRect(50, 180, 495, 76));

    gfx.DrawString("The trap worth knowing before you hit it", label, XBrushes.Firebrick, 50, 275);

    prose.DrawString(
        "SaveIncremental writes the whole file - original bytes and all - so the destination "
        + "has to be empty. Handing it the file it was read from is the tempting mistake and "
        + "the damaging one: the original is rewritten over itself, the revision is "
        + "appended, and because nothing truncates, whatever of the old file ran past the "
        + "new end survives. That includes its startxref, which a reader scanning backwards "
        + "finds first - and the appended revision is then ignored in silence, signature and "
        + "all. So it throws on a non-empty stream rather than letting that happen.",
        body, XBrushes.Black, new XRect(50, 290, 495, 90));
}

var revisionOne = new MemoryStream();
original.Save(revisionOne, false);
var sizeOfOne = revisionOne.Length;

// ----- revision two: opened for appending, a page added ------------------------------------

revisionOne.Position = 0;
using var appended = PdfReader.Open(revisionOne, PdfDocumentOpenMode.Append);

// Changing something that was already there, as well as adding. The title is in the
// information dictionary, which is an object like any other: the appended revision carries a
// second definition of it and the reader takes that one.
appended.Info.Subject = "Amended by revision two";

var third = appended.AddPage();
using (var gfx = XGraphics.FromPdfPage(third))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("Revision two", heading, XBrushes.Black, 50, 60);

    gfx.DrawString("Added by appending, not by rewriting", stamp, XBrushes.Firebrick, 50, 84);

    prose.DrawString(
        "This page did not exist when the bytes above it were written. The document was "
        + "opened again with PdfDocumentOpenMode.Append, this page was added, the subject "
        + "line in the information dictionary was changed, and SaveIncremental wrote the "
        + "original bytes through untouched with a new cross-reference section after them.",
        body, XBrushes.Black, new XRect(50, 104, 495, 62));

    (string What, string Value)[] facts =
    {
        ("Revision one", Format(sizeOfOne) + " bytes"),
        // Counted after the page above was added, so this is both revisions' objects and
        // not revision one's. Named for what it counts rather than for the row above it.
        ("Objects reachable now", appended.Internals.GetAllObjects().Length
            .ToString(CultureInfo.InvariantCulture) + " across both revisions"),
        ("Pages before this one", "2"),
        ("Changed as well as added", "/Info /Subject, which revision one had left empty")
    };

    double y = 190;
    foreach (var fact in facts)
    {
        gfx.DrawString(fact.What, label, XBrushes.Black, 50, y);
        gfx.DrawString(fact.Value, mono, XBrushes.Black, 200, y);
        y += 16;
    }

    gfx.DrawString("More changed than the page", label, XBrushes.Black, 50, y + 20);

    prose.DrawString(
        "Adding a page is not only a new page object. The /Pages node that lists them has to "
        + "say so, and its /Count has to agree, so the appended revision carries a second "
        + "definition of the page tree node as well. The larger cost is the fonts: a "
        + "document opened for appending does not adopt the font objects already in the "
        + "file, so a page drawn in a face the first revision also used embeds that face "
        + "again. Compare the two sizes on the next page - the revision is far larger than "
        + "anything visible on this one, and almost all of it is a second copy of the type.",
        body, XBrushes.Black, new XRect(50, y + 34, 495, 76));
}

var revisionTwo = new MemoryStream();
appended.SaveIncremental(revisionTwo);
var afterTwo = revisionTwo.ToArray();

// ----- revision three: opened again, and this is the one written to the file ----------------

revisionTwo.Position = 0;
var document = PdfReader.Open(revisionTwo, PdfDocumentOpenMode.Append);

var fourth = document.AddPage();
using (var gfx = XGraphics.FromPdfPage(fourth))
{
    var prose = new XTextFormatter(gfx);

    gfx.DrawString("Revision three", heading, XBrushes.Black, 50, 60);

    prose.DrawString(
        "And this one was added to the file the last one produced. Below is what a byte "
        + "scanner finds in those bytes - counted, not described, by looking through the "
        + "file revision two wrote for the markers a reader uses to walk it.",
        body, XBrushes.Black, new XRect(50, 80, 495, 44));

    (string What, string Value)[] found =
    {
        ("Revision one", Format(sizeOfOne) + " bytes"),
        ("After revision two", Format(afterTwo.Length) + " bytes"),
        ("Appended by revision two", Format(afterTwo.Length - sizeOfOne) + " bytes"),
        ("startxref, at a line start", CountAtLineStart(afterTwo, "startxref")
            .ToString(CultureInfo.InvariantCulture) + " - one per revision"),
        ("%%EOF, at a line start", CountAtLineStart(afterTwo, "%%EOF")
            .ToString(CultureInfo.InvariantCulture) + " - one per revision"),
        ("/Prev, anywhere in the bytes", Count(afterTwo, "/Prev")
            .ToString(CultureInfo.InvariantCulture) + " - one fewer, and rightly so"),
        ("This file", "one revision deeper again")
    };

    double y = 145;
    foreach (var fact in found)
    {
        gfx.DrawString(fact.What, label, XBrushes.Black, 50, y);
        gfx.DrawString(fact.Value, mono, XBrushes.Black, 200, y);
        y += 16;
    }

    gfx.DrawString("Why /Prev is one short, and why two rows say where", label,
        XBrushes.Firebrick, 50, y + 22);

    prose.DrawString(
        "Two revisions have two startxrefs and one /Prev, not two. Every revision writes a "
        + "cross-reference section; every section but the first points at the one before it, "
        + "and the first has nothing behind it to point at. So the chain has one fewer link "
        + "than it has sections, and a file with a /Prev per revision would be one with a "
        + "link into nothing.",
        body, XBrushes.Black, new XRect(50, y + 36, 495, 62));

    prose.DrawString(
        "The other caveat is how these were counted. A byte scan cannot tell a marker in a "
        + "trailer from the same characters inside a compressed stream or an embedded font, "
        + "and this file carries both - so the first two rows count the marker only where it "
        + "begins a line, which is where a trailer puts it and where a stream almost never "
        + "does. The third is a plain count and is worth exactly that much. Structure is what "
        + "a reader parses for; this page is looking at the file rather than reading it.",
        body, XBrushes.Black, new XRect(50, y + 104, 495, 76));

    gfx.DrawString("Reading it back", label, XBrushes.Black, 50, y + 192);

    prose.DrawString(
        "Nothing special is needed. A reader that understands PDF at all understands an "
        + "incrementally updated file, because following /Prev is how cross-reference "
        + "sections have always been chained - a linearised file has more than one section "
        + "too. Open the finished PDF in anything and it is a four page document; the three "
        + "revisions are visible only to something looking at the bytes.",
        body, XBrushes.Black, new XRect(50, y + 206, 495, 62));

    gfx.DrawString("What an earlier revision still holds", label, XBrushes.Black, 50, y + 282);

    prose.DrawString(
        "Everything it ever said. Redacting a document by drawing a black rectangle over a "
        + "name and appending the change leaves the name in the file, in plain text, one "
        + "revision back - and tools that recover it are not sophisticated. Redaction means "
        + "rewriting, which means Save, which means any signature goes with it. That "
        + "tension is real and has no trick to it: the two features want opposite things.",
        body, XBrushes.Black, new XRect(50, y + 296, 495, 76));

    gfx.DrawString("Where this meets signing", label, XBrushes.Black, 50, y + 386);

    prose.DrawString(
        "PdfSigner does exactly what this page does - it appends a revision - and that is "
        + "the whole reason signing a document that was already signed does not destroy the "
        + "first signature. See the Signing demo, whose output is one revision written the "
        + "same way with a hole patched into it.",
        body, XBrushes.Black, new XRect(50, y + 400, 495, 48));
}

Open for appending, change, save​

Why the mode has to be Append​

Save and SaveIncremental compared​

What a revision contains​

Things to know​

See it in action​