Incremental save rollback

Introduction

When a PDF is saved incrementally, changes are appended to the end of the file, and pointers to the original content are updated to reference the new content. An incremental save is usually faster than rewriting the entire file, and the original content remains in the PDF. The only downside is that each incremental save will grow the size of the PDF.

Mako’s IPDFOutput class has long supported incremental save with setEnableIncrementalOutput(), for example:

CPP

IJawsMakoPtr jawsMako = IJawsMako::create();
...
const IPDFOutputPtr pdfOutput = IPDFOutput::create(jawsMako);
pdfOutput->setEnableIncrementalOutput(true);

C#

using var jawsMako = IJawsMako.create();
...
var pdfOutput = IPDFOutput.create(jawsMako);
pdfOutput.setEnableIncrementalOutput(true);

Three new APIs

Mako 7.3.0 introduces three new IPDFInput APIs:

API	Purpose
`IPDFInput::getNumIncrementalSaves()`	Returns the number of incremental saves found in a given PDF input stream.
`IPDFInput::openIncremental()`	Opens an incremental PDF stream, returning the `IDocumentAssembly` representing the contents.
`IPDFInput::getIncrementalSaves()`	Returns a list of `IRAInputStream`objects representing each incremental save found in a given PDF input stream.

Each API function has variants that will take an IInputStream, U8String or String
Incremental saves are added in order of the last save. For example, to obtain the PDF as it was before the last save, use:
- openIncremental(filename.pdf, 0)
- or use the first element in the array returned by getIncrementalSaves()

The new APIs will work with incrementally-saved PDFs from any other producer, not just Mako
There is no metadata associated with an incremental save
- If there is a requirement to store information about an incremental save, for example the date and time, it must be added as custom metadata

Code sample

This sample generates PDFs, each with an increasing number of incremental saves, and also a single PDF with multiple streams each representing an incremental save.

It then shows how to interrogate the available version of the PDF, allowing the choice of version to be saved as a separate PDF.

In addition, it shows how to add custom metadata that can be used to further identify a specific version.

Finally, the ILayout class is used to generate the PDF content. A ZIP file is attached with the test files referenced by this code.

Click here to expand...

CPP

/* -----------------------------------------------------------------------
 * <copyright file="main.cpp" company="Global Graphics Software Ltd">
 *  Copyright (c) 2022-2024 Global Graphics Software Ltd. All rights reserved.
 * </copyright>
 * <summary>
 *  This example is provided on an "as is" basis and without warranty of any kind.
 *  Global Graphics Software Ltd. does not warrant or make any representations
 *  regarding the use or results of use of this example.
 * </summary>
 * -----------------------------------------------------------------------
 */

#include <iostream>
#include <Windows.h>
#include <conio.h>
#include <format>
#include <filesystem>
using namespace std;
namespace fs = std::filesystem;

#include <jawsmako/jawsmako.h>
#include <jawsmako/layout.h>
#include <jawsmako/pdfinput.h>
#include <jawsmako/pdfoutput.h>

using namespace JawsMako;
using namespace EDL;

#define M2X(value) ((value) / 25.4 * 96.0)
#define P2X(value) ((value) / 72.0 * 96.0)

// Forward declarations
IDOMImagePtr getImage(const IJawsMakoPtr& mako, const U8String& imageFile);
IPagePtr getPageContent(const IJawsMakoPtr& mako, uint32 pageIndex);
void addMetadata(const IJawsMakoPtr& mako, const IDocumentPtr& document);
U8String getMetadata(const IDocumentPtr& document);
void clearMetadata(const IDocumentPtr& document);

int main()
{
    try
    {
        const auto mako = IJawsMako::create();
        mako->enableAllFeatures(mako);

        // New assembly
        auto assembly = IDocumentAssembly::create(mako);
        auto document = IDocument::create(mako);
        assembly->appendDocument(document);

        // Input. We need this to reload the assembly after an incremental save
        const auto pdfInput = IPDFInput::create(mako);

        // Output
        const auto pdfOutput = IPDFOutput::create(mako);
        pdfOutput->setEnableIncrementalOutput(true);

        /****************************************************************************
         *
         * Method 1: Write multiple files to disk
         *
         ****************************************************************************
         */

        std::wcout << L"Method 1: Multiple files" << std::endl;

        // Timer
        clock_t begin = clock();

        // Create a 10-page document, incrementally saving after each page is added
        for (uint32 pageIndex = 0; pageIndex < 20; pageIndex++)
        {
            document->appendPage(getPageContent(mako, pageIndex));

            // Add a date stamp
            addMetadata(mako, document);

            // Write the PDF
            U8String outputFile = std::format(R"(TestPDF_{:03}.pdf)", pageIndex + 1).c_str();
            pdfOutput->writeAssembly(assembly, outputFile);
            
            // Reopen: output becomes input
            assembly = pdfInput->open(outputFile);
            document = assembly->getDocument();
        }

        clock_t end = clock();
        clock_t elapsed_msecs = (end - begin);
        if (elapsed_msecs < 1000L)
            std::wcout << L"-- Elapsed time: less than a second." << std::endl;
        else
            std::wcout << L"-- Elapsed time: " << elapsed_msecs / 1000.0 << L" seconds." << std::endl;

        system("pause");

        /****************************************************************************
         *
         * Method 2: Use a shared stream
         *
         ****************************************************************************
         */

        std::wcout << L"Method 2: Shared stream" << std::endl;

        // New assembly
        assembly = IDocumentAssembly::create(mako);
        document = IDocument::create(mako);
        assembly->appendDocument(document);

        // Timer
        begin = clock();

        // Create a 10-page document, incrementally saving after each page is added
        IRAInputStreamPtr reader;
        IRAOutputStreamPtr writer;
        for (uint32 pageIndex = 0; pageIndex < 20; pageIndex++)
        {
            // ReaderWriter we'll use to test this.
            mako->getTempStore()->createTemporaryReaderWriterPair(reader, writer);

            // Add content
            document->appendPage(getPageContent(mako, pageIndex));

            // Add a date stamp
            addMetadata(mako, document);

            // Write the PDF
            pdfOutput->writeAssembly(assembly, writer);

            // Reopen: output becomes input
            assembly = pdfInput->open(reader);
            document = assembly->getDocument();
        }

        // Now write the final PDF to disk
        IOutputStream::copy(reader, IOutputStream::createToFile(mako, "TestPDF.pdf"));

        end = clock();
        elapsed_msecs = (end - begin);
        if (elapsed_msecs < 1000L)
            std::wcout << L"-- Elapsed time: less than a second." << std::endl;
        else
            std::wcout << L"-- Elapsed time: " << elapsed_msecs / 1000.0 << L" seconds." << std::endl;

        system("pause");

        /****************************************************************************
         *
         * Now look at the created doc
         *
         ****************************************************************************
         */

        const auto inputFilePath = "TestPDF.pdf";
        std::wcout << L"Processing file " << inputFilePath << L"..." << std::endl;

        const auto numSaves = pdfInput->getNumIncrementalSaves(inputFilePath);
        std::cout << "This file has " << numSaves << " incremental saves." << std::endl;

        // Get all of them
        const CIRAInputStreamVect incrementalSaves = pdfInput->getIncrementalSaves(inputFilePath);

        // Display their metadata, if present
        for (uint32 i = 0; i < numSaves; i++)
        {
            const auto isDocument = pdfInput->open(incrementalSaves[i])->getDocument();
            std::cout << "Increment " << i + 1 << " - Date stamp: " << getMetadata(isDocument);
        }
        
        if (numSaves > 1)
        {
            // Which incremental save?
            std::cout << "\nChoose one to save as a separate PDF: ";
            uint32 saveIndex;
            std::wcin >> saveIndex;
            saveIndex--;
            if (saveIndex < 1)
                saveIndex = 0;
            if (saveIndex > numSaves - 1)
                saveIndex = numSaves - 1;

            // Save the stream as is, which will be quick
            IOutputStream::copy(incrementalSaves[saveIndex], IOutputStream::createToFile(mako, "TestOut1.pdf"));

            // Reload as an assembly and rewrite, allowing the incremental saves to be removed.
            const auto inAssembly = pdfInput->openIncremental(inputFilePath, saveIndex);
            clearMetadata(inAssembly->getDocument());
            pdfOutput->setEnableIncrementalOutput(false);
            pdfOutput->writeAssembly(inAssembly, "TestOut2.pdf");
        }
    }
    catch (IError& e)
    {
        const String errorFormatString = getEDLErrorString(e.getErrorCode());
        std::wcerr << L"Exception thrown: " << e.getErrorDescription(errorFormatString) << std::endl;
        return static_cast<int>(e.getErrorCode());
    }
    catch (std::exception& e)
    {
        std::wcerr << L"std::exception thrown: " << e.what() << std::endl;
        return 1;
    }

    return 0;
}

// Get image 
IDOMImagePtr getImage(const IJawsMakoPtr& mako, const U8String& imageFile)
{
    IDOMImagePtr image = IDOMImagePtr();
    const auto imagePath = fs::path(imageFile);
    if (!exists(imagePath))
    {
        std::string msg = "Image file ";
        msg += imageFile;
        msg += " not found.";
        throw std::exception(msg.c_str());
    }

    if (imagePath.extension() == ".jpg")
        image = IDOMJPEGImage::create(mako, IInputStream::createFromFile(mako, imageFile));

    if (imagePath.extension() == ".png")
        image = IDOMPNGImage::create(mako, IInputStream::createFromFile(mako, imageFile));

    if (imagePath.extension() == ".tif")
        image = IDOMTIFFImage::create(mako, IInputStream::createFromFile(mako, imageFile));

    if (!image)
    {
        std::string msg = "Image file ";
        msg += imageFile;
        msg += " could not be loaded.";
        throw std::exception(msg.c_str());
    }

    return image;
}

IPagePtr getPageContent(const IJawsMakoPtr& mako, uint32 pageIndex)
{
    // Get a font
    uint32 fontIndex;
    const auto font = edlobj2IDOMFontOpenType(mako->findFont("Arial", fontIndex));

    // Create a colour
    const auto darkBlue = IDOMColor::createSolidRgb(mako, 0.0f, 0.0f, 0.5f);

    // Create template paragraphs
    const auto header = ILayoutParagraph::create(ILayoutParagraph::eHALeft, P2X(10));
    const auto body = ILayoutParagraph::create(ILayoutParagraph::eHAJustified, P2X(7));

    // Create a page
    const auto page = IPage::create(mako);

    const auto fixedPage = IDOMFixedPage::create(mako);
    page->setContent(fixedPage);

    // A vector to point to each paragraph to be added to the frame(s)
    auto paragraphs = CEDLVector<ILayoutParagraphPtr>();
    uint32 paraIndex = 0;

    // Create a layout
    const auto layout = ILayout::create(mako);

    // Add frame to hold content
    layout->addFrame(ILayoutFrame::create(FRect(M2X(12), M2X(12), M2X(186), M2X(273))));

    // Create paragraphs and text runs 
    const std::string text = std::format("This is page {}", pageIndex + 1);
    auto run = ILayoutTextRun::create(text.c_str(), font, fontIndex, P2X(72), darkBlue);
    paragraphs.append(header->clone());
    paragraphs[paraIndex]->addRun(run);

    // Load some text
    std::ifstream inputFile(R"(..\..\TestFiles\text.txt)");
    if (inputFile.is_open())
    {
        const std::string fileContents((std::istreambuf_iterator<char>(inputFile)),
                                       std::istreambuf_iterator<char>());
        run = ILayoutTextRun::create(fileContents.c_str(), font, fontIndex, P2X(14));
        inputFile.close();
    }

    paragraphs.append(body->clone());
    paragraphs[++paraIndex]->addRun(run);

    // Picture
    const auto pic = getImage(mako, std::format(R"(..\..\TestFiles\Image_{:03}.jpg)", pageIndex % 5 + 1).c_str());
    paragraphs.append(ILayoutParagraph::create());
    paragraphs[++paraIndex]->addRun(ILayoutImageRun::create(mako, pic, M2X(185)));

    // Add content to the page
    fixedPage->appendChild(layout->layout(paragraphs));

    return page;
}

// Add metadata entry to help identify a given version of the PDF
void addMetadata(const IJawsMakoPtr& mako, const IDocumentPtr& document)
{
    // Get the time
    const std::string s1 = std::format("{:%F %T}", std::chrono::system_clock::now());
    const std::string s2 = std::format("{:%F %T %Z}", std::chrono::zoned_time{
                                           std::chrono::current_zone(), std::chrono::system_clock::now()
                                           });

    // Get the document store
    const IPDFObjectStorePtr documentStore = document->getObjectStore();

    // Create a dictionary
    const auto dictionary = IPDFDictionary::create(mako);
    dictionary->put("SmartSaveDate", IPDFString::create(s2.c_str()));

    // Put that in the catalog
    const auto catalog = pdfObj2IPDFDictionary(documentStore->getRootObject());

    // Store the reference to our new dictionary under the key "SmartSave"
    catalog->put("SmartSave", dictionary);

    // And the catalog is now dirty, so we need to note this for an incremental update.
    documentStore->dirtyRootObject();
}

U8String getMetadata(const IDocumentPtr& document)
{
    // Open the document store
    const IPDFObjectStorePtr documentStore = document->getObjectStore();

    // Fetch the catalog, which is the root of this store
    const auto catalog = pdfObj2IPDFDictionary(documentStore->getRootObject());

    // Find a dictionary entry 
    const IPDFObjectPtr object = catalog->get("SmartSave");
    if (!object)
        return "No entry found";

    // Which we expect is a PDF dictionary
    const auto customDictionary = pdfObj2IPDFDictionary(object);

    U8String result;
    for (IPDFDictionary::Iterator pos = customDictionary->begin(); pos != customDictionary->end(); ++pos)
    {
        if (pos.getValue()->getType() == ePOTString)
        {
            result += pdfObj2IPDFString(pos.getValue())->getValue().c_str();
            result += '\n';
        }
    }

    return result;
}

void clearMetadata(const IDocumentPtr& document)
{
    // Get the document store
    const IPDFObjectStorePtr documentStore = document->getObjectStore();

    // Put that in the catalog
    const auto catalog = pdfObj2IPDFDictionary(documentStore->getRootObject());

    // Look for the custom dictionary
    const IPDFObjectPtr object = catalog->get("SmartSave");
    if (!object)
        return;

    // Remove the reference to the key "SmartSave"
    const auto smartSaveDictionary = pdfObj2IPDFDictionary(object);
    if (smartSaveDictionary)
    {
        catalog->undefine("SmartSave");

        // And the catalog is now dirty
        documentStore->dirtyRootObject();
    }
}

TestFiles.zip