...master 2026-11-12 (1037e2069+unstaged)

The MuPDF C API

Quick Start

How to open a document and render some pages

For a simple example of how to open a document and render some pages, see docs/example.c.

The concepts you meet in this example are explained and expanded upon in the following chapters. It may be useful to have this example to hand as you read on, to give you concrete illustrations of the ideas discussed.

Naming Conventions

The function and variable names within MuPDF have been carefully chosen to follow standard conventions. By consistently using the same terms, we hope that it will become easy to remember things like the reference counting behaviour of functions, and minimise the need to look things up in the documentation.

We require any code submitted to MuPDF to follow the same conventions, and would encourage people to follow the same style in their own code that interfaces with MuPDF.

Prefixes

Historically, the graphics library upon which MuPDF relies was known as ‘fitz’. Accordingly, all MuPDF’s API calls and most types start with ‘fz_’.

The exception to this is where we have APIs and types provided by particular format handlers (e.g. PDF or XPS). These functions/types are prefixed with the format handlers name, for example ‘pdf_’ or ‘xps_’.

All exported functions and types should be prefixed in this way to avoid the possibility of clashing with symbols in calling applications. Internal functions are not required to be prefixed, but we encourage it for clarity.

Naming

All functions are named according to one of the following schemes:

The sole exceptions to this are where we have MuPDF specific functions that emulate (or extend) ‘well known’ functions, where we parrot those functions. For example ‘fz_printf’ or ‘fz_strdup’.

In addition, we avoid using ‘get’ in function names as this is generally redundant. In contrast, however, we do use ‘set’ where required. Consider for instance fz_aa_level (to retrieve the current anti-aliasing level), and fz_set_aa_level (to update it).

MuPDF makes extensive use of reference counting (see for more details). We reserve various words to indicate that reference counting is being used:

All of these calls return an object reference. It is the callers responsibility to store this reference safely somewhere for the duration of the required lifespan of the object, and to destroy that reference when the caller no longer requires it.

No function should return ownership of a reference without being named with one of the reserved words above.

Once all the references to a given object are released, the system will remove the object itself.

Failure to drop references will result in memory leaks. Dropping references too early may result in crashes due to objects being accessed after they have been destroyed.

API functions to destroy objects that are not subject to reference counted can be destroyed by calling functions using the words ‘drop’, ‘close’ or ‘free’.

In contrast to ‘find’ described above, we have one other reserved word regarding searching:

When we have a structure already allocated, and we wish to initialise some or all of its internal details, we use ‘init’. The matching pair to this should be named ’fin’. For example, fz_cmm_init_profile is matched by fz_cmm_fin_profile.

Some objects are created using functions using the verb ‘create’. Sometimes these can be reference counted objects (e.g. ‘pdf_create_document’), in which case they should be ‘drop’ped as usual. Non reference counted objects should be ’``destroy`ed`’ (e.g. fz_destroy_mutex.

Types

Various different integer types are used throughout MuPDF.

In general:

Previously MuPDF used a fz_off_t type for file offsets. This changed size according to build type or FZ_LARGEFILE being defined, but this has been abandoned due to potential pitfalls. Having the API change between builds according to different symbol definitions is poor form as it can lead to unexpected crashes.

In addition, we use floats (and doubles internally, though we avoid these in the API where possible). These are assumed to be IEEE compliant.

The Context

Overview

The core MuPDF library is designed for simplicity, portability, and ease of integration. For all these reasons, it has no global variables, has no thread library dependencies, and has a well defined exception system to handle runtime errors. Nonetheless, in order to be as useful as possible, clearly the library must have some state and needs to be able to take advantage of multi-threaded environments.

The solution to these seemingly conflicting requirements is the Context (fz_context).

Every caller to MuPDF should create a Context at the start of its use of the library, and destroy it at the end. This Context (or one ‘cloned’ from it) will then be passed in to every MuPDF API call.

Global State

At its simplest, the Context contains global settings for the library. For instance, the default levels of anti-aliasing used by the text and line art rendering routines are set in the Context, as is the default style sheet for EPUB or FB2 files. In addition, the library stores its own private information there too.

Error handling

All error handling within MuPDF is done using the fz_try/fz_catch constructs; see for more details.
These constructs can be nested, so rely on an exception stack maintained within the context. As such it is vitally important that no two threads use the same context at the same time. See for more information.

Allocation

When embedding MuPDF into a system it is often desirable to control the allocators used. A set of allocator functions can be provided to the Context at creation time, and all allocations will be performed using these. See for more information.

The Store

MuPDF uses a memory cache to aid performance, and to avoid repeated decoding of resources from the file. The store is maintained using the context, and shared between a context and its clones. See for more information.

Multi-threading

MuPDF does not rely on threading itself, but it can be used in a multi-threaded environment to give significant performance improvements. Any thread library can be used with MuPDF. A set of locking/unlocking functions must be passed to the context at creation time, and the library will use these to ensure it is thread safe. See for more information.

Creation

To create a context, use fz_new_context:

/*
        fz_new_context: Allocate context containing global state.

        The global state contains an exception stack, resource store,
        etc. Most functions in MuPDF take a context argument to be
        able to reference the global state. See fz_drop_context for
        freeing an allocated context.

        alloc: Supply a custom memory allocator through a set of
        function pointers. Set to NULL for the standard library
        allocator. The context will keep the allocator pointer, so the
        data it points to must not be modified or freed during the
        lifetime of the context.

        locks: Supply a set of locks and functions to lock/unlock
        them, intended for multi-threaded applications. Set to NULL
        when using MuPDF in a single-threaded applications. The
        context will keep the locks pointer, so the data it points to
        must not be modified or freed during the lifetime of the
        context.

        max_store: Maximum size in bytes of the resource store, before
        it will start evicting cached resources such as fonts and
        images. FZ_STORE_UNLIMITED can be used if a hard limit is not
        desired. Use FZ_STORE_DEFAULT to get a reasonable size.

        Does not throw exceptions, but may return NULL.
*/
fz_context *fz_new_context(const fz_alloc_context *alloc, const fz_locks_context *locks, unsigned int max_store);

For example, a simple, single threaded program using the standard allocator can just use:

fz_context *ctx = fz_new_context(NULL, NULL, FZ_STORE_UNLIMITED);

Custom Allocators

In some circumstances it can be desirable to force all allocations through a set of ‘custom’ allocators. These are defined as a fz_alloc_context structure whose address is passed in to fz_new_context. This structure must exist for the life time of the returned fz_context (and any clones).

typedef struct
{
        void *user;
        void *(*malloc)(void *, size_t);
        void *(*realloc)(void *, void *, size_t);
        void (*free)(void *, void *);
} fz_alloc_context;

The malloc, realloc and free function pointers have essentially the same semantics as the standard malloc, realloc and free standard functions, with the exception that they take an additional initial argument - that of the user value specified in the fz_alloc_context.

Multi-threading

MuPDF itself does not rely on a thread system, but it will make use of one if one is present. This is crucial to ensure that MuPDF can be called from multiple threads at once.

A typical example of this might be in a multi-core processor on a printer. We can interpret the PDF file to a display list, and then render ‘bands’ from that display list to send to the printer. By using multiple threads we can render multiple bands at once, thus vastly improving processing times.

In this example, although each thread will be rendering different things, they will probably share some information - for instance the same font is likely to be used in multiple bands. Rather than have every thread render all the glyphs that it needs from the font independently, it would be nice if they could collaborate and share results.

We therefore arrange that data structures such as the font cache can be shared between the different threads. This, however, brings dangers; what if two threads try to write to the same data structure at once?

To save this being a problem, we rely on the user providing some locking functions for us.

/*
        Locking functions

        MuPDF is kept deliberately free of any knowledge of particular
        threading systems. As such, in order for safe multi-threaded
        operation, we rely on callbacks to client provided functions.

        A client is expected to provide FZ_LOCK_MAX number of mutexes,
        and a function to lock/unlock each of them. These may be
        recursive mutexes, but do not have to be.

        If a client does not intend to use multiple threads, then it
        may pass NULL instead of a lock structure.

        In order to avoid deadlocks, we have one simple rule
        internally as to how we use locks: We can never take lock n
        when we already hold any lock i, where 0 <= i <= n. In order
        to verify this, we have some debugging code, that can be
        enabled by defining FITZ_DEBUG_LOCKING.
*/

typedef struct
{
        void *user;
        void (*lock)(void *user, int lock);
        void (*unlock)(void *user, int lock);
} fz_locks_context;

enum {
    ...
        FZ_LOCK_MAX
};

If MuPDF is to be used in a multi-threaded environment, then the user is expected to define FZ_LOCK_MAX locks (currently 4, though this may change in future), together with functions to lock and unlock them.

In pthreads, a lock might be implemented by pthread_mutex_t. In windows, either Mutex or a CriticalSection might be used (the latter being more lightweight).

These locks are not assumed to be recursive (though recursive locks will work just fine).

To avoid deadlocks, MuPDF guarantees never to take lock n if that thread already holds lock m (for n>mn > m).

There are 3 simple rules to follow when using MuPDF in a multi-threaded environment:

  1. No simultaneous calls to MuPDF in different threads are allowed to use the same context.

    Most of time it is simplest just to use a different context for every thread; just create a new context at the same time as you create the thread. See for more information.

  2. No simultaneous calls to MuPDF in different threads are allowed to use the same document.

    Only one thread can be accessing an document at a time. Once display lists are created from that document, multiple threads can operate on them safely.

    The document can safely be used from several different threads as long as there are safeguards in place to prevent the usages being simultaneous.

  3. No simultaneous calls to MuPDF in different threads are allowed to use the same device.

    Calling a device simultaneously from different threads will cause it to get confused and may crash. Calling a device from several different threads is perfectly acceptable as long as there are safeguards in place to prevent the calls being simultaneous.

Cloning

The context contains the exception stack for the fz_try/fz_catch constructs. As such trying to use the same context from multiple threads at the same time will lead to crashes.

The solution to this is to ‘clone’ the context. Each clone will share the same underlying store (and will inherit the same settings, such as allocators, locks etc), but will have its own exception stack. Other settings, such as anti-alias levels, will be inherited from the original at the time of cloning, but can be changed to be different if required.

For example, in a viewer application, we might want to have a background process that runs through the file generating page thumbnails. In order for this not to interfere with the foreground process, we would clone the context, and use the cloned context in the thumbnailing thread. We might choose to disable anti-aliasing for the thumbnailing thread to trade quality for speed.

Any images decoded for the thumbnailing thread would live on in the store though, and would hence be available should the viewers normal render operations need them.

To clone a context, use fz_clone_context:

/*
        fz_clone_context: Make a clone of an existing context.

        This function is meant to be used in multi-threaded
        applications where each thread requires its own context, yet
        parts of the global state, for example caching, are shared.

        ctx: Context obtained from fz_new_context to make a copy of.
        ctx must have had locks and lock/functions setup when created.
        The two contexts will share the memory allocator, resource
        store, locks and lock/unlock functions. They will each have
        their own exception stacks though.

        Does not throw exception, but may return NULL.
*/
fz_context *fz_clone_context(fz_context *ctx);

For example:

fz_context *worker_ctx = fz_clone_context(ctx);

In order for cloned contexts to work safely, they rely on being able to take locks around certain operations to make them atomic. Accordingly, fz_clone_context will return NULL (to indicate failure) if the base context did not have locking functions defined.

Destruction

Once you have finished with a fz_context (either your original one, or a ‘cloned’ one) you can destroy it using fz_drop_context.

/*
        fz_drop_context: Free a context and its global state.

        The context and all of its global state is freed, and any
        buffered warnings are flushed (see fz_flush_warnings). If NULL
        is passed in nothing will happen.

        Does not throw exceptions.
*/
void fz_drop_context(fz_context *ctx);

For example:

fz_drop_context(ctx);

Tuning

Some of MuPDF’s functionality relies on heuristics to make decisions. Rather than hard code these decisions in the library code, the tuning context allows callers to override the defaults with their own ‘tuned’ versions.

Currently, we have just 2 calls defined here, both to do with image handling, but this may expand in future.

The first tuning function enables fine control over how much of an image MuPDF should decode if it only requires a subarea:

/*
        fz_tune_image_decode_fn: Given the width and height of an image,
        the subsample factor, and the subarea of the image actually
        required, the caller can decide whether to decode the whole image
        or just a subarea.

        arg: The caller supplied opaque argument.

        w, h: The width/height of the complete image.

        l2factor: The log2 factor for subsampling (i.e. image will be
        decoded to (w>>l2factor, h>>l2factor)).

        subarea: The actual subarea required for the current operation.
        The tuning function is allowed to increase this in size if required.
*/
typedef void (fz_tune_image_decode_fn)(void *arg, int w, int h, int l2factor, fz_irect *subarea);

The purpose of allowing larger areas to be decoded than are immediately required, is so that these larger areas can be placed into the cache. This may mean that future requests can be satisfied from the cache rather than requiring complete new decodes. An example of such a situation might be where MuPDF is powering a viewer application, and a page is slowly panned onto screen revealing more and more of an image. These tuning functions put control over such decisions back into the hands of the application author.

Having defined a function of this type to implement the desired strategy, it can be set into the context using:

/*
        fz_tune_image_decode: Set the tuning function to use for
        image decode.

        image_decode: Function to use.

        arg: Opaque argument to be passed to tuning function.
*/
void fz_tune_image_decode(fz_context *ctx, fz_tune_image_decode_fn *image_decode, void *arg);

The second function allows fine control over the scaling used when images are scaled:

/*
        fz_tune_image_scale_fn: Given the source width and height of
        image, together with the actual required width and height,
        decide whether we should use mitchell scaling.

        arg: The caller supplied opaque argument.

        dst_w, dst_h: The actual width/height required on the target device.

        src_w, src_h: The source width/height of the image.

        Return 0 not to use the Mitchell scaler, 1 to use the Mitchell
        scaler. All other values reserved.
*/
typedef int (fz_tune_image_scale_fn)(void *arg, int dst_w, int dst_h, int src_w, int src_h);

Essentially this routine allows the application author to exercise control over whether images are displayed with interpolation or not. Rather than simple linear interpolation, MuPDF uses the ‘Mitchell’ sampling function. This provides subjectively better quality.

The default is to use the Mitchell scaler only when downscaling, to avoid details ‘dropping out’ of images, but by providing a tuning function, the application author can choose to use it in more (or fewer) cases as desired.

Having defined a function of this type to implement the desired strategy, it can be set into the context using:

/*
        fz_tune_image_scale: Set the tuning function to use for
        image scaling.

        image_scale: Function to use.

        arg: Opaque argument to be passed to tuning function.
*/
void fz_tune_image_scale(fz_context *ctx, fz_tune_image_scale_fn *image_scale, void *arg);

Summary

The basic usage of Contexts is as follows:

  1. Call fz_new_context to create a context. Pass in any custom allocators required. If you wish to use MuPDF from multiple threads at the same time, you must also pass in locking functions. Set the store size appropriately.

  2. Call fz_clone_context to clone the context as many times as you need; typically once for each ‘worker’ thread.

  3. Perform the operations required using MuPDF within fz_try/fz_catch constructs.

  4. Call fz_drop_context with each cloned context.

  5. Call fz_drop_context with the original context.

Things to remember:

  1. A fz_context can only be used in 1 thread at a time.

  2. A fz_document can only be used in 1 thread at a time.

  3. A fz_device can only be used in 1 thread at a time.

  4. A fz_context shares the store with all the `fz_context`s cloned from it.

Error handling

Overview

MuPDF handles all its errors using an exception system. This is superficially similar to C++ exceptions, but (as MuPDF is written in C) it is implemented using macros that wrap the setjmp/longjmp standard C functions.

It is probably best not to peek behind the curtain, and just to think of these constructs as being extensions to the language. Indeed, we have worked very hard to ensure that the complexities involved are minimised.

Unless otherwise specified, all MuPDF API functions can throw exceptions, and should therefore be called within a fz_try/fz_always/fz_catch construct.

Specific functions that never throw exceptions include all those named fz_keep_…, fz_drop_… and fz_free. This, coupled with the fact that all such ‘destructor’ functions will silently accept a NULL argument, makes the fz_always block an excellent place to clean up resources used throughout processing.

The general anatomy of such a construct is as follows:

fz_try(ctx)
{
    /* Do stuff in here that might throw an exception.
     * NEVER return from here. 'break' can be used to
     * continue execution (either in the always block or
     * after the catch block). */
}
fz_always(ctx)
{
    /* Anything in here will always be executed, regardless
     * of whether the fz_try clause exited normally, or an
     * exception was thrown. Try to avoid calling functions
     * that can themselves throw exceptions here, or the rest
     * of the fz_always block will be skipped - this is rarely
     * what is wanted! NEVER return from here. 'break' can be
     * used to continue execution in, or after the catch block
     * as appropriate. */
}
fz_catch(ctx)
{
    /* This block will execute if (and only if) anything in
     * the fz_try block calls fz_throw. We should clean up
     * anything we need to. If we are in a nested fz_try/
     * fz/catch block, we can call fz_rethrow to propagate
     * the error to the enclosing catch. Unless the exception
     * is rethrown (or a fresh exception thrown), execution
     * continues after this block. */
}

The fz_always block is completely optional. The following is perfectly valid:

fz_try(ctx)
{
    /* Do stuff here */
}
fz_catch(ctx)
{
    /* Clean up from errors here */
}

In an ideal world, that would be all there is to it. Unfortunately, there are 2 wrinkles.

The first one, relatively simple, is that you must not return from within a fz_try block. To do so will corrupt the exception stack and cause problems and crashes. To mitigate this, you can safely break out of the fz_try, and execution will pass into the fz_always block (if there is one, or continue after the fz_catch block if not).

Similarly, you can break out of a fz_always block, and execution will correctly pass into or after the fz_catch block as appropriate, but this is less useful in practise.

The second one, is more convoluted. If you do not wish to understand the long and complex reasons behind this, skip the following subsection, and just read the corrected example that follows. As long as you follow the rules given in the summary at the end, you will be fine.

Why is fz_var necessary?

As stated before `fz_try/``fz_catch` are implemented using setjmp/longjmp, and these can ‘lose’ changes to variables.

For example:

house_t *build_house(fz_context *ctx)
{
    walls_t  *w = NULL;
    roof_t   *r = NULL;
    house_t  *h = NULL;

    fz_try(ctx)
    {
        w = make_walls();
        r = make_roof();
        h = combine(w, r); /* Note, NOT: return combine(w,r); */
    }
    fz_always(ctx)
    {
        drop_walls(w);
        drop_roof(r);
    }
    fz_catch(ctx)
    {
        /* Handle the error somehow. If we are nested within another
         * layer of fz_try/fz_catch, we can simply fz_rethrow. If
         * not, handle it in a way appropriate for this application,
         * perhaps by simply returning NULL. */
        return NULL;
    }
    return h;
}

In the above code (as well as throughout MuPDF), we follow the convention that destructors always accept NULL. This makes cleanup code much simpler.

Reading through this code, it is fairly obvious what will happen if everything works correctly. First we’ll make some walls, w, and a roof, r. Then we combine the walls and the roof, to get our house, h. As part of this process, the house would take references to the walls and roof as required. Next we tidy up our local references to the walls and the roof, and we return the completed house to our caller.

It’s more interesting to consider what will happen if we have failures.

First let’s consider what happens if the make_walls fails. This will fz_throw an exception, and control will jump immediately to the fz_always. This will drop w and r (both of which are still NULL). The fz_catch can then handle the error, either by returning NULL, to indicate failure, or perhaps by `fz_rethrow`ing the error to an enclosing fz_try/fz_catch construct. No problems there.

So what happens when the failure occurs in make_roof? Let’s run through the code again.

This time, make_walls succeeds, and w is set to this new value. Then make_roof fails, `fz_throw`ing an exception, and control will jump immediately to the fz_always. This will then try to drop w (now a valid value) and r (which is still NULL). The fz_catch can then handle the error, either by returning NULL, to indicate failure, or perhaps by `fz_rethrow`ing the error to an enclosing fz_try/fz_catch construct. All sounds quite plausible.

Unfortunately, if you try it, on some systems you will find that you have a memory leak (or worse). When drop_walls is called, sometimes you will find that w has ‘lost’ its value.

This is due to an obscure part of the C specification that states that any changes to the values of local variables made between a setjmp and a longjmp can be lost. (In fact, the C specification goes further than this, and says that such variables become ‘undefined’).

In fz_try/fz_catch terms, this means that any local variables set within the fz_try block can be ‘lost’ when either fz_always or fz_catch are reached.

Fortunately, there is a fix for this, fz_var. By calling fz_var(w); before the fz_try we can ‘protect’ variable w from such unwanted behaviour.

It’s not really necessary to know how this works, but for those interested, a quick explanation. The ‘loss’ of the value occurs because the compiler can postpone writing the value back into the storage location for the variable (or can choose to just hold it in a register). The call to fz_var passes the address of the variable out of scope; this forces the compiler not to hold it in a register. Further, the compiler has no way of knowing whether any functions it calls might access that location, so it needs to make sure that the variable value is written back on every function call - such as longjmp. Hence the variable is magically protected, and is guaranteed not to lose its value, whether an exception is thrown or not.

Calls to fz_var are very low cost (but are not NOPs), so erring on the side of caution and calling fz_var on more than you need to will probably not hurt.

Example: How to protect local variables with fz_var

A corrected version of the above example is therefore:

house_t *build_house(fz_context *ctx)
{
    walls_t  *w = NULL;
    roof_t   *r = NULL;
    house_t  *h = NULL;

    fz_var(w);
    fz_var(r);

    fz_try(ctx)
    {
        w = make_walls();
        r = make_roof();
        h = combine(w, r); /* Note, NOT: return combine(w,r); */
    }
    fz_always(ctx)
    {
        drop_walls(w);
        drop_roof(r);
    }
    fz_catch(ctx)
    {
        /* Handle the error somehow. If we are nested within another
         * layer of fz_try/fz_catch, we can simply fz_rethrow. If
         * not, handle it in a way appropriate for this application,
         * perhaps by simply returning NULL. */
        return NULL;
    }
    return h;
}

Note the calls to fz_var. These warn the compiler that it should take care not to lose updates to w or r if an exception is thrown in the fz_try. See Rule 5 in below.

Throwing exceptions

Most client code need never worry about anything more than catching exceptions thrown by the core. If you are implementing your own devices or extending the core of MuPDF, then you will need to know how to generate (and pass on) your own exceptions.

An exception is constructed and thrown from an integer code and a printf like string:

enum
{
        FZ_ERROR_NONE = 0,
        FZ_ERROR_MEMORY = 1,
        FZ_ERROR_GENERIC = 2,
        FZ_ERROR_SYNTAX = 3,
        FZ_ERROR_TRYLATER = 4,
        FZ_ERROR_ABORT = 5,
        FZ_ERROR_COUNT
};

void fz_throw(fz_context *ctx, int errcode, const char *, ...);

In almost all cases, you should be using FZ_ERROR_GENERIC, for example:

fz_throw(ctx, FZ_ERROR_GENERIC, "Failed to open file '%s'", filename);

FZ_ERROR_MEMORY is reserved for exceptions thrown due to a memory allocation failing. This will rarely be thrown by application code; the typical generators of such exceptions are fz_malloc/fz_calloc/fz_realloc etc.

FZ_ERROR_SYNTAX is reserved for exceptions thrown during interpretation of document files due to syntax errors. This enables the interpreter code to keep track of how many syntax errors have been found in a file, and to abort interpretation after a reasonable number have been passed.

FZ_ERROR_TRYLATER is reserved for exceptions thrown due to lack of data in progressive mode (see for more details). Catching an error of this type can trigger different handling, whereby the operation is retried when more data has arrived.

FZ_ERROR_ABORT is reserved for exceptions that should stop any ongoing operations; for instance, while looping over the annotations on a page to render them, most exceptions are caught at the top level and ignored, ensuring that a single broken annotation doesn’t cause subsequent annotations to be skipped. FZ_ERROR_ABORT can be used to override this behaviour and cause the annotation rendering process to end as swiftly as possible.

Handling exceptions

Once you have caught an exception, most code will simply tidy up any loose resources (to prevent leaks), and rethrow the exception up to a higher layer handler.

At the top level of the program, clearly this is not an option. The catch clause needs to return the error using whatever process the calling program is using for error handling.

Details of the message from the caught error can be read (from inside the fz_catch block) using:

const char *fz_caught_message(fz_context *ctx);

The error will remain readable in this way until the next use of fz_try/fz_catch on that same context.

Some code may choose to swallow the error and retry the same code again in a different manner. To facilitate this, we can find out the type of error using:

int fz_caught(fz_context *ctx);

See for a list of the possible exception types.

For example, if an exception was thrown whilst attempting to render a page to a full page bitmap, it is entirely possible that this might be due to running out of memory. An application might reasonably decide to retry the render doing a strip at a time.

If, however, the render failed because of a corrupt file we’d gain nothing by retrying - hence the application should check the type of the exception to ensure it was a FZ_ERROR_MEMORY before trying the alternative technique.

To simplify the job of deciding whether to pass on exceptions of a given type, we have a convenience function that with rethrow just a particular type:

void fz_rethrow_if(fz_context *ctx, int errcode);

Summary

The basic exception handling rules are as follows:

  1. All MuPDF functions except those that explicitly state otherwise, throw exceptions on errors, and must therefore be called from within a fz_try/fz_catch construct.

  2. A fz_try block must be paired with a fz_catch block, and optionally a fz_always block can appear between them.

  3. Never return from a fz_try block.

  4. A fz_try block will terminate when control reaches the end of the block, or when break is called.

  5. Any local variable that is changed within a fz_try block may lose its value if an exception occurs, unless protected by fz_var call.

  6. The contents of the fz_always block will always be executed (after the fz_try block and before the fz_catch block, if appropriate).

  7. If an exception is thrown during the fz_try block, control will jump to the fz_always block (if there is one) and then continue to the fz_catch block.

Memory Management and The Store

Overview

While MuPDF is running, it holds various objects in memory, and passes them between its various components. For instance, MuPDF might read a path definition in in the PDF interpreter, and pass it first into the display list and then on into the renderer.

To avoid needless copying of data, a reference counting scheme is used. Each significant object has a reference count, so that when one area of the code retains a reference to something (perhaps the display list), the data need not be copied wholesale. In the above example, the PDF interpreter might hold one reference, and first the display list and then the renderer might take others. Some references are held just for a short length of time, but others can persist for a much longer period.

During the course of displaying files, MuPDF loads various resources into memory, such as fonts and images. By holding these resources in memory throughout the processing of the file we can avoid reloading them each time they are required.

As the document is rendered, more memory is needed to hold rendered versions of glyphs from the font, or decoded versions of images. By keeping these decoded versions around in memory, we can avoid the need to re-decode them the next time we need the same glyph, or the same image.

Keeping all this data around can end up using a large amount of memory, which may be infeasible for some systems. Equally, not keeping any of it around will result in a drastic performance drop.

The solution is to keep as much around as can conveniently fit in memory, but not so much that we start to run out for other needs. MuPDF achieves this using a mechanism known as “The Store”.

The Store is a mechanism for holding blocks of data likely to be reusable. Whenever MuPDF needs such a block of data, it checks the Store to see if the data is there already - if it is, it can be instantly reused. If not the code forms the data itself (loading it, calculating it, or decoding it etc), and then puts it into the Store.

The MuPDF allocation code is tied into the Store, so that if an allocation ever fails, objects are evicted from the Store, and the allocation retried. This ‘scavenging’ of memory means that we can safely keep lots of cached data around without ever worrying that it will cause us to run out of memory.

Creating the Store

The Store is created as part of the fz_new_context call, (see the Context chapter) and is shared with any contexts obtained with fz_clone_context. The ‘store limit’ is specified as a byte size as part of this call. A special value of FZ_STORE_UNLIMITED is used to indicate that no amount of memory is too much.

Reacting to Out of Memory events

As a last resort, applications using MuPDF can react to low memory events by changing their strategy. For example, if we fail to render a band of data due to an allocation failure, we might back off and try a smaller band size. Alternatively, we might choose to dispense with the display list, and to reinterpret the underlying file directly each time, trading speed for memory.

To this end, all exceptions thrown due to allocation failures have the FZ_ERROR_MEMORY type, enabling callers to easily distinguish them using fz_caught and to react accordingly.

Implementation

Further information on The Store can be found in in Part II.

The Document interface

Overview

Although MuPDF handles multiple different file formats, it offers a unified API for dealing with them. The fz_document API allows all the common operations to be performed on a document, hiding the implementation specifics away from the caller.

Not all functions are available on all document types (for instance, JPEG files do not support annotations), but the API returns sane values.

Opening/Closing a document

The simplest way to load a document is to load it from the local filing system:

/*
        fz_open_document: Open a PDF, XPS or CBZ document.

        Open a document file and read its basic structure so pages and
        objects can be located. MuPDF will try to repair broken
        documents (without actually changing the file contents).

        The returned fz_document is used when calling most other
        document related functions.

        filename: a path to a file as it would be given to open(2).
*/
fz_document *fz_open_document(fz_context *ctx, const char *filename);

For embedded systems, or secure applications, the use of a local filing system may be inappropriate, so an alternative is available whereby documents can be opened from a fz_stream. See for more details on `fz_stream`s.

/*
        fz_open_document_with_stream: Open a PDF, XPS or CBZ document.

        Open a document using the specified stream object rather than
        opening a file on disk.

        magic: a string used to detect document type; either a file name or mime-type.
*/
fz_document *fz_open_document_with_stream(fz_context *ctx, const char *magic, fz_stream *stream);

Almost any data source can be wrapped up as a fz_stream; see for more details.

In common with most other objects in MuPDF, `fz_document`s are reference counted:

/*
        fz_keep_document: Keep a reference to an open document.

        Does not throw exceptions.
*/
fz_document *fz_keep_document(fz_context *ctx, fz_document *doc);

/*
        fz_drop_document: Release an open document.

        The resource store in the context associated with fz_document
        is emptied, and any allocations for the document are freed when
        the last reference is dropped.

        Does not throw exceptions.
*/
void fz_drop_document(fz_context *ctx, fz_document *doc);

Once the last reference to the document is dropped, all resources used by that document will be released, including those in the Store.

Handling password protected documents

Some document types (such as PDF) can require passwords to allow the file to be opened. After you have obtained a fz_document, you should therefore check whether it needs a password using fz_needs_password:

/*
        fz_needs_password: Check if a document is encrypted with a
        non-blank password.

        Does not throw exceptions.
*/
int fz_needs_password(fz_context *ctx, fz_document *doc);

If a password is required, you can supply one using fz_authenticate_password:

/*
        fz_authenticate_password: Test if the given password can
        decrypt the document.

        password: The password string to be checked. Some document
        specifications do not specify any particular text encoding, so
        neither do we.

        Returns 0 for failure to authenticate, non-zero for success.

        For PDF documents, further information can be given by examining
        the bits in the return code.

                Bit 0 => No password required
                Bit 1 => User password authenticated
                Bit 2 => Owner password authenticated

        Does not throw exceptions.
*/
int fz_authenticate_password(fz_context *ctx, fz_document *doc, const char *password);

Handling reflowable documents

Some document types (such as EPUB) require the contents to be laid out before they can be rendered. This is done by calling fz_layout_document:

/*
        fz_layout_document: Layout reflowable document types.

        w, h: Page size in points.
        em: Default font size in points.
*/
void fz_layout_document(fz_context *ctx, fz_document *doc, float w, float h, float em);

Any non-reflowable document types (such as PDF) will ignore this layout request.

The results of the layout will depend both upon a target width and height, a given font size, and the CSS styles in effect. MuPDF has an inbuilt set of default CSS styles that will be used if a document does not provide its own. In addition, the user can provide a final set that will override any rules found in the default sets. In this way, the appearance of the rendered document can be changed (perhaps by changing document colours or font styles/sizes).

Documents can be laid out multiple times to allow changes in these properties to take effect.

MuPDF provides its own default CSS style sheet, but this can be overridden by the user CSS style sheet in the context:

/*
        fz_user_css: Get the user stylesheet source text.
*/
const char *fz_user_css(fz_context *ctx);

/*
        fz_set_user_css: Set the user stylesheet source text for use with HTML and EPUB.
*/
void fz_set_user_css(fz_context *ctx, const char *text);

The user CSS style sheet is supplied as a null terminated C string.

When the CSS or the screen size is changed, and the document relaid out, content moves. In order for applications to be able to not lose the readers place, MuPDF offers a mechanism for making a bookmark and then looking it up again after the content has been laid out to a new position.

/*
        Create a bookmark for the given page, which can be used to find the
        same location after the document has been laid out with different
        parameters.
*/
fz_bookmark fz_make_bookmark(fz_context *ctx, fz_document *doc, int page);

/*
        Find a bookmark and return its page number.
*/
int fz_lookup_bookmark(fz_context *ctx, fz_document *doc, fz_bookmark mark);

Getting Pages from a document

Once you have a laid out document, you presumably want to be able to do something with it. The first thing to know is how many pages it contains. This is achieved by calling fz_count_pages:

/*
        fz_count_pages: Return the number of pages in document

        May return 0 for documents with no pages.
*/
int fz_count_pages(fz_context *ctx, fz_document *doc);

For document types like images, they appear as a single page. If you forget to lay out a reflowable document, this will trigger a layout for a default size and return the required number of pages.

Once you know how many pages there are, you can fetch the fz_page object for each page required:

/*
        fz_load_page: Load a page.

        After fz_load_page is it possible to retrieve the size of the
        page using fz_bound_page, or to render the page using
        fz_run_page_*. Free the page by calling fz_drop_page.

        number: page number, 0 is the first page of the document.
*/
fz_page *fz_load_page(fz_context *ctx, fz_document *doc, int number);

The pages of a document with n pages are numbered from 0 to n-1.

In common with most other object types, `fz_page`s are reference counted:

/*
        fz_keep_page: Keep a reference to a loaded page.

        Does not throw exceptions.
*/
fz_page *fz_keep_page(fz_context *ctx, fz_page *page);

/*
        fz_drop_page: Free a loaded page.

        Does not throw exceptions.
*/
void fz_drop_page(fz_context *ctx, fz_page *page);

Once the last reference to a page is dropped, the resources it consumes are all released automatically.

Anatomy of a Page

In MuPDF terminology (largely borrowed from PDF) Pages consist of Page Contents, Annotations, and Links.

Page Contents (or just Contents) are typically the ordinary printed matter that you would get on a page; the text, illustrations, any headers or footers, and maybe some printers marks.

Annotations are normally extra information that is overlaid on the top of these page contents. Examples include freehand scribbles on the page, highlights/underlines/strikeouts overlaid on the text, sticky notes etc. Annotations can be included when the document is originally authored, but, frequently, are added later by people as they read the document.

Annotations also serve as the key components that allow interactivity in PDF documents.

More details can be found in .

Color Considerations

Some formats, notably PDF, contain significant extra information to enable a high quality color managed workflow. The document interface (and the related page interface) have some methods to enable this.

Documents can have a defined ‘output intent’ that governs the color space (and profile) used for rendered output:

/*
        Find the output intent colorspace if the document has defined one.
*/
fz_colorspace *fz_document_output_intent(fz_context *ctx, fz_document *doc);

Callers will typically interrogate this before creating their output pixmaps if they want to honour it.

Each page in PDF can be authored with specific spot colors (inks) in mind. Details of these can be obtained from:

/*
        fz_page_separations: Get the separations details for a page.
        This will be NULL, unless the format specifically supports
        separations (such as gproof, or PDF files). May be NULL even
        so, if there are no separations on a page.

        Returns a reference that must be dropped.
*/
fz_separations *fz_page_separations(fz_context *ctx, fz_page *page);

The returned object will be NULL for all document formats that do not support spot colors (at the time of writing, all but PDF). For PDF, the object will be NULL for all pages that do not make use of separations.

More information about using these objects can be found in .

Rendering Pages

To render a page, you first need to know how big it is. This can be discovered by calling fz_bound_page, passing a fz_rect in to be populated:

/*
        fz_bound_page: Determine the size of a page at 72 dpi.

        Does not throw exceptions.
*/
fz_rect *fz_bound_page(fz_context *ctx, fz_page *page, fz_rect *rect);

MuPDF operates on page contents (and annotations/widgets) by processing them to a Device. There are various different devices in MuPDF (and you can implement your own). See for more information. For now, just consider devices to be things that are called with each of the graphical items on the page in turn.

The simplest way to process a page is to call fz_run_page:

/*
        fz_run_page: Run a page through a device.

        page: Page obtained from fz_load_page.

        dev: Device obtained from fz_new_*_device.

        transform: Transform to apply to page. May include for example
        scaling and rotation, see fz_scale, fz_rotate and fz_concat.
        Set to fz_identity if no transformation is desired.

        cookie: Communication mechanism between caller and library
        rendering the page. Intended for multi-threaded applications,
        while single-threaded applications set cookie to NULL. The
        caller may abort an ongoing rendering of a page. Cookie also
        communicates progress information back to the caller. The
        fields inside cookie are continually updated while the page is
        rendering.
*/
void fz_run_page(fz_context *ctx, fz_page *page, fz_device *dev, const fz_matrix *transform, fz_cookie *cookie);

This will cause each graphical object from the page contents and annotations to be transformed, and fed to the device.

For finer control, you may wish to run the page contents and the annotations for a page separately:

/*
        fz_run_page_contents: Run a page through a device. Just the main
        page content, without the annotations, if any.

        page: Page obtained from fz_load_page.

        dev: Device obtained from fz_new_*_device.

        transform: Transform to apply to page. May include for example
        scaling and rotation, see fz_scale, fz_rotate and fz_concat.
        Set to fz_identity if no transformation is desired.

        cookie: Communication mechanism between caller and library
        rendering the page. Intended for multi-threaded applications,
        while single-threaded applications set cookie to NULL. The
        caller may abort an ongoing rendering of a page. Cookie also
        communicates progress information back to the caller. The
        fields inside cookie are continually updated while the page is
        rendering.
*/
void fz_run_page_contents(fz_context *ctx, fz_page *page, fz_device *dev, const fz_matrix *transform, fz_cookie *cookie);

/*
        fz_run_page_annots: Run the annotations on a page through a device.
*/
void fz_run_page_annots(fz_context *ctx, fz_page *page, fz_device *dev, const fz_matrix *transform, fz_cookie *cookie);

These functions enable viewer applications to generate separate display lists for page contents and annotations. This can be useful if annotations are frequently changed, as it allows regeneration/redraw to happen on a per-annotation rather than per-page level.

All of these functions (fz_run_page, fz_run_page_contents, fz_run_page_annots) take a fz_cookie pointer. The Cookie is a lightweight way of controlling the processing of the page. For more details, see . For most simple cases this can be NULL.

Presentations

Some file formats, such as PDF can be used as ‘presentations’, where pages are displayed as a slideshows - a form of poor man’s PowerPoint if you will. Essentially each page contains a record that says how long it should be displayed for before transitioning to the next page with a given graphical effect.

The core MuPDF library is never responsible for actually presenting a page to the user, so it is therefore not possible to expect it to cope with handling all the work required by such transitions.

What it can do is to help in 2 particular areas. Firstly, it can provide some functions to aid the caller in the task of querying the transitions required. Secondly, it can help in providing some helper functions to generate bitmaps of various stages of common transitions.

Querying

We define a structure type to hold the details of arbitrary transitions, together with some opaque state:

enum {
        FZ_TRANSITION_NONE = 0, /* aka 'R' or 'REPLACE' */
        FZ_TRANSITION_SPLIT,
        FZ_TRANSITION_BLINDS,
        FZ_TRANSITION_BOX,
        FZ_TRANSITION_WIPE,
        FZ_TRANSITION_DISSOLVE,
        FZ_TRANSITION_GLITTER,
        FZ_TRANSITION_FLY,
        FZ_TRANSITION_PUSH,
        FZ_TRANSITION_COVER,
        FZ_TRANSITION_UNCOVER,
        FZ_TRANSITION_FADE
};

typedef struct fz_transition_s
{
        int type;
        float duration; /* Effect duration (seconds) */

        /* Parameters controlling the effect */
        int vertical; /* 0 or 1 */
        int outwards; /* 0 or 1 */
        int direction; /* Degrees */
        /* Potentially more to come */

        /* State variables for use of the transition code */
        int state0;
        int state1;
} fz_transition;

Armed with such a structure, we can call a function to get it filled out:

/*
        fz_page_presentation: Get the presentation details for a given page.

        transition: A pointer to a transition struct to fill out.

        duration: A pointer to a place to set the page duration in seconds.
        Will be set to 0 if no transition is specified for the page.

        Returns: a pointer to the transition structure, or NULL if there is no
        transition specified for the page.
*/
fz_transition *fz_page_presentation(fz_context *ctx, fz_page *page, fz_transition *transition, float *duration);

This structure is defined to be sufficient to encapsulate the currently defined PDF transition types; it may be extended in future if other formats require more expressiveness.

Callers are free to directly implement their transitions using the information herein, or else they can make use of a helper function.

Helper functions

Details of a helpful routine for displaying some of these transitions can be found in .

The Device interface

Overview

In many ways, the Device interface is the heart of MuPDF.

When any given document handler is told to run the page (fz_run_page) the appropriate document interpreter serialises the page contents as a series of graphical operations, and calls the device interface to perform these operations.

Many different implementations of the device interface exist within MuPDF. The most obvious one is the Draw device. When this is called, it renders the graphical objects in turn into a Pixmap.

Alternatively we have the Structured Text device that captures the text output and forms it into an easily processable structure (for searching, or text extraction).

Some devices, such as the SVG Output device, repackage the graphical objects into a different format. The end product of these devices is a new document with (as much as possible) the same overall appearance as the initial page.

Finally, devices such as the Display List device manage to be both implementers of the interface, and callers of it. Callers can run page contents to the Display List device just once, and then replay it quickly many times over to other devices; ideal for rendering pages in bands, or repeatedly redrawing as a viewer pans and zooms around a document.

By implementing new devices callers can tap the power of MuPDF in new and interesting ways, perhaps to harness specific hardware facilities of a device.

Device Methods

Every Device in MuPDF is an extension of the fz_device structure. This contains a series of function pointers to implement the handling of different types of graphical object.

These function pointers are exposed to callers via convenience functions. These convenience functions should always be used in preference to calling the function pointers direct, as they perform various behind the scenes housekeeping functions. They also cope with the function pointers being NULL, as can permissibly happen when a device is not interested in a particular class of graphical object.

We will not describe these device functions here, but rather defer them to in Part 2. While it is perfectly permissible for callers to call the device convenience functions themselves, the vast majority of application authors will never do so, and will simply treat each fz_device as a ‘black box’ to be passed to the interpretation functions (see ).

The cookie is a lightweight mechanism for controlling and detecting the behaviour of a given interpretation call (i.e. fz_run_page, fz_run_page_contents, fz_run_annot, fz_run_display_list etc).

To use the cookie, a caller should simply define:

fz_cookie *cookie = { 0 };

set any required fields, for example:

cookie.incomplete_ok = 1;

and then pass &cookie as the last parameter to the interpretation call, for example:

fz_run_page(ctx, page, dev, transform, &cookie);

The contents and definition of fz_cookie are even more subject to change than other structures, so it is important to always initialise all the subfields to zero. The safest way to do this is as given above. If new fields are added to the structure, callers code should not need to change, and the default behaviour of zero-valued new fields will always remain the same.

Detecting errors

When displaying a page, if we hit an error, what should we do?

We could choose to stop interpretation entirely, but that would mean that a relatively unimportant error (such as a missing or broken font) would prevent us getting anything useful out of a page.

We could choose to ignore the errors and continue, but that would be a problem for (say) a print run, where it would undesirable for us to print 1000 copies of a document only to discover that it’s missing an image.

The strategy taken by MuPDF is to swallow errors during interpretation, but keep a count of them in the errors field within the cookie. That way callers can check that cookie.errors == 0 at the end to know whether a run completed without incident.

Content interpretations can take a (relatively) long time. Once one has been started, it can be useful a) to know how far through processing we are, and b) to be able to abort processing should the results of a run no longer be required.

As a run progresses, 2 fields in the cookie are updated. Firstly, progress will be set to a number that increases as progress is made. Think of this informally as being the number of objects that have been processed so far. In some cases (notably when processing a display list) we can know an upper bound for this value, and this value will be given as progress_max. In cases where no upper bound is known, progress_max will be set to -1. It is possible that the upper bound may start as -1, and then change to a known value later.

These values are intended to enable user feedback to be given, and should not be taken as guarantees of performance.

While running content, the interpreter periodically checks the abort field of the cookie. If it is discovered to be non zero, the rest of the content is ignored.

If the caller decides that it does not need the results of a run once it has been started (perhaps the user changes the page, or closes the file), then it should therefore set the abort field of the cookie to 1.

No guarantees are made about how often the cookie is checked, nor about how fast an interpreter will respond to the abort field once it is set. Setting the abort flag will never hurt, and will frequently help, however. Once the flag has been set to 1, it should never be reset to 0, as the results will be unpredictable.

Resources used by a run cannot be released until the end of a run, regardless of the setting of abort. Callers still need to wait for the fz_run_page (or other) call to complete before the page etc can be safely dropped.

The cookie also has a role to play when working in Progressive Mode. The incomplete_ok and incomplete fields are used for this. See for more details.

Device Hints

Device Hints are a mechanism that enables control over the behaviour of a device, and to interpreters calling to that device. Informally they offer hints about what a device is going to do and therefore what callers need to worry about.

Device hints take the form of bits in an int that can be enabled (set) or disabled (cleared). Callers can query these hints to customise their behaviour.

/*
        fz_enable_device_hints : Enable hints in a device.

        hints: mask of hints to enable.
*/
void fz_enable_device_hints(fz_context *ctx, fz_device *dev, int hints);

/*
        fz_disable_device_hints : Disable hints in a device.

        hints: mask of hints to disable.
*/
void fz_disable_device_hints(fz_context *ctx, fz_device *dev, int hints);

Some devices set the hints to non-zero default values.

For example, when running a text-extraction operation (as used to implement text search), there is little point in handling images, or shadings. The text extraction device therefore sets FZ_IGNORE_IMAGE and FZ_IGNORE_SHADE. The interpretation functions (such as fz_run_page or fz_run_display_list can then not bother to prepare images for calling into the device, improving performance.

If, however, you wish to extract the page content to an html file, you might want to include images in this output. So for this, you would disable the FZ_IGNORE_IMAGE hint before running the extraction, and the text extraction device would know to include them in its output structures.

The set of hints is subject to expansion in future, but is currently defined to be:

enum
{
        /* Hints */
        FZ_DONT_INTERPOLATE_IMAGES = 1,
        FZ_MAINTAIN_CONTAINER_STACK = 2,
        FZ_NO_CACHE = 4,
};

FZ_DONT_INTERPOLATE_IMAGES being enabled prevents the draw device performing interpolation. MuTool Draw uses this to inhibit interpolation when anti-aliasing is disabled. Finer control over this can now be given using the Tuning Context (see ).

FZ_MAINTAIN_CONTAINER_STACK being enabled helps devices by causing MuPDF to maintain a stack of containers. This effectively moves some logic that would have to be in several devices into a place where it can be easily reused. Currently the only device that makes use of this is the SVG device, but it is hoped that more will use it in future.

FZ_NO_CACHE being enabled tells the interpreter to try to avoid caching any objects after the end of the content run. This can be used, for example, when searching a PDF for a text string to avoid pulling all the images, shadings, fonts etc and other resources for pages into memory at the expense of those that are used on the current page.

Inbuilt Devices

MuPDF comes with a selection of devices built in, though this should not be taken as a definitive list. It is expected that other devices will be written to extend MuPDF - indeed some embeddings of MuPDF already include their own devices.

BBox Device

The BBox device is a simple device that calculates the bbox of all the marking operations1 on a page.

/*
        fz_new_bbox_device: Create a device to compute the bounding
        box of all marks on a page.

        The returned bounding box will be the union of all bounding
        boxes of all objects on a page.
*/
fz_device *fz_new_bbox_device(fz_context *ctx, fz_rect *rectp);

The fz_rect passed to the fz_new_bbox_device must obviously stay in scope for the duration of the life of the device as it will be updated when the device is closed with the bounding box for the contents.

Draw Device

The Draw device is the core renderer for MuPDF. Every draw device instance is constructed with a destination Pixmap (see for more details), and each graphical object passed to the device is rendered into that pixmap.

/*
        fz_new_draw_device: Create a device to draw on a pixmap.

        dest: Target pixmap for the draw device. See fz_new_pixmap*
        for how to obtain a pixmap. The pixmap is not cleared by the
        draw device, see fz_clear_pixmap* for how to clear it prior to
        calling fz_new_draw_device. Free the device by calling
        fz_drop_device.
*/
fz_device *fz_new_draw_device(fz_context *ctx, fz_pixmap *dest);

Most of the time we render complete pixmaps, but a mechanism exists to allow us to render a given bbox within a pixmap:

/*
        fz_new_draw_device_with_bbox: Create a device to draw on a pixmap.

        dest: Target pixmap for the draw device. See fz_new_pixmap*
        for how to obtain a pixmap. The pixmap is not cleared by the
        draw device, see fz_clear_pixmap* for how to clear it prior to
        calling fz_new_draw_device. Free the device by calling
        fz_drop_device.

        clip: Bounding box to restrict any marking operations of the
        draw device.
*/
fz_device *fz_new_draw_device_with_bbox(fz_context *ctx, fz_pixmap *dest, const fz_irect *clip);

This can be useful for updating particular areas of a page (for instance when an annotation has been edited or moved) without redrawing the whole thing.

During the course of rendering, the draw device may create new temporary internal pixmaps to cope with transparency and grouping. This is invisible to the caller, and can safely be considered an implementation detail, but should be considered when estimating the memory use for a given rendering operation. The exact number and size of internal pixmaps required depends on the exact complexity and makeup of the graphical objects being displayed.

To limit memory use, a typical strategy is to render pages in bands; rather than creating a single pixmap the size of the page and rendering that, create pixmaps for ’slices’ across the page, and render them one at a time. The memory savings are not just seen in the cost of the basic pixmap, but also serve to limit the sizes of the internal pixmaps used during rendering.

The cost for this is that the page contents do need to be run through repeatedly. This can be achieved by reinterpreting directly from the file, but that can be expensive. The next device provides a route to help with this.

Advanced Rendering - Overprint and Spots

Most formats define pages in terms of some fairly simple ‘well known’ colorspaces, like RGB and CMYK. Some formats (notably PDF) are much more powerful, and allow pages to be constructed with a range of non-standard ‘spot’ inks.

When combined with advanced features such as overprinting, care needs to be taken to ensure that the rendering is exactly as expected.

For example, if a PDF page is constructed to render a page using overprinting it only makes strict sense to render this to a CMYK (or a CMYK + Spots) pixmap. With (say) an RGB pixmap, CMYK colors would be mapped down to RGB as they are plotted, losing the information required to correctly overprint later graphical objects.

Nonetheless, while we might want to get a ‘true’ rendition of the page, we might require it ultimately to appear as an RGB pixmap. As such what we really want is to get a ‘simulation’ of how the overprint would work.

One way to work would be to call the draw device and request a CMYK + Spots rendering, and then to require the caller to convert this to their desired target colorspace manually. This is not in keeping with the general desire in MuPDF to encapsulate functionality in a friendly way.

Therefore, the draw device examines the ‘separations’ field of the pixmap that it is called with to decide how to render.

If there is no separations value supplied (i.e. it is NULL), then the draw device assumes that no form of overprint (or overprint simulation) is required.

If there is a separations value, and there is at least one separation that is not entirely disabled, then the draw device will draw internally to a CMYK + Spots pixmap (where the spots are the non-disabled separations from the separations value). This rendering can safely proceed with overprint processing enabled.

At the end of the render, the draw device will convert down from the CMYK + Spots pixmap to the colorspace of the initial pixmap. Any spot colorants present in the initial pixmap will be populated from the rendered one; any spots that aren’t will be converted down to process colors.

Thus by creating the initial pixmap passed into the draw device using a separations object with the colorants correctly set to be composite/spots/disabled as required, overprint or overprint simulation can be controlled as required.

Display List Device

The Display list device simply records all the calls made to it in a list. This list can then be played back later, potentially multiple times and with different transforms, to other devices.

/*
        fz_new_list_device: Create a rendering device for a display list.

        When the device is rendering a page it will populate the
        display list with drawing commsnds (text, images, etc.). The
        display list can later be reused to render a page many times
        without having to re-interpret the page from the document file
        for each rendering. Once the device is no longer needed, free
        it with fz_drop_device.

        list: A display list that the list device takes ownership of.
*/
fz_device *fz_new_list_device(fz_context *ctx, fz_display_list *list);

For more details of the uses of Display Lists, see .

PDF Output Device

The PDF Output device is still a work in progress, as its handling of fonts is incomplete. Nonetheless for certain classes of files it can be useful.

End users will probably prefer to use the document writer interface (see ) which wraps this class up, rather than call it directly. Nonetheless this can be useful in specific circumstances when generating particular sections of a PDF file (such as appearance streams for annotations).

The PDF Output device takes the sequence of graphical operations it is called with, and forms it back into a sequence of PDF operations, together with a set of required resources. These can then be formed into a completely new PDF page (or a PDF annotation) which can then be inserted into a document.

/*
        pdf_page_write: Create a device that will record the
        graphical operations given to it into a sequence of
        pdf operations, together with a set of resources. This
        sequence/set pair can then be used as the basis for
        adding a page to the document (see pdf_add_page).

        doc: The document for which these are intended.

        mediabox: The bbox for the created page.

        presources: Pointer to a place to put the created
        resources dictionary.

        pcontents: Pointer to a place to put the created
        contents buffer.
*/
fz_device *pdf_page_write(fz_context *ctx, pdf_document *doc, const fz_rect *mediabox, pdf_obj **presources, fz_buffer **pcontents);

Structured Text Device

The Structured Text device is used to extract the text from a given graphical stream, together with the position it inhabits on the output page. It can also optionally include details of images and their positions within its output.

/*
        fz_new_stext_device: Create a device to extract the text on a page.

        Gather and sort the text on a page into spans of uniform style,
        arranged into lines and blocks by reading order. The reading order
        is determined by various heuristics, so may not be accurate.

        sheet: The text sheet to which styles should be added. This can
        either be a newly created (empty) text sheet, or one containing
        styles from a previous text device. The same sheet cannot be used
        in multiple threads simultaneously.

        page: The text page to which content should be added. This will
        usually be a newly created (empty) text page, but it can be one
        containing data already (for example when merging multiple pages, or
        watermarking).
*/
fz_device *fz_new_stext_device(fz_context *ctx, fz_stext_sheet *sheet, fz_stext_page *page);

This can be used as the basis for searching (including highlighting the text as matches are found), for exporting text files (or text and image based files such as HTML), or even to do more complex page analysis (such as spotting what regions of the page are text, what are graphics etc).

An (initially empty) fz_stext_sheet should be created using fz_new_stext_sheet, and an empty fz_stext_page created using fz_new_stext_page. These are used in the call to fz_new_stext_device. After the contents have been run to that device, the sheet will be populated with the common styles used by the page, and the page will be populated with details of the text extracted and its position.

SVG Output Device

The SVG output device is used to generate SVG pages from arbitrary input.

End users will probably prefer to use the document writer interface (see ) which wraps this class up, rather than call it directly.

/*
        fz_new_svg_device: Create a device that outputs (single page)
        SVG files to the given output stream.

        output: The output stream to send the constructed SVG page
        to.

        page_width, page_height: The page dimensions to use (in
        points).
*/
fz_device *fz_new_svg_device(fz_context *ctx, fz_output *out, float page_width, float page_height);

The device currently generates SVG 1.1 compliant files. SVG Fonts are NOT used due to poor client support. Instead glyphs are sent as reusable symbols. Shadings are sent as rasterised images. JPEGs will be passed through unchanged, and all other images will be converted to PNG.

Test Device

The Test device, as its name suggests, tests a given set of page contents for which features are used. Currently this is restricted to testing for whether the graphical objects used are greyscale or colour. Testing for additional features may be added in future.

/*
        fz_new_test_device: Create a device to test for features.

        Currently only tests for the presence of non-grayscale colors.

        is_color: Possible values returned:
                0: Definitely greyscale
                1: Probably color (all colors were grey, but there
                were images or shadings in a non grey colorspace).
                2: Definitely color

        threshold: The difference from grayscale that will be tolerated.
        Typical values to use are either 0 (be exact) and 0.02 (allow an
        imperceptible amount of slop).

        options: A set of bitfield options, from the FZ_TEST_OPT set.

        passthrough: A device to pass all calls through to, or NULL.
        If set, then the test device can both test and pass through to
        an underlying device (like, say, the display list device). This
        means that a display list can be created and at the end we'll
        know if its color or not.

        In the absence of a passthrough device, the device will throw
        an exception to stop page interpretation when color is found.
*/
fz_device *fz_new_test_device(fz_context *ctx, int *is_color, float threshold, int options, fz_device *passthrough);

The expected purpose of the colour detecting functionality is to allow applications (e.g. printers) to easily detect if a given page requires the use of colour inks, or whether a greyscale rendering will suffice.

This device can either be used by itself, or in the form of a pass-through device.

Standalone use

In the simplest form, the device can be created standalone, by passing passthrough as NULL.

As each subsequent device call is made, the device will test the graphic object passed to it to see if it is within the given threshold of being a neutral colour. If it is, then the device continues. If not, then it sets the int pointed to by is_color to be non zero.

For graphical objects such as paths or text, this is an easy evaluation that takes almost no time. For Images or Shadings however, it is slightly trickier. An image may be defined in a colour space capable of non-neutral colours (perhaps RGB or CMYK) and yet the image itself may only use neutral colours within that space. To properly establish whether colours are required or not, requires much more CPU intensive processing.

Accordingly, the device will, by default, just look at the colour space. The value of is_color returned at the end may be examined to establish the confidence level of the test. 0 means “definitely greyscale”, 1 means “probably colour” (i.e. “an image or shading was seen that potentially contains non neutral colours”), and 2 means “definitely colour”.

If the caller wishes to spend the CPU cycles to get a definite answer, options can be set to FZ_TEXT_OPT_IMAGES | FZ_TEXT_OPT_SHADINGS and images and shadings will be exhaustively checked.

As an optimisation, given how much faster is is to check non-images and shadings, it can be worth running the device once without the options set, and then only running it again with them set if required.

If the device is run with passthrough as NULL, then as soon as it encounters a “definite” non-neutral colour, it will throw a FZ_ABORT error. This can save a considerable amount of time, as it avoids the interpreter needing to run through an entire page when observation of one of the very first graphical operations is enough to know that colour is being used.

Passthrough use

As discussed above, the envisaged use case for this device is to detect whether page contents require colour or not to allow printers to decide whether to rasterise for colour inks or a faster/cheaper greyscale pass.

Such printers will normally be operating in banded mode, which requires (or at least greatly benefits from) the use of a display list. By using the device in passthrough mode, the testing can be performed at the same time as the list is built.

Simply create the display list device as you would normally, and pass it into fz_new_test_device as passthrough. Then run the page contents through the returned test device. The test device will pass each call through to the underlying list device and so the display list be built as normal.

When run in this mode, the device can no longer use the ‘early-exit’ optimisation of throwing a FZ_ABORT error.

Trace Device

The Trace device is a simple debugging device that allows an XML-like representation of the device calls made to be output.

/*
        fz_new_trace_device: Create a device to print a debug trace of all device calls.
*/
fz_device *fz_new_trace_device(fz_context *ctx, fz_output *out);

This is a useful tool to visualise the contents of display lists.

Building Blocks

Overview

MuPDF uses many constructs and concepts that, while not deserving of chapters in their own rights, do deserve mention.

Colorspaces

In order to represent a given color for a graphical object, we need both the color component values and details of the colorspace that the color is specified in. Color values are defined simply as floats (normally between 0 and 1 inclusive), and colorspaces are defined using the fz_colorspace structure.

As with many other such structures in MuPDF, these are reference counted objects (see ).

Basic Colorspaces

MuPDF contains a set of inbuilt colorspaces that cover most simple requirements. These are the ‘device’ colorspaces:

/*
        fz_device_gray: Get colorspace representing device specific gray.
*/
fz_colorspace *fz_device_gray(fz_context *ctx);

/*
        fz_device_rgb: Get colorspace representing device specific rgb.
*/
fz_colorspace *fz_device_rgb(fz_context *ctx);

/*
        fz_device_bgr: Get colorspace representing device specific bgr.
*/
fz_colorspace *fz_device_bgr(fz_context *ctx);

/*
        fz_device_cmyk: Get colorspace representing device specific CMYK.
*/
fz_colorspace *fz_device_cmyk(fz_context *ctx);

/*
        fz_device_lab: Get colorspace representing device specific LAB.
*/
fz_colorspace *fz_device_lab(fz_context *ctx);

Indexed Colorspaces

MuPDF allows for indexed colorspaces - those where a palette is used to select color values from a (normally) larger colorspace.

These are created using the fz_new_indexed_colorspace call:

fz_colorspace *fz_new_indexed_colorspace(fz_context *ctx, fz_colorspace *base, int high, unsigned char *lookup);

Separation and DeviceN Colorspaces

MuPDF Colorspaces are extensible, so specific document handlers can implement their own new spaces. A good example of this is how PDF implements Separation and DeviceN colorspaces.

These are special spaces which represent arbitrary sets of 1 or more colorants. These can either be mapped down to ‘equivalent’ colors in a more standard space, or (depending on the capabilities of the underlying device) processed in their raw form.

Further information

Further information on Colorspaces can be found within .

Pixmaps

Overview

The fz_pixmap structure is used to represent a 2 dimensional array of contone pixels. This is used throughout MuPDF, as the target of rendering from the draw device, as internal buffers during processing, and during image decoding.

A pixmap can have an arbitrary number of colour components, together with an optional alpha plane. Every component sample is represented by an unsigned char.

Pixmaps contain a set of nn values per pixel, where n=c+s+an = c + s + a. cc is the number of color components in the colorspace of a pixmap (or 0, if the colorspace is NULL). ss is the number of spot colors in a pixmap (frequently 0). aa is 0 if there is no alpha plane, and 1 otherwise.

The initial cc entries are referred to as the ‘process’ color components. These can be either additive or subtractive dependent on the colorspace of the pixmap. Additive spaces (such as Gray, or RGB) have value 0 as dark, 255 as light. Subtractive spaces (such as CMYK) have value 0 as light (no ink), 255 as dark (full ink).

The next ss entries are the spot colors represented by a pixmap. These are always in subtractive form.

The final entry (if a=1a = 1) is the alpha value. This is 0 for completely transparent, 255 for completely opaque.

The data within a pixmap is always stored packed in ‘chunky’ format. For instance, an RGB pixmap would have data in the form: RGBRGBRGBRGB…

Alpha data is always sent as the last byte in the set corresponding to a pixel. An RGB pixmap with an alpha plane would be therefore have data of the form: RGBARGBARGBA…

A CMYK pixmap with spots for Orange and Green would have data of the form: CMYKOGCMYKOGCMYKOG…

To allow greater flexibility in the layout of the underlying memory blocks used by pixmaps, they have a ‘stride’ field. This gives the number of bytes difference from the address of the start of the representation of a pixel to the address of the start of the representation of the same pixel on the scanline below.

Normally you’d expect stride to be the same as width multiplied by the number of components in the image (including alpha), but for some cases (notably when we have pixmaps that represent a sub-rectangle of larger pixmaps) these can be much larger.

Pixmaps can frequently map onto operating system specific bitmap representations, but these sometimes require each scanline to be word aligned - again the provision of stride allows for this. Bottom up bitmaps can be implemented using a negative stride.

Premultiplied alpha

By convention MuPDF holds pixmaps in ‘premultipled alpha’ form. This means that when an alpha plane is present, the values for the process and spot colors are stored scaled by the alpha value.

So for a pixel with `R`=`G`=`B`=1, with solid alpha, we’d have values of 255, but with an alpha value of 0.5 we’d have values of 127 stored.

This format is used because it simplifies many of the plotting and compositing operations used within MuPDF.

Saving

For information on saving pixmaps, see .

Bitmaps

The fz_bitmap structure is used to represent a 2 dimensional array of monochrome pixels. They are the 1 bit per component equivalent of the fz_pixmap structure.

The core rendering engine of MuPDF does not currently make use of `fz_bitmap`s, but rather they are used as a step along the way for outputting rendered information.

Functions exist within MuPDF to create `fz_bitmap`s from fz_pixmaps by halftoning. See .

/*
        fz_new_bitmap_from_pixmap: Make a bitmap from a pixmap and a halftone.

        pix: The pixmap to generate from. Currently must be a single color
        component + alpha (where the alpha is assumed to be solid).

        ht: The halftone to use. NULL implies the default halftone.

        Returns the resultant bitmap. Throws exceptions in the case of
        failure to allocate.
*/
fz_bitmap *fz_new_bitmap_from_pixmap(fz_context *ctx, fz_pixmap *pix, fz_halftone *ht);

fz_bitmap *fz_new_bitmap_from_pixmap_band(fz_context *ctx, fz_pixmap *pix, fz_halftone *ht, int band_start, int bandheight);

Both functions work by applying a fz_halftone to the contone values to make the bitmap. The latter function is a more general version of the former, that allows for correct operation when rendering in bands - namely that the correct offset into the halftone table is used.

The data for each Bitmap is packed into bytes most significant bit first. Multiple components are packed into the same byte, so a CMYK pixmap converted to a bitmap would have 2 pixels worth of data in the first byte, CMYKCMYK, with the first pixel in the highest nibble.

The usual reference counting behaviour applies to `fz_bitmap`s, with fz_keep_bitmap and fz_drop_bitmap claiming and releasing references respectively.

Halftones

The fz_halftone structure represents a set of tiles, one per component, each of a potentially different size. Each of these tiles is a 2-dimensional array of threshold values (actually implemented as a single component fz_pixmap). During the halftoning (bitmap creation) process, if the contone value is smaller than the threshold value, then it remains unset in the output. If it is larger or equal then it is set in the output.

For convenience, a NULL pointer can be used to signify the default halftone. The default halftone can also be fetched by using:

/*
        fz_default_halftone: Create a 'default' halftone structure
        for the given number of components.

        num_comps: The number of components to use.

        Returns a simple default halftone. The default halftone uses
        the same halftone tile for each plane, which may not be ideal
        for all purposes.
*/
fz_halftone *fz_default_halftone(fz_context *ctx, int num_comps);

The creation of halftones is a specialised field upon which much research has been done. The mechanisms in MuPDF are designed to allow people the freedom to create and tune the halftones for their particular application.

The usual reference counting behaviour applies to `fz_halftone`s, with fz_keep_halftone and fz_drop_halftone claiming and releasing references respectively.

Images

The fz_image structure is used to represent a generic Image object in MuPDF. It can be viewed as an encapsulation from which both a rendering of an image (as a fz_pixmap) and (often) the original source data can be retrieved.

Further discussion of `fz_image`s is deferred to in Part 2. While it is perfectly permissible for a caller to create images, in most cases they will treat them as ‘black boxes’ to just be passed around.

Buffers

The fz_buffer structure is used to represent arbitrary buffers of data. Essentially they are a representation for arbitrary blocks of bytes (in whatever encoding required), with simple functions for extending, concatenating, and writing in byte, char, utf8 and bitwise fashion.

Both the internals and API level functions of MuPDF use `fz_buffer`s extensively.

The usual reference counting behaviour applies to `fz_buffer`s, with fz_keep_buffer and fz_drop_buffer claiming and releasing references respectively.

Transforms

The fz_matrix structure is used to represent 2 dimensional matrices used for transforming points, shapes and other geometry.

The six fields of the fz_matrix structure correspond to a matrix of the form:

(ab0cd0ef1)\left( \begin{array}{ccc} a & b & 0 \\ c & d & 0 \\ e & f & 1 \end{array} \right)

Such transformation matrices can be used to represent a wide range of different operations, including translations, rotations, scales, sheers, and any combination thereof.

Typically, a matrix will be created for a specific purpose, such as a scale, or a translation. For this reason, we have dedicated construction calls.

/*
        fz_scale: Create a scaling matrix.

        The returned matrix is of the form [ sx 0 0 sy 0 0 ].

        m: Pointer to the matrix to populate

        sx, sy: Scaling factors along the X- and Y-axes. A scaling
        factor of 1.0 will not cause any scaling along the relevant
        axis.

        Returns m.

        Does not throw exceptions.
*/
fz_matrix *fz_scale(fz_matrix *m, float sx, float sy);

/*
        fz_shear: Create a shearing matrix.

        The returned matrix is of the form [ 1 sy sx 1 0 0 ].

        m: pointer to place to store returned matrix

        sx, sy: Shearing factors. A shearing factor of 0.0 will not
        cause any shearing along the relevant axis.

        Returns m.

        Does not throw exceptions.
*/
fz_matrix *fz_shear(fz_matrix *m, float sx, float sy);

/*
        fz_rotate: Create a rotation matrix.

        The returned matrix is of the form
        [ cos(deg) sin(deg) -sin(deg) cos(deg) 0 0 ].

        m: Pointer to place to store matrix

        degrees: Degrees of counter clockwise rotation. Values less
        than zero and greater than 360 are handled as expected.

        Returns m.

        Does not throw exceptions.
*/
fz_matrix *fz_rotate(fz_matrix *m, float degrees);

/*
        fz_translate: Create a translation matrix.

        The returned matrix is of the form [ 1 0 0 1 tx ty ].

        m: A place to store the created matrix.

        tx, ty: Translation distances along the X- and Y-axes. A
        translation of 0 will not cause any translation along the
        relevant axis.

        Returns m.

        Does not throw exceptions.
*/
fz_matrix *fz_translate(fz_matrix *m, float tx, float ty);

Mathematically, points are transformed by multiplying them (extended to 3 elements long). For example (x’,y’), the point given by mapping (x,y) through such a matrix is calculated as follows:

(x′y′1)=(xy1)(ab0cd0ef1)\left( \begin{array}{ccc} x' & y' & 1 \end{array} \right) % = % \left( \begin{array}{ccc} x & y & 1 \end{array} \right) % \left( \begin{array}{ccc} a & b & 0 \\ c & d & 0 \\ e & f & 1 \end{array} \right) %

There are various functions in MuPDF to perform such transformations:

/*
        fz_transform_point: Apply a transformation to a point.

        transform: Transformation matrix to apply. See fz_concat,
        fz_scale, fz_rotate and fz_translate for how to create a
        matrix.

        point: Pointer to point to update.

        Returns transform (unchanged).

        Does not throw exceptions.
*/
fz_point *fz_transform_point(fz_point *restrict point, const fz_matrix *restrict transform);
fz_point *fz_transform_point_xy(fz_point *restrict point, const fz_matrix *restrict transform, float x, float y);

Rectangles can be transformed using the following function, which allows for the fact that the image of a rectangle may ‘flip’ the rectangle (i.e. that a minimum coordinate may end up as a maximum one after translation, and vice versa):

/*
        fz_transform_rect: Apply a transform to a rectangle.

        After the four corner points of the axis-aligned rectangle
        have been transformed it may not longer be axis-aligned. So a
        new axis-aligned rectangle is created covering at least the
        area of the transformed rectangle.

        transform: Transformation matrix to apply. See fz_concat,
        fz_scale and fz_rotate for how to create a matrix.

        rect: Rectangle to be transformed. The two special cases
        fz_empty_rect and fz_infinite_rect, may be used but are
        returned unchanged as expected.

        Does not throw exceptions.
*/
fz_rect *fz_transform_rect(fz_rect *restrict rect, const fz_matrix *restrict transform);

Also, it can be useful to transform a point, ignoring the translation components of a transformation, so we have a convenience function for this:

/*
        fz_transform_vector: Apply a transformation to a vector.

        transform: Transformation matrix to apply. See fz_concat,
        fz_scale and fz_rotate for how to create a matrix. Any
        translation will be ignored.

        vector: Pointer to vector to update.

        Does not throw exceptions.
*/
fz_point *fz_transform_vector(fz_point *restrict vector, const fz_matrix *restrict transform);

Transformations can be combined by multiplying their representative matrices together. Transforming a point by applying matrix A then matrix B, will give identical results to transforming the point by AB.

MuPDF provides an API for combining matrices in this way:

/*
        fz_concat: Multiply two matrices.

        The order of the two matrices are important since matrix
        multiplication is not commutative.

        Returns result.

        Does not throw exceptions.
*/
fz_matrix *fz_concat(fz_matrix *result, const fz_matrix *left, const fz_matrix *right);

Alternatively, operations can be specifically applied to existing matrices. Because of the non-commutative nature of matrix operations, it matters whether the new operation is applied before or after the existing matrix.

For example, if you have a matrix that performs a rotation, and you wish to combine that with a translation, you must decide whether you want the translation to occur before the rotation (‘pre’) or afterwards (‘post’).

MuPDF has various API functions for such operations:

/*
        fz_pre_scale: Scale a matrix by premultiplication.

        m: Pointer to the matrix to scale

        sx, sy: Scaling factors along the X- and Y-axes. A scaling
        factor of 1.0 will not cause any scaling along the relevant
        axis.

        Returns m (updated).

        Does not throw exceptions.
*/
fz_matrix *fz_pre_scale(fz_matrix *m, float sx, float sy);

/*
        fz_post_scale: Scale a matrix by postmultiplication.

        m: Pointer to the matrix to scale

        sx, sy: Scaling factors along the X- and Y-axes. A scaling
        factor of 1.0 will not cause any scaling along the relevant
        axis.

        Returns m (updated).

        Does not throw exceptions.
*/
fz_matrix *fz_post_scale(fz_matrix *m, float sx, float sy);

/*
        fz_pre_shear: Premultiply a matrix with a shearing matrix.

        The shearing matrix is of the form [ 1 sy sx 1 0 0 ].

        m: pointer to matrix to premultiply

        sx, sy: Shearing factors. A shearing factor of 0.0 will not
        cause any shearing along the relevant axis.

        Returns m (updated).

        Does not throw exceptions.
*/
fz_matrix *fz_pre_shear(fz_matrix *m, float sx, float sy);

/*
        fz_pre_rotate: Rotate a transformation by premultiplying.

        The premultiplied matrix is of the form
        [ cos(deg) sin(deg) -sin(deg) cos(deg) 0 0 ].

        m: Pointer to matrix to premultiply.

        degrees: Degrees of counter clockwise rotation. Values less
        than zero and greater than 360 are handled as expected.

        Returns m (updated).

        Does not throw exceptions.
*/
fz_matrix *fz_pre_rotate(fz_matrix *m, float degrees);

/*
        fz_pre_translate: Translate a matrix by premultiplication.

        m: The matrix to translate

        tx, ty: Translation distances along the X- and Y-axes. A
        translation of 0 will not cause any translation along the
        relevant axis.

        Returns m.

        Does not throw exceptions.
*/
fz_matrix *fz_pre_translate(fz_matrix *m, float tx, float ty);

Finally, sometimes it is useful to find the matrix that would represent the reverse of a given transformation. This can be achieved by ‘inverting’ the matrix. This is not possible in all cases, but can be achieved for most ‘well-behaved’ transformations.

/*
        fz_invert_matrix: Create an inverse matrix.

        inverse: Place to store inverse matrix.

        matrix: Matrix to invert. A degenerate matrix, where the
        determinant is equal to zero, can not be inverted and the
        original matrix is returned instead.

        Returns inverse.

        Does not throw exceptions.
*/
fz_matrix *fz_invert_matrix(fz_matrix *inverse, const fz_matrix *matrix);

/*
        fz_try_invert_matrix: Attempt to create an inverse matrix.

        inverse: Place to store inverse matrix.

        matrix: Matrix to invert. A degenerate matrix, where the
        determinant is equal to zero, can not be inverted.

        Returns 1 if matrix is degenerate (singular), or 0 otherwise.

        Does not throw exceptions.
*/
 int fz_try_invert_matrix(fz_matrix *inverse, const fz_matrix *matrix);

Paths

Postscript (or equivalently PDF) style paths are represented using the fz_path structure. A postscript path consists of a sequence of instructions describing the movement of a ‘pen’ around a given path.

The first instruction is always a ‘move’ to a specified location. Subsequent instructions move the pen position onwards to new positions on the page, either via straight lines, or via curves described by given control points. Such instructions can either be made with the pen up or down.

Once created paths can then be rendered by MuPDF either by being filled, or by being stroked. The path itself has no knowledge of how it will be used - the details of the fill or the stroke attributes are supplied externally to this structure. A description of the exact rules used for filling and stroking are beyond the scope of this document. For more information see “The PDF Reference Manual” or “The Postscript Language Reference Manual”.

Further discussion of paths is postponed to Part 2 in . While is is entirely permissible for application callers to want to create their own paths (for passing to device functions) it’s far more typical for callers to simply treat them as ‘black boxes’ to be passed around.

Text

MuPDFs central text type is a fz_text structure. The exact definition of this structure has evolved considerably in the past to accommodate the needs of different input formats, and it is possible this will continue in future. Accordingly we have hidden the implementation behind an interface.

Nonetheless, it is worthwhile mentioning some of the design goals that have influenced the development of this area of the code.

As fz_text objects are the only text objects passed across the device interface, they need to encode several layers of information. For simple rendering devices, they need to be expressive enough to allow us to exactly render the exact specified glyphs. For text output devices, they need to be expressive enough to allow the unicode values to be extracted.

Ideally, given any input format we would like to be able to extract any output format from it (including the same format) with no loss of data. This means that our fz_text objects need to be expressive enough to represent the super-set of functionality of all input formats out there, even if we do not currently make use of all the information.

While the idea of a single representation being enough to encapsulate each glyph from the text on the page in turn is attractive, this is not the case. Indeed, it’s not even possible to trivially define the order in which glyphs will be sent!

It would be nice to think that text would be held in the source file in the order in which it should be displayed on the page, but this is frequently not the case.

The ‘logical order’ of text can be thought of as being the order in which text would be read out loud, if you were reading from the page. In many cases (such as for EPUB files), this is the order in which the information is stored within the file itself. Sadly, for other formats this is not always the case.

PDF files in particular have no particular defined ordering in which text is sent - as each glyph is individually positioned on the page, files can (and do) send them in any order they feel like. While most PDF files containing European languages will tend to send text in the expected logical ordering, there is no guarantee that this will always be the case. This likelihood gets even more remote as we start to deal with right-to-left text, top-to-bottom text, far eastern scripts, or texts in multiple different scripts or languages.

The classic case where logical order may differ noticeably from rendered order, is for ‘bidirectional’ text2. Even if the internal document representation is in logical order, the order in which the text will actually be displayed can be quite different. Consider, for instance, some source text in Hebrew. If the individual glyphs are A,B,C,D etc, then the right-to-left nature of Hebrew means that these will be displayed in the order ‘DCBA’ on the page.

If, however, we have conventional western (arabic) numerals on the page, interspersed within the Hebrew text, this is still written left-to-right. So A,B,C,D,1,2,3,4,E,F,G would appear as ‘GFE12DCBA’.

The algorithm dealing with such strings is fairly complex, and so further discussion of this for the interested reader is best redirected to the ‘Unicode Bidirectional Algorithm’ as defined in Technical Report 9 at http://unicode.org/reports/tr9.

The final dose of complexity comes from scripts that require ‘shaping’. While simple western scripts (broadly) have a direct relationship between the character sent (e.g. the letter ’A’) and the shape used to represent it on the page (e.g. the glyph ’A’), this does not hold true for all scripts.

The simplest example for this is that of a ligature. A piece of source text might contain the letter ‘f’ followed by the letter ‘i’ (perhaps in the word ‘file’). When typeset onto a page, rather than displaying the glyphs individual, a combined glyph is generally used, ‘fi’.

This concept of their being a ‘transformation’ step from the input text to the output rendered form is extended massively when dealing with non-western scripts. For Arabic and Indic scripts in particular (and Eastern scripts in general), groups of characters are frequently combined together to give increasingly complex glyphs. This process is referred to as shaping, and it is generally applied after the bidirectional algorithm has been run.

Different source formats cope with this in different ways. The text strings within a PDF file have already had the layout and shaping process applied - they are literally a list of positioned glyphs to be displayed on the page. Each glyph is identified by a ‘glyph-id’ - a simple index of the glyph within a font, with no meaning other than that. The unicode values for the original text are frequently not there at all (and when they are they require specific work to derive).

Other formats, such as EPUB, take the opposite approach, by specifying the Unicode values directly, and leaving the displaying application (i.e. MuPDF) to do the conversion to glyphs (including the ‘shaping’ operation).

To cope with these different input requirements, and to allow us to translate one format into another, we require fz_text objects to encapsulate both forms of data at the same time.

Accordingly, our fz_text object represents a block of text, including font style and position, together with both unicode and glyph data (subject to the availability of the information in the original file). Where possible we try to provide this information in logical order, though no guarantee can be made of this.

If more information is required, then details of the current implementation are included in in Part 2, otherwise just use it as a simple black box.

Shadings

One of the most powerful graphical effects within PDF and other input formats is that of Shadings. Our central type representing shadings, fz_shade is all that we have to pass details of shadings across the fz_device interface.

Consequently, we need fz_shade to be expressive enough to cope with shadings from all possible sources, and yet we would like to avoid having to reproduce the shade handling code in all devices.

Accordingly, fz_shade is defined to be expressive enough to encapsulate all the different shading representations found in PDF with the data essentially unchanged. PDF is currently the super-set of shadings found in other formats. If this changes, fz_shade will be extended as required.

Further discussion is deferred to in Part 2, as it would be unusable (though not inconceivable) for applications authors to want to author their own shadings.

Display Lists

Overview

While MuPDF is engineered to be as fast as possible at interpreting page contents, there is inevitably some overhead in converting from the documents native format to the stream of graphical operations (calls over the fz_device interface).

If you are planning to redraw the same page several times (perhaps because you are panning and zooming around a page in a viewer), then it can be advantageous to use a display List.

A display list is simply a way of packaging up a stream of graphical operations so that they can be efficiently played back, possibly with different transforms or clip rectangles.

Display lists are optimised to use as little memory as possible, but clearly are (typically) a greater user of memory than just reinterpreting the file. The big advantage of display lists, other than their speed, is that they can safely be played back without touching the underlying file. This means they can be used in other threads without having to worry about contention.

Display lists are implemented within using MuPDF using the fz_display_list type.

Creation

An empty display list can be created by the fz_new_display_list call.

/*
        fz_new_display_list: Create an empty display list.

        A display list contains drawing commands (text, images, etc.).
        Use fz_new_list_device for populating the list.

        mediabox: Bounds of the page (in points) represented by the display list.
*/
fz_display_list *fz_new_display_list(fz_context *ctx, const fz_rect *mediabox);

Once created it can be populated by creating a display list device instance that writes to it.

/*
        fz_new_list_device: Create a rendering device for a display list.

        When the device is rendering a page it will populate the
        display list with drawing commands (text, images, etc.). The
        display list can later be reused to render a page many times
        without having to re-interpret the page from the document file
        for each rendering. Once the device is no longer needed, free
        it with fz_drop_device.

        list: A display list that the list device takes ownership of.
*/
fz_device *fz_new_list_device(fz_context *ctx, fz_display_list *list);

Once you have created such a display list device, any calls made to that device (such as by calling fz_run_page or similar) will be recorded into the display list.

When you have finished writing to the display list (remembering to call fz_close_device), you dispose of the device as normal (by calling fz_drop_device). This leaves you holding the sole reference to the display list itself.

Writing to a display list is not thread safe. That is to say, do not attempt to write to a display list from more than one thread at a time. Similarly, do not attempt to read from display lists while write operations are ongoing.

Playback

To playback from a list, just call fz_run_display_list.

/*
        fz_run_display_list: (Re)-run a display list through a device.

        list: A display list, created by fz_new_display_list and
        populated with objects from a page by running fz_run_page on a
        device obtained from fz_new_list_device.

        dev: Device obtained from fz_new_*_device.

        ctm: Transform to apply to display list contents. May include
        for example scaling and rotation, see fz_scale, fz_rotate and
        fz_concat. Set to fz_identity if no transformation is desired.

        area: Only the part of the contents of the display list
        visible within this area will be considered when the list is
        run through the device. This does not imply for tile objects
        contained in the display list.

        cookie: Communication mechanism between caller and library
        running the page. Intended for multi-threaded applications,
        while single-threaded applications set cookie to NULL. The
        caller may abort an ongoing page run. Cookie also communicates
        progress information back to the caller. The fields inside
        cookie are continually updated while the page is being run.
*/
void fz_run_display_list(fz_context *ctx, fz_display_list *list, fz_device *dev, const fz_matrix *ctm, const fz_rect *area, fz_cookie *cookie);

Reference counting

In common with most other objects in MuPDF, `fz_display_list`s are reference counted. This means that once you have finished with a reference to a display list, it can safely be disposed of by calling fz_drop_display_list.

/*
        fz_drop_display_list: Drop a reference to a display list, freeing it
        if the reference count reaches zero.

        Does not throw exceptions.
*/
void fz_drop_display_list(fz_context *ctx, fz_display_list *list);

Should you wish to keep a new reference to a display list, you can generate one using fz_keep_display_list.

/*
        fz_keep_display_list: Keep a reference to a display list.

        Does not throw exceptions.
*/
fz_display_list *fz_keep_display_list(fz_context *ctx, fz_display_list *list);

In general, it is rare for you to want to make a new reference to a display list until write operations on one have finished. It is good form to avoid this.

Miscellaneous operations

There are a few other operations that can be performed efficiently on a display list. Firstly, one can request the bounds of a list.

/*
        fz_bound_display_list: Return the bounding box of the page recorded in a display list.
*/
fz_rect *fz_bound_display_list(fz_context *ctx, fz_display_list *list, fz_rect *bounds);

Note that this is the bounding box of the page that was written to the display list, not the bounding box of the contents of the list; the latter will typically (but not always) be smaller than the former due to page borders etc.

Secondly, one can create a new fz_image from a display list. This is useful for creating scalable content to embed in other document types; for instance MuPDF makes use of this to turn SVG files embedded within EPUB files (for illustrations and cover pages etc) into convenient objects for adding into the flow of text.

/*
        Create a new image from a display list.

        w, h: The conceptual width/height of the image.

        transform: The matrix that needs to be applied to the given
        list to make it render to the unit square.

        list: The display list.
*/
fz_image *fz_new_image_from_display_list(fz_context *ctx, float w, float h, fz_display_list *list);

Finally, it is possible to very quickly check if a given display list is empty or not.

/*
        Check for a display list being empty

        list: The list to check.

        Returns true if empty, false otherwise.
*/
int fz_display_list_is_empty(fz_context *ctx, const fz_display_list *list);

The Stream interface

Overview

MuPDF is designed to run in a variety of different environments. As such, this means input can come from many different sources. On desktop computers input may come as files on backing store. For web served files, input may be streamed over a network. For systems with DRM embedded, the data may need to be decoded on the fly.

Similarly, data can be encapsulated within different formats in different ways, with multiple layers of encoding.

Accordingly, MuPDF abstracts the idea of an ‘input stream’ to a reusable class, fz_stream. Many implementations of `fz_stream`s are given by default in the core library, but the abstract nature of this class allows callers to provide implementations of their own to seamlessly extend the systems capabilities as required.

Creation

The exact mechanism for creating a stream depends upon the source for that particular stream, but typically it will involve a call to a creation function, such as fz_open_file.

/*
        fz_open_file: Open the named file and wrap it in a stream.

        filename: Path to a file. On non-Windows machines the filename should
        be exactly as it would be passed to fopen(2). On Windows machines,
        the path should be UTF-8 encoded so that non-ASCII characters can be
        represented. Other platforms do the encoding as standard anyway (and
        in most cases, particularly for MacOS and Linux, the encoding they
        use is UTF-8 anyway).
*/
fz_stream *fz_open_file(fz_context *ctx, const char *filename);

Alternative functions exist to allow creating streams from C level FILE pointers:

/*
        fz_open_file: Wrap an open file descriptor in a stream.

        file: An open file descriptor supporting bidirectional
        seeking. The stream will take ownership of the file
        descriptor, so it may not be modified or closed after the call
        to fz_open_file_ptr. When the stream is closed it will also close
        the file descriptor.
*/
fz_stream *fz_open_file_ptr(fz_context *ctx, FILE *file);

from direct memory blocks:

/*
        fz_open_memory: Open a block of memory as a stream.

        data: Pointer to start of data block. Ownership of the data block is
        NOT passed in.

        len: Number of bytes in data block.

        Returns pointer to newly created stream. May throw exceptions on
        failure to allocate.
*/
fz_stream *fz_open_memory(fz_context *ctx, unsigned char *data, size_t len);

and from `fz_buffer`s:

/*
        fz_open_buffer: Open a buffer as a stream.

        buf: The buffer to open. Ownership of the buffer is NOT passed in
        (this function takes its own reference).

        Returns pointer to newly created stream. May throw exceptions on
        failure to allocate.
*/
fz_stream *fz_open_buffer(fz_context *ctx, fz_buffer *buf);

There are too many other options for creating streams to list them all here, but their use should be self evident from the header file definitions. Once created, all streams can be used in the same ways.

Usage

Reading bytes

The simplest way to read bytes from a stream is to call fz_read_byte to read the next byte from a file. Akin to the standard fgetc, this returns -1 for end of data, or the next byte available.

/*
        fz_read_byte: Read the next byte from a stream.

        stm: The stream t read from.

        Returns -1 for end of stream, or the next byte. May
        throw exceptions.
*/
int fz_read_byte(fz_context *ctx, fz_stream *stm);

To read more than 1 byte at a time, there are two different options.

Firstly, and most efficiently, bytes can be read directly from the streams underlying buffer. For a given fz_stream *stm, the current position in the stream is pointed to by stm->rp. Bytes can simply be read out, and the pointer incremented by the number read.

To do this, you must first know how many bytes there are available to be read out. This is achieved by calling fz_available. If there are no bytes already decoded and awaiting reading, this call will trigger a refill of the underlying buffer, which may take noticeable time.

/*
        fz_available: Ask how many bytes are available immediately from
        a given stream.

        stm: The stream to read from.

        max: A hint for the underlying stream; the maximum number of
        bytes that we are sure we will want to read. If you do not know
        this number, give 1.

        Returns the number of bytes immediately available between the
        read and write pointers. This number is guaranteed only to be 0
        if we have hit EOF. The number of bytes returned here need have
        no relation to max (could be larger, could be smaller).
*/
size_t fz_available(fz_context *ctx, fz_stream *stm, size_t max);

To avoid needless work, a ‘max’ value can be supplied as a hint, telling any buffer refill operation that is triggered how many bytes are actually required. Specifying a max value does not guarantee you anything about the number of bytes actually made available.

Some callers may find this awkward - the need to potentially repeatedly call until you get enough bytes to fill a buffer of the required length may be tedious. Therefore as an alternative, we provide a simpler call, fz_read.

Designed to be similar to the standard fread call, this attempts to read as many bytes as possible into a supplied data block, returning the actual number of bytes successfully read.

/*
        fz_read: Read from a stream into a given data block.

        stm: The stream to read from.

        data: The data block to read into.

        len: The length of the data block (in bytes).

        Returns the number of bytes read. May throw exceptions.
*/
size_t fz_read(fz_context *ctx, fz_stream *stm, unsigned char *data, size_t len);

Typically the only reason that fz_read will not return the requested number of bytes is if we hit the end of the stream. This implies that calls to fz_read will block until such data is ready. For streams based on ‘fast’ sources like files or memory, this is an unimportant distinction.

For streams based on (say) an http download, this might result in significant delays, and an unacceptable user experience. To alleviate this problem we have a mechanism whereby such streams can signal a temporary end of data by throwing the FZ_ERROR_TRYLATER error. See for more details.

To facilitate reading without blocking (or using buffers larger than required), fz_available can be called to find out the number of bytes that can safely be requested.

If data within a stream is not required, it can be skipped over using fz_skip:

/*
        fz_skip: Read from a stream discarding data.

        stm: The stream to read from.

        len: The number of bytes to read.

        Returns the number of bytes read. May throw exceptions.
*/
size_t fz_skip(fz_context *ctx, fz_stream *stm, size_t len);

As a special case, after a single byte is read, it can be pushed back into the stream, using fz_unread_byte:

/*
        fz_unread_byte: Unread the single last byte successfully
        read from a stream. Do not call this without having
        successfully read a byte.
*/
void fz_unread_byte(fz_context *ctx FZ_UNUSED, fz_stream *stm);

The act of reading a byte, and then, if successful pushing it back again is encapsulated in a convenience function, fz_peek_byte:

/*
        fz_peek_byte: Peek at the next byte in a stream.

        stm: The stream to peek at.

        Returns -1 for EOF, or the next byte that will be read.
*/
int fz_peek_byte(fz_context *ctx, fz_stream *stm);

Reading objects

Often, when parsing different document formats, it can be useful to read specific objects from streams, so convenience functions exist for this too. Firstly, integers of different size and endianness are catered for:

/*
        fz_read_[u]int(16|24|32|64)(_le)?

        Read a 16/32/64 bit signed/unsigned integer from stream,
        in big or little-endian byte orders.

        Throws an exception if EOF is encountered.
*/
uint16_t fz_read_uint16(fz_context *ctx, fz_stream *stm);
uint32_t fz_read_uint24(fz_context *ctx, fz_stream *stm);
uint32_t fz_read_uint32(fz_context *ctx, fz_stream *stm);
uint64_t fz_read_uint64(fz_context *ctx, fz_stream *stm);

uint16_t fz_read_uint16_le(fz_context *ctx, fz_stream *stm);
uint32_t fz_read_uint24_le(fz_context *ctx, fz_stream *stm);
uint32_t fz_read_uint32_le(fz_context *ctx, fz_stream *stm);
uint64_t fz_read_uint64_le(fz_context *ctx, fz_stream *stm);

int16_t fz_read_int16(fz_context *ctx, fz_stream *stm);
int32_t fz_read_int32(fz_context *ctx, fz_stream *stm);
int64_t fz_read_int64(fz_context *ctx, fz_stream *stm);

int16_t fz_read_int16_le(fz_context *ctx, fz_stream *stm);
int32_t fz_read_int32_le(fz_context *ctx, fz_stream *stm);
int64_t fz_read_int64_le(fz_context *ctx, fz_stream *stm);

We have functions to read both C style strings, and newline/return terminated lines:

/*
        fz_read_string: Read a null terminated string from the stream into
        a buffer of a given length. The buffer will be null terminated.
        Throws on failure (including the failure to fit the entire string
        including the terminator into the buffer).
*/
void fz_read_string(fz_context *ctx, fz_stream *stm, char *buffer, int len);

/*
        fz_read_line: Read a line from stream into the buffer until either a
        terminating newline or EOF, which it replaces with a null byte ('\0').

        Returns buf on success, and NULL when end of file occurs while no characters
        have been read.
*/
char *fz_read_line(fz_context *ctx, fz_stream *stm, char *buf, size_t max);

Reading bits

Streams (or sections of streams) can be treated as a string of bits, packed either most significant or least significant bits first.

To read from an msb packed stream, use fz_read_bits:

/*
        fz_read_bits: Read the next n bits from a stream (assumed to
        be packed most significant bit first).

        stm: The stream to read from.

        n: The number of bits to read, between 1 and 8*sizeof(int)
        inclusive.

        Returns (unsigned int)-1 for EOF, or the required number of bits.
*/
unsigned int fz_read_bits(fz_context *ctx, fz_stream *stm, int n);

Conversely, to read from a lsb packed stream, use fz_read_rbits:

/*
        fz_read_rbits: Read the next n bits from a stream (assumed to
        be packed least significant bit first).

        stm: The stream to read from.

        n: The number of bits to read, between 1 and 8*sizeof(int)
        inclusive.

        Returns (unsigned int)-1 for EOF, or the required number of bits.
*/
unsigned int fz_read_rbits(fz_context *ctx, fz_stream *stm, int n);

;

Whichever of these is used, reading n bits will return the results in the lowest n bits of the returned value.

After reading bits using these functions, if a return to reading bytewise (or objectwise) is required, then fz_sync_bits must be called.

/*
        fz_sync_bits: Called after reading bits to tell the stream
        that we are about to return to reading bytewise. Resyncs
        the stream to whole byte boundaries.
*/
void fz_sync_bits(fz_context *ctx FZ_UNUSED, fz_stream *stm);

This function skips as many bits as as required to align with a byte boundary.

Reading whole streams

As a convenience function, MuPDF provides a mechanism for reading the entire contents of a stream into a fz_buffer.

/*
        fz_read_all: Read all of a stream into a buffer.

        stm: The stream to read from

        initial: Suggested initial size for the buffer.

        Returns a buffer created from reading from the stream. May throw
        exceptions on failure to allocate.
*/
fz_buffer *fz_read_all(fz_context *ctx, fz_stream *stm, size_t initial);

This will throw an error (and hence not return any data) if an error is encountered during the decode of the stream. Sometimes it can be preferable to ‘do the best we can’ and tolerate problematic data. For such cases, we provide fz_read_best:

/*
        fz_read_best: Attempt to read a stream into a buffer. If truncated
        is NULL behaves as fz_read_all, otherwise does not throw exceptions
        in the case of failure, but instead sets a truncated flag.

        stm: The stream to read from.

        initial: Suggested initial size for the buffer.

        truncated: Flag to store success/failure indication in.

        Returns a buffer created from reading from the stream.
*/
fz_buffer *fz_read_best(fz_context *ctx, fz_stream *stm, size_t initial, int *truncated);

Seeking

Most stream operations simply advance the stream pointer as the stream is read. The current stream position can always be obtained using fz_tell (deliberately similar to the standard ftell call):

/*
        fz_tell: return the current reading position within a stream
*/
int64_t fz_tell(fz_context *ctx, fz_stream *stm);

Some streams allow you to seek within them, that is, to change the current stream pointer to a given offset. To do this, use fz_seek (deliberately similar to fseek):

/*
        fz_seek: Seek within a stream.

        stm: The stream to seek within.

        offset: The offset to seek to.

        whence: From where the offset is measured (see fseek).
*/
void fz_seek(fz_context *ctx, fz_stream *stm, int64_t offset, int whence);

In the event that a stream does not support seeking, an error will be thrown.

As fz_seek and fz_tell work at byte granularity, care should be exercised when reading streams bitwise. Always fz_sync_bits before expecting fz_tell to give you a value that you can safely fz_seek back to.

Meta data

Occasionally, it can be useful to interrogate the properties of a stream, for example the length of the stream, or whether it is coming from a progressive source (see ).

While not implemented currently, perhaps in future a particular stream user might want to interrogate information about the Mimetype of the stream, or its compression ratios.

To allow this, we have an extensible system to request Meta operations on a stream. The fz_stream_meta function allows such calls to be made, with a reason code to identify the required operation, and pointer and size parameters to identify data to be passed:

/*
        fz_stream_meta: Perform a meta call on a stream (typically to
        request meta information about a stream).

        stm: The stream to query.

        key: The meta request identifier.

        size: Meta request specific parameter - typically the size of
        the data block pointed to by ptr.

        ptr: Meta request specific parameter - typically a pointer to
        a block of data to be filled in.

        Returns -1 if this stream does not support this meta operation,
        or a meta operation specific return value.
*/
int fz_stream_meta(fz_context *ctx, fz_stream *stm, int key, int size, void *ptr);

Destruction

In common with most other MuPDF objects, `fz_stream`s are reference counted.

As such additional references can be taken using fz_keep_stream and they can be destroyed using fz_drop_stream.

Note that care must be taken not to use fz_stream objects simultaneously in more than one thread. Not only does the act of reading in one thread upset the point at which the next read will happen in another thread, no protection is provided to make operations atomic - thus the internal data can become corrupted and cause crashes.

The Output interface

Overview

In the same way as `fz_stream`s abstracts input streams, MuPDF uses a reusable class, fz_output, to abstract output streams.

Creation

The exact function to call to create an output stream depends on the specific stream required, but they generally follow a similar format. Some common examples are:

/*
        fz_new_output_with_file: Open an output stream that writes to a
        FILE *.

        file: The file to write to.

        close: non-zero if we should close the file when the fz_output
        is closed.
*/
fz_output *fz_new_output_with_file_ptr(fz_context *ctx, FILE *file, int close);

/*
        fz_new_output_with_path: Open an output stream that writes to a
        given path.

        filename: The filename to write to (specified in UTF-8).

        append: non-zero if we should append to the file, rather than
        overwriting it.
*/
fz_output *fz_new_output_with_path(fz_context *, const char *filename, int append);

/*
        fz_new_output_with_buffer: Open an output stream that appends
        to a buffer.

        buf: The buffer to append to.
*/
fz_output *fz_new_output_with_buffer(fz_context *ctx, fz_buffer *buf);

One of the most common use cases is to get an output stream that goes to stdout or stderr, and we provide convenience functions for exactly this. In addition we allow the streams for stdout and stderr to be replaced by other `fz_output`s, thus allowing redirection to be changed simply for any of our existing tools:

/*
        fz_stdout: The standard out output stream. By default
        this stream writes to stdout. This may be overridden
        using fz_set_stdout.
*/
fz_output *fz_stdout(fz_context *ctx);

/*
        fz_stderr: The standard error output stream. By default
        this stream writes to stderr. This may be overridden
        using fz_set_stderr.
*/
fz_output *fz_stderr(fz_context *ctx);

/*
        fz_set_stdout: Replace default standard output stream
        with a given stream.

        out: The new stream to use.
*/
void fz_set_stdout(fz_context *ctx, fz_output *out);

/*
        fz_set_stderr: Replace default standard error stream
        with a given stream.

        err: The new stream to use.
*/
void fz_set_stderr(fz_context *ctx, fz_output *err);

Usage

Writing bytes

Single bytes can be written to fz_output streams using fz_write_byte:

/*
        fz_write_byte: Write a single byte.

        out: stream to write to.

        x: value to write
*/
void fz_write_byte(fz_context *ctx, fz_output *out, unsigned char x);

Blocks of bytes can be written to fz_output streams using fz_write:

/*
        fz_write: Write data to output. Designed to parallel
        fwrite.

        out: Output stream to write to.

        data: Pointer to data to write.

        size: Length of data to write.
*/
void fz_write(fz_context *ctx, fz_output *out, const void *data, size_t size);

Writing objects

We have convenience functions for outputting 16 and 32bit integers in both big and little endian forms:

/*
        fz_write_int32_be: Write a big-endian 32-bit binary integer.
*/
void fz_write_int32_be(fz_context *ctx, fz_output *out, int x);

/*
        fz_write_int32_le: Write a little-endian 32-bit binary integer.
*/
void fz_write_int32_le(fz_context *ctx, fz_output *out, int x);

/*
        fz_write_int16_be: Write a big-endian 16-bit binary integer.
*/
void fz_write_int16_be(fz_context *ctx, fz_output *out, int x);

/*
        fz_write_int16_le: Write a little-endian 16-bit binary integer.
*/
void fz_write_int16_le(fz_context *ctx, fz_output *out, int x);

And a function for outputting utf-8 encoded unicode characters:

/*
        fz_write_rune: Write a UTF-8 encoded unicode character.
*/
void fz_write_rune(fz_context *ctx, fz_output *out, int rune);

Writing strings

To output printable strings, we have the simple fputc, fputs and fputrune equivalents:

/*
        fz_putc: fputc equivalent for output streams.
*/
#define fz_putc(C,O,B) fz_write_byte(C, O, B)

/*
        fz_puts: fputs equivalent for output streams.
*/
#define fz_puts(C,O,S) fz_write(C, O, (S), strlen(S))

/*
        fz_putrune: fputrune equivalent for output streams.
*/
#define fz_putrune(C,O,R) fz_write_rune(C, O, R)

We also provide a family of enhanced output functions, patterned after fprintf:

/*
        fz_vsnprintf: Our customised vsnprintf routine.
        Takes %c, %d, %o, %s, %u, %x, as usual.
        Modifiers are not supported except for zero-padding
        ints (e.g. %02d, %03o, %04x, etc).
        %f and %g both output in "as short as possible hopefully lossless
        non-exponent" form, see fz_ftoa for specifics.
        %C outputs a utf8 encoded int.
        %M outputs a fz_matrix*.
        %R outputs a fz_rect*.
        %P outputs a fz_point*.
        %q and %( output escaped strings in C/PDF syntax.
        %ll{d,u,x} indicates that the values are 64bit.
        %z{d,u,x} indicates that the value is a size_t.
*/
size_t fz_vsnprintf(char *buffer, size_t space, const char *fmt, va_list args);

/*
        fz_snprintf: The non va_list equivalent of fz_vsnprintf.
*/
size_t fz_snprintf(char *buffer, size_t space, const char *fmt, ...);

/*
        fz_printf: fprintf equivalent for output streams. See fz_snprintf.
*/
void fz_printf(fz_context *ctx, fz_output *out, const char *fmt, ...);

/*
        fz_vprintf: vfprintf equivalent for output streams. See fz_vsnprintf.
*/
void fz_vprintf(fz_context *ctx, fz_output *out, const char *fmt, va_list ap);

Seeking

As with `fz_stream`s, `fz_output`s normally move linearly, but in special cases, can be seekable.

/*
        fz_seek_output: Seek to the specified file position. See fseek
        for arguments.

        Throw an error on unseekable outputs.
*/
void fz_seek_output(fz_context *ctx, fz_output *out, fz_off_t off, int whence);

Unlike `fz_stream`s, which support fz_tell in all cases, `fz_output`s can only fz_tell_output if they are seekable:

/*
        fz_tell_output: Return the current file position. Throw an error
        on unseekable outputs.
*/
fz_off_t fz_tell_output(fz_context *ctx, fz_output *out);

Rendered Output Formats

Overview

MuPDFs built in renderer (see ) produces in-memory arrays of contone values for areas of document pages. The MuPDF library includes routines to be able to output these areas to a number of different output formats.

Typically these devices all follow a similar pattern, enabling either full page or banded rendering to be performed according to the requirements of the particular application.

For a given format XXX, there tend to be 3 functions defined:

void fz_save_pixmap_as_XXX(fz_context *ctx, fz_pixmap *pixmap, char *filename);

void fz_write_pixmap_as_XXX(fz_context *ctx, fz_output *out, fz_pixmap *pixmap);

fz_band_writer *fz_new_XXX_band_writer(fz_context *ctx, fz_output *out);

The first function outputs a pixmap to a utf-8 encoded filename as a XXX formatted file. If the pixmap is not in a suitable colorspace/alpha configuration, then an exception will be thrown.

The second function does the same thing, but to a given fz_output rather than to a named file. The use of a fz_output allows for writing to memory buffers, or even potentially to encrypt or compress further as the write progresses.

The third function returns a fz_band_writer to do the same thing.

Band Writers

The purpose of the fz_band_writer mechanism is to allow banded rendering; rather than having to allocate a pixmap large enough to hold the entire page at once, we instead render bands across the page and feed those to the fz_band_writer which assembles them into a properly formed XXX format output stream.

Typically a band writer is created using a call such as fz_new_png_band_writer:

/*
        fz_new_png_band_writer: Obtain a fz_band_writer instance
        for producing PNG output.
*/
fz_band_writer *fz_new_png_band_writer(fz_context *ctx, fz_output *out);

The page output starts by calling fz_write_header. This both configures the band writer for the type of data that is being sent, and triggers the output of the file header:

/*
        fz_write_header: Cause a band writer to write the header for
        a banded image with the given properties/dimensions etc. This
        also configures the bandwriter for the format of the data to be
        passed in future calls.

        w, h: Width and Height of the entire page.

        n: Number of components (including spots and alphas).

        alpha: Number of alpha components.

        xres, yres: X and Y resolutions in dpi.

        pagenum: Page number

        cs: Colorspace (NULL for bitmaps)

        seps: Separation details (or NULL).

        Throws exception if incompatible data format.
*/
void fz_write_header(fz_context *ctx, fz_band_writer *writer, int w, int h, int n, int alpha, int xres, int yres, int pagenum, const fz_colorspace *cs, fz_separations *seps);

This has the effect of setting the size and format of the data for the complete image. The caller then proceeds to render the page in horizontal strips from the top to the bottom, and pass them in to fz_write_band:

/*
        fz_write_band: Cause a band writer to write the next band
        of data for an image.

        stride: The byte offset from the first byte of the data
        for a pixel to the first byte of the data for the same pixel
        on the row below.

        band_height: The number of lines in this band.

        samples: Pointer to first byte of the data.
*/
void fz_write_band(fz_context *ctx, fz_band_writer *writer, int stride, int band_height, const unsigned char *samples);

The band writer keeps track of how much data has been written, and when an entire page has been sent, it writes out any image trailer required.

For formats that can accommodate multiple pages, a new call to fz_write_header will start the process again. Otherwise (or after the final image), the band writer can be neatly discarded by calling:

void fz_drop_band_writer(fz_context *ctx, fz_band_writer *writer);

PNM

The simplest output format supported is that of PNM. The pixmap can be greyscale, or RGB, with or without alpha (though the alpha plane is always ignored on writing).

/*
        fz_save_pixmap_as_pnm: Save a pixmap as a PNM image file.
*/
void fz_save_pixmap_as_pnm(fz_context *ctx, fz_pixmap *pixmap, char *filename);

void fz_write_pixmap_as_pnm(fz_context *ctx, fz_output *out, fz_pixmap *pixmap);

fz_band_writer *fz_new_pnm_band_writer(fz_context *ctx, fz_output *out);

PAM

Related to PNM we have PAM. The pixmap formats here can be greyscale, RGB or CMYK, with or without alpha (and the alpha plane is written to the file). The TUPLTYPE in the image header reflects the color and alpha configuration, though not all readers support all variants.

/*
        fz_save_pixmap_as_pam: Save a pixmap as a PAM image file.
*/
void fz_save_pixmap_as_pam(fz_context *ctx, fz_pixmap *pixmap, char *filename);

void fz_write_pixmap_as_pam(fz_context *ctx, fz_output *out, fz_pixmap *pixmap);

fz_band_writer *fz_new_pam_band_writer(fz_context *ctx, fz_output *out);

PBM

Bitmaps suitable for output to the PBM format are generated by drawing to greyscale contone (with no alpha), and then halftoning down to monochrome.

/*
        fz_save_bitmap_as_pbm: Save a bitmap as a PBM image file.
*/
void fz_save_bitmap_as_pbm(fz_context *ctx, fz_bitmap *bitmap, char *filename);

void fz_write_bitmap_as_pbm(fz_context *ctx, fz_output *out, fz_bitmap *bitmap);

fz_band_writer *fz_new_pbm_band_writer(fz_context *ctx, fz_output *out);

PKM

Bitmaps suitable for output to the PKM format are generated by drawing to CMYK contone (with no alpha), and then halftoning down to give 1bpc cmyk.

/*
        fz_save_bitmap_as_pkm: Save a 4bpp cmyk bitmap as a PAM image file.
*/
void fz_save_bitmap_as_pkm(fz_context *ctx, fz_bitmap *bitmap, char *filename);

void fz_write_bitmap_as_pkm(fz_context *ctx, fz_output *out, fz_bitmap *bitmap);

fz_band_writer *fz_new_pkm_band_writer(fz_context *ctx, fz_output *out);

PNG

The PNG format will accept either greyscale or RGB pixmaps, with or without alpha. As a special case, alpha only pixmaps are accepted and written as greyscale.

/*
        fz_save_pixmap_as_png: Save a pixmap as a PNG image file.
*/
void fz_save_pixmap_as_png(fz_context *ctx, fz_pixmap *pixmap, const char *filename);

/*
        Write a pixmap to an output stream in PNG format.
*/
void fz_write_pixmap_as_png(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap);

/*
        fz_new_png_band_writer: Obtain a fz_band_writer instance
        for producing PNG output.
*/
fz_band_writer *fz_new_png_band_writer(fz_context *ctx, fz_output *out);

Because PNG is such a useful and widely used format, we have another couple of functions. These take either a fz_image or a fz_pixmap and produce a fz_buffer containing a PNG encoded version. This is very useful when converting between document formats as we can frequently use a PNG version of an image as a replacement for other image formats that may not be supported.

/*
        Create a new buffer containing the image/pixmap in PNG format.
*/
fz_buffer *fz_new_buffer_from_image_as_png(fz_context *ctx, fz_image *image);
fz_buffer *fz_new_buffer_from_pixmap_as_png(fz_context *ctx, fz_pixmap *pixmap);

PSD

The PSD format is used by photoshop. It is especially useful as it allows the inclusion of large numbers of spot colors, together with their ‘equivalent color’ information. This means it is the format of choice for outputting color correct renders with spot colors and correct overprint processing. PSD files can be written with grayscale, RGB or CMYK pixmaps, plus optional spots, with or without alpha.

/*
        fz_save_pixmap_as_psd: Save a pixmap as a PSD image file.
*/
void fz_save_pixmap_as_psd(fz_context *ctx, fz_pixmap *pixmap, const char *filename);

/*
        Write a pixmap to an output stream in PSD format.
*/
void fz_write_pixmap_as_psd(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap);

/*
        fz_new_psd_band_writer: Obtain a fz_band_writer instance
        for producing PSD output.
*/
fz_band_writer *fz_new_psd_band_writer(fz_context *ctx, fz_output *out);

PWG/CUPS

The PWG format is intended to encapsulate output for printers. As such there are many values that can be set in the headers. To allow for this, we expose these fields as an options structure that can be fed into the output functions.

typedef struct fz_pwg_options_s fz_pwg_options;

struct fz_pwg_options_s
{
        /* These are not interpreted as CStrings by the writing code, but
         * are rather copied directly out. */
        char media_class[64];
        char media_color[64];
        char media_type[64];
        char output_type[64];

        unsigned int advance_distance;
        int advance_media;
        int collate;
        int cut_media;
        int duplex;
        int insert_sheet;
        int jog;
        int leading_edge;
        int manual_feed;
        unsigned int media_position;
        unsigned int media_weight;
        int mirror_print;
        int negative_print;
        unsigned int num_copies;
        int orientation;
        int output_face_up;
        unsigned int PageSize[2];
        int separations;
        int tray_switch;
        int tumble;

        int media_type_num;
        int compression;
        unsigned int row_count;
        unsigned int row_feed;
        unsigned int row_step;

        /* These are not interpreted as CStrings by the writing code, but
         * are rather copied directly out. */
        char rendering_intent[64];
        char page_size_name[64];
};

No documentation for these fields is given here - for more information see the PWG specification.

There are 2 sets of output functions available for PWG, those that take `fz_pixmap`s (for contone output) and those that take `fz_bitmap`s (for halftoned output).

PWG files are structured as a header (to identify the format), followed by a stream of pages (images). Those functions that save (or write) a complete file include the file header as part of their output. If the option is used to append to a file, then the header is not added, as we presume we are appending new page information to the end of an existing file.

In circumstances when the header is not output automatically (such as when using the band writer) the header output must be triggered manually, by calling:

/*
        Output the file header to a pwg stream, ready for pages to follow it.
*/
void fz_write_pwg_file_header(fz_context *ctx, fz_output *out);

Contone

The PWG writer can accept pixmaps in greyscale, RGB and CMYK format, with no alpha planes.

PWG files can be saved to a file using:

/*
        fz_save_pixmap_as_pwg: Save a pixmap as a pwg

        filename: The filename to save as (including extension).

        append: If non-zero, then append a new page to existing file.

        pwg: NULL, or a pointer to an options structure (initialised to zero
        before being filled in, for future expansion).
*/
void fz_save_pixmap_as_pwg(fz_context *ctx, fz_pixmap *pixmap, char *filename, int append, const fz_pwg_options *pwg);

The file header will only be sent in the case where we are not appending to an existing file.

Alternatively, pages may be sent to an output stream. Two functions exist to do this. The first always sends a complete PWG file (including header):

/*
        Output a pixmap to an output stream as a pwg raster.
*/
void fz_write_pixmap_as_pwg(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap, const fz_pwg_options *pwg);

The second sends just the page data, and is therefore suitable for sending the second or subsequent pages in a file. Alternatively, the header can be sent manually, and then this function can be used for all the pages in a file.

/*
        Output a page to a pwg stream to follow a header, or other pages.
*/
void fz_write_pixmap_as_pwg_page(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap, const fz_pwg_options *pwg);

Finally, a standard band writer can be used:

/*
        fz_new_pwg_band_writer: Generate a new band writer for
        contone PWG format images.
*/
fz_band_writer *fz_new_pwg_band_writer(fz_context *ctx, fz_output *out, const fz_pwg_options *pwg);

In all cases, a NULL value can be sent for the fz_pwg_options field, in which case default values will be used.

Mono

The monochrome version of the PWG writer parallels the contone one. It can accept monochrome bitmaps only.

PWG files can be saved to a file using:

/*
        fz_save_bitmap_as_pwg: Save a bitmap as a pwg

        filename: The filename to save as (including extension).

        append: If non-zero, then append a new page to existing file.

        pwg: NULL, or a pointer to an options structure (initialised to zero
        before being filled in, for future expansion).
*/
void fz_save_bitmap_as_pwg(fz_context *ctx, fz_bitmap *bitmap, char *filename, int append, const fz_pwg_options *pwg);

The file header will only be sent in the case where we are not appending to an existing file.

Alternatively, pages may be sent to an output stream. Two functions exist to do this. The first always sends a complete PWG file (including header):

/*
        Output a bitmap to an output stream as a pwg raster.
*/
void fz_write_bitmap_as_pwg(fz_context *ctx, fz_output *out, const fz_bitmap *bitmap, const fz_pwg_options *pwg);

The second sends just the page data, and is therefore suitable for sending the second or subsequent pages in a file. Alternatively, the header can be sent manually, and then this function can be used for all the pages in a file.

/*
        Output a bitmap page to a pwg stream to follow a header, or other pages.
*/
void fz_write_bitmap_as_pwg_page(fz_context *ctx, fz_output *out, const fz_bitmap *bitmap, const fz_pwg_options *pwg);

Finally, a standard band writer can be used:

/*
        fz_new_mono_pwg_band_writer: Generate a new band writer for
        PWG format images.
*/
fz_band_writer *fz_new_mono_pwg_band_writer(fz_context *ctx, fz_output *out, const fz_pwg_options *pwg);

In all cases, a NULL value can be sent for the fz_pwg_options field, in which case default values will be used.

TGA

The TGA writer can accept pixmaps in greyscale, RGB and BGR formats, with and without alpha.

/*
        fz_save_pixmap_as_tga: Save a pixmap as a TGA image file.
        Can accept RGB, BGR or Grayscale pixmaps, with or without
        alpha.
*/
void fz_save_pixmap_as_tga(fz_context *ctx, fz_pixmap *pixmap, const char *filename);

/*
        Write a pixmap to an output stream in TGA format.
        Can accept RGB, BGR or Grayscale pixmaps, with or without
        alpha.
*/
void fz_write_pixmap_as_tga(fz_context *ctx, fz_output *out, fz_pixmap *pixmap);

/*
        fz_new_tga_band_writer: Generate a new band writer for TGA
        format images. Note that image must be generated vertically
        flipped for use with this writer!

        Can accept RGB, BGR or Grayscale pixmaps, with or without
        alpha.

        is_bgr: True, if the image is generated in bgr format.
*/
fz_band_writer *fz_new_tga_band_writer(fz_context *ctx, fz_output *out, int is_bgr);

PCL

PCL is not a standard image format, rather it is a page description language for printers. Unfortunately, the exact implementation of PCL varies from printer to printer, so it can be necessary to tweak the output according to the exact intended destination.

Accordingly, we have a pcl_options structure to allow this to happen. To use this, you simply define a pcl_options structure on the stack:

pcl_options options = { 0 };

Next you populate those options. Typically this is done by requesting a preset from our current defined set.

/*
         fz_pcl_preset: Retrieve a set of fz_pcl_options suitable for a given
         preset.

         opts: pointer to options structure to populate.

         preset: Preset to fetch. Currently defined presets include:
                ljet4   HP DeskJet
                dj500   HP DeskJet 500
                fs600   Kyocera FS-600
                lj              HP LaserJet, HP LaserJet Plus
                lj2             HP LaserJet IIp, HP LaserJet IId
                lj3             HP LaserJet III
                lj3d    HP LaserJet IIId
                lj4             HP LaserJet 4
                lj4pl   HP LaserJet 4 PL
                lj4d    HP LaserJet 4d
                lp2563b HP 2563B line printer
                oce9050 Oce 9050 Line printer

        Throws exception on unknown preset.
*/
void fz_pcl_preset(fz_context *ctx, fz_pcl_options *opts, const char *preset);

These options can then be tweaked further using fz_pcl_option:

/*
        fz_pcl_option: Set a given PCL option to a given value in the
        supplied options structure.

        opts: The option structure to modify,

        option: The option to change.

        val: The value that the option should be set to. Acceptable ranges of
        values depend on the option in question.

        Throws an exception on attempt to set an unknown option, or an
        illegal value.

        Currently defined options/values are as follows:

                spacing,0                               No vertical spacing capability
                spacing,1                               PCL 3 spacing (<ESC>*p+<n>Y)
                spacing,2                               PCL 4 spacing (<ESC>*b<n>Y)
                spacing,3                               PCL 5 spacing (<ESC>*b<n>Y and clear seed
                                                                row)
                mode2,0 or 1                    Disable/Enable mode 2 graphics compression
                mode3,0 or 1                    Disable/Enable mode 3 graphics compression
                mode3,0 or 1                    Disable/Enable mode 3 graphics compression
                eog_reset,0 or 1                End of graphics (<ESC>*rB) resets all
                                                                parameters
                has_duplex,0 or 1               Duplex supported (<ESC>&l<duplex>S)
                has_papersize,0 or 1    Papersize setting supported
                                                                (<ESC>&l<sizecode>A)
                has_copies,0 or 1               Number of copies supported
                                                                (<ESC>&l<copies>X)
                is_ljet4pjl,0 or 1              Disable/Enable HP 4PJL model-specific output
                is_oce9050,0 or 1               Disable/Enable Oce 9050 model-specific
                                                                output
*/
void fz_pcl_option(fz_context *ctx, fz_pcl_options *opts, const char *option, int val);

Color

Color PCL output can be generated from RGB pixmaps with alpha (though the alpha is ignored) using:

void fz_save_pixmap_as_pcl(fz_context *ctx, fz_pixmap *pixmap, char *filename, int append, const fz_pcl_options *pcl);

void fz_write_pixmap_as_pcl(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap, const fz_pcl_options *pcl);

fz_band_writer *fz_new_color_pcl_band_writer(fz_context *ctx, fz_output *out, const fz_pcl_options *options);

This is 24bpp RGB output, relying on the printers ability to dither. Blank lines are skipped, repeated lines are coded efficiently, and other lines are coded using deltas. Nonetheless file sizes can still be large with this output method.

Mono

Monochrome PCL output can be generated from monochrome bitmaps. These are generated by rendering to greyscale (no alpha) pixmaps and dithering down. The functions in question are:

fz_band_writer *fz_new_mono_pcl_band_writer(fz_context *ctx, fz_output *out, const fz_pcl_options *options);

void fz_write_bitmap_as_pcl(fz_context *ctx, fz_output *out, const fz_bitmap *bitmap, const fz_pcl_options *pcl);

void fz_save_bitmap_as_pcl(fz_context *ctx, fz_bitmap *bitmap, char *filename, int append, const fz_pcl_options *pcl);

Postscript

Postscript output is currently done as image output rather than high-level objects.

Pixmaps suitable for PS image output are greyscale, RGB or CMYK with no alpha.

void fz_write_pixmap_as_ps(fz_context *ctx, fz_output *out, const fz_pixmap *pixmap);

void fz_save_pixmap_as_ps(fz_context *ctx, fz_pixmap *pixmap, char *filename, int append);

fz_band_writer *fz_new_ps_band_writer(fz_context *ctx, fz_output *out);

Postscript requires file level headers and trailers, over and above that produced by the band writer itself. These can be generated using the following functions:

void fz_write_ps_file_header(fz_context *ctx, fz_output *out);

void fz_write_ps_file_trailer(fz_context *ctx, fz_output *out, int pages);

These are not generated by the band writer itself to allow a single output stream to be generated containing many pages, but a single file header and trailer.

The Document Writer interface

Usage

As well as opening existing documents, MuPDF contains functions to allow the easy creation of new documents. The most general form of this functionality takes the form of the fz_document_writer interface.

A document writer is obtained by calling a generation function. The most general purpose one is:

/*
        fz_new_document_writer: Create a new fz_document_writer, for a
        file of the given type.

        path: The document name to write (or NULL for default)

        format: Which format to write (currently cbz, pdf, pam, pbm,
        pgm, pkm, png, ppm, pnm, svg, tga)

        options: NULL, or pointer to comma separated string to control
        file generation.
*/
fz_document_writer *fz_new_document_writer(fz_context *ctx, const char *path, const char *format, const char *options);

Alternatively, direct calls to generate specific document writers can be used, such as:

fz_document_writer *fz_new_cbz_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pdf_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_svg_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_png_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_tga_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pam_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pnm_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pgm_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_ppm_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pbm_pixmap_writer(fz_context *ctx, const char *path, const char *options);
fz_document_writer *fz_new_pkm_pixmap_writer(fz_context *ctx, const char *path, const char *options);

Once a fz_document_writer has been created, pages can be written to the document one at a time. The process is started by calling fz_begin_page:

/*
        fz_begin_page: Called to start the process of writing a page to
        a document.

        mediabox: page size rectangle in points.

        Returns a fz_device to write page contents to.
*/
fz_device *fz_begin_page(fz_context *ctx, fz_document_writer *wri, const fz_rect *mediabox);

This function returns a fz_device pointer that should be used to write the page contents to. This can be done by making a sequence of normal device calls (see ) to paint the page with its content. One of the most common ways of doing this is by calling fz_run_page_contents on another open document. This therefore offers a quick mechanism for converting documents from one format to another.

Once the page contents have all been written, the page is finalized by calling fz_end_page:

/*
        fz_end_page: Called to end the process of writing a page to a
        document.
*/
void fz_end_page(fz_context *ctx, fz_document_writer *wri);

At this point, many formats will allow more pages to be written, simply by repeating the fz_begin_page, output, fz_end_page loop.

When all the pages have been written, the produced document can be finalized by calling fz_close_document_writer:

/*
        fz_close_document_writer: Called to end the process of writing
        pages to a document.

        This writes any file level trailers required. After this
        completes successfully the file is up to date and complete.
*/
void fz_close_document_writer(fz_context *ctx, fz_document_writer *wri);

Finally, the document writer itself can be freed in the usual fashion by calling fz_drop_document_writer:

/*
        fz_drop_document_writer: Called to discard a fz_document_writer.
        This may be called at any time during the process to release all
        the resources owned by the writer.

        Calling drop without having previously called drop may leave
        the file in an inconsistent state.
*/
void fz_drop_document_writer(fz_context *ctx, fz_document_writer *wri);

Implementation

Support for a new type of document writer requires a new structure, derived from fz_document_writer:

typedef struct
{
        fz_document_writer_begin_page_fn *begin_page;
        fz_document_writer_end_page_fn *end_page;
        fz_document_writer_close_writer_fn *close_writer;
        fz_document_writer_drop_writer_fn *drop_writer;
        fz_device *dev;
} fz_document_writer;

For instance:

typedef struct
{
        fz_document_writer super;
        <foo specific fields>
} foo_document_writer;

A generator function should be defined to return such an instance, perhaps:

fz_document_writer *fz_new_foo_document_writer(fz_context *ctx, const char *path, <foo specific params>) {
    foo_document_writer *foo = fz_new_derived_document_writer(ctx, foo_document_writer, foo_begin_page, foo_end_page, foo_close, foo_drop);

    <initialise foo specific fields>

    return &foo->super;
}

This uses a friendly macro that allocates a structure of the required size, initialises the function pointers as required, and zeroes the extra values in the structure.

/*
        fz_new_document_writer_of_size: Internal function to allocate a
        block for a derived document_writer structure, with the base
        structure's function pointers populated correctly, and the extra
        space zero initialised.
*/
fz_document_writer *fz_new_document_writer_of_size(fz_context *ctx, size_t size, fz_document_writer_begin_page_fn *begin_page,
        fz_document_writer_end_page_fn *end_page, fz_document_writer_close_writer_fn *close, fz_document_writer_drop_writer_fn *drop);

#define fz_new_derived_document_writer(CTX,TYPE,
                                           BEGIN_PAGE,END_PAGE,CLOSE,DROP)\
        ((TYPE *)Memento_label(\
                      fz_new_document_writer_of_size(CTX,sizeof(TYPE),\
                                                                                                        BEGIN_PAGE,END_PAGE,\
                                                                                                        CLOSE,DROP),#TYPE))

The actual work for the document writer is done in the functions that are passed to fz_new_derived_document_writer. In the example above these were foo_begin_page, foo_end_page, foo_close, and foo_drop. These have the following 4 types respectively.

/*
        fz_document_writer_begin_page_fn: Function type to start
        the process of writing a page to a document.

        mediabox: page size rectangle in points.

        Returns a fz_device to write page contents to.
*/
typedef fz_device *(fz_document_writer_begin_page_fn)(fz_context *ctx, fz_document_writer *wri, const fz_rect *mediabox);

/*
        fz_document_writer_end_page_fn: Function type to end the
        process of writing a page to a document.

        dev: The device created by the begin_page function.
*/
typedef void (fz_document_writer_end_page_fn)(fz_context *ctx, fz_document_writer *wri, fz_device *dev);

/*
        fz_document_writer_close_writer_fn: Function type to end
        the process of writing pages to a document.

        This writes any file level trailers required. After this
        completes successfully the file is up to date and complete.
*/
typedef void (fz_document_writer_close_writer_fn)(fz_context *ctx, fz_document_writer *wri);

/*
        fz_document_writer_drop_writer_fn: Function type to discard
        an fz_document_writer. This may be called at any time during
        the process to release all the resources owned by the writer.

        Calling drop without having previously called close may leave
        the file in an inconsistent state.
*/
typedef void (fz_document_writer_drop_writer_fn)(fz_context *ctx, fz_document_writer *wri);

Once defined, if this is intended to be a generally useful document writer, it should probably be hooked into fz_new_document_writer, where it can be selected by appropriate format and options strings.

Stories

Overview

Stories provide a way to easily layout styled content for use with devices, such as those offered by Document Writers (see ).

The concept of a story comes from desktop publishing, which in turn (I believe) gets it from newspapers. If you consider a traditional newspaper layout, it will consist of various news articles (stories) that are laid out into multiple columns, possibly across multiple pages.

Accordingly, MuPDF uses a story to represent a flow of text with styling information. The user of the story can then supply a sequence of rectangles into which the story will be laid out, and the positioned text can then be drawn to an output device. This keeps the concept of the text itself (the story) to be separated from the areas into which the text should be flowed (the layout).

Anyone who has ever tried to put together such a document will spot a problem here; you can’t know how exactly how much space is going to be required for a given story without trying to lay it out, and thus you can’t be sure your proposed layout is perfect without having tried it at least once.

Accordingly, MuPDF allows stories to be positioned multiple times with different layouts, and will feed back information about how much of each rectangle has been used so the caller can refine the layout and try again.

If this all sounds complex, don’t worry. It’s really quite simple, and many applications will never need to worry about much of it.

Styled Text

So, stories are to be created from styled text. Clearly we’ll want to allow for common constructs such as as headings, paragraphs and lists, and we’ll want to have ways to specify styles like fonts, colours, text alignment and spacing.

Rather than reinventing the wheel and coming up with our own custom language to specify styled text, MuPDF uses HTML. It has the benefit of being well understood, well documented, and well-provided with a range of tutorials. Almost every programmer will have a least a rudimentary understanding of the basics of HTML.

The HTML engine in MuPDF is very simple; basically HTML4 with CSS2. Javascript is not supported, nor is absolute positioning of content. Table support is present, but rudimentary at the current time. Think of HTML as being simple styled content, and you won’t go far wrong.

Consequently, stories are created from a block of HTML with an optional block of CSS.

Programmatic content

Another common use case for stories is where content is generated by a program; perhaps pulling information from a database and laying it out for human reading.

As an example, consider that we are generating a receipt for a list of purchases. The story content might be a list with an item for each of the things purchased.

The caller could construct an HTML document textually, but this can be time consuming, awkward and error prone. In the world of the web, such document manipulations would generally be done using the HTML DOM.

Accordingly, once a story has been created, the user can retrieve a pointer to a DOM-like view on the document. This allows sections of the document to be searched for, cloned, edited, rewritten, and otherwise manipulated.

This DOM-like view persists until the first layout call (or until the warnings are retrieved for the document), at which point the story content is fixed.

Life cycle

The typical life cycle for a story is as follows:

  1. Create a story.

  2. Optionally: Get the DOM pointer for that story and manipulate the document content with it.

  3. Optionally: Get the warnings associated with parsing the story.

  4. Place the story into a rectangle. This returns the bounds of the content that fitted, and whether there is more that didn’t fit.

  5. If the fit wasn’t good enough, the caller can repeat step 4 multiple times until they are happy.

  6. Draw the laid out chunk of story to the output device (which can be NULL).

  7. If stage 4 said there was more content still to be fitted, then the caller can jump back to stage 4 to lay out the rest of the story, automatically continuing where the previous attempt to place the story left off.

  8. At any point during this process, the caller can reset the story and jump back to stage 3 to start placing the story from the start again. This allows ’dry runs’ to take place with a dummy (null) output device where the number/size/positions of rectangles can be tweaked until the caller is happy with the result.

  9. Finally, the story is dropped.

Creating a story

A story is represented as usual by a fz_story structure:

typedef struct fz_story_s fz_story;

These are created from a buffer containing utf-8 encoded HTML, an optional C-string containing any CSS styling required, and the default ’em’ size to use for the text.

/*
        Create a text story using styled html.
*/
fz_story *fz_new_story(fz_context *ctx, fz_buffer *buf, const char *user_css, float em, fz_archive *archive);

Passing a NULL buffer will be treated as an empty document. This may be useful for creating empty stories that can be added to using the DOM.

The user_css string allows extra CSS definitions to be supplied over and above the default CSS built into MuPDF. Again, NULL can be used to indicate the empty string.

See for notes on recommended styles to use.

The archive pointer is used as a source for loading data for any images and fonts that may be referenced in the document. For simple HTML documents that use only inbuilt fonts and no images (or images that use data: URLs) then this can be NULL.

See for more details.

If a non-NULL pointer is passed in, the caller retains its reference, and hence must still call fz_drop_archive. This can happen at any point after calling fz_new_story() as a reference to the archive will be taken to ensure the underlying story is not freed until the story has finished with it.

Document manipulations

Once created, the HTML can be programmatically manipulated using a DOM. The first step is to get a fz_xml pointer to the top node of the document:

/*
        Get a borrowed reference to the DOM document pointer for this
        story. Do not destroy this reference, it will be destroyed
        on the first call to fz_place_story or fz_story_warnings.
*/
fz_xml *fz_story_document(fz_context *ctx, fz_story *story);

Node contents

Every node in the DOM can either be a tagged node, or a text node. Only tagged nodes can have attributes (see ) and children.

To find out which, call:

/**
        Return tag of XML node. Return NULL for text nodes.
*/
char *fz_xml_tag(fz_xml *item);

This returns a pointer to borrowed memory - do not free it.

If that returns NULL then it is a text node. The textual content for a node can be retrieved using:

/**
        Return the text content of an XML node.
        Return NULL if the node is a tag.
*/
char *fz_xml_text(fz_xml *item);

Again the pointer returned is to borrowed memory - do not free it.

Navigation

Armed with that document pointer, we can navigate around the xml nodes that make up the structure of the document. The most basic operations are:

/**
        Return a borrowed reference to the first child of a node,
        or NULL if there isn't one.
*/
fz_xml *fz_dom_first_child(fz_context *ctx, fz_xml *elt);

/**
        Return a borrowed reference to the parent of a node,
        or NULL if there isn't one.
*/
fz_xml *fz_dom_parent(fz_context *ctx, fz_xml *elt);

/**
        Return a borrowed reference to the next sibling of a node,
        or NULL if there isn't one.
*/
fz_xml *fz_dom_next(fz_context *ctx, fz_xml *elt);

/**
        Return a borrowed reference to the previous sibling of a node,
        or NULL if there isn't one.
*/
fz_xml *fz_dom_previous(fz_context *ctx, fz_xml *elt);

Each of these will return NULL if there is no such node. Note that the pointers returned from each of these are borrowed references, so should not be dropped. Their lifespan is the same as that of the original document pointer (i.e. they are guaranteed to exist only until the warnings are retrieved, or the first placement operation is attempted).

Note that only tagged nodes have children; attempting to ask for the first child of a text node will always return NULL.

If you intend to walk the xml tree using these, it is generally best to start at a known position. The most useful node (other than the top level document pointer itself) is the node that represents the BODY tag from the HTML. This can be retrieved using:

/**
        Return a borrowed reference for the 'body' element of
        the given DOM.
*/
fz_xml *fz_dom_body(fz_context *ctx, fz_xml *dom);

Searching

While manually walking the tree may be acceptable for many applications, as documents grow in complexity this can be a time consuming business. Accordingly, we can also traverse the tree by searching for matching nodes. There are various sets of functions for such searches depending on exactly where you want to search.

The first, and simplest set of functions allow you to search the siblings of a node looking for specific tags:

/**
        Search the siblings of XML nodes starting with item looking for
        the first with the given tag.

        Return NULL if none found.
*/
fz_xml *fz_xml_find(fz_xml *item, const char *tag);

Once you have found such a tag, the following call can be used to find second (and subsequent) matches:

/**
        Search the siblings of XML nodes starting with the first sibling
        of item looking for the first with the given tag.

        Return NULL if none found.
*/
fz_xml *fz_xml_find_next(fz_xml *item, const char *tag);

The final function in this set is a convenience wrapper for starting the search from the first child of a node.

/**
        Search the siblings of XML nodes starting with the first child
        of item looking for the first with the given tag.

        Return NULL if none found.
*/
fz_xml *fz_xml_find_down(fz_xml *item, const char *tag);

The next set of search functions perform the same 3 tasks as the above ones, but with the ability to search for nodes with matching attributes. If tag is NULL then all nodes will be considered, otherwise the search will be restricted to only nodes with matching tags.

/**
        Search the siblings of XML nodes starting with item looking for
        the first with the given tag (or any tag if tag is NULL), and
        with a matching attribute.

        Return NULL if none found.
*/
fz_xml *fz_xml_find_match(fz_xml *item, const char *tag, const char *att, const char *match);

/**
        Search the siblings of XML nodes starting with the first sibling
        of item looking for the first with the given tag (or any tag if tag
        is NULL), and with a matching attribute.

        Return NULL if none found.
*/
fz_xml *fz_xml_find_next_match(fz_xml *item, const char *tag, const char *att, const char *match);

/**
        Search the siblings of XML nodes starting with the first child
        of item looking for the first with the given tag (or any tag if
        tag is NULL), and with a matching attribute.

        Return NULL if none found.
*/
fz_xml *fz_xml_find_down_match(fz_xml *item, const char *tag, const char *att, const char *match);

Both of those sets of functions search fairly restricted areas of a document. The final set is much more broad ranging, allowing the whole document to be searched for matches at all levels.

/**
        Perform a depth first search from item, returning the first
        child that matches the given tag (or any tag if tag is NULL),
        with the given attribute (if att is non NULL), that matches
        match (if match is non NULL).
*/
fz_xml *fz_xml_find_dfs(fz_xml *item, const char *tag, const char *att, const char *match);

/**
        Perform a depth first search onwards from item, returning the first
        child that matches the given tag (or any tag if tag is NULL),
        with the given attribute (if att is non NULL), that matches
        match (if match is non NULL).
*/
fz_xml *fz_xml_find_next_dfs(fz_xml *item, const char *tag, const char *att, const char *match);

This is the Swiss Army Knife of search functions. Firstly, either all tags to be searched, or just specific ones, based upon the value of tag. Secondly, the results can be limited to nodes that have a given attribute based upon the value of att. Finally, in the event that a node has the requested attribute, we can choose to limit the results to only those nodes where the attribute value matches match.

These functions follow a depth first traversal of the tree, meaning that the children of a node will be considered before the nodes successor siblings. Also, in the event that a match is not found in any of the initial nodes children, or siblings, the search will contine with its ancestor nodes (and their siblings/children) too.

The simple way to understand this is to think that if you call fz_xml_find_dfs on the root node of a document, and then call fz_xml_find_next_dfs repeatedly on all matches until it returns NULL you will have checked the entire document for matches.

Modifying the structure

Nodes within the XML tree can either be nodes with a tag, or simple text nodes. These, respectively, are created by using the following functions:

/**
        Create an element of a given tag type for the given DOM.

        The element is not linked into the DOM yet.
*/
fz_xml *fz_dom_create_element(fz_context *ctx, fz_xml *dom, const char *tag);

/**
        Create a text node for the given DOM.

        The element is not linked into the DOM yet.
*/
fz_xml *fz_dom_create_text_node(fz_context *ctx, fz_xml *dom, const char *text);

The result of both of these is a pointer to a new node that is not yet linked into the DOM. Such nodes will not have any effect on the document until they are inserted using one of the following self-explanatory functions:

/**
        Insert an element as the last child of a parent, unlinking the
        child from its current position if required.
*/
void fz_dom_append_child(fz_context *ctx, fz_xml *parent, fz_xml *child);

/**
        Insert an element (new_elt), before another element (node),
        unlinking the new_elt from its current position if required.
*/
void fz_dom_insert_before(fz_context *ctx, fz_xml *node, fz_xml *new_elt);

/**
        Insert an element (new_elt), after another element (node),
        unlinking the new_elt from its current position if required.
*/
void fz_dom_insert_after(fz_context *ctx, fz_xml *node, fz_xml *new_elt);

It is worth noting that every node in the DOM can appear just once in the DOM. This means that if you navigate through the document to find a node, node1, say, and you then use one of these insertion functions to put node1 somewhere else within the document, it (and all its children) will implicitly be unlinked from its old position first.

Cloning a node

Sometimes it can be very useful to be able to copy a node and all its children. The following function allows exactly this:

/**
        Clone an element (and its children).

        A borrowed reference to the clone is returned. The clone is not
        yet linked into the DOM.
*/
fz_xml *fz_dom_clone(fz_context *ctx, fz_xml *elt);

For instance, a document might be created with a ’template’ entry in it. This could be found using fz_xml_find_dfs, and then that template could be cloned. This returns a complete subtree rooted at the cloned node, initially disjoint from the document itself.

More ’find’ operations can be carried out within this tree, where different nodes can be located and rewritten to convert the clone of the template to being a fully fleshed-out instance.

Then this tree can be inserted into the document as a whole.

An example of constructing a document and filling it out using templates can be seen in docs/examples/storytest.c

Deletion

Finally, nodes can be removed from the document using:

/**
        Remove an element from the DOM. The element can be added back elsewhere
        if required.

        No reference counting changes for the element.
*/
void fz_dom_remove(fz_context *ctx, fz_xml *elt);

The node is not actually deleted (i.e. it remains valid) and can be reinserted. The storage used for the node will be retrieved when the XML tree as a whole is removed on the first call to either fz_story_warnings or fz_place_story.

To continue the example from the previous subsection, a typical use of this might be to delete the template node from the document after we have cloned it enough times.

Node attributes

Every (non-text) XML node can have a set of attributes. Each attribute is a key/value pair, with both key and values being strings. Each key is guaranteed to be unique for that given node.

If you are looking for a particular attribute you can query it directly, using:

/**
        Retrieve the value of a given attribute from a given element.

        Returns a borrowed pointer to the value or NULL if not found.
*/
const char *fz_dom_attribute(fz_context *ctx, fz_xml *elt, const char *att);

Alternatively, if you want to know all the attributes that a given node has, you can enumerate them by repeatedly calling:

/**
        Enumerate through the attributes of an element.

        Call with i=0,1,2,3... to enumerate attributes.

        On return *att and the return value will be NULL if there are not
        that many attributes to read. Otherwise, *att will be filled in
        with a borrowed pointer to the attribute name, and the return
        value will be a borrowed pointer to the value.
*/
const char *fz_dom_get_attribute(fz_context *ctx, fz_xml *elt, int i, const char **att);

It is worth noting that attributes are not ordered in any particular way.

An attribute can be added (or changed) for a node using:

/**
        Add an attribute to an element.

        Ownership of att and value remain with the caller.
*/
void fz_dom_add_attribute(fz_context *ctx, fz_xml *elt, const char *att, const char *value);

And finally, an attribute can be removed from a node using:

/**
        Remove an attribute from an element.
*/
void fz_dom_remove_attribute(fz_context *ctx, fz_xml *elt, const char *att);

Parsing warnings

As a convenience, we provide a way to retrieve a string full of any warnings that might be given as we parse the HTML and CSS. This is entirely optional.

/*
        Retrieve the warnings given from parsing this story.

        If there are warnings, this will be returned as a NULL terminated
        C string. If there are no warnings, this will return NULL.

        These warnings will not be complete until AFTER any DOM manipulations
        have been completed.

        This function does not need to be called, but once it has been
        the DOM is no longer accessible, and any fz_xml pointer
        retrieved from fz_story_docment is no longer valid.
*/
const char *fz_story_warnings(fz_context *ctx, fz_story *story);

Because the document manipulation stage can change the CSS embedded within the document (in terms of inline styles), we can only retrieve the warnings after such manipulation is complete.

If the function returns NULL, then no warnings were generated.

If your program is taking user supplied HTML/CSS, then displaying these warnings would seem sensible given MuPDFs limitations in terms of features supported. In common with HTML consumers the world over, the fact that no warnings are given should not be taken as a guarantee that the HTML will not have problems!

Placing the story

As well as the textual content, stories notionally contain a ’current point’. This is the position within the story where the next placement attempt will start from. When first created this current point is set to the start of the story.

Callers should then figure out the first rectangle into which the story should be placed, and pass that into the following:

/*
        Place (or continue placing) a story into the supplied rectangle
        'where', updating 'filled' with the actual area that was used.
        Returns zero if all the content fitted, non-zero if there is
        more to fit.

        Note, that filled may not be returned as a strict subset of
        where, due to padding/margins at the bottom of pages, and
        non-wrapping content extending to the right.

        Subsequent calls will attempt to place the same section of story
        again and again, until the placed story is drawn using fz_draw_story,
        whereupon subsequent calls to fz_place_story will attempt to place
        the unused remainder of the story.

        After this function is called, the DOM is no longer accessible,
        and any fz_xml pointer retrieved from fz_story_document is no
        longer valid.
*/
int fz_place_story(fz_context *ctx, fz_story *story, fz_rect where, fz_rect *filled);

The return value tells you whether there is ’more’ still to lay out, and the rectangle returned within filled tells you how much space was actually used.

Generally the filled rectangle should be a subset of the rectangle passed in, but this may not always be the case. Callers may want to place a clip box around the draw call if this is a problem.

After consulting the filled box, the caller can choose to retry with different boxes by just calling the function again (as many times as required). Each placement attempt will place the same content again (starting at the current point).

When the caller is happy, they simply continue to next stage.

Inspecting the placed story

In some cases, the caller may wish to inspect the positions at which different elements have been laid out.

The classic example of this would be where a large story is being laid out over multiple pages - perhaps the complete text of a book. In order to generate a table of contents for the book, the caller would like to know whereabouts chapters, sections and subsections start. Clearly this cannot be know until the story has been placed.

Accordingly, we offer a function to enumerate the positions of different elements placed in the last call to fz_place_story:

/*
        Enumerate the positions for key blocks in the story.

        This will cause the supplied function to be called with details of each
        element in the story that is either a header, or has an id.
*/
void fz_story_positions(fz_context *ctx, fz_story *story, fz_story_position_callback *cb, void *arg);

This function takes a callback function as a parameter, and calls that for every ’interesting’ element placed by the most recent call to fz_place_story. An element is currently deemed to be interesting if it is either a heading, or has an id attribute. This definition may be extended to other elements in future.

The callback is provided both with the argument passed to fz_story_positions and with the pointer to a structure filled with information:

typedef struct
{
        /* The overall depth of this element in the box structure.
         * This can be used to compare the relative depths of different
         * elements, but shouldn't be relied upon not to change between
         * different versions of MuPDF. */
        int depth;

        /* The heading level of this element. 0 if not a header, or 1-6 for h1-h6. */
        int heading;

        /* The id for this element. */
        const char *id;

        /* The rectangle for this element. */
        fz_rect rect;

        /* The immediate text for this element. */
        const char *text;

        /* This indicates whether this opens and/or closes this element.
         *
         * As we traverse the tree we do a depth first search. In order for
         * the caller of fz_story_positions to know whether a given element
         * is inside another element, we therefore announce 'start' and 'stop'
         * for each element. For instance, with:
         *
         *   <div id="part1">
         *    <h1>Chapter 1</h1>...
         *    <h1>Chapter 2</h1>...
         *    ...
         *   </div>
         *   <div id="part2">
         *    <h1>Chapter 10</h1>...
         *    <h1>Chapter 11</h1>...
         *    ...
         *   </div>
         *
         * We would announce:
         *   + id='part1' (open)
         *   + header=1 "Chapter 1" (open/close)
         *   + header=1 "Chapter 2" (open/close)
         *   ...
         *   + id='part1' (close)
         *   + id='part2' (open)
         *   + header=1 "Chapter 10" (open/close)
         *   + header=1 "Chapter 11" (open/close)
         *   ...
         *   + id='part2' (close)
         *
         * If bit 0 is set, then this 'opens' the element.
         * If bit 1 is set, then this 'closes' the element.
         */
        int open_close;

        /* A count of the number of rectangles that the layout code has split the
         * story into so far. After the first layout, this will be 1. If a
         * layout is repeated, this number is not incremented. */
        int rectangle_num;
} fz_story_element_position;

typedef void (fz_story_position_callback)(fz_context *ctx, void *arg, const fz_story_element_position *);

The different entries in the fz_story_element_position are worthy of discussion.

depth is a simple numerical count of the number of layers down the heirarchy this element is. The exact meaning of this may change in future, dependent as it is on the way that the HTML parser behaves, but it should suffice in simple comparisons to spot whether elements are at the same level or not.

If an element is a heading (from a h1..h6 tag) then heading will be set to the appropriate value (1 to 6), otherwise it will be zero.

If an element has an id attribute then id will be a pointer to a null terminated string with the id in it. If not, it will be NULL. Ownership of this string remains with MuPDF - do not free it, or rely on it being valid beyond the life of the callback.

The element bounds are given in rect.

If an element immediately contains any text, this will be collated into a string and a pointer returned to it in text. Again, ownership of this string remains with MuPDF - do not free it, or rely on it being valid beyond the life of the callback.

The open_close field is perhaps less immediately obvious, and is best explained with the help of an example.

The laid out structure is a heirarchical one, with elements that may encapsulate other elements. Consider the layout created by the following HTML fragment.

<div id="part1">
 <h1>Chapter 1</h1>
  ...
 <h1>Chapter 2</h1>
  ...
</div>
<div id="part2">
 <h1>Chapter 6</h1>
  ...
 <h1>Chapter 7</h1>
  ...
</div>

The headings for Chapter 1 and 2 are contained within the div with id="part1", so to indicate this nesting we perform the callbacks as follows:

The best way to understand this is to consider the nodes of the tree in the order in which they would be visited during a depth first traversal. When a node is visited and found to be ’interesting’, we send a callback for it, with bit 0 of open_close set. This indicates that this node is ’opening’.

We then continue to traverse all its children, performing the callbacks for them as required. Before we pass onto our next sibling (or our parent, as appropriate), we perform another callback, this time with bit 1 of open_close set. This indicates that the node is ’closing’.

Thus any callbacks that happen between the ’opening’ of a node and the ’closing’ of it are known to be nested within it.

Note that in future, MuPDF may make callbacks with both bits 0 and 1 set at the same time (to indicate that a node has no children).

Similarly, future versions of MuPDF may define more bits within open_close, so care should be taken to test bitwise, rather than just for the value as a whole. Finally, such future versions of MuPDF may make calls on a node that are neither opens or closes, so callers should cope with neither bit 0 or 1 being set.

The caller is free to make as many calls to fz_story_positions as they want to (including none!). Once finished they can either return to the placing stage (perhaps to modify the placement rectangle based upon information gathered from the callbacks), or they can proceed to draw the placed content.

Drawing the placed story

After having successfully placed (some of) the story into a rectangle, the next stage is to draw that content out. This is done by calling:

/*
        Draw the placed story to the given device.

        This moves the point at which subsequent calls to fz_place_story
        will restart placing to the end of what has just been output.
*/
void fz_draw_story(fz_context *ctx, fz_html_story *story, fz_device *dev, fz_matrix ctm);

If this is just a dry run to evaluate the complete layout, dev can be passed as NULL to avoid any drawing actually being done.

This call moves the story’s current point to be the furthest point within the story that was successfully placed.

Typically a caller would consult the ’more’ value returned from the fz_place_story call at this point. If non-zero (more content remains to be placed), the caller would return to call fz_place_story again.

Eventually, when all the story has been placed and drawn, they simply continue to the next stage.

Resetting the story

Some callers may wish to do a complete ’dry-run’ of the placement/drawing process, perhaps using a NULL device to swallow the output. If so, then at this point (or indeed at any point during the placement/drawing process), they can reset the current point to the start of the document by simply calling:

/*
        Reset the position within the story at which the next layout call
        will continue to the start of the story.
*/
void fz_reset_story(fz_context *ctx, fz_story *story);

Once this is called the placement/drawing process can be restarted anew.

Dropping the story

Once a caller is finished with a story, it should be dropped in the usual way:

/*
        Drop the story.
*/
void fz_drop_story(fz_context *ctx, fz_story *story);

Styling stories

The PDF engine in MuPDF supports styling both with inline styles, and with cascading style sheets. It should be noted that our CSS only supports CSS2 selectors, not the enhanced selectors available in CSS3.

Similarly, our styling engine does not support every style defined in the specification. Historically we have been focussed on supporting styles commonly used for ebooks, so some of the more esoteric styles remain unimplemented for now.

It is therefore probably worth us saying a few words about what styling we do recommend for use with stories.

Font selection and sizing

Styles can be used to select from the different fonts built into MuPDF. For instance:

<span style="font-family: 'sans-serif'">

will select whichever sans-serif font MuPDF has been built with (typically a Nimbus Sans variant).

The question of what fonts are available to you will depend upon how MuPDF is built; see for more details.

Any font built in to MuPDF can be selected by name here. Alternatively, fonts can be loaded using the font-face style. For example:

@font-face {
  font-family: myFontName;
  font-style: italic;
  font-weight: bold;
  font-variant: small-caps;
  src: url(path/to/my/font.ttf)
}

MuPDF understands bold and bolder weights, (all others are held to be normal). Similarly it understands italic and oblique styles. The only supported variant is small-caps.

Padding, Margins and Borders

Margins, Padding and Borders are supported, for positive numbers only.

Line Height and Leading

The standard line-height style is supported. This sets the minimum line height between lines, but HTML layout allows for larger heights to be used if required.

To allow for cases where this automatic expansion is not desirable, MuPDF provides an extra style, -mupdf-leading. This sets the exact height to be used for line to line spacing, and therefore matches the concept of leading used within the PDF specification.

Page breaks

MuPDF supports page-break-before and page-break-after styles on HTML elements. It recognises many values as specified in the CSS documentation but these are not all acted upon. auto and avoid currently have no effect. Because fz_place_story has no idea what page we are laying out for, the behaviour of left and right should be considered undefined, currently. The only really useful options is always which will cause the laid out content to break until the next rectangle passed to fz_place_story.

Text colour

MuPDF supports both color and background-color properties, as you’d expect.

Visibility

MuPDF recognises visibility values of visible, hidden and collapse, but simply treats anything other than visible as hidden.

White space

MuPDF recognises and acts on white-space values of normal, pre, nowrap, pre-wrap and pre-line.

Text indent

MuPDF recognises the text-indent property and indents the first line of each paragraph by it. The correct behaviour of negative values should not be relied on.

Alignment

MuPDF support text-align values of left, right, center and justify.

Similarly it supports vertical-align values of baseline, sub, super, top, bottom, text-top, and text-bottom. Currently text-top behaves identically to top and text-bottom behaves identically to bottom.

Overflow wrap

MuPDF supports overflow-wrap operation for normal and break-word modes. Other values are treated as normal.

Archives

Overview

Some document types, such as XPS, ebooks or CBZ, consist of a container that groups multiple different files together. Other document types, such as HTML, contain all the basic document information in a single file, but can refer to externally supplied resources (such as fonts or image data).

MuPDF offers the fz_archive type as an abstraction to cope with both these cases.

An fz_archive is a source from which files can be read (either as streams or as raw data). Currently `fz_archive`s are read-only.

API

Creating an archive

The exact mechanism for creating an archive will vary according to archive type. See for the currently defined archive types and how to create them.

A typical example would be to open an archive pointer to the current directory:

archive = fz_open_directory(ctx, ".");

Actions on this archive will permit files within (or reachable below) the current directory to be accessed.

Reference counting

Creating an archive will return a pointer with a single reference taken. Additional references to archives can be taken and released in the usual fashion:

/**
        Drop a reference to an archive.

        When the last reference is dropped, this closes and releases
        any memory or filehandles associated with the archive.
*/
void fz_drop_archive(fz_context *ctx, fz_archive *arch);

/**
        Keep a reference to an archive.
*/
fz_archive *
fz_keep_archive(fz_context *ctx, fz_archive *arch);

When the final reference to an archive is released the storage associated with the archive will be released, and the pointer invalidated.

Checking if an archive entry exists

All archives support checking if a given named entry exists:

/**
        Check if entry by given name exists.

        If named entry does not exist 0 will be returned, if it does
        exist 1 is returned.

        name: Entry name to look for, this must be an exact match to
        the entry name in the archive.
*/
int fz_has_archive_entry(fz_context *ctx, fz_archive *arch, const char *name);

Opening an archive entry as a stream

All archives support opening a given named entry as a stream:

/**
        Opens an archive entry as a stream.

        name: Entry name to look for, this must be an exact match to
        the entry name in the archive.

        Throws an exception if a matching entry cannot be found.
*/
fz_stream *fz_open_archive_entry(fz_context *ctx, fz_archive *arch, const char *name);

Reading an archive entry as a buffer

All archives support reading the contents of a given named entry as a buffer:

/**
        Reads all bytes in an archive entry
        into a buffer.

        name: Entry name to look for, this must be an exact match to
        the entry name in the archive.

        Throws an exception if a matching entry cannot be found.
*/
fz_buffer *fz_read_archive_entry(fz_context *ctx, fz_archive *arch, const char *name);

Archive format

In some cases, such as opening a directory for reading, the format of the archive opened is known by construction. In other cases, such as when opening an archive file on disk, the format can vary according to data within the archive itself.

A printable string representing the format of the archive can be retrieved using:

/**
        Return a pointer to a string describing the format of the
        archive.

        The lifetime of the string is unspecified (in current
        implementations the string will persist until the archive
        is closed, but this is not guaranteed).
*/
const char *fz_archive_format(fz_context *ctx, fz_archive *arch);

This is primarily of use for debugging purposes. In the case of multi archives, this will report "multi" rather than the format of any of the subarchives.

Counting archive entries

Some archives, such as .zip and .tar files, allow the number of entries within the archive to be counted:

/**
        Number of entries in archive.

        Will always return a value >= 0.

        May throw an exception if this type of archive cannot count the
        entries (such as a directory).
*/
int fz_count_archive_entries(fz_context *ctx, fz_archive *arch);

Others, such as virtual archives pointing at directories will throw an exception. This is primarily of use for debugging purposes.

Enumerating archive entries

Some archives, such as .zip and .tar files, allow the entries within the archive to be be enumerated:

/**
        Get listed name of entry position idx.

        idx: Must be a value >= 0 < return value from
        fz_count_archive_entries. If not in range NULL will be
        returned.

        May throw an exception if this type of archive cannot list the
        entries (such as a directory).
*/
const char *fz_list_archive_entry(fz_context *ctx, fz_archive *arch, int idx);

Others, such as virtual archives pointing at directories, or multi archives, will throw an exception. This is primarily of use for debugging purposes.

Archive Types

Overview

A variety of different archive implementations exist within MuPDF, and callers can even implement their own.

Some archive types are ’virtual’, in that they represent a collection of files where the collection, at least, and sometimes the files themselves, are not represented in permanent storage. These types can only be created using type specific APIs documented below.

Other archive types are based upon a traditional archive file, and the content type can be detected automatically. These can be opened by using the generic functions:

/**
        Open a zip or tar archive

        Open a file and identify its archive type based on the archive
        signature contained inside.

        filename: a path to a file as it would be given to open(2).
*/
fz_archive *fz_open_archive(fz_context *ctx, const char *filename);

/**
        Open zip or tar archive stream.

        Open an archive using a seekable stream object rather than
        opening a file or directory on disk.
*/
fz_archive *fz_open_archive_with_stream(fz_context *ctx, fz_stream *file);

Currently, only zip and tar archives are supported, but this may be extended in future.

Zip archives

As well as being accessible using the generic functions above, zip archives can be detected and opened using zip-specific API functions.

/**
        Detect if stream object is a zip archive.

        Assumes that the stream object is seekable.
*/
int fz_is_zip_archive(fz_context *ctx, fz_stream *file);

/**
        Open a zip archive file.

        An exception is throw if the file is not a zip archive as
        indicated by the presence of a zip signature.

        filename: a path to a zip archive file as it would be given to
        open(2).
*/
fz_archive *fz_open_zip_archive(fz_context *ctx, const char *path);

/**
        Open a zip archive stream.

        Open an archive using a seekable stream object rather than
        opening a file or directory on disk.

        An exception is throw if the stream is not a zip archive as
        indicated by the presence of a zip signature.

*/
fz_archive *fz_open_zip_archive_with_stream(fz_context *ctx, fz_stream *file);
Zip writer

While, as stated, archive support is currently read-only, we do have an API to allow zip files to be created.

This is achieved by a fz_zip_writer:

/**
        fz_zip_writer offers methods for creating and writing zip files.
        It can be seen as the reverse of the fz_archive zip
        implementation.
*/
typedef struct fz_zip_writer fz_zip_writer;

/**
        Create a new zip writer that writes to a given file.

        Open an archive using a seekable stream object rather than
        opening a file or directory on disk.
*/
fz_zip_writer *fz_new_zip_writer(fz_context *ctx, const char *filename);

/**
        Create a new zip writer that writes to a given output stream.

        Ownership of out passes in immediately upon calling this function.
        The caller should never drop the fz_output, even if this function throws
        an exception.
*/
fz_zip_writer *fz_new_zip_writer_with_output(fz_context *ctx, fz_output *out);

Once a fz_zip_writer has been created, individual entries can then be output:

/**
        Given a buffer of data, (optionally) compress it, and add it to
        the zip file with the given name.
*/
void fz_write_zip_entry(fz_context *ctx, fz_zip_writer *zip, const char *name, fz_buffer *buf, int compress);

When all the entries have been written, the zip file should be closed:

/**
        Close the zip file for writing.

        This flushes any pending data to the file. This can throw
        exceptions.
*/
void fz_close_zip_writer(fz_context *ctx, fz_zip_writer *zip);

And finally, the reference to the writer should be dropped:

/**
        Drop the reference to the zipfile.

        In common with other 'drop' methods, this will never throw an
        exception.
*/
void fz_drop_zip_writer(fz_context *ctx, fz_zip_writer *zip);

The zip writer can be dropped at any point during usage to cleanly free up the resources, but unless it has been properly closed first the data within the written zipfile may be truncated.

As can be seen from the API, the zip file writer is independent to that of archives. Once a zip file has been fully written, it can then be opened and read using the archive mechanism, but neither the writer not the archive mechanisms can be used to modify an existing zipfile.

Should such functionality be desired, the best we can offer is to create a writer to a new zip file, open an archive view on an existing zip file, and then to copy entries across, whereupon new entries can be added.

Tar archives

As well as being accessible using the generic functions above, tar archives can be detected and opened using tar-specific API functions.

/**
        Detect if stream object is a tar achieve.

        Assumes that the stream object is seekable.
*/
int fz_is_tar_archive(fz_context *ctx, fz_stream *file);

/**
        Open a tar archive file.

        An exception is throw if the file is not a tar archive as
        indicated by the presence of a tar signature.

        filename: a path to a tar archive file as it would be given to
        open(2).
*/
fz_archive *fz_open_tar_archive(fz_context *ctx, const char *filename);

/**
        Open a tar archive stream.

        Open an archive using a seekable stream object rather than
        opening a file or directory on disk.

        An exception is throw if the stream is not a tar archive as
        indicated by the presence of a tar signature.

*/
fz_archive *fz_open_tar_archive_with_stream(fz_context *ctx, fz_stream *file);

‘Virtual’ directory archives

In some cases, it can be useful to make whole directories on the local filing system appear as ‘virtual’ archives.

/**
        Open a directory as if it was an archive.

        A special case where a directory is opened as if it was an
        archive.

        Note that for directories it is not possible to retrieve the
        number of entries or list the entries. It is however possible
        to check if the archive has a particular entry.

        path: a path to a directory as it would be given to opendir(3).
*/
fz_archive *fz_open_directory(fz_context *ctx, const char *path);

Care should be taken by callers of this function to consider the security implications of allowing read-only access to the local filing system. In many cases this will not be a problem, but in others (perhaps where a MuPDF-based app might run user submitted scripts on a server), there are potential data-leakage implications.

To mitigate against such issues, all ‘paths’ passed to fz_archive functions are ‘cleaned’ to avoid such constructs as ‘.’, ‘..’, and absolute paths (paths beginning with ‘/’).

‘Virtual’ tree archives

On occasion, it can be useful to construct a virtual archive of entries that exist in memory. One such example of this would be to make image or font resources available for an Story (see ).

For this, we provide the tree archive, so called because it is built upon the fz_tree structure.

A tree archive can be created using:

/**
        Create an archive that holds named buffers.

        tree can either be a preformed tree with fz_buffers as values,
        or it can be NULL for an empty tree.
*/
fz_archive *fz_new_tree_archive(fz_context *ctx, fz_tree *tree);

The tree can either be a pointer to an existing fz_tree where the values for each node within the tree are fz_buffer references, or it can be NULL to create an initially empty archive.

Whichever option is taken, new entries can be added to the archive by either passing in fz_buffer pointers:

/**
        Add a named buffer to an existing tree archive.

        The tree will take a new reference to the buffer. Ownership
        is not transferred.
*/
void fz_tree_archive_add_buffer(fz_context *ctx, fz_archive *arch, const char *name, fz_buffer *buf);

or by passing blocks of data in directly:

/**
        Add a named block of data to an existing tree archive.

        The data will be copied into a buffer, and so the caller
        may free it as soon as this returns.
*/
void fz_tree_archive_add_data(fz_context *ctx, fz_archive *arch, const char *name, const void *data, size_t size);

Using `fz_buffer`s is more desirable to passing in raw data, as this avoids the need to make a new copy the data as `fz_buffer`s have a better defined lifespan.

Multi archives

The final type of archive, is that of a multi archive. This allows multiple existing archives to be joined together to appear as one.

As an example of why this might be useful, consider using stories (see ) to create multiple documents - perhaps daily reports of some kind. Each document might want to share the same fonts, but might have different image data associated with it.

Whenever the program runs, we’d create the image(s) for the report in memory, where the font data (constant between runs) might be held on disc as a zip file. We can use a tree archive (see ) to hold the images, but we need some way to join both the contents of the tree archive and the zip file.

This can be achieved using a multi-archive.

/**
        Create a new multi archive (initially empty).
*/
fz_archive *fz_new_multi_archive(fz_context *ctx);

This call creates an initially empty archive. We can then ‘mount’ existing archives into it:

/**
        Add an archive to the set of archives handled by a multi
        archive.

        If path is NULL, then the archive contents will appear at the
        top level, otherwise, the archives contents will appear prefixed
        by path.
*/
void fz_mount_multi_archive(fz_context *ctx, fz_archive *arch_, fz_archive *sub, const char *path);

In our example above, we might first mount the zip file containing the font data, and then make another call to mount the tree archive containing the image data.

When called with non-NULL paths, this is very similar to the idea of mounting filing systems on unix. For instance, mounting the zip file at ‘resources’, would mean that every entry of the zip file (e.g. ‘Courier’, ‘Helvetica/Bold’ etc) would appear to users of the multi archive prefixed by ‘resources’ (e.g. ‘resources/Courier’, ‘resources/Helvetica/Bold’ etc).

When called with a NULL path, the contents of the mounted archive will appear without a prefix.

If two archives contain entries with the same filename (including any prefix) then the later entry will be the one found in the multi-archive. That is to say, entries in later archives override conflicting entries in earlier ones.

Progressive Mode

Overview

When used in the normal way, MuPDF requires the entirety of a file to be present before it can be opened. For some applications, this can be a significant restriction - for instance, when downloading a PDF file over a slow internet link, being able to view just the first page or two may be enough to know whether it is the correct file or not.

Normal PDF files require the end of the file to be present before file reading can begin, as this is where the ‘trailer’ lives (effectively the index for the entire file). In an effort to allow early display of the first page, Adobe (the originators of the PDF format) introduced the concept of a ‘linearized’ PDF file. This is a PDF file that, while constructed in accordance with the original specification, also has some extra information contained within the file to allow fast access to the first page. This information is known as the ‘hint stream’. In addition, extra constraints are placed upon the ordering of data within the file in an effort to ensure that the first page will download quickly.

Unfortunately, Linearized PDF files are far from a panacea. The specification is overly-complex, unclear and consequently poorly supported in both readers and writers of the format. Even when implemented correctly, it is of limited use for pages other than the first one.

MuPDF therefore attempts to solve the problem using a combination of mechanisms, known together as “progressive mode”. When run in this mode, MuPDF can not only take advantage of the linearization information (if present) in a file, but is also capable of directing the actual download mechanism used by a file. By controlling the order in which sections of a file are fetched, any page required can be viewed before the whole fetch is complete.

For optimum performance a file should be both linearized and be available over a byte-range supporting link, but benefits can still be had with either one of these alone.

Coupled with the ability to render pages ignoring (and detecting) errors, this means that ‘rough renderings’ of pages can be given even before all the content (such as images and fonts) for a page have been downloaded.

Implementation

MuPDF has made various extensions to its mechanisms for handling progressive loading. They rely on some special properties built into a type of fz_stream known as a ‘progressive’ stream.

Progressive Streams

At its lowest level MuPDF reads file data from a fz_stream, using the fz_open_document_with_stream call. The alternative entrypoint fz_open_document is implemented by calling this.

The PDF interpreter uses the fz_lookup_metadata call to check for its stream being progressive or not. Any non-progressive stream will be read as normal, with the system assuming that the entire file is present immediately.

If it is found to be progressive, another fz_lookup_metadata call is made to find out what the length of the stream will be once the entire file is fetched. An HTTP fetcher can know this by consulting the Content-Length header before any data has been fetched.

With this information MuPDF can decide whether a file is linearized or not. (Technically, knowing the length enables us to check with the length value given in a linearized object - if these differ, the assumption is that an incremental save has taken place, thus the file is no longer linearized.)

Other than supporting the required metadata responses, the key thing that marks a stream as being progressive, is that it will not block when attempting to read data it does not have. Instead, it will throw a FZ_ERROR_TRYLATER error. This particular error code will be interpreted by the caller as an indication that it should retry the parsing of the current objects at a later time.

When a MuPDF call is made on a progressive stream, such as fz_open_document_with_stream, or fz_load_page, the caller should be prepared to handle a FZ_ERROR_TRYLATER error as meaning that more data is required before it can continue. No indication is directly given as to exactly how much more data is required, but as the caller will be implementing the progressive fz_stream that it has passed into MuPDF to start with, it can reasonably be expected to figure out an estimate for itself.

With these mechanisms in place, a caller can repeatedly try to render each page in turn until it gets a successful result.

Rough renderings

Once a page has been loaded, if its contents are to be ‘run’ as normal (using e.g. fz_run_page) any error (such as failing to read a font, or an image, or even a content stream belonging to the page) will result in a rendering that aborts with a FZ_ERROR_TRYLATER error. The caller can catch this and display a placeholder instead.

If each pages data was entirely self-contained and sent in sequence this would perhaps be acceptable, with each page appearing one after the other. Unfortunately, the linearization procedure as laid down by Adobe does NOT do this: objects shared between multiple pages (other than the first) are not sent with the pages themselves, but rather AFTER all the pages have been sent.

This means that a document that has a title page, then contents that share a font used on pages 2 onwards, will not be able to correctly display page 2 until after the font has arrived in the file, which will not be until all the page data has been sent.

To mitigate against this, MuPDF provides a way whereby callers can indicate that they are prepared to accept an ‘incomplete’ rendering of the file (perhaps with missing images, or with substitute fonts).

Callers prepared to tolerate such renderings should set the ‘incomplete_ok’ flag in the cookie, then call fz_run_page etc as normal. If a FZ_ERROR_TRYLATER error is thrown at any point during the page rendering, the error will be swallowed, the ‘incomplete’ field in the cookie will become non-zero and rendering will continue. When control returns to the caller the caller can check the value of the ‘incomplete’ field and know that the rendering it received is not authoritative.

Directed downloads

If the caller has control over the fetch of the file (be it http or some other protocol), then it is possible to use byte range requests to fetch the document ‘out of order’. This enables non-linearized files to be progressively displayed as they download, and fetches complete renderings of pages earlier than would otherwise be the case. This process requires no changes within MuPDF itself, but rather in the way the progressive stream learns from the attempts MuPDF makes to fetch data.

Consider for example, an attempt to fetch a hypothetical file from a server.

Example implementation

An example implementation of a fetcher process can be found in curl-stream.c. This implements a fz_stream using the popular ‘curl’ http fetching library.

The structure of this process broadly behaves as follows:

Fonts

Overview

Fonts are represented in MuPDF by the abstract fz_font type. This reference counted structure, encapsulates the basic information about a font, specifically:

Glyph list

Each font consists of a list of glyphs.

Glyph data

How to draw each glyph. In traditional fonts this information is known as the ‘Outline data’ (or ‘Outlines’), but some font types (such as Type 3 fonts from PDF) can encapsulate other data, such as images and colors too.

Unicode map

Most (but not all) fonts contain information that enables glyphs to be mapped to/from the Unicode code points they represent. Without such information, it can be impossible to meaningfully extract text information from a document (such as for cut and paste).

Font BBox

All fonts include information for a bounding box that covers all the glyphs within a font. Sadly this can frequently be inaccurate or incorrect, so should be treated with distrust.

Glyph advances

All fonts contain simple Glyph advance information - how far to move the text cursor after having drawn a given glyph. This information ensures that successive characters are properly spaced w.r.t. each other.

Kerning data

Most fonts contain simple kerning data; this allows for the glyph advance between any 2 glyphs to be adjusted based upon particular glyph values. The classic example of kerning is noting that the spacing between A and the left hand edge of its following letter is typically different between AV and AN.

Shape data

Some fonts allow for the automatic ‘shaping’ of glyph sequences. The trivial example of this in western fonts is that the letters ‘f’ and ‘i’ can be combined into a single ligature glyph ‘fi’. For many non Latin scripts (especially Indic and south east Asian scripts), this procedure happens to a far greater extent. This can be as simple as the incorporation of diacritical marks, or as complex as the complete rearrangement or replacement of glyph sequences to give different appearances on the final rendered page. This process is know as ‘font shaping’ and the data required to perform this is font specific, and is optionally encapsulated within fonts themselves.

The fz_font does not include information about the particular size that a font is used at on the page, nor the basic colour used to render a font. It is therefore typical to see `fz_font`s passed around the system paired with both a size (and/or transformation matrix), a colorspace, and color definition.

MuPDF uses Freetype to handle most of its font rendering. For Type 3 PDF fonts, it renders them itself. Font shaping is done using the HarfBuzz library.

Inbuilt Fonts

The exact set of fonts built in to any version of MuPDF is configurable (See ).

For PDF, we build in a basic set of fonts licensed from URW. Originally the PDF specification suggested that 14 standard fonts should be available on all readers, and that any other fonts should be embedded within files. This recommendation has since been updated to suggest that all fonts (or at least the required subset thereof) are embedded in all files. Nonetheless it is still common to find many PDF files that ask for fonts that are not embedded.

In order to cope with the widest range of scripts possible, MuPDF is supplied with (and can optionally include) a selection of the ‘Noto’ fonts from Google.

If this would take too much space on your target system, an alternative is to use the supplied DroidSansFallback font (also from Google) which is a good trade-off between size and coverage.

MuPDF does not, by default, make use of fonts present on the underlying system (for instance, on Windows, MuPDF will not look for fonts in C:/Windows/). Should you wish to implement this kind of ‘system font’ loading, however, MuPDF does provide hooks for this to be done. The fz_install_load_system_font_funcs call takes a set of function pointers that can be used for this purpose.

Implementation

The implementation details of `fz_font`s are still in flux, so it would be inappropriate to document them in any more detail at the moment.

Build configuration

Overview

By default, MuPDF builds almost the most capable version of itself that it can. This can result in larger library sizes than are actually required for any given application. Some care has been taken in the design of MuPDF to try to allow linkers to intelligently drop sections of the code that are not actually required, but nonetheless it can still be worthwhile tuning builds.

Configuration file

To simply the task of tuning builds, all the configuration options have been gathered into a single header file, include/mupdf/fitz/config.h.

This file is in two halves; the top half consists of a sequence of pairs of comments, and commented out `#define`s, the second half consists of #ifdef logic to make sense of the #define values set.

The configuration options available at build time are therefore described by the pairs in the first half of the file.

Each one of these pairs has first a comment that describes what the following configuration options do, and then commented out `#define`s that sets the configuration options to their defaults.

If an integrator wants to change a build option from the default, they have two options. Firstly, and most simply, they can edit the first half of the file to set the required `#define`s by uncommenting the relevant line and editing it. Alternatively, they can arrange for these values to be predefined by the compiler, normally by editing the CFLAGS defined in the Makefile. This latter method has the advantage of not requiring any edits to the source itself, and of allowing different configurations to be built from the same source tree.

The second half of the file should never need to be edited.

Plotter selection

The first section of the file deals with the plotters that are to be built into MuPDF.

/*
        Enable the following for spot (and hence overprint/overprint
        simulation) capable rendering. This forces FZ_PLOTTERS_N on.
*/
#define FZ_ENABLE_SPOT_RENDERING

If this is enabled (i.e. if spot rendering capabilities are required) then all the plotters in the following section are implictly enabled. For people that do not need spot rendering (i.e. anyone dealing with on screen display where overprint simulation is not required) can turn this off, and further configure the next options:

/*
        Choose which plotters we need.
        By default we build the greyscale, RGB and CMYK plotters in,
        but omit the arbitrary plotters. To avoid building
        plotters in that aren't needed, define the unwanted
        FZ_PLOTTERS_... define to 0.
*/
/* #define FZ_PLOTTERS_G 1 */
/* #define FZ_PLOTTERS_RGB 1 */
/* #define FZ_PLOTTERS_CMYK 1 */
/* #define FZ_PLOTTERS_N 0 */

The plotter selection built into MuPDF will determine which formats can be rendered too. If, for instance, you are using MuPDF to target a greyscale only device (say, an e-ink screen for a greyscale ebook reader) then there is no need to ever render in color, and the RGB and CMYK options can be disabled.

Similarly, if you know you are targeting screen display, then the greyscale and CMYK plotters can be omitted.

The N plotters are generic plotters capable of coping with any color depth, but are not as optimised for rendering to G, RGB or CMYK as the specific sets of plotters for those depths. You may find that you can trade some speed for space by enabling the N plotters and disabling the other plotters.

If you wish to render to colorspaces other than G, RGB or CMYK, you must enable the N plotters.

At least 1 plotter family must be defined in any build. If all the plotters are specifically disabled, then the N plotters will be enabled.

Document handlers

Support for the different document types supported by MuPDF is given by a set of document handlers (see ). Each handler type can be enabled or disabled as required:

/*
        Choose which document handlers to include.
        By default all but GPRF are enabled. To avoid building unwanted
        ones, define FZ_ENABLE_... to 0.
*/
/* #define FZ_ENABLE_PDF 1 */
/* #define FZ_ENABLE_XPS 1 */
/* #define FZ_ENABLE_SVG 1 */
/* #define FZ_ENABLE_CBZ 1 */
/* #define FZ_ENABLE_IMG 1 */
/* #define FZ_ENABLE_TIFF 1 */
/* #define FZ_ENABLE_HTML 1 */
/* #define FZ_ENABLE_EPUB 1 */
/* #define FZ_ENABLE_GPRF 1 */

For instance to omit support for HTML, we would define FZ_ENABLE_HTML to be 0. Note that some document handlers share substantial parts of their code with other ones; for example the HTML and EPUB document ages share a large amount of code, so disabling just one of them will not make much difference to the overall code size.

JPEG 2000 support

Some security minded applications choose to disable JPEG2000 decoding. While MuPDF ships with the latest version of the JPEG2000 decoder we use, with all known security patches applied, it is possible to disable this entirely if required.

/*
        Choose whether to enable JPEG2000 decoding.
        By default, it is enabled, but due to frequent security
        issues with the third party libraries we support disabling
        it with this flag.
*/
/* #define FZ_ENABLE_JPX 1 */

Note that if this is disabled, MuPDF will no longer be able to decode all PDF files. If you are authoring your own PDF files (or you know that they do not make use of JPEG2000, then disabling this will have no effect).

Javascript

PDF files can make use of Javascript for form data validation. If such interactive features are not required then the inclusion of an (albeit tiny) javascript engine serves no purpose. Accordingly, it can be disabled by setting FZ_ENABLE_JS to 0.

/*
        Choose whether to enable JavaScript.
        By default JavaScript is enabled both for mutool and PDF interactivity.
*/
/* #define FZ_ENABLE_JS 1 */

Fonts

By far the largest impact on MuPDFs size is given by choosing which fonts to include. Accordingly, there are a range of different options:

/*
        Choose which fonts to include.
        By default we include the base 14 PDF fonts,
        DroidSansFallback from Android for CJK, and
        Charis SIL from SIL for epub/html.
        Enable the following defines to AVOID including
        unwanted fonts.
*/
/* To avoid all noto fonts except CJK, enable: */
/* #define TOFU */

/* To skip the CJK font, enable: (this implicitly enables TOFU_CJK_EXT and TOFU_CJK_LANG) */
/* #define TOFU_CJK */

/* To skip CJK Extension A, enable: (this implicitly enables TOFU_CJK_LANG) */
/* #define TOFU_CJK_EXT */

/* To skip CJK language specific fonts, enable: */
/* #define TOFU_CJK_LANG */

/* To skip the Emoji font, enable: */
/* #define TOFU_EMOJI */

/* To skip the ancient/historic scripts, enable: */
/* #define TOFU_HISTORIC */

/* To skip the symbol font, enable: */
/* #define TOFU_SYMBOL */

/* To skip the SIL fonts, enable: */
/* #define TOFU_SIL */

/* To skip the Base14 fonts, enable: */
/* #define TOFU_BASE14 */
/* (You probably really don't want to do that except for measurement purposes!) */

If documents require a font that is not present, systems will try to ‘fallback’ to alternative ones. When this is not always successful (or indeed possible) unknown glyphs are often rendered as empty boxes, known informally in the typographic world as ‘Tofu’.

By default MuPDF includes all the fonts it knows about. The configuration options are therefore a matter of choosing which scripts should instead be rendered as tofu.

Accordingly, to drop support for rendering emoji characters, you’d define TOFU_EMOJI.

The largest set of fonts are those for the wide range of worldwide scripts given by the Google Noto fonts. These can be omitted by defining TOFU.

See for more details.

Annotations, Links and Interaction

Overview

Different document formats offer different levels of interactivity. Some formats (such as images) are simple static content that (other than being panned and zoomed) offers no interaction at all. Others (such as ebooks) offer interaction in that they might have internal links to allow fast navigation from a "Contents" page to the chapters concerned. PDF takes interaction to a completely different level by offering a rich mechanism for annotations, including a complete JavaScript engine to allow arbitrarily complex calculations and interactions to be built in to each document.

Accordingly, the fitz APIs allow access to both inter- and intra-document links using an fz_link abstraction. However the links are defined in the underlying document they will be exposed through this same abstraction.

As described earlier in 6, annotations are conceptually extra information overlaid on the basic page content.

The simplest examples are of content to be overlaid on the document at specific locations. For instance, an Ink annotation allows a user defined path to be displayed - typically used for freehand signatures, or scrawled freehand drawings. Stamp annotations allow preset text strings to be displayed in the form of a ’rubber stamp’ (such as ’Approved’).

More complex examples include the ’sticky note’, where an icon is displayed on the document that when clicked reveals some textual notes.

PDF Forms are implemented with each field being an annotation, capable of having their values edited by the user, including being processed by embedded JavaScript to format and validate responses.

Finally, buttons can trigger JavaScript that modifies other annotations programmatically, enabling arbitrarily complex interactions to be constructed. A simple document used to demonstrate this implements a pocket calculator that performs a range of arithmetic functions. Another might be a loan calculator, where the last page of a document describing the loans available from a bank would let people choose the loan amount, repayment frequency, loan term etc, and see how their payments would change.

Links describe ‘active’ regions on the page; if the user ‘clicks’ within such a region typically the viewer should respond. Some links move to other places in the document, others launch external clients such as mail or web sites.

The links on a page can be read by calling fz_load_links:

/*
        fz_load_links: Load the list of links for a page.

        Returns a linked list of all the links on the page, each with
        its clickable region and link destination. Each link is
        reference counted so drop and free the list of links by
        calling fz_drop_link on the pointer return from fz_load_links.

        page: Page obtained from fz_load_page.
*/
fz_link *fz_load_links(fz_context *ctx, fz_page *page);

This returns a linked list of fz_link structures. link->next gives the next one in the chain.

Links are reference counted, and fz_load_links returns ownership of a reference to the caller. The caller must eventually fz_drop_link it to avoid a leak.

/**
        Increment the reference count for a link. The same pointer is
        returned.

        Never throws exceptions.
*/
fz_link *fz_keep_link(fz_context *ctx, fz_link *link);

/**
        Decrement the reference count for a link. When the reference
        count reaches zero, the link is destroyed.

        When a link is freed, the reference for any linked link (next)
        is dropped too, thus an entire linked list of fz_link's can be
        freed by just dropping the head.
*/
void fz_drop_link(fz_context *ctx, fz_link *link);

Currently the API requires users to access the bounds for the link to be accessed within the fz_link structure as rect (in untransformed document coordinates), and the destination of the link as uri. This may change with the addition of an API for accessing these fields in future.

A convenience function is given to determine if a link is an external one.

/**
        Query whether a link is external to a document (determined by
        uri containing a ':', intended to match with '://' which
        separates the scheme from the scheme specific parts in URIs).
*/
int fz_is_external_link(fz_context *ctx, const char *uri);

Fitz-level Annotations

In the MuPDF universe, the rendered version of a page contains the page contents, overlaid by content from the annotations.

Accordingly, as seen in 6, the fitz layer allows page contents and annotations to be drawn separately using fz_run_page_contents and fz_run_page_annots.

This is currently the sole access provided to annotations using the fitz layer; to exert finer control, a caller will need to use the PDF level API.

PDF-level Annotations and Widgets

How to operate at the PDF level

In order to operate on a document at the PDF level, we require a pdf_document pointer. Such a pointer can be obtained from a the fz_document pointer, by using the pdf_specifics call.

pdf_document *pdf_specifics(fz_context *ctx, fz_document *doc};

If pdf_specifics is called on a non-PDF file, it will return NULL, so callers can safely check that a given document can be operated on.

Note that pdf_specifics does not return a new reference to the document; if one is required, callers should pdf_keep_document as usual and pdf_drop_document it later.

PDF Annotations

Each annotation is attached to a specific area of a page. The annotations on a page are found by enumerating them one at a time, by first calling pdf_first_annot, and then pdf_next_annot:

/*
        pdf_first_annot: Return a pointer to the first annotation on a page.

        Does not throw exceptions.
*/
pdf_annot *pdf_first_annot(fz_context *ctx, fz_page *page);

/*
        fz_next_annot: Return a pointer to the next annotation on a page.

        Does not throw exceptions.
*/
pdf_annot *pdf_next_annot(fz_context *ctx, pdf_annot *annot);

When either of these returns NULL, you have reached the end of the list.

Annotations are reference counted and can be kept and dropped as usual. It’s important to realise that the references returned by pdf_first_annot and pdf_next_annot are ’borrowed’. That is to say, the ownership of the reference does not pass back to the caller, and therefore does not need to be dropped. If the caller wants to keep hold of such a reference for any period of time, it should ’keep’ the reference and then ’drop’ it later when it has finished with it.

/*
        pdf_keep_annot: Take a new reference to an annotation.
*/
pdf_annot *pdf_keep_annot(fz_context *ctx, pdf_annot *annot);

/*
        pdf_drop_annot: Drop a reference to an annotation. If the
        reference count reaches zero, annot will be destroyed.
*/
void pdf_drop_annot(fz_context *ctx, pdf_annot *annot);

They can also be bounded, by passing a rectangle to pdf_bound_annot:

/*
        pdf_bound_annot: Return the bounding rectangle of the annotation.

        Does not throw exceptions.
*/
fz_rect *pdf_bound_annot(fz_context *ctx, pdf_annot *annot, fz_rect *rect);

On return, the rectangle is populated with the bounding box of the annotation.

Every annotation has a type, which can be retrieved using:

/*
        Return the type of an annotation.
*/
enum pdf_annot_type pdf_annot_type(fz_context *ctx, pdf_annot *annot);

If the type returned is PDF_ANNOT_WIDGET, then it is a particular subtype of annotation, known as a widget. Widgets allow a wider range of operations to be performed on them, described in 19.

Annotation Values vs Appearances

In general Annotations have both a ’value’ and an ’appearance’. These are related, but different.

As an example, consider a text annotation. The value of the annotation will be the text contained in the annotation, coupled with some styling parameters (perhaps the color to be used for the text). The appearance will be the actual appearance of that text once laid out with those styles and displayed on the page.

Similarly, an Ink (or ’freehand’) annotation might have a value that is the list of points that should be stroked, together with the color of the path, and the line caps to use. The appearance will be the actual stream of PDF operators that are used to draw that path with those styles.

In other cases, multiple appearances may be given to an annotation, and the one used will be selected by the value. In such cases, there are no styles do not apply, as the appearances are not recreated to take account of them. For instance, a CheckBox widget will have ’On’ and ’Off’ states, and the PDF viewer will select between them according to the value.

Very often PDF generators will create an annotation with both value and appearance set within the file. Naive PDF viewers can then just ignore the value and display the appearance. Sometimes, however, PDF files are generated with just the value being set. Accordingly, PDF viewers must ’synthesise’ an appearance from the value in order to have something to display.

Similarly, if an annotation’s value is edited, unless we are in the position of just choosing between multiple existing appearances, we’ll need to ’resynthesise’ the appearance. For example, changing the value of a CheckBox widget will not require resynthesising an appearance, but changing the value of a text field will.

To add to the complexity, changing the value of one field can cause other fields to change too (consider setting a radio button that unsets the other buttons grouped with it). This can even cause changes across multiple pages! Accordingly a mechanism is needed for clients to ask MuPDF what annotations have changed since they were last rendered.

Updating annotations

Whenever a call is made to change the value of a field (or to pass an ’event’ in, see 19), the caller needs to check with the document as to which fields have been updated. Changing one value can potentially change many other field in the document, possibly on many pages. This can happen by an annotation being ’grouped’ with others, or even by document level JavaScript ’recalculating’ values anywhere in the document.

The caller thus needs to check for every annotation on every open page whether it needs to be ’updated’ or not.

This can be done wholesale, by calling pdf_update_page for each open page. If this returns true, then the page needs to be redrawn.

If the caller wishes to do finer-grained redraws, it can walk over the annotations on a page using pdf_first_annot and pdf_last_annot, calling pdf_update_annot as it goes. This call will return true for every annotation that requires redrawing (which can be done using pdf_run_annot). Care must be taken to ensure that redraws work properly though, as annotations can overlap, and not all of them may be flagged as having been changed.

The updating of annotations on pages that are open (i.e. to which handles are held) but not visible can be deferred until the page is displayed again.

PDF Annotation operations

MuPDF provides an extensive API for manipulating annotations within a document. Most obviously, they can be created:

/*
        create a new annotation of the specified type on the
        specified page. Populate it with sensible defaults per the type.

        Returns a reference that the caller must drop when finished with it.
*/
pdf_annot *pdf_create_annot(fz_context *ctx, pdf_page *page, enum pdf_annot_type type);

and destroyed:

/*
        Delete an annotation from the page.

        This unlinks the annotation from the page structure and drops
        the pages reference to it. Any reference held by the caller
        will not be dropped automatically, so this can safely be used
        on a borrowed reference.
*/
void pdf_delete_annot(fz_context *ctx, pdf_page *page, pdf_annot *annot);

The exact operations available to work on an annotation depend upon its type, and can be found in include/mupdf/pdf/annot.h. Which operations can be called on which annotation depends largely upon its type, which can be found using:

/*
        Return the type of an annotation.
*/
enum pdf_annot_type pdf_annot_type(fz_context *ctx, pdf_annot *annot);

In general, callers can check for the existence of certain parameters using calls of the form pdf_annot_has_XXX, fetch them using pdf_annot_XXX, and set them using pdf_annot_set_XXX.

PDF Widgets

PDF encapsulates a whole range of interactivity into annotations of subtype Widget. These Widgets are used particularly for form elements and digital signatures. As such they have a richer set of operations that can be done on them.

As well as the annotation operations from include/mupdf/pdf/annot.h there are more operations that can performed, as listed in include/mupdf/pdf/form.h.

Widgets can be identified by an annotation having type PDF_ANNOT_WIDGET. The exact type of widget can then be established by calling:

/*
        Return the type of a widget.
*/
enum pdf_widget_type pdf_widget_type(fz_context *ctx, pdf_annot *annot);

Events

PDF offers interactivity that goes far beyond that of simple static pages; by using JavaScript embedded in the document that triggers according to the inputs of the user viewing it, it can make documents appear very responsive.

Accordingly, viewers need to tell MuPDF when certain things happen. These are termed ’Events’.

First off, we have document level events:

void pdf_document_event_will_close(fz_context *ctx, pdf_document *doc);
void pdf_document_event_will_save(fz_context *ctx, pdf_document *doc);
void pdf_document_event_did_save(fz_context *ctx, pdf_document *doc);
void pdf_document_event_will_print(fz_context *ctx, pdf_document *doc);
void pdf_document_event_did_print(fz_context *ctx, pdf_document *doc);

The viewer should call these at the appropriate times. These actions cannot ’succeed’ or ’fail’, but they can put up appropriate alert boxes via a registered event callback (see 19).

There are events at the page level too:

void pdf_page_event_open(fz_context *ctx, pdf_page *page);
void pdf_page_event_close(fz_context *ctx, pdf_page *page);

These should be called when the page is opened and closed, respectively.

There are events at the annotation level too. The first set are related to the page:

void pdf_annot_event_page_open(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_page_close(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_page_visible(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_page_invisible(fz_context *ctx, pdf_annot *annot);

These should be called when the page containing the annotation is opened, close, becomes visible, and becomes invisible respectively.

Next, there are a set of annotation level events to do with the users interaction with a document:

void pdf_annot_event_enter(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_exit(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_down(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_up(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_focus(fz_context *ctx, pdf_annot *annot);
void pdf_annot_event_blur(fz_context *ctx, pdf_annot *annot);

These are based upon the concept of a user interface with a pointing device (such as a traditional desktop interface equipped with a mouse). These should be called when the mouse enters and exits the annotation, when the mouse button goes down or up on the annotation, and when the annotation gains or loses focus.

Finally, there are events to do with actively editing text fields. In order to provide inline editing, we have events that enable keystrokes to be validated and formatted as they are edited.

As editing begins in a field, we should call a keystroke event as characters arrive:

typedef struct
{
        const char *value;
        const char *change;
        int selStart, selEnd;
        int willCommit;
        char *newChange;
        char *newValue;
} pdf_keystroke_event;

int pdf_field_event_keystroke(fz_context *ctx, pdf_document *doc, pdf_obj *field, pdf_keystroke_event *evt);

The handling here is somewhat involved, and is defined (poorly) by Adobe in their “JavaScript for Acrobat API Reference" document.

On each call, value contains the current value of the field, and change contains the propose new character(s) to be added. selStart and selEnd describe the region of value that is currently selected. In the case of no selection selStart and selEnd should both be set to the caret position within value. Where the selection is completely undefined selStart and selEnd should both be set to -1. It is believed that Adobe intends this to mean “the whole of change".

The proposal therefore is that we would like to insert change between selStart and selEnd within value to get a new value.

Initially this call is made with willCommit set to 0, and the call made.

The JavaScript (if there is any) will run, and the routine will return 0 or 1.

If 0, then the proposed change is rejected, and the keystroke should be ignored.

If 1, then the JavaScript has deemed that the change can go ahead, using the updated selStart and selEnd, and the updated values of value and change returned in newValue and newChange.

The caller should form this updated value, and present it to the user in whatever form it is doing the editing. Future keystrokes go through the same process until the user indicates that editing is complete (perhaps by clicking an ’OK’ button or pressing RETURN).

At this point the proposed ‘final version’ of the is presented to the event again, this time with value being the string, change being the empty string, selStart and selEnd both being -1, and willCommit being 1.

The JavaScript will again assess the proposed string (with possibly more stringent tests), and will return 0 or 1.

If it returns 0 then the string is not acceptable. If it returns 1, then the (possibly updated) version of the string in newValue should be used.

Event Callback

As part of executing the JavaScript within a document, the JavaScript engine within MuPDF can need to call out to the environment for help. This is achieved by the user of MuPDF registering an ‘Event Callback’.

Such a callback is registered using:

void pdf_set_doc_event_callback(fz_context *ctx, pdf_document *doc, pdf_doc_event_cb *event_cb, pdf_free_doc_event_data_cb *free_event_data_cb, void *data);

The user of MuPDF should call this to provide a suitable event_cb handler function that will be called to handle a range of events. The opaque data value will be passed in each time, and will be freed on closedown of the library by a call to free_event_data_cb.

The callback event encapsulates a range of scenarios:

enum
{
        PDF_DOCUMENT_EVENT_ALERT,
        PDF_DOCUMENT_EVENT_PRINT,
        PDF_DOCUMENT_EVENT_LAUNCH_URL,
        PDF_DOCUMENT_EVENT_MAIL_DOC,
        PDF_DOCUMENT_EVENT_SUBMIT,
        PDF_DOCUMENT_EVENT_EXEC_MENU_ITEM,
};

It is envisaged that the callback handler will check the type field of the supplied pdf_doc_event structure, and behave accordingly.

For each event type there is a function to get the event specific data. For instance, for alert events, it would be:

/*
        access the details of an alert event
        The returned pointer and all the data referred to by the
        structure are owned by mupdf and need not be freed by the
        caller.
*/
pdf_alert_event *pdf_access_alert_event(fz_context *ctx, pdf_doc_event *evt);

Note that the execution will be stalled until this function returns, and there is currently no way to return which button of an alert was pressed. This is a flaw, but is not easy to fix. As such alerts are useful mainly for showing errors ’after the fact’.

For details of other events see include/mupdf/pdf/event.h.


  1. A marking operation is any graphical operation that causes a mark to appear on the page.↩︎

  2. Text which has a mixture of left-to-right and right-to-left blocks↩︎