• Places
    • Home
    • Graphs
    • Prefixes
  • Admin
    • Users
    • Settings
    • Plugins
    • Statistics
  • CPACK
    • Home
    • List packs
    • Submit pack
  • Repository
    • Load local file
    • Load from HTTP
    • Load from library
    • Remove triples
    • Clear repository
  • Query
    • YASGUI SPARQL Editor
    • Simple Form
    • SWISH Prolog shell
  • Help
    • Documentation
    • Tutorial
    • Roadmap
    • HTTP Services
  • Login

12.4.9 BLOBS: Using atoms to store arbitrary binary data
All Application Manual Name SummaryHelp

  • Documentation
    • Reference manual
      • Foreign Language Interface
        • The Foreign Include File
          • BLOBS: Using atoms to store arbitrary binary data
            • Defining a BLOB type
              • PL_unregister_blob_type()
              • PL_register_blob_type()
            • Accessing blobs
    • Packages
th additional type information. The type is represented by a structure holding C function pointers that tell Prolog how to handle releasing the atom, writing it, sorting it, etc. Two atoms created with different types can represent the same sequence of bytes. Atoms are first ordered on the rank number of the type and then on the result of the compare() function. Rank numbers are assigned when the type is registered. This implies that the results of inequality comparisons between blobs of different types is undefined and can change if the program is run twice (the ordering within a blob type will not change, of course).

While the blob is alive, neither its handle nor the location of the contents (see PL_blob_data()) change. If the blob's type has the PL_BLOB_UNIQUE feature, the content of the blob must remain unmodified. If the blob's type does not have the PL_BLOB_UNIQUE feature, then none of the instances will be equal. Blobs are only reclaimed by the atom garbage collector. In this case the atom garbage collector calls the release() function associated with the blob type and reclaims the memory allocated for the content unless this is owned by the creator of the blob indicated by the PL_BLOB_NOCOPY flag. After an atom_t value is reclaimed by the atom garbage collector, the value may be reused for allocating a new blob or atom.

If foreign code stores the atom_t handle in some permanent location it must make sure the handle is registered to prevent it from being garbage collected. If the handle is obtained from a term_t object it is not registered because it is protected by the term_t object. This applies to e.g., PL_get_atom(). Functions that create a handle from data, such as PL_new_atom(), return a registered handle to prevent the asynchronous atom garbage collector from reclaiming it immediately. Note that many of the API functions create an atom or blob handle and use this to fill a term_t object, e.g., PL_unify_blob(), PL_unify_chars(), etc. In this scenario the handle is protected by the term_t object. Registering and unregistering atom_t handles is done by PL_register_atom() and PL_unregister_atom().

Note that during program shutdown using PL_cleanup(), all atoms and blobs are reclaimed as described above. These objects are reclaimed regardless of their registration count. The order in which the atoms or blobs are reclaimed under PL_cleanup() is undefined. However, when these objects are reclaimed using garbage_collect_atoms/0, registration counts are taken into account.

12.4.10.1 Defining a BLOB type

The type PL_blob_t represents a structure with the layout displayed below. The structure contains additional fields at the ... for internal bookkeeping as well as future extensions.

typedef struct PL_blob_t
{ uintptr_t     magic;          /* PL_BLOB_MAGIC */
  uintptr_t     flags;          /* Bitwise or of PL_BLOB_* */
  const char *  name;           /* name of the type */
  int           (*release)(atom_t a);
  int           (*compare)(atom_t a, atom_t b);
  int           (*write)(IOSTREAM *s, atom_t a, int flags);
  void          (*acquire)(atom_t a);
  int           (*save)(atom_t a, IOSTREAM *s);
  atom_t        (*load)(IOSTREAM *s);
  ...
} PL_blob_t;

For each type, exactly one such structure should be allocated and must not be moved because the address of the structure determines the blob's "type". Its first field must be initialised to PL_BLOB_MAGIC. If a blob type is registered from a loadable object (shared object or DLL) the blob type must be deregistered using PL_unregister_blob_type() before the object may be released.

The flags is a bitwise or of the following constants:

PL_BLOB_TEXT
If specified, the blob is assumed to contain text and is considered a normal Prolog atom. The (currently) two predefined blob types that represent atoms have this flag set. User-defined blobs may not specify this, even if they contain only text. Applications should not use the blob API to create normal text atoms or get access to the text represented by normal text atoms. Most applications should use PL_get_nchars() and PL_unify_chars() to get text from Prolog terms or create Prolog terms that represent text.
PL_BLOB_UNIQUE
If specified the system ensures that the blob-handle is a unique reference for a blob with the given type, length and content. If this flag is not specified, each lookup creates a new blob. Uniqueness is determined by comparing the bytes in the blobs unless PL_BLOB_NOCOPY is also specified, in which case the pointers are compared. Note that the lookup does not use the blob's compare function when testing for equality, but only tests the bytes; this means that terms from the recorded database or C++-style strings will typically not compare as equal when doing blob lookup.
PL_BLOB_NOCOPY
By default the content of the blob is copied. Using this flag the blob references the external data directly. The user must ensure the provided pointer is valid as long as the atom lives. If PL_BLOB_UNIQUE is also specified, uniqueness is determined by comparing the pointer rather than the data pointed at. Using PL_BLOB_UNIQUE|PL_BLOB_NOCOPY can be used to make a blob reference an arbitrary pointer where the pointer data may be reclaimed in the release() handler.
PL_BLOB_WCHAR
If PL_BLOB_TEXT is also set, then the text is made up of pl_wchar_t items and the blob's lenght is the number of bytes (that is, the number of characters times sizeof(pl_wchar_t)). As PL_BLOB_TEXT, this flag should not be set in user-defined blobs.

The name field represents the type name as available to Prolog. See also current_blob/2. The other fields are function pointers that must be initialised to proper functions or NULL to get the default behaviour of built-in atoms. Below are the defined member functions:

void acquire(atom_t a)
Called if a new blob of this type is created through PL_put_blob(), PL_unify_blob(), or PL_new_blob(). Note this this call is done as part of creating the blob. The call to PL_unify_blob() may fail if the unification fails or cannot be completed due to a resource error. PL_put_blob() has no such error conditions. This callback is typically used to store the atom_t handle into the content of the blob. Given a pointer to the content, we can now use PL_unify_atom() to bind a Prolog term with a reference to the pointed to object. If the content of the blob can be modified (PL_BLOB_UNIQUE is not present) this is the only way to get access to the atom_t handle that belongs to this blob. If PL_BLOB_UNIQUE is provided and respected, PL_unify_blob() given the same pointer and length will produce the same atom_t handle.
int release(atom_t a)
The blob (atom) a is about to be released. This function can retrieve the data of the blob using PL_blob_data(). If it returns FALSE, the atom garbage collector will not reclaim the atom. The release() function is called when the atom is reclaimed by the atom garbage collector. For critical resources such as file handles or significant memory resources it may be desirable to have an explicit call to dispose (most of) the resources. For example, close/1 reclaims the file handle and most of the resources associated with a stream, leaving only a tiny bit of content to the garbage collector. See also setup_call_cleanup/3.

The release() callback is called in the context of the thread executing the atom garbage collect. Normally the thread gc runs all atom and clause garbage collections. The release() function may not call any of the PL_*() functions except for PL_unregister_atom() to unregister other atoms that are part data associated to the blob. Calling any of the other PL_* functions may return in deadlocks or crashes. The release() function should not call any potentially slow or blocking functions as this may cause serious slowdowns in the rest of the system.

Blobs that require cleanup that is slow, blocking or requires calling Prolog must pass the data to be cleaned to another thread. Be aware that if the blob uses PL_BLOB_NOCOPY the user is responsible for discarding the data, otherwise the atom garbage collector will free the data.

As SWI-Prolog atom garbage collector is conservative, there is no guarantee that the release() function will ever be called. If it is important to clean up some resource, there should be an explicit predicate for doing that, and calling that predicate should be guaranteed by using setup_call_cleanup/3 or some a process finalization hook such as at_halt/1.

Normally, Prolog does not clean memory during shutdown. It does so on an explicit call to PL_cleanup().222Or if the system is compiled with the cmake build type Debug. In such a situation, there is no guarantee of the order in which atoms are released; if a blob contains an atom (or another blob), those atoms (or blobs) may have already been released. See also PL_blob_data().

int compare(atom_t a, atom_t b)
Compare the blobs a and b, both of which are of the type associated to this blob type. Return values are as memcmp(): < 0 if a is less than b, = 0 if both are equal, and > 0 otherwise. The default implementation is a bitwise comparison of the blobs' contents. This default implementation suffices if PL_BLOB_UNIQUE is set and the blob follows the requirement that its contents do not change, although it might give an unexpected ordering, and the ordering may change if the blob is saved and restored using save_program/2.

If the compare() function is defined, the sort/2 predicate uses that to determine if two blobs are equal and only keeps one of them. This can cause unexpected results with blobs that are actually different; if you cannot guarantee that the blobs all have unique contents, then you should incorporate the blob address (the system guarantees that blobs are not shifted in memory after they are allocated). This function should not call any PL_*() functions other than PL_blob_data().

The following minimal compare function gives a stable total ordering:

static int
compare_my_blob(atom_t a, atom_t b)
{ const struct my_blob_data *blob_a = PL_blob_data(a, NULL, NULL);
  const struct my_blob_data *blob_b = PL_blob_data(b, NULL, NULL);
  return (blob_a < blob_b) ? -1 : (blob_a > blob_b) ? 1 : 0;
}
int write(IOSTREAM *s, atom_t a, int flags)
Write the content of the blob a to the stream s respecting the flags. The return value is TRUE or FALSE and does not follow the Unix convention of the number of bytes (where zero is possible) and negative for errors. Any I/O operations to s are in the context of a PL_acquire_stream(); upon return, the PL_release_stream() handles any errors, so it is safe to not check return codes from Sfprintf(), etc.

In general, the output from the write() callback should be minimal. If you wish to output more debug information, it is suggested that you either add a debug option to your "open" predicate to output more information, or provide a "properties" predicate. A typical implementation is:

static int write_my_blob(IOSTREAM *s, atom_t symbol, int flags)
{ (void)flags; /* unused */
  Sfprintf(s, "<my_blob>(%p)", PL_blob_data(symbol, NULL, NULL));
  return TRUE;
}

The flags are a bitwise or of zero or more of the PL_WRT_* flags that were passed in to the calling PL_write_term() that called write(), and are defined in SWI-Prolog.h. The flags do not have the PL_WRT_NEWLINE bit set, so it is safe to call PL_write_term() and there is no need for writing a t

ClioPatria (version V3.1.1-42-gd6a756b-DIRTY)