1.6 Overview
All Application Manual Name SummaryHelp

1.6.8 Blobs

Nomenclature warning:

There are two different release() functions:

Disclaimer:

The blob API for C++ is not completely general, but is designed to make common use cases easy to write. For other use cases, the underlying C API can still be used. The use case is:

A Prolog blob consists of five parts:

For the PL_blob_t structure, the C++ API provides the PL_BLOB_DEFINITION(blob_class,blob_name) macro, which references a set of template functions that allow easily setting up the callbacks. The C interface allows more flexibility by allowing some of the callbacks to default; however, the C++ API for blobs provides suitable callbacks for all of them, using the PL_BLOB_DEFINITION() macro.

For the data, which is subclassed from PlBlob, the programmer defines the various fields, a constructor that initializes them, and a destructor. Optionally, override methods can be defined for one of more of the methods PlBlob::compare_fields(), PlBlob::write_fields(), PlBlob::save(), PlBlob::load(), PlBlob::pre_delete(). More details on these are given later.

There is a mismatch between how Prolog does memory management (and garbage collection) and how C++ does it. In particular, Prolog assumes that cleanup will be done in the release() callback function associated with the blob whereas C++ typically does cleanup in a destructor. The blob interface gets around this mismatch by providing a default release() callback that assumes that the blob was created using PL_BLOB_NOCOPY and manages memory using a std::unique_ptr.15This release() function has nothing to do with std::unique_ptr::release(). More details on this are in section 1.6.8.1.

The C blob interface has a flag that determines how memory is managed: PL_BLOB_NOCOPY. The PL_BLOB_DEFINITION() macro sets this, so Prolog will call the C++ destructor when the blob is garbage collected. (This call is done indirectly, using a callback that is registeered with Prolog.)

The C++ API for blobs only supports blobs with PL_BLOB_NOCOPY.16The API can probably also support blobs with PL_BLOB_UNIQUE, but there seems to be little point in setting this flag for non-text blobs.

1.6.8.1 A review of C++ features used by the API

Some slightly obscure features of C++ are used with PlBlob and ContextType, and can easily cause subtle bugs or memory leaks if not used carefully.

When a C++ object is created, its memory is allocated (either on the stack or on the heap using new), and the constructors are called in this order:

When the object is deleted (either by stack pop or the delete operator), the destructors are called in the reverse order.

There are special forms of the constructor for copying, moving, and assigning. The “copy constructor” has a signature Type(const Type& and is used when an object is created by copying, for example by assignment or passing the object on the stack in a function call. The “move constructor” has the signature Type(Type&& and is equivalent to the copy constructor for the new object followed by the destructor for the old object. (Assignment is usually allowed to default but can also be specified).

Currently, the copy and move constructors are not used, so it is best to explicitly mark them as not existing: 

Type(const Type&) = delete;
Type(Type&&) = delete;
Type& operator =(const Type&) = delete;
Type& operator =(Type&&) = delete;

A constructor may throw an exception - good programming style is to not leave a “half constructed” object but to throw an exception. Destructors are not allowed to throw exceptions,17because the destructor might be invoked by another exception, and C++ has no mechanism for dealing with a second exception. which complicates the API somewhat.

More details about constructors and destructors can be found in the FAQs for constructors and destructors.

Many classes or types have a constructor that simply assigns a default value (e.g., 0 for int) and the destructor does nothing. In particular, the destructor for a pointer does nothing, which can lead to memory leaks. To avoid memory leaks, the smart pointer std::unique_ptr18The name “unique” is to distinguish this from a “shared” pointer. A shared pointer can share ownership with multiple pointers and the pointed-to object is deleted only when all pointers to the object have been deleted. A unique pointer allows only a single pointer, so the pointed-to object is deleted when the unique pointer is deleted. can be used, whose destructor deletes its managed object. Note that std::unique_ptr does not enforce single ownership; it merely makes single ownership easy to manage and it detects most common mistakes, for example by not having copy constructor or assignment operator.

For example, in the following, the implicit destructor for p does nothing, so there will be a memory leak when a Ex1 object is deleted: 

class Ex1 {
public:
  Ex1() : p(new int) { }
  int *p;
};

To avoid a memory leak, the code could be changed to this: 

class Ex1 {
public:
  Ex1() p(new int) { }
  ~Ex1() { delete p; }
  int *p;
};

but it is easier to do the following, where the destructor for std::unique_ptr will free the memory: 

class Ex1 {
public:
  Ex1() p(new int) { }
  std::unique_ptr<int> p;
};

The same concept applies to objects that are created in code - if a C++ object is created using new, the programmer must manage when its destructor is called. In the following, if the call to data->validate() fails, there will be a memory leak: 

MyData *foo(int some_value) {
  MyData *data = new MyData(...);
  data->some_field = some_value;
  if (! data->validate() )
    throw std::runtime_error("Failed to validate data");
  return data;
}

Ths could fixed by adding delete data before throwing the runtime_error; but this doesn't handle the situation of data->validate() throwing an exception (which would require a catch/throw). Instead, it's easiser to use std::unique_ptr, which takes care of every return or exception path: 

MyData *foo(int some_value) {
  std::unique_ptr<MyData> data(new MyData(...));
  data->some_field = some_value;
  if (! data->validate() )
    throw std::runtime_error("Failed to validate data");
  return data.release(); // don't delete the new MyData
}

The destructor for std::unique_ptr will delete the data when it goes out of scope (in this case, by return or throw) unless the std::unique_ptr::release() method is called.19The call to unique_ptr<MYData>::release doesn't call the destructor; it can be called using std::unique_ptr::get_deleter().

In the code above, the throw will cause the unique_ptr’s destructor to be called, which will free the data; but the data will not be freed in the return statement because of the unique_ptr::release(). Using this style, a pointer to data on the heap can be managed as easily as data on the stack. The current C++ API for blobs takes advantage of this - in particular, there are two methods for unifying a blob:

unique_ptr allows specifying the delete function. For example, the following can be used to manage memory created with PL_malloc(): 

  std::unique_ptr<void, decltype(&PL_free)> ptr(PL_malloc(...), &PL_free);

or, when memory is allocated within a PL_*() function (in this case, using the Plx_*() wrapper for PL_get_nchars()): 

  size_t len;
  char *str = nullptr;
  Plx_get_nchars(t, &len, &str.get(), BUF_MALLOC|CVT_ALL|CVT_WRITEQ|CVT_VARIABLE|REP_UTF8|CVT_EXCEPTION);
  std::unique_ptr<char, decltype(&PL_free)> _str(str, &PL_free);

The current C++ API assumes that the C++ blob is allocated on the heap. If the programmer wishes to use the stack, they can use std::unique_ptr to automatically delete the object if an error is thrown - PlTerm::unify_blob(std::unique_ptr<PlBlob>*) prevents the automatic deletion if unification succeeds.

A unique_ptr needs a bit of care when it is passed as an argument. The unique_ptr::get() method can be used to get the “raw” pointer; the delete must not be used with this pointer. Or, the unique_ptr::release() method can be used to transfer ownership without calling the object's destructor.

Using unique_ptr::release() is a bit incovenient, so instead the unique_ptr can be passed as a pointer (or a reference). This does not create a new scope, so the pointer must be assigned to a local variable. For example, the code for unify_blob() is something like: 

bool PlTerm::unify_blob(std::unique_ptr<PlBlob>* b) const
{ std::unique_ptr<PlBlob> blob(std::move(*b));
  if ( !unify_blob(blob.get()) )
    return false;
  (void)blob.release();
  return true;
}

The line declaration for blob uses the “move constructor” to set the value of a newly scoped variable (std::move(*b) is a cast, so unique_ptr’s move constructor is used). This has the same effect as calling b->reset(), so from this point on, b has the value nullptr.

Alternatively, the local unique_ptr could be set by 

std::unique_ptr<PlBlob> blob(b->release());

or 

std::unique_ptr<PlBlob> blob;
blob.swap(*b);

If the call to PlTerm::unify_blob() fails or throws an exception, the virtual destructor for blob is called. Otherwise, the call to blob.release() prevents the destructor from being called - Prolog now owns the blob object and can call its destructor when the garbage collector reclaims it.

1.6.8.2 How to define a blob using C++

TL;DR: Use PL_BLOB_DEFINITION() to define the blob with the flag PL_BLOB_NOCOPY and the default PlBlob wrappers; define your struct as a subclass of PlBlob with no copy constructor, move constructor, or assignment operator; create a blob using std::unique_ptr<PlBlob>(new ...), call PlTerm::unify_blob(). Optionally, define one or more of: compare_fields(), write_fields(), save(), load() methods (these are described after the sample code).

1.6.8.3 The life of a PlBlob

In this section, the blob is of type MyBlob, a subclass of PlBlob. (Example code is given in section 1.6.8.5) and section 1.6.8.7.

A blob is typically created by calling a predicate that does the following:

At this point, the blob is owned by Prolog and may be freed by its atom garbage collector, which will call the blob's destructor (if the blob shouldn't be deleted, it can override the the PlBlob::pre_delete() method to return false).

Whenever a predicate is called with the blob as an argument (e.g., as A1), the blob can be accessed by PlBlobv<MyBlob>::cast_check(A1.as_atom()).

Within a method, the Prolog blob can be accessed as a term (e.g., for constructing an error term) using the method MyBlob::symbol_term(). This field is initialized by the call to PlTerm::unify_blob(); if MyBlob::symbol_term() is called before a successful call to PlTerm::unify_blob(), MyBlob::symbol_term() returns a PlTerm_var.

When the atom garbage collector runs, it frees the blob by first calling the release() callback, which does delete, which calls the destructor MyBlob::~MyBlob(). Note that C++ destructors are not supposed to raise exception; they also should not cause a Prolog error, which could cause deadlock unless the real work is done in another thread.

Often it is desired to release the resources before the garbage collector runs. To do this, the programmer can provide a “close” predicate that is the inverse of the “open” predicate that created the blob. This typically has the same logic as the destructor, except that it can raise a Prolog error.

1.6.8.4 C++ exceptions and blobs

When a blob is used in the context of a PREDICATE() macro, it can raise a C++ exception (PlFail or PlException) and the PREDICATE() code will convert the exception to the appropriate Prolog failure or error; memory allocation exceptions are also handled.

Blobs have callbacks, which can run outside the context of a PREDICATE(). Their exception handling is as follows:

void PlBlob::acquire()
, which is called from PlBlobV<MyBlob>::acquire(), can throw a C++ exception. The programmer cannot override this.
int PlBlob::compare_fields(const PlBlob *_b)
, which is called from PlBlobV<MyBlob>::compare(), should not throw an exception. A Prolog error won't work as it uses “raw pointers” and thus a GC or stack shift triggered by creating the exception will upset the system.
bool PlBlob::write_fields(IOStream *s, int flags)
, which is called from PlBlobV<MyBlob>::write(), can throw an exception, just like code inside a PREDICATE(). In particular, you can wrap calls to Sfprintf() in PlCheckFail(), although the calling context will check for errors on the stream, so checking the Sfprintf() result isn't necessary.
void PlBlob::PlBlob::save(IOStream *fd)
can throw a C++ exception, including PlFail().
PlAtom PlBlob::PlBlob::load(IOSTREAM *fd)
can throw a C++ exception, which is converted to a return value of PlAtom::null, which is interpreted by Prolog as failure.
bool PlBlob::PlBlob::pre_delete()
, which is called from PlBLobV<MyBLOB>::release(), can return false (or throw a PlException or PlExceptinFailBase, which will be interpreted as a return value of false), resulting in the blob not being garbage collected, and the destructor not being called. Note that this doesn't work well with final clean-up atom garbage collection, which disregards the return value and also doesn't respect the ordering of blob dependencies (e.g., if an iterator blob refers to a file-like blob, the file-like blob might be deleted before the iterator is deleted).

This code runs in the gc thread. The only PL_*() function that can safely be called are PL_unregister_atom() (which is what PlAtom::unregister_ref() calls).

1.6.8.5 Sample PlBlob code (connection to database)

Here is minimal sample code for creating a blob that owns a connection to a database. It has a single field (connection) and defines compare_fields() and write_fields().

A second sample code shows how to wrap a system pointer - section 1.6.8.7 

struct MyConnection
{ std::string name;

  explicit MyConnection();
  explicit MyConnection(const std::string& _name);
  bool open();
  bool close() noexcept;
  void portray(PlStream& strm) const;
};

struct MyBlob;

static PL_blob_t my_blob = PL_BLOB_DEFINITION(MyBlob, "my_blob");

struct MyBlob : public PlBlob
{ std::unique_ptr<MyConnection> connection;

  explicit MyBlob()
    : PlBlob(&my_blob) { }

  explicit MyBlob(const std::string& connection_name)
    : PlBlob(&my_blob),
      connection(std::make_unique<MyConnection>(connection_name))
  { if ( !connection->open() )
      throw MyBlobError("my_blob_open_error");
  }

  PL_BLOB_SIZE

  ~MyBlob() noexcept
  { if ( !close() )
      Sdprintf("***ERROR: Close MyBlob failed: %s\n", name().c_str()); // Can't use PL_warning()
  }

  inline std::string
  name() const
  { return connection ? connection->name : "";
  }

  bool close() noexcept
  { if ( !connection )
      return true;
    bool rc = connection->close();
    connection.reset(); // Can be omitted, leaving deletion to ~MyBlob()
    return rc;
  }

  PlException MyBlobError(const char* error) const
  { return PlGeneralError(PlCompound(error, PlTermv(symbol_term())));
  }

  int compare_fields(const PlBlob* _b_data) const override
  { auto b_data = static_cast<const MyBlob*>(_b_data); // See note about cast
    return name().compare(b_data->name());
  }

  bool write_fields(IOSTREAM *s, int flags) const override
  { PlStream strm(s);
    strm.printf(",");
    return write_fields_only(strm);
  }

  bool write_fields_only(PlStream& strm) const
  { if ( connection )
      connection->portray(strm);
    else
      strm.printf("closed");
    return true;
  }

  bool portray(PlStream& strm) const
  { strm.printf("MyBlob(");
    write_fields_only(strm);
    strm.printf(")");
    return true;
  }
};

// %! create_my_blob(+Name: atom, -MyBlob) is semidet.
PREDICATE(create_my_blob, 2)
{ // Allocating the blob uses std::unique_ptr<MyBlob> so that it'll be
  // deleted if an error happens - the auto-deletion is disabled by
  // ref.release() inside unify_blob() before returning success.

  auto ref = std::unique_ptr<PlBlob>(new MyBlob(A1.as_atom().as_string()));
  return A2.unify_blob(&ref);
}

// %! close_my_blob(+MyBlob) is det.
// % Close the connection, silently succeeding if is already
// % closed; throw an exception if something goes wrong.
PREDICATE(close_my_blob, 1)
{ auto ref = PlBlobV<MyBlob>::cast_ex(A1, my_blob);
  if ( !ref->close() )
    throw ref->MyBlobError("my_blob_close_error");
  return true;
}

// %! portray_my_blob(+Stream, +MyBlob) is det.
// % Hook predicate for
// %   user:portray(MyBlob) :-
// %     blob(MyBlob, my_blob), !,
// %     portray_my_blob(current_output, MyBlob).
PREDICATE(portray_my_blob, 2)
{ auto ref = PlBlobV<MyBlob>::cast_ex(A2, my_blob);
  PlStream strm(A1, 0);
  return ref->portray(strm);
}

1.6.8.6 Discussion of the sample PlBlob code

1.6.8.7 Sample PlBlob code (wrapping a pointer)

struct MyFileBlob;

static PL_blob_t my_file_blob = PL_BLOB_DEFINITION(MyFileBlob, "my_file_blob");

static const PlOptionsFlag<int>
MyFileBlob_options("MyFileBlob-options",
                   { {"absolute", PL_FILE_ABSOLUTE},
                     {"ospath",   PL_FILE_OSPATH},
                     {"search",   PL_FILE_SEARCH},
                     {"exist",    PL_FILE_EXIST},
                     {"read",     PL_FILE_READ},
                     {"write",    PL_FILE_WRITE},
                     {"execute",  PL_FILE_EXECUTE},
                     {"noerrors", PL_FILE_NOERRORS} });

struct MyFileBlob : public PlBlob
{ std::FILE* file_;

  std::string mode_;
  int flags_;
  std::string filename_;
  std::vector<char> buffer_; // used by read(), to avoid re-allocation

  explicit MyFileBlob()
    : PlBlob(&my_file_blob) { }

  explicit MyFileBlob(PlTerm filename, PlTerm mode, PlTerm flags)
    : PlBlob(&my_file_blob),
      mode_(mode.as_string())
  { flags_ = MyFileBlob_options.lookup_list(flags);
    filename_ = filename.get_file_name(flags_);
    file_ = fopen(filename_.c_str(), mode_.c_str());
    if ( !file_ ) // TODO: get error code (might not be existence error)
      throw PlExistenceError("my_file_blob_open", PlTerm_string(filename_));
    // for debugging:
    //   PlTerm_string(filename.as_string() + "\" => \"" +
    //                 filename_ + "\", \"" + mode_ +
    //                 ", flags=" + MyFileBlob_options.as_string(flags_) + "\")")
  }

  PL_BLOB_SIZE

  std::string read(size_t count)
  { assert(sizeof buffer_[0] == sizeof (char));
    assert(sizeof (char) == 1);

    buffer_.reserve(count);
    return std::string(buffer_.data(),
                       std::fread(buffer_.data(), sizeof buffer_[0], count, file_));
  }

  bool eof() const
  { return std::feof(file_);
  }

  bool error() const
  { return std::ferror(file_);
  }

  virtual ~MyFileBlob() noexcept
  { if ( !close() )
      // Can't use PL_warning()
      Sdprintf("***ERROR: Close MyFileBlob failed: (%s)\n", filename_.c_str());
  }

  bool close() noexcept
  { if ( !file_ )
      return true;
    int rc = std::fclose(file_);
    file_ = nullptr;
    return rc == 0;
  }

  PlException MyFileBlobError(const std::string error) const
  { return PlGeneralError(PlCompound(error, PlTermv(symbol_term())));
  }

  int compare_fields(const PlBlob* _b_data) const override
  { // dynamic_cast is safer than static_cast, but slower (see documentation)
    // It's used here for testing (the documentation has static_cast)
    auto b_data = dynamic_cast<const MyFileBlob*>(_b_data);
    return filename_.compare(b_data->filename_);
  }

  bool write_fields(IOSTREAM *s, int flags) const override
  { PlStream strm(s);
    strm.printf(",");
    return write_fields_only(strm);
  }

  bool write_fields_only(PlStream& strm) const
  { // For debugging:
    // strm.printf("%s mode=%s flags=%s", filename_.c_str(), mode_.c_str(),
    //             MyFileBlob_options.as_string(flags_).c_str());
    strm.printf("%s", filename_.c_str());
    if ( !file_ )
      strm.printf("-CLOSED");
    return true;
  }

  bool portray(PlStream& strm) const
  { strm.printf("MyFileBlob(");
    write_fields_only(strm);
    strm.printf(")");
    return true;
  }
};

PREDICATE(my_file_open, 4)
{ auto ref = std::unique_ptr<PlBlob>(new MyFileBlob(A2, A3, A4));
  return A1.unify_blob(&ref);
}

PREDICATE(my_file_close, 1)
{ auto ref = PlBlobV<MyFileBlob>::cast_ex(A1, my_file_blob);
  if ( !ref->close() ) // TODO: get the error code
    throw ref->MyFileBlobError("my_file_blob_close_error");
  return true;
}

1.6.8.8 Discussion of the sample PlBlob code (wrapping a pointer)

1.6.8.9 Identifying blobs by atoms

Passing a Prolog blob around can be inconvenient; it is easier if a blob can be identified an atom. An example of this is with streams, which are identified by atoms such as user_input.

A utility class AtomMap is provided for this situation. See section 1.17.4.

ClioPatria (version V3.1.1-51-ga0b30a5)