Component of the Week #38: bdld_datum

Summary:

  • Provides a discriminated variant type with a small footprint, particularly useful for interfacing with or creating scripting languages.

The bdld_datum component defines a mechanism, bdld::Datum, that provides a space-efficient discriminated union (i.e., a variant) that can hold values of many different types - from scalars like integers and doubles to aggregates like arrays and maps. Unlike typical BDE types, Datum is not a value-semantic type but behaves more like a pointer, requiring explicit memory management similar to raw pointers. However, Datum does provide equality comparison operators (operator== and operator!=) that compare the values to which the Datum objects refer, not just the addresses.

This design makes Datum ideal for applications that create and copy very large numbers of variant values, such as:

  • Interfacing with dynamic languages like JavaScript

  • Creating scripting languages like Bloomberg’s HSL (Hypersheet Scripting Language)

  • Spreadsheet implementations where cells can contain different types

  • Data interchange and serialization scenarios

The key characteristic that distinguishes Datum from other BDE types is its pointer-like behavior: copying a Datum performs a shallow copy (like copying a pointer), and memory must be explicitly managed using static Datum::create* and Datum::destroy functions. When deep copying is needed, Datum provides a clone method that creates independent, self-contained copies where external references are converted to owned data.

Understanding Datum’s Memory Model

Datum is a POD (Plain Old Data) type with a memory model fundamentally different from typical BDE value-semantic types. The analogy to raw pointers is instructive: Datum::create* is like new, Datum::destroy is like delete, and copying a Datum is like copying a pointer (both point to the same memory).

#include <bdld_datum.h>
#include <bslma_testallocator.h>
#include <bsl_iostream.h>

using namespace BloombergLP;

bslma::TestAllocator ta;

// Create a long string that exceeds short-string optimization
const char* longText = "This is a long string that will definitely "
                       "exceed the short string optimization limit "
                       "on 64-bit platforms and require allocation";

bdld::Datum original = bdld::Datum::copyString(longText, &ta);
bsl::cout << "Original: " << original.theString() << bsl::endl;
bsl::cout << "Address:  "
          << static_cast<const void*>(original.theString().data())
          << bsl::endl;

// This is a shallow copy - both point to the same string data!
bdld::Datum shallowCopy = original;
bsl::cout << "Shallow copy: " << shallowCopy.theString() << bsl::endl;
bsl::cout << "Address:      "
          << static_cast<const void*>(shallowCopy.theString().data())
          << bsl::endl;
bsl::cout << "Same address? "
          << (original.theString().data() == shallowCopy.theString().data()
              ? "yes" : "no") << bsl::endl;

// Only destroy once (destroying twice would be an error)
bdld::Datum::destroy(original, &ta);
// shallowCopy now refers to deallocated memory - do not use it!

Deep Copying with clone

When deep copying is needed, the clone method creates an independent copy where external references are converted to owned data:

// Deep copy - creates independent copy of the string
bdld::Datum deepCopy = original.clone(&ta);
bsl::cout << "Deep copy: " << deepCopy.theString() << bsl::endl;
bsl::cout << "Address:   "
          << static_cast<const void*>(deepCopy.theString().data())
          << bsl::endl;
bsl::cout << "Different address? "
          << (original.theString().data() != deepCopy.theString().data()
              ? "yes" : "no") << bsl::endl;

// Both can now be destroyed independently
bdld::Datum::destroy(original, &ta);
bdld::Datum::destroy(deepCopy, &ta);

Efficient Temporary Management with Managed Allocators

A common pattern uses a managed allocator (like bdlma::SequentialAllocator) for temporary Datum values during computation, then uses clone to extract the final result before releasing all temporaries at once. This approach avoids double-delete errors from shallow copies, eliminates individual deallocation overhead, and handles self-referencing structures safely since all temporaries vanish together atomically. This pattern is extensively used in expression evaluators and scripting implementations.

See Component of the Week #34: bdlma_sequentialallocator for more information on bdlma::SequentialAllocator.

Working with Arrays and Maps

The bdld package provides builder components ( DatumArrayBuilder, DatumMapBuilder, and DatumIntMapBuilder) for constructing arrays and maps of Datum objects in an exception-safe manner. These builders handle all memory allocation complexity and are the recommended approach for creating aggregate Datum values.

The bdld Package Ecosystem

The bdld package provides several supporting components:

External References vs. Owned Data

Datum can reference external data without taking ownership (for performance) or own its data. Factory methods like createStringRef and createArrayReference create references where isExternalReference() returns true, while methods like copyString and adoptArray create owned data.

When using clone to deep-copy a Datum, external references are converted to owned data (except for user-defined types which remain opaque pointers).

For More Information

For comprehensive details on memory management, all supported types, and advanced usage patterns, see: