- Generated by
1.9.8
|
BDE 4.39.x Production Release
|
Basic Application Library Logging (ball)
Provide thread-safe logging toolkit suitable for all platforms.
Basic Application Library Logging (ball)
The 'ball' package provides a toolkit for logging messages in applications and library code. The logger toolkit has an administration layer that allows configuration both at start-up and during program execution, a basic logger API for the most general possible use, and two sets of macros for somewhat less flexible but simpler, more convenient use. In particular, messages may be logged via these macros in either a C++ stream-style syntax (i.e., with the '<<' operator) or a 'printf'-style syntax. Users are encouraged to use the macros exclusively, because they provide uniformity, and because they are less error-prone. See the {Appendix: Macro Reference} section below.
This document contains a number of code examples, explained below:
The following section provides short examples highlighting some important features of logging.
The following trivial example shows how to use the logging macros to log messages at various levels of severity.
First, we include 'ball_log.h', then create an example function where we initialize the log category within the context of this function. The logging macros such as 'BALL_LOG_ERROR' will not compile unless a category has been specified in the current lexical scope:
Then, we record messages at various levels of severity. These messages will be conditionally written to the log depending on the current logging threshold of the category (configured using the 'ball::LoggerManager' singleton):
Next, we demonstrate how to use proprietary code within logging macros. Suppose you want to add the content of a vector to the log trace:
Notice that the code block will be conditionally executed depending on the current logging threshold of the category. The code within the block must not produce any side effects, because its execution depends on the current logging configuration. The special macro 'BALL_LOG_OUTPUT_STREAM' provides access to the log stream within the block.
Then we show a simple class that declares a log category for the class (log categories can also be configured at namespace scope in a '.cpp' file):
Finally we can use a 'ball::ScopedAttribute' to associate an attribute with the current thread's logging context.
Notice that the attribute "mylibrary.security" will be associated with each log message generated by the current thread until the destruction of the 'securityAttribute' object (including the log message created by 'privateRetrieveData'). To publish the attribute to the log the 'ball::Observer' must be configured correctly (e.g., using the "%a" format specification with 'ball::FileObserver' or 'ball::RecordStringFormatter'), as we do in the subsequent example.
Clients that perform logging must first instantiate the singleton logger manager using the 'ball::LoggerManagerScopedGuard' class. This example shows how to create a logger manager with basic "default behavior". Subsequent examples will show more customized behavior.
The following snippets of code illustrate the initialization sequence (typically performed near the top of 'main').
First, we create a 'ball::LoggerManagerConfiguration' object, 'configuration', and set the logging "pass-through" level – the level at which log records are published to registered observers – to 'WARN' (see {'Categories, Severities, and Threshold Levels'}):
Next, create a 'ball::LoggerManagerScopedGuard' object whose constructor takes the configuration object just created. The guard will initialize the logger manager singleton on creation and destroy the singleton upon destruction. This guarantees that any resources used by the logger manager will be properly released when they are not needed:
Note that the application is now prepared to log messages using the 'ball' logging subsystem, but until the application registers an observer, all log messages will be discarded.
Finally, we create a 'ball::FileObserver' object 'observer' that will publish records to a file, and exceptional records to 'stdout'. We configure the log format to publish log attributes (see {Key Example 1: Write to a Log}, enable the logger to write to a log file, and then register 'observer' with the logger manager. Note that observers must be registered by name; this example simply uses "default" for a name:
The application is now prepared to log messages using the 'ball' logging subsystem:
The 'ball' package currently has 64 components having 18 levels of physical dependency. The list below shows the hierarchical ordering of the components. The order of components within each level is not architecturally significant, just alphabetical.
ball_administration : Provide a suite of utility functions for logging administration.
ball_asyncfileobserver : Provide an asynchronous observer that logs to a file and stdout.
ball_attribute : Provide a representation of (literal) name/value pairs.
ball_attributecollectorregistry : Provide a registry for attribute collector functors.
ball_attributecontainer : Provide a protocol for containers holding logging attributes.
ball_attributecontainerlist : Provide a list of attribute container addresses.
ball_attributecontext : Provide a container for storing attributes and caching results.
ball_broadcastobserver : Provide a broadcast observer that forwards to other observers.
ball_category : Provide a container for a name and associated thresholds.
ball_categorycallbacks : Provide category related callback function types.
ball_categorymanager : Provide a manager of named categories each having "thresholds".
ball_categorymanager_radixtree : !PRIVATE! Provide a space-efficient associative container for string keys.
ball_categorymanager_radixtree_cpp03 : !PRIVATE! Provide C++03 implementation for ball_categorymanager_radixtree.h
ball_context : Provide a container for the context of a transmitted log record.
ball_countingallocator : Provide a concrete allocator that keeps count of allocated bytes.
ball_cstdioobserver : Provide an observer that emits log records to a FILE *.
ball_defaultattributecontainer : Provide a default container for storing attribute name/value pairs.
ball_fileobserver : Provide a thread-safe observer that logs to a file and to stdout.
ball_fileobserver2 : Provide a thread-safe observer that emits log records to a file.
ball_filteringobserver : Provide an observer that filters log records.
ball_fixedsizerecordbuffer : Provide a thread-safe fixed-size buffer of record handles.
'ball_fmt': Provide macros to facilitate bsl::format logging.
ball_hierarchicalcategorysetting : Provide a container for a name prefix and associated thresholds.
'ball_log': Provide macros and utility functions to facilitate logging.
ball_logfilecleanerutil : Provide a utility class for removing log files.
ball_loggercategoryutil : Provide a suite of utility functions for category management.
ball_loggerfunctorpayloads : Provide a suite of logger manager singleton functor payloads.
ball_loggermanager : Provide a manager of core logging functionality.
ball_loggermanagerconfiguration : Provide a constrained-attribute class for the logger manager.
ball_loggermanagerdefaults : Provide constrained default attributes for the logger manager.
ball_logthrottle : Provide throttling equivalents of some of the ball_log macros.
ball_managedattribute : Provide a wrapper for ball::Attribute with managed name storage.
ball_managedattributeset : Provide a container for managed attributes.
ball_multiplexobserver : !DEPRECATED! Provide a multiplexing observer that forwards to other observers.
ball_observer : Define a protocol for receiving and processing log records.
ball_observeradapter : Provide a helper for implementing the ball::Observer protocol.
ball_observerformatterimp : Provide common methods for scheme-based formatters for observers
ball_patternutil : Provide a utility class for string pattern matching.
ball_predicate : !DEPRECATED! Provide a predicate object that consists of a name/value pair.
ball_predicateset : !DEPRECATED! Provide a container for managed attributes.
ball_record : Provide a container for the fields and attributes of a log record.
ball_recordattributes : Provide a container for a fixed set of fields suitable for logging.
ball_recordbuffer : Provide a protocol for managing log record handles.
ball_recordformatterfunctor : Provide a typedef for the record formatter functor.
ball_recordformatteroptions : Provides log record formatter option values.
ball_recordformatterregistryutil : Provide utilities for creating log record formatters by scheme.
ball_recordformattertimezone : Enumerate a set of timezone defaults for log timestamps.
ball_recordjsonformatter : Provide a formatter for log records that renders output in JSON.
ball_recordstringformatter : Provide a record formatter that uses a printf-style format spec.
ball_rule : Provide an object having a pattern, thresholds, and attributes.
ball_ruleset : Provide a set of unique rules.
ball_scopedattribute : Provide a scoped guard for a single BALL attribute.
ball_scopedattributes : Provide a class to add and remove attributes automatically.
ball_severity : Enumerate a set of logging severity levels.
ball_severityutil : Provide a suite of utility functions on ball::Severity levels.
ball_streamobserver : Provide an observer that emits log records to a stream.
ball_testobserver : Provide an instrumented observer for testing.
ball_thresholdaggregate : Provide an aggregate of the four logging threshold levels.
ball_thresholddefaults : Provide default threshold values.
ball_transmission : Enumerate the set of states for log record transmission.
ball_userfields : Provide a container of user supplied field values.
ball_userfieldtype : Enumerate the set of data types for a user supplied attribute.
ball_userfieldvalue : Provide a type for the value of a user supplied field.
The 'ball' logging toolkit is thread-enabled and suitable for multi-threaded applications. At the user's option, the multi-threaded toolkit permits each thread to install a distinct instance of 'ball::Logger'; the process-wide "default" logger is available to any thread that does not install its own logger.
This section provides a brief overview of the features of the 'ball' logging toolkit, and introduces (without formal definition) some of the terminology used in 'ball'. Refer to the hierarchical and alphabetical Synopsis sections above to associate these features with the overall 'ball' design, and see the various sections below for more detailed descriptions.
The 'ball' package provides a flexible logging toolkit that supports several standard and non-standard features. Perhaps most notable among the non-standard features is the ability to write messages to an in-memory "circular" (finite) buffer with the expectation that those messages will be overwritten and never actually "logged" to any permanent medium. With this feature, during normal production operation a large quantity of "trace-back" information can be "logged" to memory, and typically be discarded, without having clogged production resources. If, however, some "error" occurs, the information just prior to that error will be available for fast diagnosis and debugging. It is easy to re-configure logger operation so that every message is archived, if that is the preferred behavior, but the efficient trace-back feature is an explicit design goal. See the "Hello world!" examples under {Usage} below for illustrations of how to configure the logger behavior.
Another key design feature is the "observer" object. 'ball' defines the 'ball::Observer' protocol (abstract interface), and provides a few concrete observers. It is expected that most users will find the concrete observers that are provided in 'ball' to be sufficient for their needs. However, users are free to define their own observers tailored to meet their specific requirements. In the current release, exactly one observer is registered with the 'ball' logger on initialization; we anticipate that multiple observers, e.g., one per thread or perhaps one per logger, may be available in future releases. An observer makes its 'publish' method available to the logger; since the 'publish' method is free to do almost anything that the user wants, the limitation of one observer per process is not very restrictive. The observer may: write the message to a simple file, write the message to a set of managed files, write the message to the console (perhaps with some information removed and/or added), process the message (complete with Category and Severity information) and take specific responsive actions, or any combination of these or other behaviors. In particular, 'ball' provides a "broadcast observer", 'ball::BroadcastObserver', that forwards log records to any number of observers registered with the broadcast observer. Note that 'ball::LoggerManager' contains an integrated broadcast observer and all observers registered with the logger manager will receive log records.
The logger supports the notions of severity levels and categories. Every message is logged at some severity level and to some category. Categories are user-defined (except for the "default category"), and can be separately managed; in particular, the behavior of any given message-logging operation depends upon specific severity level threshold settings for the category to which the message is being logged. See the ball_loggermanager component for more details about categories and category administration.
Severity levels are, from the perspective of the basic logger API, user-settable in the range '[0 .. 255]'. Much more commonly, however, (and necessarily when using the 'ball' convenience macros defined in the 'ball_log' component), the user will choose to use the fixed set of enumerated severity levels as defined in the ball_severity component. The enumerator names suggest a meaning that is consistent with common practice, but no policy is enforced by the basic logger API or the macros.
For convenient reference only, the 'ball::Severity::Level' 'enum' implementation is presented below. Note that this implementation and the specific numerical values may not be relied upon. Also note that the enumerator 'NONE' is deprecated, since its name has proven to be confusing. As an argument to 'logMessage', it would satisfy "none" of the thresholds, but when provided as a threshold value itself, it would enable "all" of the enumerated 'logMessage' requests (and all of the macros). Use 'e_OFF' as a threshold level to disable all logging events for a particular logging behavior (see the "Log Record Storage and Publication" section below).
Note that numerically lower 'Level' values correspond to conditions having greater severity.
The entity that the logger logs is called a "log record" in 'ball'. A log record consists of a number of fixed and user-defined fields; one of the fixed fields is called a "log message" (or "message" for short). In casual usage, we use "record" and "message" interchangeably, but note that they are distinct concepts. The set of fixed fields that constitute a log record are defined in the ball_recordattributes component. A brief description of the fixed fields (or "record attributes") is given in the following table.
The user may wish to specify a set of user-defined fields to be included within every log record for a given program. Such user-defined fields are represented in each log record by an instance of 'ball::UserFields'. A 'ball::LoggerManagerConfiguration::UserFieldsPopulatorCallback' functor may be provided to the logger manager on construction (via a 'ball::LoggerManagerConfiguration' object) that, when invoked, "populates" (i.e., provides the values for) all elements of the 'ball::UserFields' object containing the user-defined fields. See the ball_record component for more information about the use of 'ball::UserFields' in logging, and see the ball_loggermanager component for more information about installing a user populator callback. Note that 'ball::UserFields' is deprecated; new code should use 'ball::Attribute' and 'ball::ScopedAttribute' instead (see {Log Attributes}).
The 'ball' infrastructure has no provisions for specifying or enforcing constraints on the encoding used for logged messages (in general, logged messages are published to registered observers in the encoding in which they were originally supplied). However, particular applications, frameworks, and consumers of log records (e.g., readers of log files) may place their own constraints on log records with respect to encoding. Clients generating log records should be aware of the encoding requirements for logged messages generated by their application (e.g., many applications at Bloomberg require UTF-8 encoded messages).
The logger achieves good run-time performance while capturing critical log messages by selectively storing log records and selectively publishing a subset of those stored records. The determination of whether a log record should be stored or published is based on the "category" and "severity" level that are associated with each record. The severity of a record is intended to indicate that record's relative urgency. A category is comprised of a name (an arbitrary string) and four severity threshold levels (see the component-level documentation of ball_severity for more information on typical severity levels). A message's text, its associated severity, and the name of its associated category are each among the fixed fields in the logged record (see the "Messages, Records, and other 'ball' Terminology" section above).
The following four settings define the log record severity levels at which the logger takes certain actions. The logger determines the disposition of a record based on its severity level in relation to the four severity threshold levels of the category associated with the message:
Record: If the severity level of the record is at least as severe as the Record threshold level of the associated category, then the record will be stored by the logger in its log record buffer (i.e., it will be recorded).
Pass: If the severity level of the record is at least as severe as the Pass threshold level of the associated category, then the record will be immediately published by the logger (i.e., it will be transmitted to the logger's downstream recipient – the observer).
Trigger: If the severity level of the record is at least as severe as the Trigger threshold level of the associated category, then the record will cause immediate publication of that record and any records in the logger's log record buffer (i.e., this record will trigger a general log record dump).
Trigger-All: If the severity level of the record is at least as severe as the Trigger-All threshold level of the associated category, then the record will cause immediate publication of that record and all other log records stored by all active loggers.
Note that more than one of the above actions can apply to a given log record since the four threshold levels are independent of one another. Also note that the determination of whether a log record should cause a Trigger or Trigger-All event is based on the trigger and trigger-all threshold levels, respectively, of the category associated with the record being logged. The categories and severities associated with log records stored earlier are not taken into account.
This section will be expanded upon in the future. For now, we provide a brief introduction to the objects that are most central to understanding basic logger operation, so that we may use their names elsewhere in this documentation.
'ball::LoggerManager' is a singleton that must be constructed with a configuration object (see below) and an optional 'bslma' allocator before any true logging operations can be performed. The logger manager's main tasks are to administer categories and to "allocate" loggers. If the user has not allocated and installed any logger instances, then the 'getLogger' method will return the "default logger" that is a specific instance of 'ball::Logger' owned and managed by the logger manager. This default logger is suitable for single-threaded and multi-threaded use, and is used "transparently" by users willing to accept default-logger behavior. See {Usage} below.
The logger manager maintains an internal broadcast observer (see below) and provides the 'registerObserver', 'deregisterObserver', and 'findObserver' methods that register, deregister, and find observers, respectively. The internal broadcast observer forwards all log records that it receives to all registered observers.
The logger manager provides the 'allocateLogger' and 'setLogger' methods that allocate arbitrarily many loggers and install (at most) one logger per thread, respectively. Any thread in which the 'setLogger' method was not called will use the default logger. See {Example 6} and {Example 7} below for a discussion of how and why to allocate multiple loggers.
'ball::Logger' provides the most central logging functionality, namely the 'logMessage' method. However, most users will never call 'logMessage' directly, but rather will use the sets of macros defined in the 'ball_log' component. See the {Appendix: Macro Reference} section below.
The "default" 'ball::Logger' instance managed by the logger manager is suitable for many purposes, so typical users need never interact with logger instances explicitly at all. However, in multi-threaded applications (and perhaps also in certain special-purpose single-threaded programs), the user may want to instantiate one or more logger instances and install at most one logger instance per thread. See {Example 6} and {Example 7} below.
'ball::LoggerManagerConfiguration' is a "configuration" object that must be supplied to the logger manager at construction. The default object configures the "default" logging behavior; the user can change the logger behavior by changing attributes in the configuration object. The name, type, and description of the configuration attributes are presented in the two tables below; the awkward repetition of 'NAME' is due to the long type names of the functor types.
Note that the configuration object is not value-semantic because its three functor attributes are not value-semantic. For this reason, the single "defaults" attribute, a value-semantic simply-constrained attribute type, was factored out from the configuration type. The defaults object holds six numerical attributes and is described next.
'ball::LoggerManagerDefaults' is a value-semantic simply-constrained attribute type that is itself an attribute of the above "configuration" object. The "defaults" object contains the following six constrained attributes.
The constraints are as follows:
'ball::LoggerManagerDefaults' was factored out of the configuration object because the former is purely value-semantic, and can be generated from, e.g., a configuration file. From this perspective, it is very convenient to set this one attribute in the configuration object. It is quite possible to ignore the independent existence of 'ball::LoggerManagerDefaults' and set the above constrained attributes directly in the 'ball::LoggerManagerConfiguration' object. The latter is more convenient in "simple" usage, and is illustrated in {Usage} below.
'ball::Observer' is the object that receives "published" log messages. Specifically, 'ball::Observer' is a protocol (abstract interface), and the user must supply a concrete implementation (separately written or chosen from existing 'ball' observers). The observer provides a 'publish' method that defines the behavior of the logical concept of "publication" of a message used throughout this document. Note that multiple observers may be registered with the logger manager and each registered observer will receive "published" log messages.
For both convenience and uniformity of programming, 'ball' provides a suite of logging macros (defined in the 'ball_log' component). All but the most demanding and customized applications will use the logging macros for basic logging operations (e.g., setting categories and logging messages). The logger manager interface is used primarily to administer global and category-specific severity threshold levels, and to allocate loggers. See the {Usage} and {Appendix: Macro Reference} sections below for examples and reference documentation, respectively.
The user should note the following two facts about macro usage:
When the logger manager singleton is created, a distinguished category known as the Default Category is created. At initialization, the Default Category is given "factory-supplied" default threshold levels. These threshold levels, values distributed in the range '[0 .. 255]', are not published and are subject to change over time. Explicit settings for these factory values may be specified when the singleton is constructed. This is done by constructing a 'ball::LoggerManagerConfiguration' object (needed for logger manager construction in any event) and then calling the 'setDefaultThresholdLevelsIfValid' method with the desired values for its four arguments before supplying the configuration object to the logger manager configuration.
The Default Category may arise during logging whenever the 'setCategory(const char <em>categoryName)' method is called. That method returns the address of the category having 'categoryName', if it exists; if no such category exists, and a category having 'categoryName' cannot be created due to a capacity limitation on the category registry maintained by the logger manager singleton, then the *Default Category is returned.
Categories that are added to the registry during logging through calls to the 'setCategory(const char *)' method are given threshold levels by one of two means. One alternative is to use the 'ball::LoggerManagerConfiguration::DefaultThresholdLevelsCallback' functor that is optionally supplied by the client when the logger manager singleton is initialized. If such a functor is provided by the client, then it is used to supply threshold levels to categories added by 'setCategory(const char *)'. Otherwise, default threshold levels maintained by the logger manager for that purpose are used. At initialization, these default threshold levels are given the same "factory-supplied" settings as those for the Default Category that, again, may be explicitly overridden at construction.
The default threshold levels can be adjusted ('setDefaultThresholdLevels') and reset to their original values ('resetDefaultThresholdLevels'). Note that if factory values are overridden at initialization, a reset will restore thresholds to the user-specified default values. In addition, there is a method to set the threshold levels of a given category to the current default threshold levels ('setCategoryThresholdsToCurrentDefaults') or to the factory-supplied (or client-overridden) default values ('setCategoryThresholdsToFactoryDefaults').
One final note regarding categories is that the client can optionally supply a 'ball::LoggerManagerConfiguration::CategoryNameFilterCallback' functor to translate category names from an external to an internal representation. For example, a project may allow programmers to refer to categories using mixed-case within an application, but provide a 'toLower' 'CategoryNameFilterCallback' to map all external upper-case letters to lower-case internally. Such a name-filtering functor is set in the 'ball::LoggerManagerConfiguration' object before that configuration object is passed to the logger manager scoped guard constructor. In this scenario, the (hypothetical) external category names "EQUITY.MARKET.NYSE" and "equity.market.nyse" would be mapped to the same category internally by the presumed 'toLower' functor.
The 'ball' logging framework provides the ability to associatiate "attributes" (name-value pairs) with the current logging context, which can be both be written to the log as part of a log record, as well as used in logging rules (see {Rule-Based Logging} below). This is typically done using the ball_scopedattribute component.
Below is a simple example using log attributes:
In the above example a logging attribute, "mylibrary.security", is associated with the current thread's logging context for the lifetime of the 'securityAttribute' object. As a result, if attributes have been enabled in the log format, the error log message generated by this example might look like:
Notice the attribute rendered does not appear in the logging message itself ("Error computing splines (-1)"), and is rendered as a name-value pair, meaning it can be easily parsed by log management systems like Humio or Splunk.
Attributes will not appear in your log unless your ball::Observer format specification is configured to render attributes (see below).
Log attributes are not rendered by default as part of the log message (for backward compatibility). Clients can enable log attributes to be rendered for observers that support log record formatting:
Log message formatting is implemented by the ball_recordstringformatter component, which supports the following (new) format specifiers to render log attributes:
The following code snippet illustrates the creation and configuration commonly used by observers:
The 'ball' logging toolkit supports scheme-based format specifications that allow you to select different log formatters and specify their configurations using a URI-like syntax. This feature is available in the following observer components:
A scheme-based format specification has the following syntax:
The scheme determines which formatter will be used and the syntax of the format specification. For backward compatibility, if no scheme is specified (i.e., the configuration doesn't contain '://'), the configuration is treated as a legacy 'printf'-style format specification for 'ball::RecordStringFormatter' (equivalent to the 'text://' scheme).
The following table lists the currently supported schemes:
The following subsections describe the format specification syntax for each supported scheme.
1.9.8