BDE 4.39.x Production Release
Loading...
Searching...
No Matches
ball_fileobserver

Detailed Description

Provide a thread-safe observer that logs to a file and to stdout.

Outline

Purpose

Provide a thread-safe observer that logs to a file and to stdout.

Classes

See also
ball_record, ball_context, ball_observer, ball_fileobserver2

Description

This component provides a concrete implementation of the ball::Observer protocol, ball::FileObserver, for publishing log records to stdout and, optionally, to a user-specified file. The following inheritance hierarchy diagram shows the classes involved and their methods:

,------------------.
`------------------'
| ctor
| disableFileLogging
| disableTimeIntervalRotation
| disableSizeRotation
| disableStdoutLoggingPrefix
| disablePublishInLocalTime
| enableFileLogging
| enableStdoutLoggingPrefix
| enablePublishInLocalTime
| forceRotation
| rotateOnSize
| rotateOnTimeInterval
| setFileLogFormat
| setLogFormat
| setLogFormats
| setOnFileRotationCallback
| setStdoutLogFormat
| setStdoutThreshold
| suppressUniqueFileNameOnRotation
| getFileLogFormat
| getLogFormat
| getStdoutLogFormat
| isFileLoggingEnabled
| isStdoutLoggingPrefixEnabled
| isPublishInLocalTimeEnabled
| isSuppressUniqueFileNameOnRotation
| rotationLifetime
| rotationSize
| stdoutThreshold
V
,--------------.
( ball::Observer )
`--------------'
dtor
publish
releaseRecords
Definition ball_fileobserver.h:579

A ball::FileObserver object processes the log records received through its publish method by writing them to stdout and, if so configured, to a user-specified file. The format of published log records is user-configurable for both destinations, stdout and file (see {Log Record Formatting} below). Although logging to a file is initially disabled following construction, the most common use-case for ball::FileObserver is to also log to a file, enabled by calling the enableFileLogging method. In addition, a file observer may be configured to perform automatic log file rotation (see {Log File Rotation} below).

File Observer Configuration Synopsis

ball::FileObserver offers several constructor arguments and manipulators that may be used to configure various aspects of a file observer object. These are summarized in the following tables along with the accessors that can be used to query the current state of the configuration. Further details are provided in the following sections and the function-level documentation.

+-----------------------+-----------------------+
| Aspect | Constructor Arguments |
+=======================+=======================+
| Log Record Timestamps | publishInLocalTime |
+-----------------------+-----------------------+
| `stdout` Logging | stdoutThreshold |
+-----------------------+-----------------------+
+-------------+------------------------------------+
| Aspect | Related Methods |
+=============+====================================+
| Log Record | setLogFormat |
| Formatting | enableStdoutLoggingPrefix |
| | disableStdoutLoggingPrefix |
| | getLogFormat |
| | isStdoutLoggingPrefixEnabled |
+-------------+------------------------------------+
| Log Record | enablePublishInLocalTime |
| Timestamps | disablePublishInLocalTime |
| | isPublishInLocalTimeEnabled |
| | |
+-------------+------------------------------------+
| File | enableFileLogging |
| Logging | disableFileLogging |
| | isFileLoggingEnabled |
| | |
+-------------+------------------------------------+
| `stdout` | setStdoutThreshold |
| Logging | enableStdoutLoggingPrefix |
| | disableStdoutLoggingPrefix |
| | stdoutThreshold |
| | isStdoutLoggingPrefixEnabled |
+-------------+------------------------------------+
| Log File | rotateOnSize |
| Rotation | rotateOnTimeInterval |
| | disableSizeRotation |
| | disableTimeIntervalRotation |
| | setOnFileRotationCallback |
| | suppressUniqueFileNameOnRotation |
| | rotationSize |
| | rotationLifetime |
| | isSuppressUniqueFileNameOnRotation |
+-------------+------------------------------------+

In general, a ball::FileObserver object can be dynamically configured throughout its lifetime (in particular, before or after being registered with a logger manager). However, note that for ball::FileObserver, configuration changes that affect how records are logged (e.g., enablePublishInLocalTime and disablePublishInLocalTime) impact only those records that are published subsequent to making the configuration change.

Log Record Formatting

By default, the output format of published log records, whether written to stdout or to a user-specified file, is:

DATE_TIME PID:THREAD-ID SEVERITY FILE:LINE CATEGORY MESSAGE USER-FIELDS

where DATE and TIME are of the form DDMonYYYY and HH:MM:SS.mmm, respectively (Mon being the 3-letter abbreviation for the month). For example, a log record will have the following appearance when the default format is in effect (assuming that no user-defined fields are present):

18MAY2005_18:58:12.076 7959:1 WARN ball_fileobserver.t.cpp:404 TEST hello!

The default format for records published to stdout (only) can be shortened by calling disableStdoutLoggingPrefix. This method has the effect of reducing the above example message to the following when output to stdout:

WARN ball_fileobserver.t.cpp:404 TEST hello!

For additional flexibility, the setFileLogFormat and setStdoutLogFormat methods can be called to independently configure the format of published records to a file and to stdout, respectively. The format specifications can be either scheme-tagged (recommended) or legacy printf-style format strings that results in a RecordStringFormatter being used.

Scheme-Based Format Specifications (Recommended)

The recommended way to specify log record formats is using URI-like scheme-tagged format configuration strings. A scheme-tagged format string begins with a scheme identifier followed by :// and then a scheme-specific format specification:

<scheme>://<format-specification>

The scheme determines which formatter will be used and the syntax of the format specification. The following schemes are currently supported: text, json, qjson. See Scheme-Based Formatters for more details of the supported schemes and their accompanying format specification syntaxes.

For example, to log records to a file in JSON format with printf-style format specification, and to stdout using text format with ISO 8601 timestamps:

fileObserver.setFileLogFormat("qjson://%d %p:%t %s %f:%l %c %m");
fileObserver.setStdoutLogFormat("text://\n%I %p:%t %s %f:%l %c %m\n");

Note that both text:// and qjson:// schemes support printf-style % specifications in their format strings, though with different output structures (plain text vs. JSON).

Legacy Format Specifications

For backward compatibility, format specifications that do not begin with a scheme tag are treated as legacy printf-style format strings. Such specifications are implicitly treated as if they had a text:// prefix and use ball::RecordStringFormatter. For example, the following two calls are equivalent:

fileObserver.setFileLogFormat("%d %p:%t %s %f:%l %c %m %a\n");
fileObserver.setFileLogFormat("text://%d %p:%t %s %f:%l %c %m %a\n");

These %-prefixed conversion specifications are defined in ball_recordstringformatter .

Legacy API: setLogFormat

The setLogFormat method sets both file and stdout formats at once. Using setFileLogFormat and setStdoutLogFormat is preferred to set formats independently for file and stdout output:

// Legacy:
fileObserver.setLogFormat(fileFormat, stdoutFormat);
// Preferred:
fileObserver.setFileLogFormat(fileFormat);
fileObserver.setStdoutLogFormat(stdoutFormat);

Interaction with enableStdoutLoggingPrefix & disableStdoutLoggingPrefix

When a format is set using setLogFormat, calling disableStdoutLoggingPrefix will change the stdout format to the default short format (i.e., "\n%s %f:%l %c %m %u\n"), regardless of whether the original format was specified as a printf-style format string or a scheme-based formatter. If enableStdoutLoggingPrefix is subsequently called, the format specified in the most recent call to setLogFormat will be reinstated, preserving the original formatter type (string-based or scheme-based).

Log Record Timestamps

By default, the timestamp attributes of published records are written in UTC time (unless true is supplied for the optional publishInLocalTime constructor flag). To write timestamps in local time instead, call the enablePublishInLocalTime method. Note that the local time offset is calculated using the UTC timestamp of each record. To revert to UTC time, call the disablePublishInLocalTime method. Whether UTC time or local time is in effect can be queried via isPublishInLocalTimeEnabled.

Local Time Offset Calculations

The calculation of the local time offset adds some overhead to the publication of each log record. If this overhead is an issue, it can be mitigated by installing a high-performance local-time offset callback for bdlt::CurrentTime in main. See bsls_systemtime for the details of installing such a callback and see baltzo_localtimeoffsetutil for an example facility. Note that such callbacks can improve performance for all users of bdlt::CurrentTime, not just the ball logger.

Log Filename Patterns

The enableFileLogging method supports the use of %-escape sequences to specify log filenames. The recognized sequences are as follows:

%Y - current year (4 digits with leading zeros)
%M - current month (2 digits with leading zeros)
%D - current day (2 digits with leading zeros)
%h - current hour (2 digits with leading zeros)
%m - current minute (2 digits with leading zeros)
%s - current second (2 digits with leading zeros)
%T - current datetime, equivalent to "%Y%M%D_%h%m%s"
%p - process ID

The date and time elements of the derived filename are based on the time when the log file is created. Furthermore, these elements are based on either UTC time or local time depending on the value returned by isPublishInLocalTimeEnabled. (See {Log Record Timestamps} for the similarity with the treatment of record timestamps.)

For example, a log filename pattern of "task.log.%Y%M%D_%h%m%s" will yield the filename task.log.20110501_123000 if the file is created on 01-May-2011 at 12:30:00 local time (assuming enablePublishInLocalTime was called).

Log File Rotation

A ball::FileObserver may be configured to perform automatic rotation of log files based on simple file rotation conditions (or rules).

File Rotation Conditions

Rotation rules may be established based on the size of the log file (i.e., a "rotation-on-size" rule), and a periodic time interval (i.e., a "rotation-on-time-interval" rule). These rules are independently enabled by the rotateOnSize and rotateOnTimeInterval methods, respectively. If both rules are in effect, log file rotation is performed whenever either rule applies.

Rotated File Naming

When a log file is rotated, a new filename is generated using the pattern supplied to enableFileLogging. If the file having the new name does not exist, the current log file is closed, and the logging continues to the new file.

If the file having the new name already exists, then the behavior of the file rotation is further controlled by the flag set with suppressUniqueFileNameOnRotation:

The two tables below illustrate the names of old and new log files when a file rotation occurs. We assume that the log file is rotated on 2011-May-21 at 12:29:59 local time and that the last rotation occurred at 12:30:00 on the previous day. We further assume that enablePublishInLocalTime was called, so that all date and time elements are rendered in local time.

The first table shows the name change (if any) of the (old) log file being rotated when the flag controlled by suppressUniqueFileNameOnRotation is set to false:

Disabled: `suppressUniqueFileNameOnRotation`
For brevity:
<TS1> = 20210520_123000
<TS2> = 20210521_122959 (aka next day, about the same time)
+----------------+-----------------+----------------+----------------------
| Pattern | Filename Before | Filename After | Rotated Filename
| | Rotation | Rotation |
+----------------+-----------------+----------------+----------------------
| "a.log" | a.log | a.log | a.log.<TS1>
| "a.log.%T" | a.log.<TS1> | a.log.<TS2> | a.log.<TS1>
| "a.log.%Y%M" | a.log.202105 | a.log.202105 | a.log.202105.<TS1>
| "a.log.%Y%M%D" | a.log.20210520 | a.log.20110521 | a.log.20210520
+----------------+-----------------+----------------+----------------------

Note that upon rotation a timestamp was appended to the name of the rotated file when the log pattern does not contain %-escape sequences indicating a time period ("a.log"), or the rotation period (in our case, one day) is less than the time period encoded in the pattern (in case of "a.log.%Y%M" the period is one month).

The next table shows the rotated name when the flag controlled by suppressUniqueFileNameOnRotation is set to true, and (possibly new) name of the (new) log file following rotation:

Enabled: `suppressUniqueFileNameOnRotation`
+----------------+-----------------+----------------+----------------------
| Pattern | Filename Before | Filename After | Rotated Filename
| | Rotation | Rotation |
+----------------+-----------------+----------------+----------------------
| "a.log" | a.log | a.log | none
| "a.log.%T" | a.log.<TS1> | a.log.<TS2> | a.log.<TS1>
| "a.log.%Y%M" | a.log.202105 | a.log.202105 | none
| "a.log.%Y%M%D" | a.log.20210520 | a.log.20110521 | a.log.20210520
+----------------+-----------------+----------------+----------------------

Note that the original filename is reused when the log pattern does not contain %-escape sequences indicating a time period ("a.log"), or the rotation period (in our case, one day) is less than the time period encoded in the pattern (in case of "a.log.%Y%M" the period is one month).

Also note, that in any cases, when the log pattern includes "%T", or encodes a time period that coincides the rotation period (in case of "a.log.%Y%M%D" the period is one day), then a unique name on each rotation is produced with the (local) time at which file rotation occurred embedded in the filename.

Thread Safety

All methods of ball::FileObserver are thread-safe, and can be called concurrently by multiple threads.

Usage

This section illustrates intended use of this component.

Example: Basic Usage

First, we create a ball::LoggerManagerConfiguration object, lmConfig, and set the logging "pass-through" level – the level at which log records are published to registered observers – to DEBUG:

int main()
{
Definition ball_loggermanagerconfiguration.h:283
int setDefaultThresholdLevelsIfValid(int passLevel)
@ e_DEBUG
Definition ball_severity.h:178

Next, create a ball::LoggerManagerScopedGuard object whose constructor takes the configuration object just created. The guard will initialize the logger manager singleton on creation and destroy the singleton upon destruction. This guarantees that any resources used by the logger manager will be properly released when they are not needed:

Definition ball_loggermanager.h:2162
Definition ball_loggermanager.h:1293
static LoggerManager & singleton()
Definition ball_loggermanager.h:2389

Next, we create a ball::FileObserver object and register it with the ball logging system;

bsl::make_shared<ball::FileObserver>();
int rc = manager.registerObserver(observer, "default");
ASSERT(0 == rc);
int registerObserver(const bsl::shared_ptr< Observer > &observer, const bsl::string_view &observerName)
Definition ball_loggermanager.h:2585
Definition bslstl_sharedptr.h:1838

The default format for outputting log records can be changed by calling the setFileLogFormat and setStdoutLogFormat methods. The statements below output record timestamps in ISO 8601 format to the log file and in bdlt-style (default) format to stdout, where timestamps are output with millisecond precision in both cases:

observer->setFileLogFormat("%I %p:%t %s %f:%l %c %m\n");
observer->setStdoutLogFormat("%d %p:%t %s %f:%l %c %m\n");

Note that both of the above format specifications omit user fields ("%u") in the output. Also note that, unlike the default, this format does not emit a blank line between consecutive log messages.

Henceforth, all messages that are published by the logging system will be transmitted to the publish method of observer. By default, only the messages with a e_WARN, e_ERROR, or e_FATAL severity will be logged to stdout:

BALL_LOG_INFO << "Will not be published on 'stdout'.";
BALL_LOG_WARN << "This warning *will* be published on 'stdout'.";
#define BALL_LOG_SET_CATEGORY(CATEGORY)
Definition ball_log.h:1210
#define BALL_LOG_WARN
Definition ball_log.h:1466
#define BALL_LOG_INFO
Definition ball_log.h:1462

This default can be changed by specifying an optional argument to the ball::FileObserver constructor or by calling the setStdoutThreshold method:

observer->setStdoutThreshold(ball::Severity::e_INFO);
BALL_LOG_DEBUG << "This debug message is not published on 'stdout'.";
BALL_LOG_INFO << "This info message *will* be published on 'stdout'.";
BALL_LOG_WARN << "This warning will be published on 'stdout'.";
#define BALL_LOG_DEBUG
Definition ball_log.h:1458
@ e_INFO
Definition ball_severity.h:177

The user can log all messages to a specified file and specify rotation rules based on the size of the log file or its lifetime:

// Create and log records to a file named "/var/log/task/task.log".
observer->enableFileLogging("/var/log/task/task.log");
// Disable 'stdout' logging.
observer->setStdoutThreshold(ball::Severity::e_OFF);
// Rotate the file when its size becomes greater than or equal to 256
// megabytes.
observer->rotateOnSize(1024 * 256);
// Rotate the file every 24 hours.
observer->rotateOnTimeInterval(bdlt::DatetimeInterval(1));
Definition bdlt_datetimeinterval.h:201
@ e_OFF
Definition ball_severity.h:173

Note that in this configuration the user may end up with multiple log files for any given day (because of the rotation-on-size rule). This feature can be disabled dynamically later:

observer->disableSizeRotation();
return 0;
}