bdlmt.h

Go to the documentation of this file.

/// @file bdlmt.h
///
///
/// @defgroup bdlmt Package bdlmt
/// @brief  Basic Development Library Multi Thread (bdlmt)
/// @addtogroup bdl
/// @{
/// @addtogroup bdlmt
/// [bdlmt]: group__bdlmt.html
/// @{
///
/// # Purpose {#bdlmt-purpose}
/// Provides thread pools and event schedulers.
///
/// # Mnemonic {#bdlmt-mnemonic}
/// Basic Development Library Multi Thread (bdlmt)
///
/// @see  bdlcc
///
/// # Description {#bdlmt-description}
/// The 'bdlmt' ("Basic Development Library Multi Thread") package
/// provides components for creating and managing thread pools, and components for
/// scheduling (time-based) events.
///
/// A "thread pool" is a collection of processor threads that are managed
/// together and used interchangeably to support user requests.  The
/// @ref bdlmt_threadpool  component allows clients to configure the pool so that it
/// grows and shrinks according to user demand, manage thread availability, and
/// schedule client "jobs" to be run independently as threads in the pool become
/// available.  It does this by placing client requests on an internal job
/// queue, and controlling multiple threads as they remove jobs from the queue
/// and execute them.
///
/// A "multi-queue thread pool" defines a dynamic, configurable pool of queues,
/// each of which is processed by a thread in a thread pool, such that elements
/// on a given queue are processed serially, regardless of which thread is
/// processing the queue at a given time.  In addition to the ability to create
/// and delete queues, clients are able to tune the underlying thread pool.
///
/// A "timer-event scheduler" defines a thread-safe event scheduler.  It
/// provides methods to schedule and cancel recurring and non-recurring events
/// (also referred to as clock).  The callbacks are processed by a separate
/// thread (called dispatcher thread).
///
/// ## Hierarchical Synopsis
///
/// The 'bdlmt' package currently has 9 components having 2 levels of physical
/// dependency.  The list below shows the hierarchical ordering of the components.
/// The order of components within each level is not architecturally significant,
/// just alphabetical.
/// @code
///  2. bdlmt_multiqueuethreadpool
///     bdlmt_threadmultiplexor
///
///  1. bdlmt_eventscheduler
///     bdlmt_fixedthreadpool
///     bdlmt_multiprioritythreadpool
///     bdlmt_signaler
///     bdlmt_threadpool
///     bdlmt_throttle
///     bdlmt_timereventscheduler
/// @endcode
///
/// ## Component Synopsis
///
///  @ref bdlmt_eventscheduler :
///       Provide a thread-safe recurring and one-time event scheduler.
/// 
///  @ref bdlmt_fixedthreadpool :
///       Provide portable implementation for a fixed-size pool of threads.
/// 
///  @ref bdlmt_multiprioritythreadpool :
///       Provide a mechanism to parallelize a prioritized sequence of jobs.
/// 
///  @ref bdlmt_multiqueuethreadpool :
///       Provide a pool of queues, each processed serially by a thread pool.
/// 
///  @ref bdlmt_signaler :
///       Provide an implementation of a managed signals and slots system.
/// 
///  @ref bdlmt_threadmultiplexor :
///       Provide a mechanism for partitioning a collection of threads.
/// 
///  @ref bdlmt_threadpool :
///       Provide portable implementation for a dynamic pool of threads.
/// 
///  @ref bdlmt_throttle :
///       Provide mechanism for limiting the rate at which actions may occur.
/// 
///  @ref bdlmt_timereventscheduler :
///       Provide a thread-safe recurring and non-recurring event scheduler.
///
/// ## Generic Overview of Thread Pools
///
/// At the current time, this generic overview applies only to the
/// 'bdlmt_MultipriorityThreadPool'.  The plan is for other threadpools to move
/// to this model at a later date.
///
/// As Figure 1 illustrates, a threadpool allows its clients to enqueue units of
/// work to be processed concurrently in multiple threads.  Each work item, or
/// "job", consists of a function along with the address of its associated input
/// data.  When executed, this address is supplied to the function as its only
/// argument; note that this function must have external linkage and return
/// 'void':
/// @code
///  extern "C" void job(void *);     // Idiomatic C-style function signature
/// @endcode
/// Alternatively both the function and its data can be encapsulated and
/// supplied in the form of an (invokable) function object, or "functor", taking
/// no arguments and returning 'void'.
/// @code
///  +-------------------------------------------------------------------------+
///  |                     ThreadPool *Control* Methods                        |
///  |                                                                         |
///  | Front Operations        Middle Operations      Back Operations          |
///  | ----------------        -----------------      ---------------          |
///  | int startThreads()      void removeJobs()      void enableQueue()       |
///  | void stopThreads()      void drainJobs()       void disableQueue()      |
///  | int resumeProcessing()                         int enqueueJob(func,arg) |
///  | int suspendProcessing()                        int enqueueJob(job)      |
///  |                                                                         |
///  +- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
///  |           +--<--+--<--+--<--+--<--+--<--+----------------+              |
///  |           |     |     |     |     |     |                |              |
///  |  Front <==| Job | Job | Job | Job | Job |                |==< Back      |
///  |           |     |     |     |     |     |                |              |
///  |           +--<--+--<--+--<--+--<--+--<--+----------------+              |
///  |                                                                         |
///  |     ,----------------.                      ,-----------------.         |
///  |    ( N Worker Threads )                    ( Thread Attributes )        |
///  |     `----------------'                      `-----------------'         |
///  +-------------------------------------------------------------------------+
///             Figure 1: Illustration of Generalized Thread Pool
/// @endcode
/// In addition to enqueuing jobs, a thread pool must supply primitive control
/// functionality such as creating and destroying worker threads, enabling and
/// disabling the enqueuing of new jobs, causing the queue to block until there
/// are no pending jobs, and removing (i.e., canceling) all pending (i.e., not
/// yet running) jobs.  Different kinds of threadpools will provide different
/// functionality and/or performance characteristics, corresponding those of the
/// underlying thread-enabled ('bdlcc') queue -- e.g., (limited-capacity)
/// 'FixedQueue', (heap-based) 'PriorityQueue', and (array-based)
/// 'MultipriorityQueue'.  Nonetheless, each of the threadpool objects in 'bdlmt'
/// should provide a suite of input and control operations that are consistent
/// in both name and behavior across the 'bdlmt' package.
///
/// Due to the intricate nature of threadpools, it is easy to convolve behaviors
/// in subtly different ways for functions having the same name.  Consider, for
/// example, the method 'void drainJobs()', the basic functionality of which is
/// to 'block' the caller until all of the pending jobs complete (i.e., the
/// queue is empty and all worker threads are idle).  Should 'drainJobs()' also
/// leave the queue in the disabled state?  Even if that is a common usage
/// pattern, it is often useful to start with simple, orthogonal behaviors, and
/// if needed, define more complex behaviors in terms of them.
///
/// In the case of a thread pool, it is instructive to break the functionality
/// into three categories of operations relative to the underlying queue: Front,
/// Middle, and Back.  At the back of the queue (refer to Figure 1), we need to
/// enable/disable clients from adding work items.  Enabling or disabling the
/// queue does not affect the items already in the queue [Middle], nor any
/// worker threads processing these items [Front].
///
/// In the middle of the queue, we have two operations that result in purging
/// all pending items in the queue: 'drainJobs()' and 'removeJobs()' If we
/// invoke 'removeJobs()', then all currently pending (i.e., not started) work
/// items will be removed (i.e., canceled).  During this process, clients
/// attempting to add work items [Back] will block, but their eventual success
/// or failure, (which is based solely on whether the queue is enabled or
/// disabled) is not affected.  Note that jobs that are already in progress
/// [Front] are also unaffected.  Similarly, invoking our orthogonal
/// 'drainJobs()' method will block enqueuing clients until all pending jobs
/// have completed, but will not affect the enabledness of the thread pool
/// [Back], nor the processing of work items [Front].
///
/// Finally we come to the front of the queue, which addresses the processing of
/// jobs.  A (typically fixed) number of worker threads is specified at
/// construction.  The thread pool "wakes up" in an enabled state, but without
/// having created the worker threads.  Invoking the 'startThreads()' method
/// attempts to create these threads (unless they are already created).  The
/// 'startThreads()' method returns 0 if all of these threads are started, and a
/// non-zero value otherwise (in which case none of the worker threads are
/// started).  Redundant calls to 'startThreads()' do nothing and return zero.
/// Invoking 'stopThreads()' destroys each worker thread (after it completes any
/// current job).  Note that the current contents of the queue [Middle], and the
/// ability to enqueue new jobs [Back] are not affected.
///
/// Whether or not started threads should be pulling jobs from the queue and
/// processing them is not necessarily the same as having the user-specified
/// number of worker threads created.  In addition to being *enabled* and
/// *started* let's consider one more possible state, *suspended*.  If a thread
/// pool is in the *suspended* state, then even when it is in the *started*
/// state, it will not attempt to pop jobs from the queue and execute them.
///
/// A created threadpool will be created enabled, not suspended, and not
/// started.  All three of these qualities are orthogonal and any one of them
/// can be changed at any time.
///
/// The vast majority of users will be uninterested in both the 'suspend' and
/// 'disable' features, so it is imperative that newly created threadpools be
/// both non-suspended and enabled so users can remain blissfully ignorant of
/// them.  It is also important the first usage examples, if not all of them,
/// omit use of these features to minimize learning time for the typical user.
///
/// To conclude this generic overview, we note that there is one common usage
/// that, although not minimal, arguably deserves to be a method of every thread
/// pool class: 'void shutdown()'.  This method is best described as a
/// composition of the simple, orthogonal functions described above.  In order
/// to shut down a thread pool, we need to first disable the enqueuing of any
/// additional jobs, then remove all of the pending work items, and finally stop
/// all of the active threads:
/// @code
///  void shutdown()
///  {
///      disableQueue();
///      removeJobs();
///      stopThreads();
///  }
/// @endcode
/// By making sure that our initial operations are simple and orthogonal, we can
/// ensure that the precise meaning of more complex operations is kept clear.
///
/// ## Synchronous Signals on Unix
///
/// A thread pool ensures that, on Unix platforms, all the threads in the pool
/// block all asynchronous signals.  Specifically all the signals, except the
/// following synchronous signals are blocked.
/// @code
///  SIGBUS
///  SIGFPE
///  SIGILL
///  SIGSEGV
///  SIGSYS
///  SIGABRT
///  SIGTRAP
///  SIGIOT
/// @endcode
///
/// @}
/** @} */