Package bdlmt [Package Group bdl]

Provides thread pools and event schedulers. More...

Components
	Component bdlmt_eventscheduler
	Provide a thread-safe recurring and one-time event scheduler.
	Component bdlmt_fixedthreadpool
	Provide portable implementation for a fixed-size pool of threads.
	Component bdlmt_multiprioritythreadpool
	Provide a mechanism to parallelize a prioritized sequence of jobs.
	Component bdlmt_multiqueuethreadpool
	Provide a pool of queues, each processed serially by a thread pool.
	Component bdlmt_signaler
	Provide an implementation of a managed signals and slots system.
	Component bdlmt_threadmultiplexor
	Provide a mechanism for partitioning a collection of threads.
	Component bdlmt_threadpool
	Provide portable implementation for a dynamic pool of threads.
	Component bdlmt_throttle
	Provide mechanism for limiting the rate at which actions may occur.
	Component bdlmt_timereventscheduler
	Provide a thread-safe recurring and non-recurring event scheduler.

---

Detailed Description

Outline

• Purpose
• MNEMONIC: Basic Development Library Multi Thread (bdlmt)
• Description
  • Hierarchical Synopsis
  • Component Synopsis
  • Generic Overview of Thread Pools
  • Synchronous Signals on Unix

Purpose:

Provides thread pools and event schedulers.

MNEMONIC: Basic Development Library Multi Thread (bdlmt):

See also:

bdlcc

Description:

The bdlmt ("Basic Development Library Multi Thread") package provides components for creating and managing thread pools, and components for scheduling (time-based) events.

A "thread pool" is a collection of processor threads that are managed together and used interchangeably to support user requests. The bdlmt_threadpool component allows clients to configure the pool so that it grows and shrinks according to user demand, manage thread availability, and schedule client "jobs" to be run independently as threads in the pool become available. It does this by placing client requests on an internal job queue, and controlling multiple threads as they remove jobs from the queue and execute them.

A "multi-queue thread pool" defines a dynamic, configurable pool of queues, each of which is processed by a thread in a thread pool, such that elements on a given queue are processed serially, regardless of which thread is processing the queue at a given time. In addition to the ability to create and delete queues, clients are able to tune the underlying thread pool.

A "timer-event scheduler" defines a thread-safe event scheduler. It provides methods to schedule and cancel recurring and non-recurring events (also referred to as clock). The callbacks are processed by a separate thread (called dispatcher thread).

Hierarchical Synopsis:

The bdlmt package currently has 9 components having 2 levels of physical dependency. The list below shows the hierarchical ordering of the components. The order of components within each level is not architecturally significant, just alphabetical.

  2. bdlmt_multiqueuethreadpool
     bdlmt_threadmultiplexor

  1. bdlmt_eventscheduler
     bdlmt_fixedthreadpool
     bdlmt_multiprioritythreadpool
     bdlmt_signaler
     bdlmt_threadpool
     bdlmt_throttle
     bdlmt_timereventscheduler

Component Synopsis:

bdlmt_eventscheduler:
Provide a thread-safe recurring and one-time event scheduler.

bdlmt_fixedthreadpool:
Provide portable implementation for a fixed-size pool of threads.

bdlmt_multiprioritythreadpool:
Provide a mechanism to parallelize a prioritized sequence of jobs.

bdlmt_multiqueuethreadpool:
Provide a pool of queues, each processed serially by a thread pool.

bdlmt_signaler:
Provide an implementation of a managed signals and slots system.

bdlmt_threadmultiplexor:
Provide a mechanism for partitioning a collection of threads.

bdlmt_threadpool:
Provide portable implementation for a dynamic pool of threads.

bdlmt_throttle:
Provide mechanism for limiting the rate at which actions may occur.

bdlmt_timereventscheduler:
Provide a thread-safe recurring and non-recurring event scheduler.

Generic Overview of Thread Pools:

At the current time, this generic overview applies only to the bdlmt_MultipriorityThreadPool. The plan is for other threadpools to move to this model at a later date.

As Figure 1 illustrates, a threadpool allows its clients to enqueue units of work to be processed concurrently in multiple threads. Each work item, or "job", consists of a function along with the address of its associated input data. When executed, this address is supplied to the function as its only argument; note that this function must have external linkage and return void:

  extern "C" void job(void *);     // Idiomatic C-style function signature

Alternatively both the function and its data can be encapsulated and supplied in the form of an (invokable) function object, or "functor", taking no arguments and returning void.

  +-------------------------------------------------------------------------+
  |                     ThreadPool *Control* Methods                        |
  |                                                                         |
  | Front Operations        Middle Operations      Back Operations          |
  | ----------------        -----------------      ---------------          |
  | int startThreads()      void removeJobs()      void enableQueue()       |
  | void stopThreads()      void drainJobs()       void disableQueue()      |
  | int resumeProcessing()                         int enqueueJob(func,arg) |
  | int suspendProcessing()                        int enqueueJob(job)      |
  |                                                                         |
  +- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
  |           +--<--+--<--+--<--+--<--+--<--+----------------+              |
  |           |     |     |     |     |     |                |              |
  |  Front <==| Job | Job | Job | Job | Job |                |==< Back      |
  |           |     |     |     |     |     |                |              |
  |           +--<--+--<--+--<--+--<--+--<--+----------------+              |
  |                                                                         |
  |     ,----------------.                      ,-----------------.         |
  |    ( N Worker Threads )                    ( Thread Attributes )        |
  |     `----------------'                      `-----------------'         |
  +-------------------------------------------------------------------------+
             Figure 1: Illustration of Generalized Thread Pool

In addition to enqueuing jobs, a thread pool must supply primitive control functionality such as creating and destroying worker threads, enabling and disabling the enqueuing of new jobs, causing the queue to block until there are no pending jobs, and removing (i.e., canceling) all pending (i.e., not yet running) jobs. Different kinds of threadpools will provide different functionality and/or performance characteristics, corresponding those of the underlying thread-enabled (bdlcc) queue -- e.g., (limited-capacity) FixedQueue, (heap-based) PriorityQueue, and (array-based) MultipriorityQueue. Nonetheless, each of the threadpool objects in bdlmt should provide a suite of input and control operations that are consistent in both name and behavior across the bdlmt package.

Due to the intricate nature of threadpools, it is easy to convolve behaviors in subtly different ways for functions having the same name. Consider, for example, the method void drainJobs(), the basic functionality of which is to block the caller until all of the pending jobs complete (i.e., the queue is empty and all worker threads are idle). Should drainJobs() also leave the queue in the disabled state? Even if that is a common usage pattern, it is often useful to start with simple, orthogonal behaviors, and if needed, define more complex behaviors in terms of them.

In the case of a thread pool, it is instructive to break the functionality into three categories of operations relative to the underlying queue: Front, Middle, and Back. At the back of the queue (refer to Figure 1), we need to enable/disable clients from adding work items. Enabling or disabling the queue does not affect the items already in the queue [Middle], nor any worker threads processing these items [Front].

In the middle of the queue, we have two operations that result in purging all pending items in the queue: drainJobs() and removeJobs() If we invoke removeJobs(), then all currently pending (i.e., not started) work items will be removed (i.e., canceled). During this process, clients attempting to add work items [Back] will block, but their eventual success or failure, (which is based solely on whether the queue is enabled or disabled) is not affected. Note that jobs that are already in progress [Front] are also unaffected. Similarly, invoking our orthogonal drainJobs() method will block enqueuing clients until all pending jobs have completed, but will not affect the enabledness of the thread pool [Back], nor the processing of work items [Front].

Finally we come to the front of the queue, which addresses the processing of jobs. A (typically fixed) number of worker threads is specified at construction. The thread pool "wakes up" in an enabled state, but without having created the worker threads. Invoking the startThreads() method attempts to create these threads (unless they are already created). The startThreads() method returns 0 if all of these threads are started, and a non-zero value otherwise (in which case none of the worker threads are started). Redundant calls to startThreads() do nothing and return zero. Invoking stopThreads() destroys each worker thread (after it completes any current job). Note that the current contents of the queue [Middle], and the ability to enqueue new jobs [Back] are not affected.

Whether or not started threads should be pulling jobs from the queue and processing them is not necessarily the same as having the user-specified number of worker threads created. In addition to being enabled and started let's consider one more possible state, suspended. If a thread pool is in the suspended state, then even when it is in the started state, it will not attempt to pop jobs from the queue and execute them.

A created threadpool will be created enabled, not suspended, and not started. All three of these qualities are orthogonal and any one of them can be changed at any time.

The vast majority of users will be uninterested in both the suspend and disable features, so it is imperative that newly created threadpools be both non-suspended and enabled so users can remain blissfully ignorant of them. It is also important the first usage examples, if not all of them, omit use of these features to minimize learning time for the typical user.

To conclude this generic overview, we note that there is one common usage that, although not minimal, arguably deserves to be a method of every thread pool class: void shutdown(). This method is best described as a composition of the simple, orthogonal functions described above. In order to shut down a thread pool, we need to first disable the enqueuing of any additional jobs, then remove all of the pending work items, and finally stop all of the active threads:

  void shutdown()
  {
      disableQueue();
      removeJobs();
      stopThreads();
  }

By making sure that our initial operations are simple and orthogonal, we can ensure that the precise meaning of more complex operations is kept clear.

Synchronous Signals on Unix:

A thread pool ensures that, on Unix platforms, all the threads in the pool block all asynchronous signals. Specifically all the signals, except the following synchronous signals are blocked.

  SIGBUS
  SIGFPE
  SIGILL
  SIGSEGV
  SIGSYS
  SIGABRT
  SIGTRAP
  SIGIOT