125 lines
5.3 KiB
Plaintext
125 lines
5.3 KiB
Plaintext
[/
|
|
Copyright Oliver Kowalke 2016.
|
|
Distributed under the Boost Software License, Version 1.0.
|
|
(See accompanying file LICENSE_1_0.txt or copy at
|
|
http://www.boost.org/LICENSE_1_0.txt
|
|
]
|
|
|
|
[/ import path is relative to this .qbk file]
|
|
[import ../examples/work_sharing.cpp]
|
|
|
|
[#migration]
|
|
[section:migration Migrating fibers between threads]
|
|
|
|
[heading Overview]
|
|
|
|
Each fiber owns a stack and manages its execution state, including all
|
|
registers and CPU flags, the instruction pointer and the stack pointer. That
|
|
means, in general, a fiber is not bound to a specific thread.[footnote The
|
|
["main] fiber on each thread, that is, the fiber on which the thread is
|
|
launched, cannot migrate to any other thread. Also __boost_fiber__ implicitly
|
|
creates a dispatcher fiber for each thread [mdash] this cannot migrate
|
|
either.][superscript,][footnote Of course it would be problematic to migrate a
|
|
fiber that relies on [link thread_local_storage thread-local storage].]
|
|
|
|
Migrating a fiber from a logical CPU with heavy workload to another
|
|
logical CPU with a lighter workload might speed up the overall execution.
|
|
Note that in the case of NUMA-architectures, it is not always advisable to
|
|
migrate data between threads. Suppose fiber ['f] is running on logical CPU
|
|
['cpu0] which belongs to NUMA node ['node0]. The data of ['f] are allocated on
|
|
the physical memory located at ['node0]. Migrating the fiber from ['cpu0] to
|
|
another logical CPU ['cpuX] which is part of a different NUMA node ['nodeX]
|
|
might reduce the performance of the application due to increased latency of
|
|
memory access.
|
|
|
|
Only fibers that are contained in __algo__[s] ready queue can migrate between
|
|
threads. You cannot migrate a running fiber, nor one that is __blocked__. You
|
|
cannot migrate a fiber if its [member_link context..is_context] method returns
|
|
`true` for `pinned_context`.
|
|
|
|
In __boost_fiber__ a fiber is migrated by invoking __context_detach__ on the
|
|
thread from which the fiber migrates and __context_attach__ on the thread to
|
|
which the fiber migrates.
|
|
|
|
Thus, fiber migration is accomplished by sharing state between instances of a
|
|
user-coded __algo__ implementation running on different threads. The fiber[s]
|
|
original thread calls [member_link algorithm..awakened], passing the
|
|
fiber[s] [class_link context][^*]. The `awakened()` implementation calls
|
|
__context_detach__.
|
|
|
|
At some later point, when the same or a different thread calls [member_link
|
|
algorithm..pick_next], the `pick_next()` implementation selects a ready
|
|
fiber and calls __context_attach__ on it before returning it.
|
|
|
|
As stated above, a `context` for which `is_context(pinned_context) == true`
|
|
must never be passed to either __context_detach__ or __context_attach__. It
|
|
may only be returned from `pick_next()` called by the ['same] thread that
|
|
passed that context to `awakened()`.
|
|
|
|
[heading Example of work sharing]
|
|
|
|
In the example [@../../examples/work_sharing.cpp work_sharing.cpp]
|
|
multiple worker fibers are created on the main thread. Each fiber gets a
|
|
character as parameter at construction. This character is printed out ten times.
|
|
Between each iteration the fiber calls __yield__. That puts the fiber in the
|
|
ready queue of the fiber-scheduler ['shared_ready_queue], running in the current
|
|
thread.
|
|
The next fiber ready to be executed is dequeued from the shared ready queue
|
|
and resumed by ['shared_ready_queue] running on ['any participating thread].
|
|
|
|
All instances of ['shared_ready_queue] share one global concurrent queue, used
|
|
as ready queue. This mechanism shares all worker fibers between all instances
|
|
of ['shared_ready_queue], thus between all participating threads.
|
|
|
|
|
|
[heading Setup of threads and fibers]
|
|
|
|
In `main()` the fiber-scheduler is installed and the worker fibers and the
|
|
threads are launched.
|
|
|
|
[main_ws]
|
|
|
|
The start of the threads is synchronized with a barrier. The main fiber of
|
|
each thread (including main thread) is suspended until all worker fibers are
|
|
complete. When the main fiber returns from __cond_wait__, the thread
|
|
terminates: the main thread joins all other threads.
|
|
|
|
[thread_fn_ws]
|
|
|
|
Each worker fiber executes function `whatevah()` with character `me` as
|
|
parameter. The fiber yields in a loop and prints out a message if it was migrated
|
|
to another thread.
|
|
|
|
[fiber_fn_ws]
|
|
|
|
|
|
[heading Scheduling fibers]
|
|
|
|
The fiber scheduler `shared_ready_queue` is like `round_robin`, except that it
|
|
shares a common ready queue among all participating threads. A thread
|
|
participates in this pool by executing [function_link use_scheduling_algorithm]
|
|
before any other __boost_fiber__ operation.
|
|
|
|
The important point about the ready queue is that it[s] a class static, common
|
|
to all instances of shared_ready_queue.
|
|
Fibers that are enqueued via __algo_awakened__ (fibers that are ready to be
|
|
resumed) are thus available to all threads.
|
|
It is required to reserve a separate, scheduler-specific queue for the thread[s]
|
|
main fiber and dispatcher fibers: these may ['not] be shared between threads!
|
|
When we[,]re passed either of these fibers, push it there instead of in the
|
|
shared queue: it would be Bad News for thread B to retrieve and attempt to
|
|
execute thread A[s] main fiber.
|
|
|
|
[awakened_ws]
|
|
|
|
When __algo_pick_next__ gets called inside one thread, a fiber is dequeued from
|
|
['rqueue_] and will be resumed in that thread.
|
|
|
|
[pick_next_ws]
|
|
|
|
|
|
The source code above is found in
|
|
[@../../examples/work_sharing.cpp work_sharing.cpp].
|
|
|
|
[endsect]
|