[cpp-threads] Asynchronous Execution Issues (caching_async)

[Doug Lea]

Boehm, Hans wrote:
> Presumably, you would basically fork a thread if there are available
> processors.  But that may mean that even for a balanced tree-structured
> computation you may end up spending a lot of time forking very short-lived
> threads corresponding to tasks at the bottom of the tree, rather than forking
> at high levels of the tree.
> 

Right. So in most lightweight task frameworks (including
simple pools), programmers do not explicitly create threads but
instead provide work to existing ones. Almost no
matter how low the overhead of a thread, it will be
cheaper to have the same thread serially reused to perform
two different tasks than to use two different threads.
However, this should be exposed via APIs: you can't fool
taxi cab riders into thinking that they own new cars.
Which is one reason to distinguish "tasks" from "threads".

In which case, the main question is whether people creating task
frameworks/libraries that do this are provided with everything
they need to implement them. Like Hans, I'd hesitate to standardize
one particular medium-grained version without at least getting
some experience with these. Still, the thread-local problems already
noted for cached_async seem likely to arise across most of
these kinds of worker-thread frameworks, so might be worth
further consideration.

> The programming model seems similarly restrictive in that I can't allow
> arbitrary synchronization between tasks.  

Mostly just as an aside: In general, restrictions come with the
territory of lightweight frameworks, and the frameworks are useful
and well-received despite them: No one complains that you can't
do IO in upcoming (java.util.concurrent) ForkJoin framework, and
can't multiply matrices using upcoming (java.nio) AsyncIO framework.
(Well, in both cases, "can't" is a little too strong, but still.)

-Doug