[cpp-threads] Asynchronous Function Proposal

[Lawrence Crowl]

On 6/19/09, Herb Sutter <hsutter at microsoft.com> wrote:
> Pablo wrote:
> > I have been silent on this discussion for a long time, partly
> > because I've been extremely busy preparing and presenting a
> > course on parallel programming.  I apologize if any of what
> > I'm about to say should have been said earlier.
>
> No problem, input is always welcome.

Agreed.

> > 1. The parallel-programming trend is just now heating up.
> > If we try to standardize sophisticated facilities for parallel
> > programming, we are
>
> Agreed, and nobody is proposing that.

Agreed.

> > 2. What we have in C++0x is a threading library, not a
> > parallel-programming library.  If it can be used as a platform
> > for a parallelism library, great, but we should not try to
> > throw one parallel-programming trick into the mix in isolation.
>
> Agreed, and nobody is proposing that.

Agreed.

> > 3. I don't think async() will actually work as a general
> > fork-join tool without a LOT more work.
>
> Agreed, and nobody is proposing that.

Agreed.

> Let me try again:
>
> N2889 already permits a work stealing implementation because
> it allows the "either"/"may-be-called" policy. We already agree
> on this point which is what enables running the work on a work
> stealing runtime, and that's the only thing in the whole proposal
> related to work stealing.
>
> The only point of disagreement that's related to runtimes at all
> is that some of us want to _allow_ running the async operation
> in a more efficient way than "on a new thread," because that
> wording doesn't permit any kind of caching of threads (including,
> but not limited to, running it on a thread pool).
>
> To illustrate how important the efficiency loss is, proponents of
> N2889 have suggested that the operating system can or should be
> changed to make "on a new thread" efficient. It is a departure
> for C++ as a systems programming language for existing systems
> to assume OS support for efficient operation.

I do not think that the efficiency loss here is that significant.
N2889 has the point to address a possible objection, not because
I think it needs doing.  The reasons are two-fold.

First, async-must-have-a-thread will in all likelyhood be that way
because of synchronization.  In that environment, I think of async
as a way to get a thread-with-return-value.

Second, async-do-what-you-want can exploit the threads created
at higher levels in the program to provide the necessary context
for sequential execution of async work.  That is, in the tree of
computation, a small number of threads will be created near the root,
and the rest of the branches and leaves will execute serially.

Put another way, async is a tool for extracting concurrency from the
program structure, not from its data structures.  More specifically,
async is not a replacement for OpenMP.

The programming model is:
   At the highest levels of the program, add async where appropriate.
   If enough concurrency has not been achieved, move down a layer.
   Repeat until you achieve the desired core utilization.

In this model, nested async is not only supported, but desired.

Be wary of trying to use this facility for massive parallelism,
it is a stop-gap mechanism for few-core systems until TR2 becomes
available.

A difference in the anticpated programming model may be at the root
of the lack of consensus.

> There's another key problem that I don't think I mentioned
> explicitly before, which is oversubscription: A big reason
> to be able to run the async task on a thread pool is that
> the pool is already in the business of staying "rightsized"
> for the machine. In an application that uses today's thread
> pools, having compute-intensive work apart from the thread pool
> penalizes performance because it makes it harder for the pool to
> accurately match ready work to available cores and oversubscribes
> the machine. So that's another key reason that an efficient
> implementation needs to be able to run the work on a pool
> especially when the application is using that pool anyway.

I agree that async must avoid the over-subscription problem.
However, thread pools are not necessary to avoid the problem.
In particular, a count of active threads, compared against
std::thread::hardware_concurrency() can provide all the information
necessary to determine if new thread should be created.

-- 
Lawrence Crowl