[cpp-threads] Asynchronous Function Proposal

[Herb Sutter]

Pablo wrote:
> I have been silent on this discussion for a long time, partly because
> I've been extremely busy preparing and presenting a course on parallel
> programming.  I apologize if any of what I'm about to say should have
> been said earlier.

No problem, input is always welcome.

> 1. The parallel-programming trend is just now heating up.  If we try to
> standardize sophisticated facilities for parallel programming, we are

Agreed, and nobody is proposing that.

> 2. What we have in C++0x is a threading library, not a
> parallel-programming library.  If it can be used as a platform for a
> parallelism library, great, but we should not try to throw one
> parallel-programming trick into the mix in isolation.

Agreed, and nobody is proposing that. 

> 3. I don't think async() will actually work as a general fork-join tool
> without a LOT more work.

Agreed, and nobody is proposing that.

Let me try again:

N2889 already permits a work stealing implementation because it allows the "either"/"may-be-called" policy. We already agree on this point which is what enables running the work on a work stealing runtime, and that's the only thing in the whole proposal related to work stealing.

The only point of disagreement that's related to runtimes at all is that some of us want to _allow_ running the async operation in a more efficient way than "on a new thread," because that wording doesn't permit any kind of caching of threads (including, but not limited to, running it on a thread pool).

To illustrate how important the efficiency loss is, proponents of N2889 have suggested that the operating system can or should be changed to make "on a new thread" efficient. It is a departure for C++ as a systems programming language for existing systems to assume OS support for efficient operation.

There's another key problem that I don't think I mentioned explicitly before, which is oversubscription: A big reason to be able to run the async task on a thread pool is that the pool is already in the business of staying "rightsized" for the machine. In an application that uses today's thread pools, having compute-intensive work apart from the thread pool penalizes performance because it makes it harder for the pool to accurately match ready work to available cores and oversubscribes the machine. So that's another key reason that an efficient implementation needs to be able to run the work on a pool especially when the application is using that pool anyway.

Herb