[cpp-threads] Asynchronous Function Proposal

[Hans Boehm]

On Fri, 19 Jun 2009, Herb Sutter wrote:

> Lawrence wrote:
>
[Herb:]
>>> There's another key problem that I don't think I mentioned
>>> explicitly before, which is oversubscription: A big reason
>>> to be able to run the async task on a thread pool is that
>>> the pool is already in the business of staying "rightsized"
>>> for the machine. In an application that uses today's thread
>>> pools, having compute-intensive work apart from the thread pool
>>> penalizes performance because it makes it harder for the pool to
>>> accurately match ready work to available cores and oversubscribes
>>> the machine. So that's another key reason that an efficient
>>> implementation needs to be able to run the work on a pool
>>> especially when the application is using that pool anyway.
>>
>> I agree that async must avoid the over-subscription problem.
>> However, thread pools are not necessary to avoid the problem.
>> In particular, a count of active threads, compared against
>> std::thread::hardware_concurrency() can provide all the information
>> necessary to determine if new thread should be created.
>
> No, that's not at all what I'm saying. I'm saying that an "in a new 
> thread" async specification doesn't play nice with thread pools, and 
> therefore doesn't play nice with apps that *do already* (or will choose 
> to) use thread pools for their compute-intensive work.
>
> Let me try to say it again with maybe slightly clearer phrasing:
>
> Applications that run their compute-intensive work on a thread pool 
> really want all their compute-intensive work to run on the pool. The 
> pool is already in the business of staying "rightsized" for the machine, 
> and having compute-intensive work outside the thread pool interferes 
> with the pool's ability to accurately match the number of ready threads 
> to the available hardware. Each compute-intensive async task in a 
> non-pool thread adds extra work that the thread pool doesn't know about 
> and so results in oversubscribing the machine, providing more ready work 
> than there is available hardware parallelism.
>
> If we mandate "in a new thread," then it will probably be unusable in 
> practice for any compute-intensive task in an application that is using 
> a thread pool to spread its compute-intensive work across the available 
> hardware.
>
I'm currently have some trouble with combining some of the different lines 
of reasoning being used here.  I think Lawrence has been assuming(?) that
thread pools will be used in part to manage object lifetimes.  If you
have a bunch of tasks that use an allocator of limited lifetime, you may
need to make sure that the thread pool goes away before the allocator 
does, because some of the libraries involved may use thread-locals.

In this model, I don't think you can limit yourself to a single thread 
pool.  There are probably other reasons you can't.  The 
java.util.concurrent executor framework has many knobs, which I think are 
there for a reason, and need to be set differently for different 
applications.

I'm not convinced it will ever be feasible to insist that all tasks are
executed by one kind of thread pool.  We may want to encourage the 
programmer to do that for each stage of program execution, since it
may maximize performance.  But I don't see this as an argument
against any particular library construct.  Some applications will
always want to live on bare threads, or some other kind of model.

Hans