Document of relevance

[Peter A. Buhr]

Andrei, thanks for the documentation. It will take we at least a week to get
through it. In the meantime, I have attached a recent conversation I had with
Herb Sutter to give people a sense of my global position with respect to
concurrency, i.e., independent of a specific language to which it is added.  To
be perfectly clear, I'm coming into this with a strong attitude about how it
should be done, and I'm not going to change that attitude. If people want to
take the Java attitude that "the language can't change" and then try to make
changes, then I'm not interested in participating. I say this up front because
I don't want to waste my time or your time on this project.

===============================================================================

Date: Mon, 10 Jan 2005 15:53:00 -0500 (EST)
From: "Peter A. Buhr" <pabuhr at plg.uwaterloo.ca>
To: hsutter at microsoft.com
CC: rcbilson at uwaterloo.ca, asharji at plg.uwaterloo.ca
Subject: Re: uC++

   My name is Herb, and I chair the ISO C++ standards committee and
   architect the MS Visual C++ product. I'm interested in the subject of
   adding clean language support for concurrency to C++, and among the
   research I've been doing in this area your name and uC++ project
   naturally also came up.

Hi Herb.

I'm flattered to be contacted by you in your official capacity as chair of the
ISO C++ standards committee. I delayed in responding hoping you would get a
chance to read some of the uC++ material you found, which is essential to
understand some of my following comments. As well, the start of term has been
very busy.

   I notice that you have papers and a draft book on your website; I've
   downloaded the latter but haven't read it yet (is it going to be
   published?).

I'm having troubling finding the time to finish the book. As a professor, I
have a lot of different kinds of work and not much money, so I have to spend
time rather than money to accomplish things, which means its hard to find the
extra time necessary to finish the book. Nevertheless, I am going to try to
finish it this Summer.

   I'd like to understand the current state of your project and how you see
   this work developing. If you have a few minutes to chat (presumably
   after the holidays) I'd appreciate it. Thanks, and have a good rest of
   the year,

uC++ is alive and well, and has been successfully used to educate 2000+
Waterloo undergraduate students over the last 12+ years. Currently, uC++ is
being funded through a research grant involving HP and the Federal and
Provincial governments.

I am always happy to chat on this topic, which is dear to my heart. Also, I
would enjoy working with the C++ standards committee on extending C++ with
advanced control flow features, like concurrency.  However, I'm only interested
depending on the direction the committee is going to take. Here's why:

1. Advanced control-flow and especially concurrency CANNOT be added to a
   language after the fact through the language itself, i.e., solely via a
   library approach. I have spent the last 10 years trying to explain this fact
   to people, often to no avail. Exception handling, coroutines, and
   concurrency have a pervasive affect throughout a programming language.
   Specifically, these advanced forms of control flow can be in direct conflict
   with other language requirements like code optimization.  To deal with these
   conflicts, a compiler must know if and where advanced control-flow is
   occurring, which is only possible if the advanced control-flow is part of
   the programming language. For example, pthreads is the de facto concurrency
   library for C, but it technically does not work. To make it work, many
   compiler writers scan for pthreads routines and perform special code
   generation to ensure things work correctly. This clandestine approach to
   ensuring correctness is a mistake, and people are fooling themselves into
   believing that a library approach can actually work.

   Doug Lea sent me a first draft of the "Memory model for multithreaded C++"
   (#WG21/N1680=J16/04-0120) several months ago, and it is clear from this
   proposal that this group understands adding concurrency to C++ requires some
   language changes. The draft proposal appears to have identified most of the
   basic low-level problem areas, but it lacked firm suggestions on how to
   solve these problems. However, a memory-model proposal only addresses
   low-level effects, albeit of crucial importance, but it does not address
   making concurrency work consistently with high-level language features
   (e.g., exception handling and inheritance), or defining a concurrency model
   and language constructs to support this model.

2. Which brings me to the last point in the draft proposal for a thread
   library. Given a correct memory model for C++, I have shown in prior work
   that advanced control-flow and especially concurrency still CANNOT be added
   into an OO language using existing language features.  For example, the C++
   Boost thread-library is possibly the best attempt at this kind of solution,
   but it is nothing more than a fancy pthreads for C++. Users must still
   follow a list of conventions to make it work, and it does not integrate with
   the OO model of C++ or other language features. Boost and all libraries like
   it only supply a simple threads and locks concurrency model, which is
   inadequate for future development of concurrency in any programming
   language.

   If you have looked at the extended control-flow features in uC++, you can
   see what I'm referring to. The uC++ model integrates new control-flow
   facilities with the existing features in C++ by building on the class
   notion, which retains all class mechanisms and features (inheritance,
   overloading and templates). As well, the uC++ model extends other features,
   like exception handling, to have meaning in a multithreaded context. When
   using uC++'s high-level concurrency features, there are no explicit locks;
   all locking is implicit in the new object and how it is used. While uC++
   does provide low-level locks, these are only for teaching and situations
   requiring very unusual control and extreme performance. Java (and other
   languages) have clearly demonstrated that even a simple concurrency
   construct like a "monitor" makes a significant improvement in the
   development and correctness of concurrent programs.

   So I'm only interested in helping if people want to add advanced control
   flow in a way that properly integrates with the existing OO model of C++ and
   the other features within the language.  Hence, if the C++ standards
   committee is going to pursue a simple library approach, I cannot be helpful,
   nor do I believe such an approach will be productive, successful, or advance
   the state of the art.

3. Advanced control-flow and especially concurrency NEED additional
   features, such as:

   * advanced concurrent exception handling
   * real-time: real-time tasks, extensible schedulers, inheritance protocols,
     timeouts
   * user-defined grouping of tasks and virtual processors (clustering)
   * object-oriented, nonblocking I/O for files and sockets
   * concurrent profiling and debugger tools (which I'm sure is out of the
     committee's purview, for better or worse)

So if the C++ standards committee has the will and mandate to extend the
language as is necessary, and focus on proposals that are truly integrated
versus bolt-on, then I'm very interested in helping, and I believe I can make a
significant contribution, even if the approaches in uC++ are not the ones
selected.

I hope this opening message helps you to understand how I feel about this issue
and how I might be useful to the standards committee.

   P.S.: UW is my alma mater; I did my undergrad there in the 80s.

I started at Waterloo in 1986, so our paths might have crossed briefly.

===============================================================================

Subject: RE: uC++
Date: Mon, 10 Jan 2005 13:54:16 -0800
From: "Herb Sutter" <hsutter at microsoft.com>
To: "Peter A. Buhr" <pabuhr at plg.uwaterloo.ca>
Cc: <rcbilson at uwaterloo.ca>, <asharji at plg.uwaterloo.ca>

Hi Peter,

> I'm flattered to be contacted by you in your official capacity as chair of
> the ISO C++ standards committee.

Just remember the chair doesn't get to vote. :-) I see you know of the proposal
to apply the JSR 133 work to C++. That's part of what sparked my interest in
the area, and as I also chair the subgroup that's responsible for safety and
performance I want to make sure the best people can be involved to contribute
if they're able, realizing that not everyone has time to participate in
often-lengthy standards work. I think you can take it for granted that the
committee members understand and agree with the points you made (at least most
of them; a small number like Boost.Threads).

But I'm also personally interested in this area, because I'm convinced we need
to develop a higher-level programming model for concurrency via clean language
extensions. Solutions to simple concurrency problems still get published as
research papers (e.g., Santa Claus) and it's really not good enough to expect
an average developer to do what amounts to advancing the state of the computing
art just to correctly implement their applications' custom concurrency
requirements that depart from the limited number of classic well-analyzed
problems. I'm doing some design work in this area.

So in part I wanted to see how involved you could (or wanted) to be with the
proposals being brought to the C++ standards committee, and in part I wanted to
see what your thinking was on the successes of uC++ and what further
higher-level extensions are on your radar in the Abstractions That Ought To
Exist department.

Re that first part, you wrote:

> So if the C++ standards committee has the will and mandate to extend the
> language as is necessary, and focus on proposals that are truly integrated
> versus bolt-on, then I'm very interested in helping, and I believe I can
> make a significant contribution, even if the approaches in uC++ are not
> the ones selected.

Great. Are you keeping in touch with Doug and the other folks working on that
proposal?

Thanks,

Herb