/ elib / concurrency

Overview: Concurrency in E

To programmers schooled in the C/Java/Python/Scheme traditions, the most unfamiliar part of E is its handling of concurrency. The conventional concurrency style -- multiple preemptive threads operating on shared data -- has been an unmitigated disaster. Although many people have learned this paradigm, very few have learned -- or could learn -- how to write complex correct programs in this paradigm. A correct program must maintain consistency while avoiding deadlock. Further, when extended to distributed systems, threads usually lead one into horribly inefficient communications patterns and consistency-recovery mechanisms (synchronous RPC and distributed transactions, respectively).

However, threads are a largely unexamined disaster, as few realize that any alternative is possible. Ironically, many people (especially UI programmers) already have lots of experience with a better alternative -- event loop programming -- without realizing it. We suffer from a blind spot. Although experience with event loops has generally been pleasant, and although the same effort on quality makes an event loop program much more reliable than a similar thread-based program, the common prejudice is to view event-loop programming as less respectable than thread-based programming. Event-loops are seen as disrespectable hacks we use only until we can build the proper solution --threads. Real computer scientists use threads.

Classic event-loops may feel dirty and hard to program in, but that's because, with this lack of respect, they have not received a fraction of the investment in tools and abstraction design that has been thrown at threads. At the same time, a different school of programming -- represented by Actors, Concurrent Logic Programming, Concurrent Constraint Programming, and Joule -- have done all their control flow with nothing but communicating event loops. These languages have shown how to provide high quality linguistic support for event-loop programming. Event loops are so successful at taming concurrency that these languages succeed at being massively concurrent while still being stateful. However, they are only concurrent -- they do not contain a sequential programming subsystem -- and so they are inaccessible to most programmers.

E reconciles these tensions. E supports event-loop programming with linguistic abstractions derived from these massively concurrent languages. At the same time, E combines event-loops and sequential programming in essentially the way familiar to UI programmers. The E programmer can easily, reliably, and efficiently maintain consistency without deadlock in the face of both concurrency and distribution.