Coordinating Asynchronous Code

JS/Pipe provides JavaScript primitives that make it possible to coordinate asynchronous code without callbacks or chained functions.

As a result, stack traces are crystal clear so debugging is easy.

Although other solutions also try to make it easier to deal with callback hell, race conditions, and code coordination, they suffer from the same problem as callbacks/events in that they intertwine control flow and communication. This results in esoteric code that is harder to grok.


JS/Pipe is inspired by Goroutines and Channels, found in the Go language.

Coordinating concurrent activities using Jobs and Pipes

Jobs and Pipes make programs with concurrent behavior (e.g. user can navigate through a list while data gets downloaded) much easier to write and understand because you can use standard control flow mechanisms like for and while loops, conditionals like if/then/else and case, higher order functions like map and reduce and error handling using try/catch.


Job

A Job is a function executing concurrently with other Jobs. Start a Job by calling it with the job function:

        job(someFunction);
      
You can also pass a function literal, e.g.
        job(function* (x, y) {
            var answer = x + y;
            console.log('answer=' + answer);
        });
      
Notice that the function is declared with an asterisk function*. This makes it a Generator function, which is new in the upcoming version of JavaScript, but that can be used today by transpiling your code to ES5 using Google Traceur. (you can declare Generators in NodeJS 0.11 when running with the -harmony flag).


Pipe

A Pipe is created with new Pipe() and it provides an API for sending/receiving data between two Jobs. Pipes are unbuffered/synchronous which has the nice effect that they combine communication (data exchange) with synchronization, guaranteeing that two Jobs are in a known state at any given time.

A Job can put data into a Pipe by calling yield pipe.put('someData'), and another Job can get data by calling yield pipe.get(). The yield keyword causes the Job to suspend executation until another Job issues an opposite request on the same Pipe. When this happens, the Jobs are said to rendezvous; they are now synchronized, and will each continue code execution until they reach another yield statement.

simple example

This example was inpired by David Nolen's example for Clojure(Script)'s core.async.

          var pipe = new JSPipe.Pipe(),
              timeout = JSPipe.timeout,
              job = JSPipe.job;

          
          job(function* () {
              while (yield timeout(250).get()) {
                  pipe.send(1);
              }   
          });

          job(function* () {
              while (yield timeout(1000).get()) {
                  pipe.send(2);
              }
          });

          job(function* () {
              while (yield timeout(1500).get()) {
                  pipe.send(3);
              }
          });

          job(function* () {
              var data;
              while (data = yield pipe.get()) {
                  console.log(data);            
              }
          });
        

We create a new pipe that the two jobs will use to communicate & synchronize on.

timeoutis a standard pipe that waits for the specified amount of time before it puts data on the pipe, signalling to a waiting job that the time as expired.


Now we start our first job. It runs every 250ms (yield timeout(250).get()), sending the value "1" to the pipe.


Similarly, we have a second and third job, with 1000ms and 1500ms cadences respectively, each sending values to the pipe.


The fourth job waits until it can get data from the pipe and then simply displays it to the user.

complex example

Building an autocomplete search input that is responsive, efficient, and correct can be tricky. For a good user experience, it requires careful coordination of user input events and API requests.

This tutorial shows how you can implement it very simply using Jobs and Pipes.

get started

Get the code.

Fork me on GitHub