Push Generator Proposal (ES2016)

Push Generators are an alternative to the Async Generator proposal currently in the strawman stage. They are proposed for ES2016.

The Problem

JavaScript programs are single-threaded and therefore must steadfastly avoid blocking on IO operations. Today web developers must deal with a steadily-increasing number of push stream APIs including...

• Server sent events
• Web sockets
• DOM events

Unfortunately consuming these data sources in JavaScript is inconvenient. Asynchronous functions (proposed for ES7) provide language support for functions that push a single result, allowing loops and try/catch to be used for control flow and error handling respectively. ES6 introduced language support for producing and consuming functions that return multiple results. However no language support is currently proposed for functions that push multiple values. The push generator proposal is intended to resolve this discrepancy, and add symmetrical support for push and pull functions in JavaScript.

Iteration and Observation

Iteration and Observation are two patterns which allow a consumer to progressively consume multiple values produced by a producer. In each of these patterns, the producer may produce one of three types of notifications:

a) a value b) an error c) a final value

When the producer notifies the consumer of an error or a final value, no further values will be produced. Furthermore in both patterns the producer or consumer should be able to short-circuit at any time and elect to produce or receive no further notifications.

Iteration

Iteration puts the consumer in control. In iteration the producer waits for the consumer to pull a value. The consumer may synchronously or asynchronously pull values from the producer (the iterator), and the producer must synchronously produce values.

In ES6, language support was added for producing and consuming values via iteration.

// data producer
function* nums() {
  yield 1;
  yield 2;
  return 3;
}

// data consumer
function consume() {
  for(var x of nums()) {
    console.log(x);
  }
}

consume();
// prints
// 1
// 2

Generator functions are considered pull functions, because the producer delivers notifications in the return position of function. This is evident in the desugared version of the for...of statement:

function nums() {
  var state = 0;
  return {
    next(v) {
      switch(state) {
        case 0:
          state = 1;
          return {value: 1, done: false};
        case 1:
          state = 2;
          return {value: 2, done: false};
        case 2:
          state = 3;
          return {value: 3, done: true};
        caase 3:
          
    }
}

function consume() {
  var generator = nums(),
    pair;
  
  // value received in return position of next()
  while(!(pair = generator.next()).done) {
    console.log(pair.value);
  }
}

consume();
// prints
// 1
// 2

The producer may end the stream by either throwing when next is called or returning an IterationResult with a done value of true. The consumer may short-circuit the iteration process by invoking the return() method on the data producer (iterator).

generator.return();

Note that there is no way for the producer to asynchronously notify the consumer that the stream has completed. Instead the producer must wait until the consumer requests a new value to indicate stream completion.

Iteration works well for consuming streams of values that can be produced synchronously, such as in-memory collections or lazily-computed values (ex. fibonnacci sequence). However it is not possible to push streams such as DOM events or Websockets as iterators, because they produce their values asynchronously. For these data sources is necessary to use observation.

Observation

Observation puts the producer in control. The producer may synchronously or asynchronouly push values to the consumer's sink (observer), but the consumer must handle each value synchronously. In observation the consumer waits for the producer to push a value.

The push generator proposal would add syntactic support for producing and consuming push streams of data to ES2016:

// data producer
function*> nums() {
  yield 1;
  yield 2;
  return 3;
}

// data consumer
function*> consume() {
  for(var x on nums()) {
    console.log(x);
  }
}

// note that two functions calls are required, one to create the Observable and another to being observation
consume()();
// prints
// 1
// 2

Push Generator functions are considered push, because the producer delivers notifications in the argument position of the function. This is evident in the desugared version of the code above:

function nums() {
  return {
    [Symbol.observer](generator = { next() {}, throw() {}, return() {} }) {
      var iterationResult = generator.next(1);
      if (iterationResult.done) break;
      iterationResult = generator.next(2);
      if (iterationResult.done) break;
      iterationResult = generator.return(3);
      
      return generator;
    }
  }
};

function consume() {
  return {
    [Symbol.observer](generator = { next() {}, throw() {}, return() {} }) {
      return nums()[Symbol.observer]({
        next(v) { console.log(v); }
      });
    }
  };
};

// note that two functions calls are required, one to create the Observable and another to being observation
consume()();
// prints
// 1
// 2

The push generator proposal would add symmetrical support for Iteration and Observation to JavaScript. Like generator functions, push generator functions allow functions to return multiple values. However push generator functions send values to consumers via Observation rather than Iteration. The for...on loop is also introduced to enable values to be consumed via observation. The process of Observation is standardized using a new interface: Observable.

Introducing Observable

ES6 introduces the Generator interface, which is a combination of two different interfaces:

1. Iterator
2. Observer

The Iterator is a data source that can return a value, an error (via throw), or a final value (value where IterationResult::done).

interface Iterator {
  IterationResult next();
}

type IterationResult = {done: boolean, value: any}

interface Iterable {
  Iterator iterator();
}

The Observer is a data sink which can be pushed a value, an error (via throw()), or a final value (return()):

interface Observer {
  void next(value);
  void return(returnValue);
  void throw(error);
}

These two data types mixed together forms a Generator:

interface Generator {
  IterationResult next(value);
  IterationResult return(returnValue);
  IterationResult throw(error);
}

Iteration and Observation both enable a consumer to progressively retrieve 0...N values from a producer. The only difference between Iteration and Observation is the party in control. In iteration the consumer is in control because the consumer initiates the request for a value, and the producer must synchronously respond.

In this example a consumer requests an Iterator from an Array, and progressively requests the next value until the stream is exhausted.

function printNums(arr) {
  // requesting an iterator from the Array, which is an Iterable
  var iterator = arr[Symbol.iterator](),
    pair;
  // consumer (this function)
  while(!(pair = iterator.next()).done) {
    console.log(pair.value);
  }
}

This code relies on the fact that in ES6, all collections implement the Iterable interface. ES6 also added special support for...of syntax, the program above can be rewritten like this:

function printNums(arr) {
  for(var value of arr) {
    console.log(value);
  }
}

ES6 added great support for Iteration, but currently there is no equivalent of the Iterable type for Observation. How would we design such a type? By taking the dual of the Iterable type.

interface Iterable {
  Generator [Symbol.iterator]()
}

The dual of a type is derived by swapping the argument and return types of its methods, and taking the dual of each term. The dual of a Generator is a Generator, because it is symmetrical. The generator can both accept and return the same three types of notifications:

1. data
2. error
3. final value

Therefore all that is left to do is swap the arguments and return type of the Iterator's iterator method and then we have an Observable.

interface Observable {
  void [Symbol.observer](Generator observer)
}

This interface is too simple. If iteration and observation can be thought of as long running functions, the party that is not in control needs a way to short-circuit the operation. In the case of observation, the producer is in control. As a result the consumer needs a way of terminating observation. If we use the terminology of events, we would say the consumer needs a way to unsubscribe. To allow for this, we make the following modification to the Observable interface:

interface Observation {
  void unobserve();
}

interface Observable {
  Observation [Symbol.observer](Generator observer)
}

This version of the Observable interface accepts a Generator and returns an Observation. The consumer can short-circuit observation (unsubscribe) by invoking the return() method on the Generator object returned for the Observable @@observer method. To demonstrate how this works, let's take a look at how we can adapt a common push stream API (DOM event) to an Observable.

// The decorate method accepts a generator and dynamically inherits a new generator from it
// using Object.create. The new generator wraps the next, return, and throw methods, 
// intercepts any terminating operations, and invokes an onDone callback.
// This includes calls to return, throw, or next calls that return a pair with done === true
fu