Stated

<!-- TOC -->

• Stated
• Licensing and documentation
• API docs
• Testing
• Intro
• Motivation
• Getting Started
  • Installation
  • Running
    • REPL Mode
    • Oneshot mode
• stated-js lib
  • API documentation
  • Basic usage
    • Stated-js package.json exports
    • Node.js based projects
• REPL Commands
  • .open
  • All commands
  • .out
  • .color
  • .log
  • .set
  • REPL command arguments
    • --options
      • strict
    • --xf
    • --ctx
• Language & Syntax
  • Expressions and Variables
  • Dollars-Moustache
  • Dollars-Variables
  • Temporary Expressions
  • References
  • Expression Scope
  • Rerooting Expressions
  • Tags
• Generative Templates
• Reactive Behavior
  • SVG command
  • YAML
  • Setting Values in the stated REPL
  • Expression Scope
  • Rerooting Expressions
  • DAG
  • visualizing the plan with .svg command
• Concurrency
  • Serialized Mutations
  • Atomic State Updates
  • Multi Version Concurrency Control (MVCC) and $forked
• YAML
• Complex Data Processing
• Functions
  • JSONata built-in
  • Stated functions
    • $timeout
    • $interval
    • $fetch
    • $import
      • Importing bits of other templates
      • Setting up local imports with --importPath
      • Import Javascript ES modules
      • The __init sidecar function
    • $open
    • $set
    • $debounce
    • $defer
    • $rateLimit
  • Custom Functions
    • Simple Custom Function Example
    • More Complex Function Example
• Error Handling
  • The error object
  • Error Reporting
  • $errorReport function
• TemplateProcessor Snapshots
• Understanding Plans
• Planning
  • MetaInfo Graph
  • DFS Tree Traversal

<!-- TOC -->

Licensing and documentation

Stated is a cisco-open, Apache 2 Licensed, Open Source project at https://github.com/cisco-open/stated, our github main page.

API docs

If you would like to see a table of contents and developer API docs, jump over to our API docs page which is generated in CI and published here. Most developers will need to interact with the TemplateProcessor. It's API Docs are here

Testing

Are the examples in this README correct/reliable? Every markdown codeblock in this readme is tested on every commit.

Intro

Stated is an engine of reactive state (a State Daemon). State is expressed as JSON or YAML. Rules for evolving state are written via embedded JSONata expressions in classic shell variable syntax ${}.

cat examples/hello.json
{
  "to": "world",
  "msg": "${'hello ' & to}"
}

You can use stated as a 'one shot' template processor that computes all bound states once. If we run the template above through stated, we see the ${} expression has been evaluated:

stated example/hello.json
{
  "to": "world",
  "msg": "hello world"
}

Variables can be passed in:

cat example/helloVar.json
{
  "msg": "${'hello ' & $TO}"
}
stated example/helloVar.json --ctx.TO=world
{
   "msg": "hello world"
}

The more interesting use of Stated is an engine of reactive state. Load a template in the REPL, .set a value and watch the state evolve.

stated
> .init -f example/hello.json
{
   "to": "world",
   "msg": "${'hello ' & to}"
}
> .set /to "Joshua"
{
   "to": "Joshua",
   "msg": "hello Joshua"
}
> .set /to "David Lightman"
{
   "to": "David Lightman",
   "msg": "hello David Lightman"
}

Stated templates can contain expressions, reusable functions, and can even use JS timeouts and intervals. Let's see a template that increments a counter every 10 ms, forever, as long it is running in the Stated engine. We will use the --tail command to tail the count variable until it reaches 100, then automatically disconnect the tail.

cat example/infiniteCount.json
{
  "count": 0,
  "counter": "${ $setInterval(function(){$set('/count', count+1)}, 10) }",
}
stated
> .init -f example/infiniteCount.json --tail "/count until $=100"
Started tailing... Press Ctrl+C to stop.
100

Stated is written in JS, and runs in the browser and in Node. Stated's REPL extends the Node.js REPL and allows you to interact with running templates. Stated uses asynchronous event loop I/O and can be used to orchestrate complex workflows:

> .init -f example/homeworlds.json
{
  "lukePersonUrl": "${ $fetch('https://swapi.tech/api/people/?name=luke').json().**.url}",
  "lukePersonDetails": "${ $fetch(lukePersonUrl).json().result[0]}",
  "lukeHomeworldURL": "${ lukePersonDetails.**.homeworld }",
  "homeworldDetails": "${ $fetch(lukeHomeworldURL).json() }",
  "homeworldName": "${ homeworldDetails.**.name }"
}
> .out /homeworldName
"Tatooine"

Unlike an ordinary program, Stated templates can be kept "alive" indefinitely. A change to any of the independent fields causes change propagation throughout the DAG. Stated includes a node REPL, stated.ts, for testing Stated json templates, and a JS library for embedding stated in applications. A typical REPL session consists of loading a template with the init command, viewing the computed output with the .out command and then setting values with the .set command and observing the changed output.

falken$ stated
> .init -f "example/ex08.json"
{
  "a": "${c}",
  "b": "${d+1+e}",
  "c": "${b+1}",
  "d": "${e+1}",
  "e": 1
}
> .out
{
  "a": 5,
  "b": 4,
  "c": 5,
  "d": 2,
  "e": 1
}
> .set /e 42
{
  "a": 87,
  "b": 86,
  "c": 87,
  "d": 43,
  "e": 42
}

Stated templates are modular and can be imported from a URL:

> .init -f "example/ex18.json"
{
  "noradCommander": "${ norad.commanderDetails  }",
  "norad": "${ $import('https://raw.githubusercontent.com/geoffhendrey/jsonataplay/main/norad.json')}"
}
> .out
{
  "noradCommander": {
    "fullName": "Jack Beringer",
    "salutation": "General Jack Beringer",
    "systemsUnderCommand": 4
  },
  "norad": {
    "commanderDetails": {
      "fullName": "Jack Beringer",
      "salutation": "General Jack Beringer",
      "systemsUnderCommand": 4
    },
    "organization": "NORAD",
    "location": "Cheyenne Mountain Complex, Colorado",
    "commander": {
      "firstName": "Jack",
      "lastName": "Beringer",
      "rank": "General"
    },
    "purpose": "Provide aerospace warning, air sovereignty, and defense for North America",
    "systems": [
      "Ballistic Missile Early Warning System (BMEWS)",
      "North Warning System (NWS)",
      "Space-Based Infrared System (SBIRS)",
      "Cheyenne Mountain Complex"
    ]
  }
}

Motivation

Consider this ordinary program:

let a=1;
let b=a;
a=42;
console.log(b); //prints out 1

In an ordinary sequential program the value of b is not affected by changes to the value of a at any point after the value of b has been assigned. But there are many situations where we do NOT want a sequential program execution Instead, we actually want b to change when a changes. Broadly, these cases fall under the rubric of "reactive" or "state driven" applications. When we try to build reactive applications upon a sequential execution model we are forced to code the data flow graph ourselves and things become very complex quickly. How could we make b change any time a changes in a sequential world? Perhaps naively like this?

let a=1;
let b=a;
function setA(val){
    a=val;
    b=a;
}

...or perhaps more generally like this:

let data = {
  a: 1,
  b: 1 
};

let handler = {
  set: function(target, property, value) {
    if (property === 'a') {
      target.b = value; // When 'a' changes, also change 'b'
    }
    target[property] = value;
    return true; // The set operation was successful
  }
};

let proxy = new Proxy(data, handler);

proxy.a = 2; // Setting a new value for 'a'
console.log(proxy.a); // Outputs: 2
console.log(proxy.b); // Outputs: 2

Every "coding" approach requires us to understand and implement code for propagating data dependencies. Stated solves for this by natively parsing and understanding dependencies.

{
  "a": 1,
  "b$": "a"
}

b$ is now declared to be continuously dependent upon a and reactive to any changes in `