This code is historical. See RDX C or RDX Go for the new generation of the technology. No JavaScript impl yet. Maybe you can help, please use Issues.

Swarm 2.0 real-time sync

<img align="right" width="400" src="https://i.imgur.com/hqGwft1.png">

Swarm.js is a JavaScript client for the Swarm database. Swarm is like "git for data" except it's real-time and never has a merge conflict. Swarm is based on Replicated Object Notation (RON), a distributed live data format. In turn, RON is based on Conflict-free Replicated Data Types (CRDTs), a new math apparatus for distributed data.

Like git, Swarm gives you a replica of your data which is as good as any other replica. You can read, you can write, you can see it updated by others. Concurrent changes merge automatically. Offline replicas re-sync on re-connection. All data is cached.

API-wise, Swarm is an object graph, although the API may depend on a particular mapper. Still, any API is a "view" of the system. The ground truth is RON.

Swarm picks its trade-offs in a rather opinionated way. It fits the world of abundant storage and unreliable wireless connections.

Swarm.js is isomorphic and is perfect for implementing synchronization, collaboration or continuity features in Web and mobile apps. Swarm works for:

• vanilla client-server sync,
• real-time collaboration,
• continuity,
• offline work,
• and other sync-related scenarios.

Unlike Firebase and others, you may run your own swarm server. If you understand how CRDT works, then you may implement your own data types. Free-as-in-freedom (MIT).

Table of contents:

• Intro
• Demos
• Setup
• CRDT implementations
• API
• GraphQL
• Using with React
• FAQ
• TODO
• Contributing
• Contacts

Demos

See our demos:

• "Mice" application - demo, source code.

  <details open="open"><summary><b>Preview</b></summary> <img src="https://i.imgur.com/tv8EITG.gif" width="600"> </details>

• Chat application - demos for webkit-backed browsers, iOS, Android and the source code

  <details><summary><b>Preview</b></summary> <img src="https://i.imgur.com/9kZDBdG.gif" width="900"> </details>

• Todo application, based on the world-famous TodoMVC - demo and source

  <details><summary><b>Preview</b></summary> <img src="https://i.imgur.com/TQKTkf2.gif" width="600"> </details>

Every app perfectly works offline.

Setup

A basic Swarm server implementation is available as a docker image. First, please make sure that you have the docker installed. Then, run the container:

$ docker run -d --name swarmdb -p 31415:31415 -v `pwd`:/var/lib/swarm olebedev/swarmdb

Once a Swarm server is listening incoming connections on ws://0.0.0.0:31415, we can initiate connections. But let's suppose that we can't talk RON over raw WebSocket.

Then, let's setup a JavaScript client project:

$ mkdir ./myapp
$ cd ./myapp
$ yarn add @swarm/db graphql-tag

Now we can initialize a client instance and connect it to the running server.

import gql from 'graphql-tag';
import SwarmDB, { LocalStorage } from '@swarm/db';

const swarm = new SwarmDB({
  storage: new LocalStorage(),
  upstream: 'ws://0.0.0.0:31415',
  db: { name: 'default' }
});

// And then subscribe to live data.
const query = gql`
  subscription ChatsList($from: Int = 0, $to: Int = 100, $uid: UUID!) {
    chats @node(id: "chats") {
      version
      length
      list: id @slice(begin: $from, end: $to) {
        title
        picture
        private
      }
    }
    user @node(id: $uid) {
      id
      version
      username
      avatar
    }
  }
`

const variables = { from: 0, to: 20, uid: 'X8Kq%~github' }
swarm.execute({ query, variables }, ({ data, error, off }) => {
  // handle updates 
  // or stop receiving updates via call `off()`
})

CRDT implementations

• [x] LWW, a last-write-wins replicated data type that may host a variety of user-land data types, like: a dictionary, a simple 1D array (no splice, no index shifts). This LWW employs client-side logical timestamps to decide which write wins, on a field-by-field basis. That is similar to e.g. Cassandra LWW.

• [x] Set, fully commutative, ordered, with tombstones set. You can either add or remove an atom.

• [ ] RGA, a replicated growable array.

• [ ] Counter, a positive-negative counter.

API

constructor(options: Options): SwarmDB

Creates a new Swarm instance. Fetches the metadata from the server, saves it locally, sets the replica up (async, see ensure() method).

Parameters:

• options - an object with properties:

  • storage - a storage interface implementation, required. There are tree implementations: in-memory, localStorage(web) and AsyncStorage(ReactNative).
  • upstream - URL string, e.g. 'wss://the.host.com', required
  • db - an object, required
    • name - A database name, required
    • auth - a JWT, makes sense if auth mode enabled at the server, ignored otherwise

Example:

const swarm = new SwarmDB({
  storage: new InMemory(),
  upstream: 'wss://example.com',
  db: { name: 'default' }
});

execute(request: Request, cbk<T>: (Response<T>) => void ): Promise<{ ok: boolean, off: () => void }>

Fetch data by a GraphQL query(see below).

Parameters:

• request - an object with properties:
  • query - GraphQL AST
  • variables - arguments for GraphQL statement
• cbk - a callback function which accepts Response<T> - object with properties:
  • data - T
  • off - a function which cancels a subscription, may be undefined for mutations and static queries
  • error - an error if any

Example:

const query = gql`
subscription GetChats($from: Int!, $to: Int!) { 
  chats @node(id: "chats") { 
    length
    list: id @slice(begin: $from, end: $to) {
      title
    }
  } 
}`

const variables = { from: 0, to: 20 }
swarm.execute({ query, variables }, ({ data, error, off }) => {
  // handle updates 
  // or stop receiving updates via call `off()`
})

close(): Promise<void>

Returns a Promise which will be resolved when the replica is closed(websocket closed and the state is written to the storage).

ensure(): Promise<void>

Returns a Promise which will be resolved when the replica is initialized and ready to use.

open(): void

(Re)Opens websocket connection to a database server.

uuid(): UUID

Returns a new timestamp-based UUID.

uuid.local()

Use local UUIDs for nodes you don't want to send to the server. Example:

const dashboard = swarm.uuid().local() // a local uuid works only for the current instance. 

Note that a local object can reference a shared object, but not the other way round. Please take a look at todo to see at fully working example of using local UUIDs for local state management.

GraphQL

Swarm has a GraphQL API on the client side. Server-client communication employs RON (state, ops, patches).

Swarm offers GraphQL as a schema-less interface to fetch and modify your data, with basic mutations operations for specific CRDTs, and with several additional directives(see below). GraphQL is used to describe(declare) the shape of the requested data tree.

Key features:

• data reactivity (full or partial)
• (sub)documents/(sub)collections as first-class citizens
• every object has a globally-unique Swarm UUID (see teh @node directive)

Basic primitives and conventions

UUID - is a Swarm UUID or a string representation of an UUID.

scalar UUID

Atom - a scalar (a union of scalar types).

union Atom = String | Int | Float | Boolean | UUID

Node - is an interface of CRDT which presented as a key/value object where a value is a union of Atom | Node | [Node]. Internally, each objects JS prototype regardless its type has id, type and version fields. Additionally, the set type has length field, and lww type can also has length in case if all the user-land field keys are numbers(say, index of array). There are no native JavaScript arrays by default, they can be produced by directives(see below).

interface Node {
  id: String!
  type: String!
  version: String!
  length: Int # only for array-like objects
  # ... various of keys and values
}

Payload - is a flat key/value object. Where keys are strings and values are Atoms.

interface Payload {
  # ... various of keys and values
}

Queries