DPP: Deep Persistent Proxy Objects for JavaScript

Rob Tweed rtweed@mgateway.com
19 May 2023, MGateway Ltd https://www.mgateway.com

Twitter: @rtweed

Google Group for discussions, support, advice etc: http://groups.google.co.uk/group/enterprise-web-developer-community

Thanks to @mpen for his original deep proxy logic on which this module depends.

What is DPP?

Deep Persistent Proxy (DPP) is a JavaScript module that allows you to create and maintain JavaScript objects/JSON that will persist automatically between browser sessions.

Next time you start a browser session, any persistent objects you define will be restored to their previous state automatically.

Quick Example

The following creates a persistent object named myObj which adds to its content every time you reload the page:

    const {createDPP} = await import('https://cdn.jsdelivr.net/gh/robtweed/DPP/src/dpp_browser.min.js');
    let dpp = await createDPP({storeName: 'demo'});
    let myObj = await dpp.start();

    if (!myObj.counter) myObj.counter = 0;
    if (!myObj.arr) myObj.arr = [];

    myObj.counter++;

    myObj.arr.push({
      time: Date.now()
    });

    console.log('myObj: ' + JSON.stringify(myObj));

Output on each page load/reload

    myObj: {"counter":1,"arr":[{"time":1691913947097}]}

    myObj: {"arr":[{"time":1691913947097},{"time":1691913998671}],"counter":2}

    myObj: {"arr":[{"time":1691913947097},{"time":1691913998671}],{"time":1691914072961}],"counter":3}

    etc...

How does DPP Manage to Persist JavaScript Objects?

DPP uses the browser's built-in indexedDB database to maintain a persistent image of your JavaScript object(s).

DPP also makes use of JavaScript Proxy Objects, allowing changes to your object(s) to be trapped and recorded to indexedDB.

Because indexedDB's APIs are asynchronous, a key challenge was how to keep the database copy of the object in synchronisation with the local proxy object, particularly if rapid sequences of sets and deletes occur.

The solution is provided by the unique characteristics of our queue-based QOper8 WebWorker management module.

DPP establishes a single QOper8 worker process, and any changes to the local object are added by DPP to the Qoper8 queue. By using a single persistent QOper8 WebWorker process, the corresponding changes to the indexedDB copy of the object can be guaranteed to be handled in strict chronological sequence within the WebWorker.

In summary, DPP decouples your local object from the indexedDB copy. The indexedDB copy is only used by the WebWorker. All you have to do is manipulate your local object and let DPP do the rest!

Do you need to understand the indexedDB API or WebWorkers in order to use DPP?

No. DPP implements all the necessary indexedDB, Qoper8 and WebWorker mechanics for you behind the scenes.

You just use a couple of simple DPP APIs to specify a persistent object, and DPP looks after everything else for you automatically.

When using a DPP-defined Persistent Object, you are accessing its Proxy Object and not the indexedDB database. The indexedDB database image of it is maintained automatically behind the scenes for you within its own WebWorker process.

The only time the indexedDB version is directly used by DPP is to automatically restore the contents of your Persistent Object(s) whenever you start a new browser session.

Will DPP Work on All Browsers?

DPP should work on all modern browsers, but not on older ones. It is usable on any browser that supports all the following:

• indexedDB
• Proxy Objects
• async/await
• ES6 Modules
• WebWorkers

Try It Out

Basic Demo

Try out this live example, running directly from the code you'll find in the /examples folder of this repo.

Each time you load/reload the page, a persistent object named myObj is extended.

It includes a button that clears down the object and reloads the page again.

Note that you can also view the persistent version of this object and and manually clear it down in the indexedDB database: use the browser's Developer Tools.

ToDo Application Demo

If you're wanting to see DPP in use in a more realistic application environment, try running this live ToDo application.

This demonstrates the use of DPP to provide both persistent state data and reactive application control.

NOTE: You must use a modern browser that supports WebComponents to run this example!

For details about and source code for this application, see the Golgi /examples/todo folder.

Installing

DPP is designed so that it can be used whether you build your front-end code manually, or build it using Node.js and WebPack. The way you install and use DPP varies depending on which approach you use.

Using a CDN Copy

You can use DPP directly from the Github CDN linked to this repository. In your main browser application module, load it using:

  const {createDPP} = await import('https://cdn.jsdelivr.net/gh/robtweed/DPP/src/dpp_browser.min.js');

Using a Local Copy

Alternatively, clone or copy the folder /src to an appropriate directory on your web server (eg /var/www/dpp).

You should now have the following files in your web server folder:

    - dpp.js
    - dpp.min.js
    - dpp_browser.js
    - dpp_browser.min.js
    - dpp_node.js
    - idb_handlers
      - delete.js
      - instantiate.js
      - put.js
      - restore.js

You can now load DPP directly from your Web Server, eg:

  const {createDPP} = await import('/dpp/dpp_browser.min.js');

Note that if you're going to use local copies of both DPP and its dependent QOper8 modules, you can bypass this step and just use the DPP class modules itself:

  const {DPP} = await import('/dpp/dpp.min.js');

See below on how to use this latter approach.

From NPM

DPP is available on NPM:

    npm install dpp-db

You can now load DPP using:

    import {createDPP} from "dpp-db/node";

Starting DPP

Using the CDN Version

You need to decide on a store name: this is the name of the store that will be used by indexedDB to maintain your object. The name is entirely for you to decide, eg

    let storeName = 'myObjCopy';

Optionally you can specify an indexedDB database name. By default, DPP will apply a database name of "DPP". Otherwise we can do the following:

    let idb_name = 'MY-DB';

Now create an instance of DPP using these as inputs:

    const {createDPP} = await import('https://cdn.jsdelivr.net/gh/robtweed/DPP/src/dpp_browser.min.js');

    let dpp = await createDPP({
      storeName: storeName,
      idb_name: idb_name
    });

You can now start DPP and attach your local object to its indexedDB copy:

    let myObj = await dpp.start();

You're ready to start using your object!

Note that this approach will use CDN versions of QOper8's resources.

Using a Local Copy of DPP, but a CDN Copy of QOper8

You can use DPP with your own local copy of DPP, but allow it to use a CDN copy of QOper8 behind the scenes.

Installing a local copy of DPP is described earlier above.

You need to decide on a store name: this is the name of the store that will be used by indexedDB to maintain your object. The name is entirely for you to decide, eg:

    let storeName = 'myObjCopy';

Optionally you can specify an indexedDB database name. By default, DPP will apply a database name of "DPP". Otherwise we can do the following:

    let idb_name = 'MY-DB';

Now create an instance of DPP using these as inputs:

    const {createDPP} = await import('/dpp/dpp_browser.min.js');
    const {DPP} = await import('/dpp/dpp.min.js');

    let dpp = await createDPP({
      storeName: storeName,
      idb_name: idb_name,
      DPP: DPP
    });

You can now start DPP and attach your local object to its indexedDB copy:

    let myObj = await dpp.start();

You're ready to start using your object!

Using a Local Copies of DPP and QOper8

if you're using a local copy of DPP, it's probably most sensible to also use a local copy of QOper8.

Installing a local copy of DPP is described earlier above.

Installing a local copy of Qoper8 is very similar: copy/clone it from its Github repository to your own web server. For example, let's say you installed it in the folder /var/www/qoper8 on your Web Server.

You need to decide on a store name: this is the name of the store that will be used by indexedDB to maintain your object. The name is entirely for you to decide, eg:

    let storeName = 'myObjCopy';

Optionally you can specify an indexedDB database name. By default, DPP will apply a database name of "DPP". Otherwise we can do the following:

    let idb_name = 'MY-DB';

You can now just use the DPP module itself and create an instance of DPP using its constructor directly:

    const {DPP} = await import('/dpp/dpp.min.js');
    const {QOper8} = await import('/qoper8/QOper8.min.js');

    let dpp = new DPP({
      storeName: storeName,
      idb_name: idb_name,
      QOper8: QOper8
    });

You can now start DPP and attach your local object to its indexedDB copy:

    let myObj = await dpp.start();

You're ready to start using your object!

Using DPP Installed Using NPM

If you're using a bundler such as WebPack to create your front-end code, you'll need to use the NPM/Node.js-based approach described below.

If you installed DPP using NPM (see above), you use it as follows:

You