NestDB

The Node.js Embedded JavaScript Database

Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, written in 100% JavaScript, no binary dependency. Originally forked from NeDB. API is similar to MongoDB's and it's <a href="#speed">plenty fast</a>.

IMPORTANT NOTE: Please don't submit issues for questions regarding your code. Only actual bugs or feature requests will be answered, all others will be closed without comment. Also, please follow the <a href="#bug-reporting-guidelines">bug reporting guidelines</a> and check the <a href="https://github.com/louischatriot/nedb/wiki/Change-log" target="_blank">change log</a> before submitting an already fixed bug :)

Installation

Node.js version via NPM

$ npm install nestdb --save

Node.js version via Yarn

$ yarn add nestdb

Browser version via Bower

$ bower install nestdb --save    # Main files under "./browser-version/out/"

Compatibility with NeDB

NestDB was originally forked from NeDB.

NestDB v2.x will maintain backward compatibility with NeDB v1.x to make for seamless transitions migrating from NeDB to NestDB.

When we eventually release NestDB v3.x, it will not be backward compatible with NeDB v1.x.

API

It is similar to MongoDB's API (the most used operations). The NestDB API is a superset of NeDB's API.

• <a href="#creatingloading-a-datastore">Creating/loading a datastore</a>
• <a href="#persistence">Persistence</a>
• <a href="#inserting-documents">Inserting documents</a>
• <a href="#finding-documents">Finding documents</a>
  • <a href="#basic-querying">Basic Querying</a>
  • <a href="#operators-lt-lte-gt-gte-in-nin-ne-exists-regex">Operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $exists, $regex)</a>
  • <a href="#array-fields">Array fields</a>
  • <a href="#logical-operators-or-and-not-where">Logical operators $or, $and, $not, $where</a>
  • <a href="#sorting-and-paginating">Sorting and paginating</a>
  • <a href="#projections">Projections</a>
• <a href="#counting-documents">Counting documents</a>
• <a href="#updating-documents">Updating documents</a>
• <a href="#removing-documents">Removing documents</a>
• <a href="#indexing">Indexing</a>
• <a href="#destroying">Destroying a datastore</a>
• <a href="#extending-with-plugins">Extending with plugins</a>
• <a href="#using-a-custom-storage-engine">Using a custom storage engine</a>

Creating/loading a datastore

You can use NestDB as an in-memory only datastore or as a persistent datastore. One datastore is the equivalent of a MongoDB collection. The constructor is used as follows new Datastore(options) where options is an object with the following fields:

• filename (optional): path to the file where the data is persisted. If left blank, the datastore is automatically considered in-memory only. It cannot end with a ~ which is used in the temporary files NestDB uses to perform crash-safe writes.
• inMemoryOnly (optional, defaults to false): as the name implies.
• timestampData (optional, defaults to false): timestamp the insertion and last update of all documents, with the fields createdAt and updatedAt. User-specified values override automatic generation, usually useful for testing.
• autoload (optional, defaults to false): if used, the datastore will automatically be loaded from the datafile upon creation (you don't need to call load). Any command issued before load is finished is buffered and will be executed when load is done.
• onload (optional): if you use autoloading, this is the handler called after the load. It takes one error argument. If you use autoloading without specifying this handler, and an error happens during load, an error will be thrown.
• idGenerator (optional): if set, this function will be used for generating IDs. It takes no arguments and should return a unique string.
• afterSerialization (optional): hook you can use to transform data after it was serialized and before it is written to disk. Can be used for example to encrypt data before writing datastore to disk. This function takes a string as parameter (one line of an NestDB data file) and outputs the transformed string, which must absolutely not contain a \n character (or data will be lost).
• beforeDeserialization (optional): inverse of afterSerialization. Make sure to include both and not just one or you risk data loss. For the same reason, make sure both functions are inverses of one another. Some failsafe mechanisms are in place to prevent data loss if you misuse the serialization hooks: NestDB checks that never one is declared without the other, and checks that they are reverse of one another by testing on random strings of various lengths. In addition, if too much data is detected as corrupt, NestDB will refuse to start as it could mean you're not using the deserialization hook corresponding to the serialization hook used before (see below).
• corruptAlertThreshold (optional): between 0 (0%) and 1 (100%), defaults to 0.1 (10%). NestDB will refuse to start if more than this percentage of the datafile is corrupt. 0 means you don't tolerate any corruption, 1 means you don't care.
• compareStrings (optional): function compareStrings(a, b) should compare strings a and b and must return -1, 0 or 1. If specified, it overrides default string comparison (===), which is not well adapted to non-US characters such as accented or diacritical letters. Using the native String.prototype.localeCompare will be the right choice most of the time.
• storage (optional): A custom storage engine for the database files. Must implement at least the handful of methods exported by the standard "storage" module included in NestDB, as detailed below in the Using a custom storage engine section.

If you use a persistent datastore without the autoload option, you need to call load manually. This function fetches the data from datafile and prepares the datastore. Do NOT forget it! If you use a persistent datastore, no command (e.g. insert, find, update, remove) will be executed before load is called, so make sure to either call it yourself or use the autoload option.

Also, if load fails, all commands registered to the executor afterwards will not be executed. They will be registered and executed, in sequence, only after a successful load.

Once the datastore is fully loaded, it also emits a "loaded" event that you can add a listener for.

Whenever any datastore is fully loaded, the Datastore object itself emits a global "created" event that you can add a listener for.

Type 1: In-memory only datastore (no need to load)

var Datastore = require('nestdb')
  , db = new Datastore();
// You can issue commands right away

Type 2: Persistent datastore with manual loading

var Datastore = require('nestdb')
  , db = new Datastore({ filename: 'path/to/datafile' });
db.load(function (err) {    // Callback is optional
  // Now commands will be executed
});

Type 3: Persistent datastore with automatic loading

var Datastore = require('nestdb')
  , db = new Datastore({
          filename: 'path/to/datafile'
        , autoload: true
        , onload: function (err) {
            if (err) {
              console.error('Failed to load the datastore:', err);
            } else {
              console.log('Loaded the datastore!');
            }
          }
         });
// You can issue commands right away because of NestDB's internal queueing

// You can also synchronously add an event listener for the 'loaded' event
// If not added synchronously, you will probably miss the event
// This event will not be emitted if an error occurs during loading
db.once('loaded', function () {
  console.log('Loaded the datastore!');
});

Type 4: Persistent datastore for a Node WebKit app

// For a Node WebKit app called 'nwtest'
// For example on Linux, the datafile will be ~/.config/nwtest/nestdb-data/something.db
var Datastore = require('nestdb')
  , path = require('path')
  , db = new Datastore({ filename: path.join(require('nw.gui').App.dataPath, 'something.db') });

Loading a datastore with an event listener

var Datastore = require('nestdb')
  , db = new Datastore({ filename: 'path/to/datafile', autoload: true });
// You can issue commands right away because of NestDB's internal queueing

// You can also synchronously add an event listener for the 'loaded' event
// If not added synchronously, you will probably miss the event
// This event will not be emitted if an error occurs during loading
db.once('loaded', function () {
  console.log('Loaded the datastore!');
});

Listening for the global "created" event

var Datastore = require('nestdb')
  , db;

Datastore.on('created', function (dbRef) {
  // dbRef === db
  console.log('Created or loaded a datastore: ' + (dbRef.filename || 'in-memory'));
});

db = new Datastore({ filename: 'path/to/datafile', autoload: true });

Loading multiple datastores

// Of course you can create