<blockquote>Please note, this project is not "dead". It's been feature-complete and stable for years. If you find a bug, file it.</blockquote> <h1><a name=toc> Javascript `Databank`</a></h1>

<a href=#introduction>The adventures of Agnes and Frederick</a>

<a href=#expressions>Expressions</a>
<a href=#browser>KISS syncing</a>
<a href=#autoincrement>AutoIncrement</a>
<a href=#mutation>Restructuring data</a>
<a href="http://9ol.es/ytwatch1">The Project This is Primarily Built For</a>

<a name=toc-inserting href=#inserting>Inserting and Removing</a> records

<a href=#initialization>Initialization</a> a new database
<a href=#transforming>Transforming</a> existing data
<a href=#insert>Insert</a> new records
<a href=#mutator>Mutator</a> functions as values
<a href=#template>Template</a> based insertion
<a href=#update>Update</a> existing records
<a href=#unset>Unset</a> existing keys
<a href=#remove>Remove</a> records and get a copy of them
<a href=#constrain>Constrain</a> insertion by a unique, primary key
<a href=#addif>AddIf</a> and only if something matches a test
<a href=#beforeadd>BeforeAdd</a> to sanitize and cleanup data prior to insertion

<a name=toc-finding href=#finding>Finding</a> and searching for data

<a href=#find>find</a> records through an expressive syntax
<a href=#findFirst>findFirst</a> record through an easy syntax
<a href=#first>first</a> as an alias to findFirst.
<a href=#not>not</a> records that match the condition
<a href=#like>like</a> finds records by substring
<a href=#isin>isin</a> finds whether the record is in a group
<a href=#has>has</a> looks inside a record stored as an array
<a href=#missing>missing</a> records that have keys not defined
<a href=#hasKey>hasKey</a> finds records that have keys defined
<a href=#select>select</a> one or more fields from a result
<a href=#invert>invert</a> gets the unary inverse of a set of results
<a href=#slice>slice</a> the records while maintaining the function chain
<a href=#view>view</a> data easily (or <a href=#lazyView>lazily</a>)

<a name=toc-manipulating href=#manipulating>Manipulating</a> retrieved data

<a href=#each>each</a> or <a href=#each>map</a> results to new values
<a href=#reduceLeft>reduceLeft</a> results to aggregate values
<a href=#reduceRight>reduceRight</a> results to aggregate values
<a href=#order>order</a> or <a href=#order>sort</a> results given some function or expression
<a href=#group>group</a> results by some key
<a href=#keyBy>keyBy</a> a certain key to make a 1-to-1 map
<a href=#distinct>distinct</a> values for a certain key
<a href=#indexBy>indexBy</a> to re-index the database by a sort constraint

<a href=#storage>Storage</a> options to importing and expoting data

<a href=#sync>sync</a> the database when things are modified
<a href=#transaction>transaction</a> to help reduce expensive indexing

<a href=#inspection>Inspect</a> execution and various data

<a href=#trace>trace</a> the database when things are modified

<a href=#example>Examples</a>

<a href=#ex-creation>Creating and Inserting</a>
<a href=#ex-more>More</a>

Miscellaneous non-awesome stuff

<a href=#buzzword>Advanced Usage</a>
<a href=#syntax>Syntax notes</a>
<a href=#support>Supported Platforms</a>
<a href=#dependencies>Dependencies</a>
<a href=#performance>Performance</a>
<a href=#license>License</a>
<a href=#contact>Contact</a>
<a href=#similar>Similar Projects</a>
<a href=#users>Users</a>

<h2><a name=introduction></a>Agnes and Frederick [ <a href=#toc>top</a> ] </h2>

We will follow two groups in our exploration:

Two time travellers from the 1700s
A secret agency of spies that are out to get them.

The time travellers have hacked into the spy's communication systems, but only have a browser to work with. The schema is a mess and they must make sense of it and find out what the spies know in order to escape their wrath.

We start our story shortly after they have discovered the large dataset.

<blockquote> Agnes: Why dearest me, Sir Frederick, this data manipulation dilemma is truly proving to be quite intractible. If only I or one such as me had for our immediate disposal and use **an expressive and flexible** syntax; much akin to SQL - for the use in the Browser or in other Javascript environments: then, we could resolve the issues that are contained within this problem with a great ease - indeed, that of which would be immeasurable and truly beneficial to the cause at hand.</blockquote>

Let's take a familiar everyday SQL query such as:

select spyname, location
  from hitlist
  where 
    target_time < NOW() and
    alive == true
  order by distance desc

Dance around a bit...

from hitlist
  where 
    target_time < NOW() and
    alive == true
  order by distance desc
  select spyname, location

Add some commas, a few parenthesis, lots of black magic, and here we go:

hitlist
  .where(
    'target_time < new Date()',
    'alive == true'
  )
  .order('distance', 'desc')
  .select('spyname', 'location')
  .each(function(row) {
    console.log(
      row.spyname + ", you have moments to live." +
      "They know you are at " + row.location "." +
      "Use rule 37."
    );
  });

Who said life wasn't easy, my dear Frederick?

<h3><a name=expressions>Expressions</a> [ <a href=#toc>top</a> ] </h3> Expressions are the *biggest, most important part here*.

Let's go back to our coders. They have now created a bunch of underscore, jquery, and backbone mess of select, without, uniq, and other weird things to manipulate their data. They are getting nowhere.

One of them says:

<blockquote> Agnes: Sir Frederick, whilst looking at the code, one is apt to imagine that she is perusing some ill-written tale or romance, which instead of natural and agreeable images, exhibits to the mind nothing but frightful and distorted shapes "Gorgons, hydras, and chimeras dire"; discoloring and disfiguring whatever it represents, and transforming everything it touches into a monster.</blockquote>

Let's clean up that mess.

Expressions are a processing engine where you can toss in things and get matching functions.

For instance, say you want to find out what parts of your complex object datastore has a structure like this:

{ 'todo': { 'murder': <string> } }

Agnes and Frederick have found this:

  { 'name': "Agnes",
    'location': 'Starbucks',
    'role': 'target',
    'kill-date': 'today',
    'hitmen' : ['Agent 86']
  },
  { 'name': "Agent 86",
    'role': 'spy',
    'distance': 80000,
    'todo': { 'murder': 'Agnes' }
  },
  { 'name': "Agent 99",
    'role': 'spy',
    'backup-for': ['Agent 86', 'Agent Orange']
  },
  { 'name': "Frederick",
    'role': 'target',
    'location': 'Starbucks',
    'kill-date': 'today',
    'hitmen' : ['Agent 86', 'Agent Orange']
  },
  { 'name': "Agent 007",
    'role': 'spy',
    'todo': { 'sleep-with': 'spy' }
  },
  { 'name': "Agent Orange",
    'distance': 10000,
    'role': 'spy',
    'todo': { 'murder' : 'Frederick' },
  },

We want to find out a few things:

DB.find([
  DB('.todo.murder == "Frederick"),
  DB('.todo.murder == "Agnes")
])

Gets you there, the Array means OR. Now they want to manipulate it further.

DB.find({
  'role': 'target',
  'kill-date': 'today'
}).update({
  'location': 'across town'
});

There's a backup agent, Agent 99, to be used in case the other two fail. Agnes and Frederick want to foil her:

DB.find({
  'backup-for': DB.find(
    DB('.todo.murder.indexOf(["Frederick", "Agnes"]) > -1')
  ).select('name')
).update(function(who) {
  delete who['backup-for'];
  who.todo = 'lie around and sun bathe';
});

They find that there is a lot more to explore, try

DB(some string).toString()

to peek at the implementation. This is how they got started:

DB(".a.b")({a:{c:1})
>> undefined

DB(".a.b")({a:{b:1})
>> 1

DB(".a.b")({b:{b:1})
>> undefined

DB(".a.b")({a:{b:[1,2,3]})
>> [1,2,3]

To debug their expressions. Much to their delight, they found they can use these expressions and data manipulations just about everywhere in this library of black magic.

<h3><a name=browser>KISS syncing in the browser</a>[ <a href=#toc>top</a> ] </h3>

Our heros are now finally getting somewhere. They can bring down their data, and manipulate it with ease.

<blockquote>Frederick: A world of hope is but a few keystrokes away for us Agnes. However, I haven't uncovered a painless way to remove our true information, place in plausibly fraudulant information, and then automatically update the remote database with ease --- surely, there must be a way to trigger a function when our data-bank is manipulated.</blockquote>

Going to the documentation, they find a convenient <a href=#sync>sync</a> function that is designed to do just that. Returning to their laptop:

var what_it_is_that_we_know = DB().sync(function(espionage_dataset) {
  $.put("/government-secrets", espionage_dataset);
});

$.get("/government-secrets", what_it_is_that_we_know);

And it's done. Now Agnes and Frederick can modify stuff in the browser and it automatically does a remote sync. It was 4 lines. That's really all it took.

<h3><a name=autoincrement>AutoIncrement</a>[ <a href=#toc>top</a> ] </h3>

Agnes and Frederick are in the clear for now. However, this isn't to last long

<blockquote>Agnes: Wouldn't it be a wonderful, and I do mean quite a pleasant reality if we had a more organized way of dealing with this immensely distraught set of informa

Db.js

Install / Use

README