Cursus Database System

CursusDB is a fast open source in-memory document oriented database offering security, persistence, distribution, availability and an SQL like query language(CDQL).

<ul> <li><a href="#features">Features</a></li> <li><a href="#design-drawings">Design Drawings</a></li> <li><a href="#docker">Docker</a></li> <li><a href="#native-clients">Native Clients</a></li> <li><a href="#native-observers">Native Observers</a></li> <li><a href="#prebuild-binaries">Prebuilt Binaries</a></li> <li><a href="#cluster-node-setup">Cluster & Node Building & Initial Setup</a></li> <li><a href="#node-replicating">Node Replicating</a></li> <li><a href="#query-language">Query Language</a> <ul> <li><a href="#ping-cluster">Ping the cluster</a></li> <li><a href="#inserts">Inserts</a></li> <li><a href="#selects">Selects</a></li> <li><a href="#updates">Updates</a></li> <li><a href="#deletes">Deletes</a></li> <li><a href="#pattern-matching">Pattern Matching</a> <ul> <li><a href="#like">LIKE</a></li> <li><a href="#not-like">NOT LIKE</a></li> </ul> </li> <li><a href="#sorting">Sorting</a></li> <li><a href="#counting">Counting</a></li> <li><a href="#uniqueness">Uniqueness</a></li> <li><a href="#delete-key">Deleting a key within documents in a collection</a></li> <li><a href="#operators">Operators</a></li> <li><a href="#conditional-symbols">Conditional Symbols</a></li> <li><a href="#actions">Actions</a></li> <li><a href="#list-collections">List collections</a></li> <li><a href="#deleting-collections">Deleting collections</a></li> <li><a href="#database-users">Database users</a> <ul> <li><a href="#listing-database-users">Listing database users</a></li> <li><a href="#remove-database-users">Removing database users</a></li> </ul> </li> </ul> </li> <li><a href="#logging">Logging</a> <li><a href="#status-codes">Status Codes</a> <ul> <li><a href="#other">Other</a></li> <li><a href="#authentication-authorization">Authentication / Authorization</a></li> <li><a href="#node-cluster">Node / Cluster</a></li> <li><a href="#document-cdql">Document & CDQL</a></li> </ul> </li> <li><a href="#default-ports">Default Ports</a> <li><a href="#reserved-doc-keys">Reserved Document Keys</a> <li><a href="#mysql-v-cursusdb">MySQL v CursusDB Benchmark</a> <li><a href="#cluster-to-node-tls-and-node-to-node-tls">Cluster to Node TLS connectivity & Node to Node replica TLS connectivity</a> <li><a href="#what-is-a-node-observer">What is a Node Observer?</a> <li><a href="#doc-expectation-relation">Document Expectation & Document Relation</a> <li><a href="#live-chat-observer">Live Chat using an Observer</a> <li><a href="#report-issues">Reporting Issues</a> </ul>

📙📙 https://cursusdb.com/documentation

The idea behind CursusDB was to create something exceedingly scalable whilst never really slowing down. Say you have 1 billion documents stored within 1 collection spread across 100 nodes the cluster will query 1 billion documents in the time it takes to query 10 million as the cluster initiates a non insert action on all nodes simultaneously. This is the power of parallel search. The Cursus(cluster) system is searching say in the users collection in multiple sections of the collection simultaneously. A cluster can query thousands of nodes at the same time. Think of main nodes as shards of many or one collection. Each collection locks on insert, update and delete but because of CursusDB's distributed design it's like a concurrent switch board that allows for large amounts of concurrent transactions. A cluster or many clusters take actions, these actions are relayed as requests to 1 or many nodes simultaneously. Consistency and reliability was one of the main goals when designing CursusDB. If you have many cluster's setup through a TCP load balancer you can imagine transactions just don't miss if the system is configured correctly. One more bit! Say you have multiple updates to one document the node will work in order of operation received. The database system is well-designed, heavily tested and very stable; It was designed and developed for my own need's with other projects/companies I have going on; Over the period of design and development it's very much turned into something special. With that I hope you all enjoy CursusDB!

~ Alex Gaetano Padula

Features

Secured cluster and node(s) communication with shared key and BASIC AUTH type implementation and OR TLS
If configured secured node replication sync with TLS using tls-replication config within .curodeconfig
In-memory data during runtime
Parallel search. Searching section of collections within multiple nodes or replicas simultaneously at the same time.
Auto generated $id key for all documents unique across all nodes
Database Users with basic (R, RW) permissions
Cluster node data replication and synchronization specifically for reads
JSON object insert
Unstructured collections
Cluster and client authentication using BASIC AUTH type implementation
Node(s) (insert, update, delete) relay to observers in real time
Node observer automatic reconnect if connection lost
SQL like query language (CDQL - Cursus Document Query Language)
Low-latency
Highly available
Unique k:v across all nodes using exclamation at end of key name email!
Secure by default with shared key and users
Highly configurable
Lightweight core code under 6000 lines of code in total
File logging and automatic log truncation based on log-max-lines config
Automatic reconnect of any lost node or node replica
Automatic node backups if automatic-backup within .curodeconfig is set to true
Automatic node backup clean up if automatic-backup-cleanup within .curodeconfig is set to true.
Automatic node recovery if data is corrupt if automatic-backup configured
Node data(.cdat) and node backups (/backups/.cdat.{unixtime}) are created by taking what's in memory serializing it, encrypting it and compressing it block by block via serialization-encryption(chacha20poly1305)-compression(DEFLATE) on shutdown or backup

There are no databases like MySQL let's say where you can have multiples. A cluster is your database that spreads data across many nodes.

A node keeps track of queries/txns and if something bad happens can re-trigger what hasn't been processed. A node syncs to a .qqueue file every 70 milliseconds this is fixed and cannot be changed. .qqueue files are encrypted.

Docker

https://hub.docker.com/repositories/cursusdb (SOON)

Native Clients

SHELL https://github.com/cursusdb/curush STABLE ✔️
GO https://github.com/cursusdb/cursusdb-go STABLE ✔️
NODE.JS https://github.com/cursusdb/cursusdb-node OR https://www.npmjs.com/package/cursusdb-node STABLE ✔️
PYTHON https://github.com/cursusdb/cursusdb-py STABLE ✔️
JAVA https://github.com/cursusdb/cursusdb-java STABLE ✔️
CSHARP https://github.com/cursusdb/cursusdb-cs & Available on nuget! STABLE ✔️

Native Observers

NODE.JS https://github.com/cursusdb/cursusdb-observer-node OR https://www.npmjs.com/package/cursusdb-observer-node STABLE ✔️
GO https://github.com/cursusdb/cursusdb-observer-go IN PROGRESS 👨‍💻

.. more coming

Prebuilt Binaries

You can find the latest stable release prebuilt binaries at https://cursusdb.com/downloads

Cluster & Node Building & Initial Setup

Getting started with CursusDB is extremely easy! First you must build a cluster and node binary. To do that clone the source and follow below:

You must make sure you have GO installed minimum version 1.21.3, once installed follow below.

git clone git@github.com:cursusdb/cursusdb.git
cd cluster
go build .
cd ..
cd node 
go build .

Now you should have a curode and a cursus binary.

Now with both we first start cursus to setup a database user, .cursusconfig and a shared key which will be used for your node as well. This key is used to authenticate your cluster and nodes also encrypt your data at rest with ChaCha!

So now that we have our credentials setup we have to setup our first node!

We can run a node on the same instance as a cluster for this example. After completion of cluster setup through the initial run you'll get a .cursusconfig which has a few configurations.

nodes: []
host: 0.0.0.0
tls-node: false
tls-cert: ""
tls-key: ""
tls: false
port: 7681
key: QyjlGfs+AMjvqJd/ovUUA1mBZ3yEq72y8xBQw94a96k=
users:
    - YWxleA==:7V8VGHNwVTVC7EktlWS8V3kS/xkLvRg/oODmOeIukDY=
node-reader-size: 2097152
log-max-lines: 1000
join-responses: false
logging: false
timezone: Local
log-query: false
node-read-deadline: 2

nodes - database cluster nodes. i.e an ip/fqdn + port combination (cluster1.example.com:7682)
tls-node - whether the cluster will connect to nodes via tls
tls-cert - path to your tls cert for cluster
tls-key - path to your tls key for cluster
tls - enable or disable tls
port - cluster port
key - encoded shared key
users - array of database users serialized, and encoded.
node-reader-size - the max size of a response from a node
join-responses - join all node responses and limit based on provided n
logging - start logging to file
timezone - Default is Local but format allowed is for example America/Toronto
**l

Cursusdb

Install / Use

README