@rxdi/xmigrate

Migration library for Mongodb and Mongoose written in TypeScript

Features

• Simple UI/UX
• Rollback support
• Templates for migrations
• TypeScript/JavaScript compatible
• async/await configuration loader
• Mongoose and Mongodb compatibility
• ACID transactions provided by MongoDB
• error and success logs for up/down migrations
• Infinite error log with append NodeJS streaming technique
• 100% TypeScript support with JIT compilation provided by esbuild

Installation

Using binary

wget https://github.com/rxdi/xmigrate/raw/master/dist/xmigrate-linux

Give it permission to execute

chmod +x xmigrate-linux

./xmigrate up|down|create|status

Using NodeJS

npm i -g @rxdi/xmigrate

Configuration

Automatic configuration

xmigrate init

Manual configuration

You can create a xmigrate.js file where you execute the xmigrate command:

import { MongoClient } from 'mongodb';
import { connect } from 'mongoose';

export default async () => {
  return {
    changelogCollectionName: 'migrations',
    migrationsDir: './migrations',
    defaultTemplate: 'es6',
    typescript: true,
    outDir: './.xmigrate',
    /* Custom datetime formatting can be applied like so */
    // dateTimeFormat: () => new Date().toISOString(),
    /* If you do need some better bundling of your migrations when there are tsconfig paths namespaces @shared/my-namespace
        You should consider using `bundler.build()` configuration.
    */
    // bundler: {
    //   build(entryPoints: string[], outdir: string) {
    //     return esbuild.build({
    //       entryPoints,
    //       bundle: true,
    //       sourcemap: false,
    //       minify: false,
    //       platform: 'node',
    //       format: 'cjs',
    //       outdir,
    //       logLevel: 'info',
    //       plugins: [pluginTsc()],
    //     })
    //   },
    // },
    logger: {
      folder: './migrations-log',
      up: {
        success: 'up.success.log',
        error: 'up.error.log',
      },
      down: {
        success: 'down.success.log',
        error: 'down.error.log',
      },
    },
    database: {
      async connect() {
        const url = 'mongodb://localhost:27017';

        await connect(url);
        const client = await MongoClient.connect(url);
        return client;
      },
    },
  };
};

Commands

First time run

xmigrate init

Creating migration

xmigrate create "my-migration"

Creating migration with template.

Templates to choose: es5, es6, native, typescript. By default xmigrate create "my-migration" executes with es6 template

xmigrate create "my-migration" --template (es5|es6|native|typescript)

xmigrate create "my-migration" --template typescript

Up migrations

Will execute all migrations which have the status PENDING

xmigrate up

Up migrations with rollback down to current errored migration

xmigrate up --rollback

Down migrations

Will execute migrations one by one starting from the last created by timestamp

xmigrate down

Status of migrations

xmigrate status

Will print inside the console a table. When there is a PENDING flag these migrations were not running against the current database.

🖥️  Database: test

💿  DBCollection: migrations

🗄️  LoggerDir: ./migrations-log

📁  MigrationsDir: migrations

👷  Script: xmigrate status

┌─────────┬───────────────────────────┬────────────────────────────┐
│ (index) │         fileName          │         appliedAt          │
├─────────┼───────────────────────────┼────────────────────────────┤
│    0    │ '20190725160010-pesho.js' │ '2019-07-25T16:07:27.012Z' │
│    1    │ '20190725160011-pesho.js' │         'PENDING'          │
│    2    │ '20190725160012-pesho.js' │         'PENDING'          │
│    3    │ '20190725160013-pesho.js' │         'PENDING'          │
└─────────┴───────────────────────────┴────────────────────────────┘

🔥  There are 3 migrations with status 'PENDING', run 'xmigrate up' command!

Migration templates

Native mongo driver template

module.exports = {

  async prepare(client) {
    return [client]
  }

  async up([client]) {
    await client
      .db()
      .collection('albums')
      .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: true } });
    await client
      .collection('albums')
      .updateOne({ artist: 'The Doors' }, { $set: { stars: 5 } });
  },

  async down([client]) {
    await client
      .db()
      .collection('albums')
      .updateOne({ artist: 'The Doors' }, { $set: { stars: 0 } });
    await client
      .collection('albums')
      .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: false } });
  },
};

ES5 template

module.exports = {

  async prepare(client) {
    return [client]
  }

  async up([client]) {
    return ['UP'];
  },

  async down([client]) {
    return ['DOWN'];
  },
};

ES6 template

export async function prepare(client) {
  return [client];
}
export async function up([client]) {
  return ['Up'];
}
export async function down([client]) {
  return ['Down'];
}

Typescript template

(Optional) type definitions for mongodb and mongoose

npm install @types/mongodb @types/mongoose -D

import { MongoClient } from 'mongodb';

export async function prepare(client: mongoClient) {
  return [client];
}

export async function up([client]: [MongoClient]) {
  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: true } });

  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Doors' }, { $set: { stars: 5 } });
}

export async function down([client]: [MongoClient]) {
  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Doors' }, { $set: { stars: 0 } });

  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: false } });
}

TypeScript migrations (Deprecated use option builder: 'ESBUILD' in your xmigrate config file)

To be able to run migrations with TypeScript you need to set typescript: true inside xmigrate.js and install @gapi/cli globally

Install @gapi/cli for runtime build using glob

npm i -g @gapi/cli

Command that will be run internally

npx gapi build --glob ./1-migration.ts,./2-migration.ts

After success transpiled migration will be started from ./.xmigrate/1-migration.js Before exit script will remove artifacts left from transpilation located inside ./.xmigrate folder

Rollback

When executing command xmigrate up --rollback this will trigger immediate rollback to DOWN migration on the current crashed migration The log will look something like this:

🖥️  Database: test

💿  DBCollection: migrations

🗄️  LoggerDir: ./migrations-log

📁  MigrationsDir: migrations

👷  Script: xmigrate up


🔥  Status: Operation executed with error
🧨  Error: {"fileName":"20190725160011-pesho.js","migrated":[]}
📨  Message: AAA


🙏  Status: Executing rollback operation xmigrate down
📁  Migration: /migrations/20190725160011-pesho.js


🚀  Rollback operation success, nothing changed if written correctly!

Logs

By default logs will append streaming content for every interaction made by migration

Down migration success Log

🚀 ********* Thu Jul 25 2019 11:23:06 GMT+0300 (Eastern European Summer Time) *********

{
  "fileName": "20190723165157-example.js",
  "appliedAt": "2019-07-25T08:23:06.668Z",
  "result": [
    {
      "result": "DOWN Executed"
    }
  ]
}

Down migration error log

🔥 ********* Thu Jul 25 2019 03:28:48 GMT+0300 (Eastern European Summer Time) *********

{
  "downgraded": [],
  "errorMessage": "AAA",
  "fileName": "20190724235527-pesho.js"
}

Up migration success log

🚀 ********* Thu Jul 25 2019 11:23:24 GMT+0300 (Eastern European Summer Time) *********

{
  "fileName": "20190723165157-example.js",
  "appliedAt": "2019-07-25T08:23:24.642Z",
  "result": [
    {
      "result": "UP Executed"
    }
  ]
}

Up migration error log

🔥 ********* Thu Jul 25 2019 03:39:00 GMT+0300 (Eastern European Summer Time) *********

{
  "migrated": [],
  "errorMessage": "AAA",
  "fileName": "20190724235545-pesho.js"
}

TypeScript configuration

When you change your configuration file to xmigrate.ts it will automatically transpile to ES5 and will be loaded

import { Config } from '@rxdi/xmigrate';
import { MongoClient } from 'mongodb';
import { connect } from 'mongoose';

export default async (): Promise<Config> => {
  return {
    changelogCollectionName: 'migrations',
    migrationsDir: './migrations',
    defaultTemplate: 'typescript',
    typescript: true,
    outDir: './.xmigrate',
    // bundler: {
    //   build(entryPoints: string[], outdir: string) {
    //     return esbuild.build({
    //       entryPoints,
    //       bundle: true,
    //       sourcemap: false,
    //       minify: false,
    //       platform: 'node',
    //       format: 'cjs',
    //       outdir,
    //       logLevel: 'info',
    //       plugins: [pluginTsc()],
    //     });
    //   },
    // },
    logger: {
      folder: './migrations-log',
      up: {
        success: 'up.success.log',
        error: 'up.error.log',
      },
      down: {
        success: 'down.success.log',
        error: 'down.error.log',
      },
    },
    database: {
      async connect() {