Darwin - helps to evolve your database schema

Darwin is one of our oldest libraries, which we still use extensively - its origins date back to 2007. It allows easy evolution of the data model for your application. Darwin runs migration SQL scripts in a specific dialect according to clearly defined rules. It probably originated much earlier than the open-source variants of Liquibase or FlywayDB. It is also much simpler, but it represents a reliable "Kalashnikov" for us, which is universal and reliable in its simplicity.

Disclaimer: we took original source code that works for more than 13 years in production and updated it to JDK 8 version and up-to-date Java classes and Spring framework. Architecture, design and tests are mostly originates several years ago.

The library allows you to maintain SQL scripts on classpath, allowing you to create the tables that your application needs, not only for one specific database, but for different ones at once (ie in parallel you can maintain both MySQL and Oracle schema for your universal libraries). Darwin decides which one to apply after starting the application and finding out the database type from the DataSource object.

As you further develop your application, you will soon find that the existing table structure does not suit you and needs to be expanded or changed. To do this, all you have to do is to save the so-called patch files in the Darwin folder, containing in your name the version number of the application for which this structure is needed. The next time you start the application, Darwin will compare the current version of your application (ie the one you just started) with the version it last applied to the database schema. If the application version is newer, it finds all patches between the two versions and applies them to the schema one by one.

At the same time, Darwin pays close attention to the "transactionality" of changes, and even if the database does not support transaction for schema changes (ie it cannot properly roll back DDL SQL statements), it remembers which SQL statements it actually executed as part of the patch and which did not. Thanks to this, it can start its activity even in the middle of a half-applied patch. This is especially handy in the development phase.

The library allows for a modular approach, so it can be instantiated in the application, for example, 20 times for different shared libraries.

Darwin also pays great attention to the traceability of everything he has done with your database. All statements he executed at your instruction are logged in its tables with all necessary information such as the date the patch was discovered in the application, the date the patch was applied to the db, the duration of individual SQL statements, any exception that fell out of the database when much more.

Darwin also includes a Locker tool that allows you to synchronize tasks within a cluster if a shared database exists. Darwin uses this Locker class to apply SQL patches to only one of the nodes when running the same application on multiple nodes in a cluster at the same time, and the other instances of Darwin obediently wait for the migration to complete. However, you can use the tool for other purposes. For example, we use it relatively actively to control the execution of asynchronous tasks not only within the cluster, but also on one instance of the JVM.

For more information see section How to use

Prerequisites

• JDK 1.8+
• Commons Logging (1.2.0+)
• Spring framework (5.3.0+), works also on 4.3.X older version although not compilable due to JUnit tests

Supported databases

• MySQL
• Oracle
• H2

Do you missing one? Fill a pull request!

How to compile

Use standard Maven 3 command:

mvn clean install

How to run tests

Start databases:

docker-compose -f docker/docker-compose.yml up 

Run your tests in IDE or run:

mvn clean test

Help us maintain at least 80% code coverage!

How to use

See separate chapters for details:

• How to integrate to your project
• How to create migration scripts
• How to switch to Darwin with existing database
• How to make programmable scripts
• How to user locker for intra-cluster synchronization

Implementation notes:

• Versioning rules
• Patch content rules and escaping
• Infrastructural tables