= Mongify

{<img src="https://badges.gitter.im/Join%20Chat.svg" alt="Join the chat at https://gitter.im/anlek/mongify">}[https://gitter.im/anlek/mongify?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge] {<img src="https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg" alt="Reviewed by Hound">}[https://houndci.com]

http://mongify.com

A data translator from sql database to mongoDB.

Supports MySQL, PostgreSQL, SQLite, Oracle, SQLServer, and DB2 (Basically anything ActiveRecord has built-in). However, I've only tested it with MySql and SQLite

=== Tested MongoDB Versions

MongoDB 4.4
MongoDB 8.0

Learn more about MongoDB at: http://www.mongodb.org

== Install

gem install mongify

=== NOTICE

You might have to install active record database gems (you'll know if you get an error message while running mongify command)

== Usage

=== Creating a configuration file

In order for Mongify to do its job, it needs to know where your databases are located.

Building a config file is as simple as this:

sql_connection do adapter "mysql" host "localhost" username "root" password "passw0rd" database "my_database" batch_size 10000 # This is defaulted to 10000 but in case you want to make that smaller (on lower RAM machines) # Uncomment the following line if you get a "String not valid UTF-8" error. # encoding "utf8" end

mongodb_connection do host "localhost" database "my_database" # Uncomment the following line if you get a "String not valid UTF-8" error. # encoding "utf8" end

You can check your configuration by running

mongify check database.config

==== Options Currently the only supported option is for the mongodb_connection.

mongodb_connection :force => true do # Forcing a mongodb_connection will drop the database before processing # ... end

You can omit the mongodb connection until you're ready to process your translation

=== Generating or creating a translation

==== Generating a translation If your database is large and complex, it might be a bit too much work to write the translation file by hand. Mongify's translate command can help with this: mongify translation database.config Or pipe it right into a file by running mongify translation database.config > translation_file.rb

==== Creating a translation Creating a translation is pretty straightforward. It looks something like this:

table "users" do column "id", :key column "first_name", :string column "last_name", :string column "created_at", :datetime column "updated_at", :datetime end

table "posts" do column "id", :key column "title", :string column "owner_id", :integer, :references => :users column "body", :text column "published_at", :datetime column "created_at", :datetime column "updated_at", :datetime end

table "comments", :embed_in => :posts, :on => :post_id do column "id", :key column "body", :text column "post_id", :integer, :references => :posts column "user_id", :integer, :references => :users column "created_at", :datetime column "updated_at", :datetime end

table "preferences", :embed_in => :users, :as => :object do column "id", :key, :as => :string column "user_id", :integer, :references => "users" column "notify_by_email", :boolean end

table "notes", :embed_in => true, :polymorphic => 'notable' do column "id", :key column "user_id", :integer, :references => "users" column "notable_id", :integer column "notable_type", :string column "body", :text column "created_at", :datetime column "updated_at", :datetime end

Save the file as <tt>"translation_file.rb"</tt> and run the command:

mongify process database.config translation_file.rb

=== Commands

Usage: mongify command database_config [database_translation.rb]

Commands: "check" or "ck" >> Checks connection for sql and no_sql databases [configuration_file] "process" or "pr" >> Takes a translation and process it to mongodb [configuration_file, translation_file] "sync" or "sy" >> Takes a translation and process it to mongodb, only syncs (insert/update) new or updated records based on the updated_at column [configuration_file, translation_file] "translation" or "tr" >> Outputs a translation file from a sql connection [configuration_file]

Examples:

mongify translation datbase.config
mongify tr database.config
mongify check database.config
mongify process database.config database_translation.rb
mongify sync database.config database_translation.rb

Common options: -h, --help Show this message -v, --version Show version

== Translation Layout and Options

When dealing with a translation, there are a few options you can set

=== Table

==== Structure

Structure for defining a table is as follows: table "table_name", {options} do # columns go here... end

==== Options

Table Options are as follow: table "table_name" # Does a straight copy of the table table "table_name", :embed_in => 'users' # Embeds table_name into users, assuming a user_id is present in table_name. # This will also assume you want the table embedded as an array.

table "table_name", # Embeds table_name into users, linking it via a owner_id :embed_in => 'users', # This will also assume you want the table embedded as an array. :on => 'owner_id'

table "table_name", # Embeds table_name into users as a one to one relationship :embed_in => 'users', # This also assumes you have a user_id present in table_name :on => 'owner_id', # You can also specify both :on and :as options when embedding :as => 'object' # NOTE: If you rename the owner_id column, make sure you # update the :on to the new column name

table "table_name", :rename_to => 'my_table' # This will allow you to rename the table as it's getting process # Just remember that columns that use :reference need to # reference the new name.

table "table_name", :ignore => true # This will ignore the whole table (like it doesn't exist) # This option is good for tables like: schema_migrations

table "table_name", # This allows you to specify the table as being polymorphic :polymorphic => 'notable', # and provide the name of the polymorphic relationship. :embed_in => true # Setting embed_in => true allows the relationship to be # embedded directly into the parent class. # If you do not embed it, the polymorphic table will be copied in to # MongoDB and the notable_id will be updated to the new BSON::ObjectID

table "table_name" do # A table can take a before_save block that will be called just before_save do |row| # before the row is saved to the no sql database. row.admin = row.delete('permission').to_i > 50 # This gives you the ability to do very powerful things like: end # Moving records around, renaming records, changing values in row based on end # some values! Checkout Mongify::Database::DataRow to learn more

table "users" do # Here is how to set new ID using the old id before_save do |row| row._id = row.delete('pre_mongified_id') end end

table "preferences", :embed_in => "users" do # As of version 0.2, embedded tables with a before_save will take an before_save do |pref_row, user_row, unset_user_row| # extra argument which is the parent row of the embedded table. user_row.email_me = pref_row.delete('email_me') # This gives you the ability to move things from an embedded table row # to the parent row. if pref_row['one_name'] unset_user_row['last_name'] = true # This will delete/unset the last name for the parent row end end end

More documentation can be found at {Mongify::Database::Table}

=== Columns

==== Structure

Structure for defining a column is as follows: column "name", :type, {options} Columns with no type given will be set to <tt>:string</tt>

==== Notes

as of version 0.2: Leaving a column out when defining a table will result in the column being ignored

==== Types

Before we cover the options, you need to know what types of columns are supported: :key # Columns that are primary keys need to be marked as :key type. You can provide an :as if your :key is not an integer column :integer # Will be converted to a integer :float # Will be converted to a float :decimal # Will be converted to a string (due to MongoDB Ruby Drivers not supporting BigDecimal, read details in Mongify::Database::Column under Decimal Storage

Mongify

Install / Use

README