Shale

Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML. It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures, as well as serialize data structures into JSON, YAML, TOML, CSV or XML.

Documentation with interactive examples is available at Shale website

Features

• Convert JSON, YAML, TOML, CSV and XML to Ruby data model
• Convert Ruby data model to JSON, YAML, TOML, CSV and XML
• Generate JSON and XML Schema from Ruby models
• Compile JSON and XML Schema into Ruby models
• Out of the box support for JSON, YAML, Tomlib, toml-rb, CSV, Nokogiri, REXML and Ox parsers
• Support for custom adapters

Installation

Shale supports Ruby (MRI) 3.0+

Add this line to your application's Gemfile:

gem 'shale'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install shale

Contents

• Simple use case
• Creating objects
• Converting JSON to object
• Converting object to JSON
• Converting YAML to object
• Converting object to YAML
• Converting TOML to object
• Converting object to TOML
• Converting Hash to object
• Converting object to Hash
• Converting XML to object
• Converting object to XML
• Converting CSV to object
• Converting object to CSV
• Converting collections
• Mapping JSON keys to object attributes
• Mapping YAML keys to object attributes
• Mapping TOML keys to object attributes
• Mapping CSV columns to object attributes
• Mapping Hash keys to object attributes
• Mapping XML elements and attributes to object attributes
• Using XML namespaces
• Rendering nil values
• Using methods to extract and generate data
• Delegating fields to child attributes
• Additional options
• Overriding attribute methods
• Using custom models
• Supported types
• Writing your own type
• Adapters
• Generating JSON Schema
• Compiling JSON Schema into Shale model
• Generating XML Schema
• Compiling XML Schema into Shale model

Usage

Simple use case

require 'shale'

class Address < Shale::Mapper
  attribute :city, :string
  attribute :street, :string
  attribute :zip, :string
end

class Person < Shale::Mapper
  attribute :first_name, :string
  attribute :last_name, :string
  attribute :age, :integer
  attribute :married, :boolean, default: -> { false }
  attribute :hobbies, :string, collection: true
  attribute :address, Address
end

• default: -> { 'value' } - add a default value to attribute (it must be a proc that returns value)
• collection: true - indicates that a attribute is a collection

Creating objects

person = Person.new(
  first_name: 'John',
  last_name: 'Doe',
  age: 50,
  hobbies: ['Singing', 'Dancing'],
  address: Address.new(city: 'London', street: 'Oxford Street', zip: 'E1 6AN'),
)

Converting JSON to object

person = Person.from_json(<<~DATA)
{
  "first_name": "John",
  "last_name": "Doe",
  "age": 50,
  "married": false,
  "hobbies": ["Singing", "Dancing"],
  "address": {
    "city": "London",
    "street": "Oxford Street",
    "zip": "E1 6AN"
  }
}
DATA

# =>
#
# #<Person:0x00007f9bc3086d60
#  @address=
#   #<Address:0x00007f9bc3086748
#    @city="London",
#    @street="Oxford Street",
#    @zip="E1 6AN">,
#  @age=50,
#  @first_name="John",
#  @hobbies=["Singing", "Dancing"],
#  @last_name="Doe",
#  @married=false>

Converting object to JSON

person.to_json

# =>
#
# {
#   "first_name": "John",
#   "last_name": "Doe",
#   "age": 50,
#   "married": false,
#   "hobbies": ["Singing", "Dancing"],
#   "address": {
#     "city": "London",
#     "street": "Oxford Street",
#     "zip": "E1 6AN"
#   }
# }

Converting YAML to object

person = Person.from_yaml(<<~DATA)
first_name: John
last_name: Doe
age: 50
married: false
hobbies:
- Singing
- Dancing
address:
  city: London
  street: Oxford Street
  zip: E1 6AN
DATA

Converting object to YAML

person.to_yaml

# =>
#
# ---
# first_name: John
# last_name: Doe
# age: 50
# married: false
# hobbies:
# - Singing
# - Dancing
# address:
#   city: London
#   street: Oxford Street
#   zip: E1 6AN

Converting TOML to object

To use TOML with Shale you have to set adapter you want to use. It comes with adapters for Tomlib and toml-rb. For details see Adapters section.

To set it, first make sure Tomlib gem is installed:

$ gem install tomlib

then setup adapter:

require 'sahle/adapter/tomlib'
Shale.toml_adapter = Shale::Adapter::Tomlib

# Alternatively if you'd like to use toml-rb, use:
require 'shale/adapter/toml_rb'
Shale.toml_adapter = Shale::Adapter::TomlRB

Now you can use TOML with Shale:

person = Person.from_toml(<<~DATA)
first_name = "John"
last_name = "Doe"
age = 50
married = false
hobbies = ["Singing", "Dancing"]
[address]
city = "London"
street = "Oxford Street"
zip = "E1 6AN"
DATA

Converting object to TOML

person.to_toml

# =>
#
# first_name = "John"
# last_name = "Doe"
# age = 50
# married = false
# hobbies = [ "Singing", "Dancing" ]
#
# [address]
# city = "London"
# street = "Oxford Street"
# zip = "E1 6AN"

Converting Hash to object

person = Person.from_hash(
  'first_name' => 'John',
  'last_name' => 'Doe',
  'age' => 50,
  'married' => false,
  'hobbies' => ['Singing', 'Dancing'],
  'address' => {
    'city'=>'London',
    'street'=>'Oxford Street',
    'zip'=>'E1 6AN'
  },
)

Converting object to Hash

person.to_hash

# =>
#
# {
#   "first_name"=>"John",
#   "last_name"=>"Doe",
#   "age"=>50,
#   "married"=>false,
#   "hobbies"=>["Singing", "Dancing"],
#   "address"=>{"city"=>"London", "street"=>"Oxford Street", "zip"=>"E1 6AN"}
# }

Converting XML to object

To use XML with Shale you have to set adapter you want to use. Shale comes with adapters for REXML, Nokogiri and OX parsers. For details see Adapters section.

require 'shale/adapter/rexml'
Shale.xml_adapter = Shale::Adapter::REXML

Now you can use XML with Shale:

person = Person.from_xml(<<~DATA)
<person>
  <first_name>John</first_name>
  <last_name>Doe</last_name>
  <age>50</age>
  <married>false</married>
  <hobbies>Singing</hobbies>
  <hobbies>Dancing</hobbies>
  <address>
    <city>London</city>
    <street>Oxford Street</street>
    <zip>E1 6AN</zip>
  </address>
</person>
DATA

Converting object to XML

person.to_xml

# =>
#
# <person>
#   <first_name>John</first_name>
#   <last_name>Doe</last_name>
#   <age>50</age>
#   <married>false</married>
#   <hobbies>Singing</hobbies>
#   <hobbies>Dancing</hobbies>
#   <address>
#     <city>London</city>
#     <street>Oxford Street</street>
#     <zip>E1 6AN</zip>
#   </address>
# </person>

Converting CSV to object

To use CSV with Shale you have to set adapter. Shale comes with adapter for csv. For details see Adapters section.

To set it, first make sure CSV gem is installed:

$ gem install csv

then setup adapter:

require 'shale/adapter/csv'
Shale.csv_adapter = Shale::Adapter::CSV

Now you can use CSV with Shale.

CSV represents a flat data structure, so you can't map properties to complex types directly, but you can use methods to map properties to complex types (see Using methods to extract and generate data section).

.from_csv method always returns an array of records.

people = Person.from_csv(<<~DATA)
John,Doe,50,false
DATA

Converting object to CSV

people[0].to_csv # or Person.to_csv(people) if you want to convert a collection

# =>
#
# John,Doe,50,false

Converting collections

Shale allows converting collections for formats that support it (JSON, YAML and CSV). To convert Ruby array to JSON:

person1 = Person.new(name: 'John Doe')
person2 = Person.new(name: 'Joe Sixpack')

Person.to_json([person1, person2], pretty: true)
# or Person.to_yaml([person1, person2])
# or Person.to_csv([person1, person2])

# =>
#
# [
#   { "name": "John Doe" },
#   { "name": "Joe Sixpack" }
# ]

To convert JSON array to Ruby:

Person.from_json(<<~JSON)
[
  { "name": "John Doe" },
  { "name": "Joe Sixpack" }
]
JSON

# =>
#
# [
#   #<Person:0x00000001033dbce8 @name="John Doe">,
#   #<Person:0x00000001033db4c8 @name="Joe Sixpack">
# ]

Mapping JSON keys to object attributes

By default keys are named the same as attributes. To use custom keys use:

:warning: Declaring custom mapping removes default mapping for given format!

class Person < Shale::Mapper
  attribute :first_name, :string
  attribute :last_name, :string

  json do
    map 'firstName', to: :first_name
    map 'lastName', to: