swaddle

[swaddle] wraps up your newly built programs (with 'swaddling') as ready-to-go signed releases: archives, packages, package repositories and even release note websites, using simple data files stored in source control laid out as you would have them in a package. It's MIT-licensed, making it friendly to adopt. And it's in pure shellscript: there's no need for a working Ruby or Python or Perl installation. You can even use it directly from git, and it'll bootstrap itself. Oh, and it eats its own dog food. Those releases on GitHub were made by swaddle.

The Final Step

[swaddle] is your final step after build and test:-

• It creates whatever packages you want, with the best possible settings
  • Deb
  • RPM
  • tarball
  • zip
  • 7z
  • etc
• It creates debian repositories with
  • complete file contents for apt-file
  • translations
  • package pools
  • components
  • priorities
  • InRelease and Release.gpg for maximal compatibility
  • and can even import packages from other sources, so you need never use reprepro or anything else again
    • you could even use it just for that alone
  • oh, and they're versioned, too
• It tags, using a signature, your source and binaries
  • Works with gpg-agent for silent signing packages and repositories
  • Except for rpm signing, which is so badly coded that it can't use gpg-agent (workarounds welcomed)
• Automatically uploads your GPG key to a keyserver after releasing, so everyone can check it's your release
• It can create a GitHub release
  • with all files attached
  • with lots of sensible default copy
  • with links to standalone websites and versioned, signed package repositories
  • with automatic installation scripts included in your release notes tested for all major distributions
• It can create a standalone website for your packages on GitHub, referenced in the release notes
• It can push and tag, using a signature, a versioned set of repositories to GitHub pages
  • so you can always rollback
  • and never deletes older released files

All this seems complex. It isn't. Take a look at the swaddling for [swaddle] itself. Not much to see, is there?

Who Uses it?

• ReleaseQueue have integrated it into their upcoming product management platform.
• bish-bosh, a pure shellscript MQTT 3.1.1 client
• And, of course, we use it to release itself. Eating our own dog food and all that. See [shellfire]'s build script, which shows how to go from a bunch of stuff in source to a complete automated release, just by typing ./build.

Differences with everything else

In many ways, we see this as the logically conclusion of [fpm]. It is to [fpm] what [fpm] was to RPM. Now, if only yum would die…

• it completely separates build from packaging (unlike, say, dh-*)
• source control is king: package files are just stored in source control wherever possible
• it is data-driven rather than script-driven
• all configuration data is just shell-like text files - not 200 shell scripts
• it produces valid, thorough Debian packages which use every ounce of cleverness Debian's packaging system provides
• it provides a one stop shop for creating signed, valid and versioned repositories with yours and third party packages, and even server config packages, which you can even roll back
• and, unlike everything else, it doesn't need C, Python, Perl or Ruby. It's pure shell script, built using [shellfire]*

* Yep, that's right. No need to have the dpkg-* or yum-utils tools installed. It'll even run on the Mac with Homebrew. The only downside is you'll need rpmbuild, because, RPM, being ~~a brilliant format~~ is ~~unusable~~ unimplementable with anything else. At least you won't have to write any more spec files, though.

How to use it

For example, image you have the [shellfire] application 'overdrive'. You have a git repository 'overdrive' (perhaps at GitHub), containing the following structure:-

overdrive\
	.git\
	README.md
	COPYRIGHT
	overdrive           # your shellfire application script
	swaddling\

Inside swaddling, you'll create a configuration. For example, to create a tarball, debian package and RPM, with apt and yum repositories, we might do:-

    swaddling\
	    swaddling.conf       # Essential configuration
		swaddling.conf.d\    # Any files .conf are loaded after swaddling.conf, a la Debian run-parts.
		                     # This is true for any .conf file (eg package.conf, deb.conf, etc, below) in swaddle
							 # Use it to have localized bits of sensitive configuration external to source control
        overdrive\           # name of your 'swaddle'. Usually the same as your GitHub repo name.
                             # You can have many of these (eg for multiple packages, etc) but most people need just one.
	        package.conf     # Configuration settings for all package kinds (tarball, debian, etc) built for this swaddle
			skeleton\        # Put files that never change and aren't built in here
			    any\         # For any architecture
				    etc\
					    overdrive.conf
				all\         # For packages without an architecture (Debian's 'all', RPM's 'noarch')
				amd64\       # For amd64 (and other architectures, as appropriate) - we use the Debian names (as these 
							 # are highly consistent), and convert as necessary for RPM
			body\            # Identical structure to skeleton\, but intended for files that are build outputs (so you can `.gitignore` it; often symlinked to your build folder).
			    …
            deb\             # Create this, and you're making debian packages
			    deb.conf     # Debian specific settings, if any; entirely optional
			    scripts\     # Package scripts folder
			       preinst\  # Folder containing pre installation script snippets
			       postinst\ # Folder containing post installation script snippets
			       prerm\    # Folder containing pre removal script snippets
			       postrm\   # Folder containing post removal script snippets
			    skeleton\    # As above. Merged using rsync. Allows per-package-kind, per-architecture-variant file differences
			    body\        # As above. Merged using rsync.
			        …
            rpm\             # Create this, and you're making RPMs
			    rpm.conf     # RPM specific settings, if any; entirely optional
				skeleton\    # As above.
			    …
            tar\
			    tar.conf     # Same again
			    …

If an architecture folder exists, say amd64, then a amd64 variant of a package is made. If it only exists, at, say, the level of deb, then it won't be made for a RPM or tarball. It is not allowed to have both all and another architecture (indeed, it makes no sense at all for Deb and RPM packages). So in the above example, we shouldn't have either all or amd64.

Surprisingly, there's actually very little to put in our conf files at this time. For example, the most complex is probably swaddling.conf. We might have:-

configure swaddle host_base_url 'https://raphaelcohn.github.io/swaddle/download'
configure swaddle maintainer_name 'Raphael Cohn'
configure swaddle maintainer_comment 'Package Signing Key'
configure swaddle maintainer_email 'raphael.cohn@stormmq.com'
configure swaddle vendor stormmq

Now it's possible we might not want those values to be used the same for every package. That's quite possible. A conf file deeper in the hierarchy, overrides one above it for that part. For example, we could change the vendor above for Deb overdrive packages by putting this into overdrive/deb.conf:-

configure swaddle vendor 'someone else'

Of course, using this couldn't be easier:-

swaddle --swaddling-path /path/to/swaddling --output-path /path/to/output -- overdrive

And off we go!

~~Education, Education, Education~~ Configuration, Configuration, Configuration

The key to [swaddle] is configuration. In [swaddle], there are configuration namespaces. Each namespace is useful at a different level in the hierarchy above. Some are global; some are only useful, for, say, a deb. Configuration uses the file system layout, as well, to be useful. All are designed to be source control friendly. Indeed, [swaddle] works best when used with git and especially GitHub.

Jargon Guide

We've tried to keep this as simple as possible.

|Name|Meaning| |----|-------| |swaddling|A folder containing 'swaddles'. Typically directly below your top-level directory| |swaddle|All the stuff needed to make wrap up code into packages. The name of your swaddle will be used as the name of your packages and other outputs. A folder below swaddling| |README.md|A file in your top-level directory, usually. Used to create man pages and READMEs in your packages if possible| |COPYRIGHT|A file containing both copyright and licensing details in Debian format. See ours. Used to automatically feed license and copyright details into your packages|

Configuration files

All .conf files are actually shell script running inside our process, so you can (although probably shouldn't) do simple code in them. If you really, really wanted to, you could even replace our logic. Not a great idea, and not one we'd support, but we wouldn't actively try to stop you, either. Handy, nonetheless. This is a dev tool, so don't run it on configuration you don't trust, ok?

For every file like NAME.conf, there is an optional folder NAME.conf.d which can contain snippets of code. They must end .conf. These are sourced after the master NAME.conf. This folder doesn't have to exist - nor does NAME.conf. One or the other or both is allowed. Exploit this to avoid storing sensitive configuration details in source contol without sacrificing it