nimph

nim package hierarchy manager from the future

or: How I Learned to Stop Worrying and Love the Search Path

Features

• truly path-agnostic dependencies
• native git integration for speed
• github api integration for comfort
• reproducible builds via lockfiles
• immutable cloud-based distributions
• wildcard, tilde, and caret semver
• absolutely zero configuration
• total interoperability with Nimble
• full-featured choosenim replacement

Usage

You can run nimph from anywhere in your project tree; it will simply search upwards until it finds a .nimble file and act as if you ran it there.

Most operations do require that you be within a project, but nimph is flexible enough to operate on local dependencies, global packages, and anything in-between. You can run it on any package, anywhere, and it will provide useful output (and optional repair) of the environment it finds itself in.

• Searching for New Nim Packages
• Adding Packages to the Environment
• Checking the Environment for Errors
• Quickly Forking an Installed Package
• Finding a Path via Nim Import Name
• Locking the Dependency Tree by Name
• Specifying Arbitrary Package Versions
• Upgrading Dependencies Automatically
• Downgrading Dependencies Automatically
• Cutting New Release Versions+Tags
• Adding Any Missing Tags Automatically
• Running Commands on All Dependencies
• Outputting the Dependency Graph
• Git Subcommand Auto-Integration
• Nimble Subcommand Auto-Integration
• Tweaking Nimph Behavior Constants
• Using choosenim to Select Nim Toolchains
• Nimph Module Documentation

Demonstration

This is a demo screencast of using Nimph to setup a project for development. Starting with nothing more than the project's repository, we'll...

1. show the bot.nimble that specifies varied dependencies
2. show the nim.cfg that specifies compilation options
3. edit the nim.cfg to configure a directory to hold local dependencies
4. create a deps directory to hold those packages
5. run nimph to evaluate the state of the environment -- verdict: 😦
6. run nimph doctor to converge the environment to our specifications
7. run nimph to confirm the environment state -- verdict: 😊
8. show the nim.cfg to reveal any changes made by nimph doctor

Installation

A bootstrap-nonimble.sh script is provided which retrieves the dependencies and builds Nimph without requiring nimble.

Windows

I no longer test Windows via the CI because I have no way to debug it. That said, Windows builds may work just fine for you using the older bootstrap.ps1 which relies upon nimble to install dependencies.

OS X

I no longer test OS X via the CI because I cannot be bothered to debug libgit2/libssh behavior there. The tests for nim-1.2 do pass, however.

GitHub Integration

You may want to create a new GitHub personal access token here and then add it to your environment as NIMPH_TOKEN or GITHUB_TOKEN.

If you skip this step, Nimph will try to use a Nimble token for searches, and it will also try to read any hub or ghi credentials. Notably, the fork subcommand will not work without adequate scope authorization.

Subcommand Usage

Search

The search subcommand is used to query GitHub for packages. Arguments should match GitHub search syntax for repositories and for convenience, a language:nim qualifier will be included.

Results are output in increasing order of relevance to reduce scrolling; the last result is the best.

$ nimph search pegs

https://github.com/GlenHertz/peg                                  pushed 2017-11-19
  645 kb            0 issues        0 stars         0 forks      created 2017-11-18
  PEG version of grep

https://github.com/lguzzon-NIM/simplePEG                          pushed 2019-09-05
   82 kb            0 issues        0 stars         0 forks      created 2017-09-05
  Simple Peg

https://github.com/zevv/npeg                                      pushed 2019-11-27
 9125 kb            2 issues       66 stars         2 forks      created 2019-03-08
  PEGs for Nim, another take

Clone

The clone subcommand performs git clones to add packages to your environment. Pass this subcommand some GitHub search syntax and it will download the best matching package, or you can supply a URL directly. Local URLs are fine, too.

Where the package ends up is a function of your existing compiler settings as recorded in relevant nim.cfg files; we'll search all --nimblePath statements, but according to a convention also adopted by Nimble...

The last specified --nimblePath, as processed by the nim.cfg files, is the "default" for the purposes of new package additions.

$ nimph clone npeg
👭cloning git://github.com/zevv/npeg.git...
👌cloned git://github.com/zevv/npeg.git

Doctor

The interesting action happens in the doctor subcommand. When run without any arguments, nimph effectively runs the doctor with a --dry-run option, to perform non-destructive evaluation of your environment and report any issues. In this mode, logging is elevated to report package versions and a summary of their last commit or tag.

$ nimph
✔️  8a7114          bot   cleanups
✔️  775047      swayipc   we can remove this notice now
✔️  v0.4.5         nesm   Version 0.4.5
✔️  5186f4       cligen   Add a test program and update release notes as per last commit to fix https://github.com/c-blake/cligen/issues/120
✔️  c7ba0f         dbus   Merge pull request #3 from SolitudeSF/case
✔️  57f244        c2nim   new option: annotate procs with `{.noconv.}`
✔️  54ed41         npeg   Added section about non-consuming operators and captures to the README. Fixes #17
✔️  183eaa    unittest2   remove redundant import
✔️  v0.3.0          irc   v0.3.0
✔️  fe276f         rest   add generated docs
✔️  5d72a4      foreach   clarify example
✔️  5493b2           xs   add some docs about google
✔️   1.0.1      cutelog   ladybug easier to see
✔️  9d75fe         bump   update docs
✔️   1.0.2       github   fix nimble again
✔️  6830ae        nimph   add asciinema demo
✔️  b6b8d5     compiler   [backport] always set `fileInfoIdx.isKnownFile` (#12773)
✔️  v0.3.3     nimterop   v0.3.3
✔️ v0.13.0        regex   bump 0.13.0 (#52)
✔️  2afc38    unicodedb   improve decomposition performance (#11)
✔️  v0.5.1  unicodeplus   Fix ascii range (#2)
✔️  v0.1.1      nimgit2   v0.1.1
✔️  v0.5.0    parsetoml   Update to version 0.5.0
👌bot version 0.0.11 lookin' good

When run as nimph doctor, any problems discovered will be fixed, if possible. This includes cloning missing packages for which we can determine a URL, adjusting path settings in the project's nim.cfg, and similar housekeeping.

$ nimph doctor
👌bot version 0.0.11 lookin' good

Fork

The fork subcommand is used to fork an installed dependency in your GitHub account and add a new git origin remote pointing at your new fork. The original origin remote is renamed to upstream by default. These constants may be easily changed; see Hacking below.

This allows you to quickly move from merely testing a package to improving it and sharing your work upstream.

$ nimph fork npeg
🍴forking npeg-#54ed418e80f1e1b14133ed383b9c585b320a66cf
🔱https://github.com/disruptek/npeg

Path

The path subcommand is used to retrieve the filesystem path to a package given the Nim symbol you might use to import it. For consistency, the package must be installed.

In contrast to Nimble, you can specify multiple symbols to search for, and the symbols are matched without regard to underscores or capitalization.

$ nimph path nimterop irc
/home/adavidoff/git/bot/deps/pkgs/nimterop-#v0.3.3
/home/adavidoff/git/bot/deps/pkgs/irc-#v0.3.0

If you want to limit your search to packages that are part of your project's dependency tree, add the --strict switch:

$ nimph path coco
/home/adavidoff/git/nimph/deps/pkgs/coco-#head

$ nimph path --strict coco
couldn't find a dependency importable as `coco`

It's useful to create a shell function to jump into dependency directories so you can quickly hack at them.

#!/bin/bash
function goto { pushd `nimph path $1`; }

or

#!/bin/fish
function goto; pushd (nimph path $argv); end

L