locus

locus is library for Erlang/OTP and Elixir that allows you to pinpoint the country, city or ASN of IP addresses using MaxMind GeoIP2 and other providers.

The databases will be loaded on-demand and, when retrieved from the network, cached on the filesystem and updated automatically.

⚠️ For instructions on how to upgrade to 2.x, check MIGRATION.md

Usage

1. Configure your license key

Skip this step if you're not loading databases directly from MaxMind.

Get a free license key from MaxMind if you haven't one already. Once logged in, you'll find the page to generate it on the left menu, under "Manage License Keys".

Then clone the repository, run make shell and declare your key:

application:set_env(locus, license_key, "YOUR_LICENSE_KEY").

If you're using Elixir, add locus as a dependency to your mix project:

defp deps do
    [
      ...
      {:locus, "~> 2.3"}
    ]
  end

Then, configure your license key in config.exs:

config :locus,
  license_key: <MAXMIND_API_KEY>

2. Start the database loader

ok = locus:start_loader(country, {maxmind, "GeoLite2-Country"}).
% You can also use:
% * an HTTP(S) URL,
% * or a local path, e.g. "/usr/share/GeoIP/GeoLite2-City.mmdb"
% * or a {custom_fetcher, Module, Args} tuple, with Module
%   implementing the locus_custom_fetcher behaviour.

Or, in Elixir, start the database loaders that you'll be using in application.ex:

  def start(_type, _args) do
    # :locus.start_loader(:asn, {:maxmind, "GeoLite2-ASN"})
    # :locus.start_loader(:country, {:maxmind, "GeoLite2-Country"})
    :locus.start_loader(:city, {:maxmind, "GeoLite2-City"})

    ...

3. Wait for the database to load (optional)

{ok, _DatabaseVersion} = locus:await_loader(country). % or `{error, Reason}'

4. Look up IP addresses

% > locus:lookup(country, "93.184.216.34").
% > locus:lookup(country, "2606:2800:220:1:248:1893:25c8:1946").

% * '{ok, Entry}' in case of success;
% * 'not_found' if no entry was found
% * '{error, _}' if something bad happened

{ok,#{<<"continent">> =>
          #{<<"code">> => <<"NA">>,
            <<"geoname_id">> => 6255149,
            <<"names">> =>
                #{<<"de">> => <<"Nordamerika">>,
                  <<"en">> => <<"North America">>,
                  <<"es">> => <<"Norteamérica"/utf8>>,
                  <<"fr">> => <<"Amérique du Nord"/utf8>>,
                  <<"ja">> => <<"北アメリカ"/utf8>>,
                  <<"pt-BR">> => <<"América do Norte"/utf8>>,
                  <<"ru">> => <<"Северная Америка"/utf8>>,
                  <<"zh-CN">> => <<"北美洲"/utf8>>}},
      <<"country">> =>
          #{<<"geoname_id">> => 6252001,
            <<"iso_code">> => <<"US">>,
            <<"names">> =>
                #{<<"de">> => <<"USA">>,
                  <<"en">> => <<"United States">>,
                  <<"es">> => <<"Estados Unidos">>,
                  <<"fr">> => <<"États-Unis"/utf8>>,
                  <<"ja">> => <<"アメリカ合衆国"/utf8>>,
                  <<"pt-BR">> => <<"Estados Unidos">>,
                  <<"ru">> => <<"США"/utf8>>,
                  <<"zh-CN">> => <<"美国"/utf8>>}},
      <<"registered_country">> =>
          #{<<"geoname_id">> => 6252001,
            <<"iso_code">> => <<"US">>,
            <<"names">> =>
                #{<<"de">> => <<"USA">>,
                  <<"en">> => <<"United States">>,
                  <<"es">> => <<"Estados Unidos">>,
                  <<"fr">> => <<"États-Unis"/utf8>>,
                  <<"ja">> => <<"アメリカ合衆国"/utf8>>,
                  <<"pt-BR">> => <<"Estados Unidos">>,
                  <<"ru">> => <<"США"/utf8>>,
                  <<"zh-CN">> => <<"美国"/utf8>>}}}}

Or, in Elixir, call the erlang library from your Elixir application:

iex> :locus.lookup(:city, "93.184.216.34")
{:ok,
 %{
   "city" => %{"geoname_id" => 4945936, "names" => %{"en" => "Norwell"}},
   ...
 }}

Documentation

1. Supported File Formats
2. Database Types and Loading
3. Database Validation
4. Remote sources: Downloading and Updating
5. Remote sources: Caching
6. Local sources: Loading and Updating
7. Logging
8. Event Subscriptions
9. API Reference
10. Tested Setup
11. License
12. Alternative Providers
13. Alternative Libraries (Erlang)
14. Alternative Libraries (Elixir)

Supported File Formats

• gzip-compressed tarballs (.tar.gz, .tgz)
• plain tarballs (.tar)
• MMDB files (.mmdb)
• gzip-compressed MMDB files (.mmdb.gz)

For tarball files, the first file to be found within it with an .mmdb extension is the one that's chosen for loading.

The implementation of MaxMind DB format is complete except for the data cache container data type.

Database Types and Loading

• The free GeoLite2 Country, City and ASN databases were all successfully tested; presumably locus can deal with any MMDB database that maps IP address prefixes to arbitrary data
• The databases are loaded into memory (mostly) as is; reference counted binaries are shared with the application callers using persistent_term, and the original binary search tree is used to lookup addresses. The data for each entry is decoded on the fly upon successful lookups.

Database Validation

Databases, local or remote, can have their compatibility validated through the locus:check/1 function after they've been loaded (see function reference.)

Alternatively, they can also be checked from the command line by use of the locus CLI utility:

1. Run make cli to build the script, named locus, which will be deployed to the current directory.

2. Check the database:

   ./locus check GeoLite2-City.mmdb
   # Loading database from "GeoLite2-City.mmdb"...
   # Database version {{2019,11,6},{11,58,0}} successfully loaded
   # Checking database for flaws...
   # Database is wholesome.
   

The script will exit with code 1 in case of failure, and 0 otherwise.

Warnings can produce failure through the --warnings-as-errors flag.

Run ./locus check --help for a description of supported options and arguments.

Remote sources: Downloading and Updating

• The downloaded database files, when compressed, are inflated in memory
• For MaxMind and HTTP downloads, the last-modified response header, if present, is used to condition subsequent download attempts (using if-modified-since request headers) in order to save bandwidth
• The downloaded databases are cached on the filesystem in order to more quickly achieve readiness on future launches of the database loader
• Database download attempts are retried upon error according to an exponential backoff policy - quickly at first (every few seconds) but gradually slowing down to every 15 minutes. Successful and dismissed download attempts will be checked for update after 6 hours. Both of these behaviours can be tweaked through the error_retries and update_period loader settings (see function reference.)
• When downloading from a MaxMind edition or an HTTP URL, the remote certificate will be authenticated against a list of known Certification Authorities and connection negotiation will fail in case of an expired certificate, mismatched hostname, self-signed certificate or unknown certification authority. These checks can be disabled by specifying the insecure loader option.

Remote sources: Caching

• Caching is a best effort; the system falls back to relying exclusively on the network if needed
• By default a caching directory named locus_erlang is created under the 'user_cache' basedir
• A cached database is named after either:
  • the MaxMind database edition name (when explicitly downloading from MaxMind), or
  • the SHA256 hash of the HTTP(S) URL, or
  • for {custom_fetcher, Module, Args} sources, a filesystem-safe version of Module's name concatenated with the 32-bit erlang:phash2/2 value of the opaque database source as returned by the callbacks.
• Modification time of