aiodnsresolver

Asyncio Python DNS resolver. Pure Python, with no dependencies other than the standard library, threads are not used, no additional tasks are created, and all code is in a single module. The nameservers to query are taken from /etc/resolv.conf, and treats hosts in /etc/hosts as A or AAAA records with a TTL of 0.

Designed for highly concurrent/HA situations. Based on https://github.com/gera2ld/async_dns.

Installation

pip install aiodnsresolver

Usage

from aiodnsresolver import Resolver, TYPES

resolve, _ = Resolver()
ip_addresses = await resolve('www.google.com', TYPES.A)

Returned are tuples of subclasses of IPv4Address or IPv6Address. Both support conversion to their usual string form by passing them to str.

Cache

A cache is part of each Resolver(), expiring records automatically according to their TTL.

import asyncio
from aiodnsresolver import Resolver, TYPES

resolve, clear_cache = Resolver()

# Will make a request to the nameserver(s)
ip_addresses = await resolve('www.google.com', TYPES.A)

# Will only make another request to the nameserver(s) if the ip_addresses have expired
ip_addresses = await resolve('www.google.com', TYPES.A)

await clear_cache()
# Will make another request to the nameserver(s)
ip_addresses = await resolve('www.google.com', TYPES.A)

The cache for each record starts on the start of each request, so duplicate concurrent requests for the same record are not made.

TTL / Record expiry

The address objects each have an extra property, expires_at, that returns the expiry time of the address, according to the loop.time() clock, and the TTL of the records involved to find that address.

import asyncio
from aiodnsresolver import Resolver, TYPES

resolve, _ = Resolver()
ip_addresses = await resolve('www.google.com', TYPES.A)

loop = asyncio.get_event_loop()
for ip_address in ip_address:
    print('TTL',  max(0.0, ip_address.expires_at - loop.time())

This can be used in HA situations to assist failovers. The timer for expires_at starts just before the request to the nameserver is made.

CNAMEs

CNAME records are followed transparently. The expires_at of IP addresses found via intermediate CNAME(s) is determined by using the minimum expires_at of all the records involved in determining those IP addresses.

Custom nameservers and timeouts

It is possible to query nameservers other than those in /etc/resolv.conf, and for each to specify a timeout in seconds to wait for a reply before querying the next.

async def get_nameservers(_, __):
    yield (0.5, ('8.8.8.8', 53))
    yield (0.5, ('1.1.1.1', 53))
    yield (1.0, ('8.8.8.8', 53))
    yield (1.0, ('1.1.1.1', 53))

resolve, _ = Resolver(get_nameservers=get_nameservers)
ip_addresses = await resolve('www.google.com', TYPES.A)

Parallel requests to multiple nameservers are also possible, where the first response from each set of requests is used.

async def get_nameservers(_, __):
    # For any record request, udp packets are sent to both 8.8.8.8 and 1.1.1.1, waiting 0.5 seconds
    # for the first response...
    yield (0.5, ('8.8.8.8', 53), ('1.1.1.1', 53))
    # ... if no response, make another set of requests, waiting 1.0 seconds before timing out
    yield (1.0, ('8.8.8.8', 53), ('1.1.1.1', 53))

resolve, _ = Resolver(get_nameservers=get_nameservers)
ip_addresses = await resolve('www.google.com', TYPES.A)

This can be used as part of a HA system: if a nameserver isn't contactable, this pattern avoids waiting for its timeout before querying another nameserver.

Custom hosts

It's possible to specify hosts without editing the /etc/hosts file.

from aiodnsresolver import Resolver, IPv4AddressExpiresAt, TYPES

async def get_host(_, fqdn, qtype):
    hosts = {
        b'localhost': {
            TYPES.A: IPv4AddressExpiresAt('127.0.0.1', expires_at=0),
        },
        b'example.com': {
            TYPES.A: IPv4AddressExpiresAt('127.0.0.1', expires_at=0),
        },
    }
    try:
        return hosts[qtype][fqdn]
    except KeyError:
        return None

resolve, _ = Resolver(get_host=get_host)
ip_addresses = await resolve('www.google.com', TYPES.A)

Exceptions

Exceptions are subclasses of DnsError, and are raised if a record does not exist, on socket errors, timeouts, message parsing errors, or other errors returned from the nameserver.

Specifically, if a record is determined to not exist, DnsRecordDoesNotExist is raised.

from aiodnsresolver import Resolver, TYPES, DnsRecordDoesNotExist, DnsError

resolve, _ = Resolver()
try:
    ip_addresses = await resolve('www.google.com', TYPES.A)
except DnsRecordDoesNotExist:
    print('domain does not exist')
    raise
except DnsError as exception:
    print(type(exception))
    raise

If a lower-level exception caused the DnsError, it will be in the __cause__ attribute of the exception.

Logging

By default logging is through the Logger named aiodnsresolver, and all messages are prefixed with [dns] or [dns:<fqdn>,<query-type>] through a LoggerAdapter. Each function accepts get_logger_adapter: the default of which results in this behaviour, and can be overridden to set either the Logger or the LoggerAdapter.

import logging
from aiodnsresolver import Resolver, ResolverLoggerAdapter

resolve, clear_cache = Resolver(
    get_logger_adapter=lambda extra: ResolverLoggerAdapter(logging.getLogger('my-application.dns'), extra),
)

The LoggerAdapter used by resolve and clear_cache defaults to the one passed to Resolver.

Chaining logging adapters

For complex or highly concurrent applications, it may be desirable that logging adapters be chained to output log messages that incorporate a parent context. So the default ouput of

[dns:my-domain.com,A] Concurrent request found, waiting for it to complete

would be prefixed with a parent context to output something like

[request:12345] [dns:my-domain.com,A] Concurrent request found, waiting for it to complete

To do this, set get_logger_adapter as a function that chains multiple LoggerAdapter.

import logging
from aiodnsresolver import Resolver, TYPES, ResolverLoggerAdapter

class RequestAdapter(logging.LoggerAdapter):
    def process(self, msg, kwargs):
        return '[request:%s] %s' % (self.extra['request-id'], msg), kwargs

def get_logger_adapter(extra):
    parent_adapter = RequestAdapter(logging.getLogger('my-application.dns'), {'request-id': '12345'})
    child_adapter = ResolverLoggerAdapter(parent_adapter, extra)
    return child_adapter

resolve, _ = Resolver()
result = await resolve('www.google.com', TYPES.A, get_logger_adapter=get_logger_adapter)

Log levels

A maximum of two messages per DNS query are logged at INFO. If a nameserver fails, a WARNING is issued [although an exception will be raised if no nameservers succeed], and the remainder of messages are logged at DEBUG. No ERROR or CRITICAL messages are issued when exceptions are raised: it is the responsiblity of client code to log these if desired.

Disable 0x20-bit encoding

By default each domain name is encoded with 0x20-bit encoding before being sent to the nameservers. However, some nameservers, such as Docker's built-in, do not support this. So, to control or disable the encoding, you can pass a custom transform_fqdn coroutine to Resolver that does not perform any additional encoding.

from aiodnsresolver import Resolver

async def transform_fqdn_no_0x20_encoding(fqdn):
    return fqdn

resolve, _ = Resolver(transform_fqdn=transform_fqdn_no_0x20_encoding)

or performs it conditionally

from aiodnsresolver import Resolver, mix_case

async def transform_fqdn_0x20_encoding_conditionally(fqdn):
    return \
        fqdn if fqdn.endswith(b'some-domain') else \
        await mix_case(fqdn)

resolve, _ = Resolver(transform_fqdn=transform_fqdn_0x20_encoding_conditionally)

Security considerations

To migitate spoofing, several techniques are used.

• Each query is given a random ID, which is checked against any response.

• By default each domain name is encoded with 0x20-bit encoding, which is checked against any response.

• A new socket, and so a new random local port, is used for each query.

• Requests made for a domain while there is an in-flight query for that domain, wait for the the in-flight query to finish, and use its result.

Also, to migitate the risk of evil responses/configuration

• Pointer loops are detected.

• CNAME chains have a maximum length.

Event loop, tasks, and yielding

No tasks are created, and the event loop is only yielded to during socket communication. Because fetching results from the cache involves no socket communication, this means that cached results are fetched without yielding. This introduces a small inconsistency between fetching cached and non-cached results, and so clients should be written to not depend on the presence or lack of a yield during resolution. This is a typically recommended process howeve