AQL

AQL is a Lucene-inspired query language for Go [currently targeted for arbitrary JSON]. It aims to provide the following features:

Simple, recognizable syntax
Expressive composition of boolean expressions
A generic query front-end for multiple data sources (eventually)

Currently it is used for searching arbitrary JSON data.

Usage

import (
    "log"

	"github.com/flowchartsman/aql/jsonmatcher"
)

const json = `{
    "date": "1970-01-02",
    "number": 2,
    "name": "Andy",
    "description": "大懒虫"
}`

func main() {
	m, err := NewMatcher(`name:"andy" AND date:><(1970-01-01,1980-01-01)`)
	if err != nil {
		log.Fatalf("error running query: %v", err)
	}

	result, err := m.Match([]byte(json))
	if err != nil {
		log.Fatalf("error during match: %v", err)
	}
	fmt.Println(result)
}
// output: true

AQL Syntax

In its simplest form, an AQL query is just a field identifier followed by a search term describing a value to match:

<field>:<value>

For example, given the following JSON:

{
    "text":"hello"
}

The following query returns true:

text: "hello"==true

Paths

Nested values are specified by a path, using dot-separated fields:

{
    "level1": {
        "level2": {
            "level3": "here!"
        }
    }
}

level1.level2.level3:"here!" == true

Array Introspection

If the targetted field is an array, all values will be tested, and the query will return true if it finds one that matches:

{
    "outer":{
        "inner": [
            "hello",
            "world",
            "from AQL"
        ]
    }
}

outer.inner:"world"==true

Intervening arrays will be transparently traversed, provided they contain objects or scalar values (deeply nested arrays are a WIP):

{
    "outer": [
        {
            "inner": "hello"
        },
        {
            "inner": [
                "world"
            ]
        },
        {
            "inner": "from AQL"
        }
    ]
}

outer.inner:"world"==true

Boolean Logic

Queries can also be combined using AND and OR or negated with NOT/!:

{
    "text":"AQL",
    "number":1
}

text:"AQL" AND number:1 == true

text:"Nope" OR number:1 == true

text:"AQL" AND !number:1 == false

Parenthetical grouping is also supported:

(text:"Nope" OR text:"AQL") AND number:1==true

Types

AQL recognizes several different types of terms:

| Type | Examples | Description | Notes | |------|----------|-------------|-------| |string|"hello"|a literal string|supports regular and unicode escaping| |integer|1|an integer number| | |floating point|1.0|a floating point number| | |timestamp|1970-01-02 1970-01-02T00:00:00Z|A string representing a moment in time, following the RFC3339 standard format|DateTime or FullDate values are supported| |CIDR|192.168.0.0/16|a network block| | |boolean|true false|a boolean literal value| | |regex|/^hello to \d{2} people$/|a regular expression for advanced string matching|uses Go regex syntax|

Note: Not all terms work with all operators, see the next section for details

Operators

AQL can also perform many different types of checks, depending on the type of data.

Equality

field:value

This is the basic equality check we've seen so far.

|Supported Types|Examples|Notes| |---------------|--------|-----| |string|field:"value"|searches for the exact string value provided| |integer|field:1|searches for a numeric value of the exact value provided| |float|field:1.0|searches for a numeric value of the exact value provided| |timestamp|field:1970-01-01 field:1970-01-02T15:53:33+00:00|searches for a string whith represnts this date. AQL attempts to detect a number of different possible time representations to make this check. For details, see here. Note that this check is currently for the exact timestamp specified, and other operations may be more useful for working with timestamps. |boolean|field:true field:false|searches for a JSON boolean of the exact value provided| |regex|field:/attack of the \d+ foot (?:cat\|dog)/|matches a string where a dog or cat of any height attacks (uses Go regex syntax)| |IP/CIDR|field:192.168.1.0/24|matches string values that correspond to network addresses. AQL will attempt to extract an IP address or CIDR block from the text and match the provided address against it. If the provided address is an IP address, AQL will check to see if any values match it, or if it finds CIDR blocks, whether they contain it. Correspondingly, if a CIDR block is provided, AQL will match if an extracted IP address is in that CIDR block. If both values are in CIDR notation, AQL will match if they overlap.

Exists/Null

AQL also supports two special operators, exists and null.

|exists|field:exists|matches if the field exists in the document in any surveyed location. This is actually a unary operation and not a special value, so it cannot be combined with other values in an equality set (not that you'd want to.) |null|field:null|matches if the field is explicitly null (or if all resolutions of the field in a nested structure are null). It behaves similarly to exists, but it is more strict.

Equality set

field:(value1, value2, ...)

Analagous to a SQL IN query, this is basically a shorthand for field:value1 OR field:value2 OR ...

|Supported Types|Examples| |---------------|--------| |string|field:("value1", "value2", "value3")| |integer|field:(1,2)| |float|field:(1.1, 2.2)| |timestamp|field:(1970-01-01, 1970-01-02T15:53:33Z)|

Values in an equality set can be of any type.

Numeric Comparison

field:>value

field:>=value

field:<value

field:<=value

These operations operate on numbers or those values which can be meaningfully compared numerically.

|Supported Types|Examples|Notes| |---------------|--------|-----| |integer|field:>1|searches for a numeric value greater than 1| |float|field:<2.5|searches for a numeric value less than 2.5 | |timestamp|field:<=1970-01-01|Attempts to match a timestamp in one of the recognized formats that occurs on or before midnight, UTC, January 1, 1970| ||field:>=1970-01-02T15:53:33−05:00|Attempts to match a timestamp in one of the recognized formats that occurs on or after 3:53 PM, EST, January 1, 1970|

Between

field:><(value1, value2)

This operation attempts to search for a value which falls between the two provided terms. Note: only supports two terms.

|Supported Types|Examples|Notes| |---------------|--------|-----| |integer|field:><(1, 2)|searches for a numeric value greater than 1 and less than 2| |float|field:><(2.1, 2.2)|searches for a numeric value less than 2.5 | |timestamp|field:><(1970-01-01, 1970-01-02)|Attempts to match a timestamp in one of the recognized formats that occurs between midnight, January 1, 1970 and midnight, January 2, 1970|

Contributing

PRs welcome. Please file issues if your PR addresses a bug.

Please make sure to update tests as appropriate.

License

MIT

TODO

Pluggable string-based user-provided types
Pluggable user-provided operators
Flexible query backend with selectable language features (backend doesn't support clause)

Aql

Install / Use

README

AQL