Filterql

<h1 align="center"><img align="top" style="color:#36BCF7; width:38px; height:38px;" src="https://raw.githubusercontent.com/adamhl8/filterql/refs/heads/main/assets/logo.svg"> FilterQL</h1>

A tiny query language for filtering structured data 🚀

<!-- https://readme-typing-svg.demolab.com/demo/?font=JetBrains+Mono&size=16&duration=3000&pause=7500&vCenter=true&width=690&height=25&lines=(genre+%3D%3D+Action+%7C%7C+genre+%3D%3D+Comedy)+%26%26+rating+%3E%3D+8.5+%7C+SORT+rating+desc -->

---

In addition to the Overview below, there are three main sections of this README:

• Queries - How to write FilterQL queries
• TypeScript Library - Usage of the FilterQL TypeScript library
• FilterQL language specification

Overview

Define a query schema and create a FilterQL instance:

import { FilterQL } from "filterql"

// The schema determines what fields are allowed in the query
const schema = {
  title: { type: "string", alias: "t" },
  year: { type: "number", alias: "y" },
  monitored: { type: "boolean", alias: "m" },
  rating: { type: "number" },
  genre: { type: "string" },
}

const filterql = new FilterQL({ schema })

const movies = [
  { title: "The Matrix", year: 1999, monitored: true, rating: 8.7, genre: "Action" },
  { title: "Inception", year: 2010, monitored: true, rating: 8.8, genre: "Thriller" },
  { title: "The Dark Knight", year: 2008, monitored: false, rating: 9.0, genre: "Action" },
]

// Filter movies by genre
const actionMovies = filterql.query(movies, "genre == Action")

// Field aliases and multiple comparisons
const recentGoodMovies = filterql.query(movies, "y >= 2008 && rating >= 8.5")

// Sort the filtered data by using the built-in SORT operation
const recentGoodMovies = filterql.query(movies, "year >= 2008 | SORT rating desc")

// Filter using boolean shorthand
const monitoredMovies = filterql.query(movies, "monitored")

---

<!-- toc -->

• Queries
  • Logical Operators
  • Comparison Operators
  • Boolean Fields
  • Quoted Values
  • Empty Value Checks
  • Match-All
  • Operations
• TypeScript Library
  • Installation
  • Example
  • Schemas
  • Handling data
  • Options
  • Custom Operations
  • API Reference
• Language Specification
  • Grammar
  • Comparison Operators
  • Logical Operators
  • Match-All
  • Operations
  • Syntax Rules
  • Implementation

<!-- tocstop -->

---

Queries

The most basic query is a single comparison: <field> <comparison operator> <value>

title == Interstellar

You can also use the alias for a field:

t == Interstellar

Combine multiple comparisons using logical operators for more complex queries:

title == Interstellar && year == 2014

Logical Operators

The following logical operators can be used in queries:

• () (parentheses for grouping)
• ! (not)
• && (and)
• || (or)

[!TIP] Note that these operators are listed in order of precedence. This is important because many queries will likely require parentheses to do what you want.

For example:

genre == Action || genre == Thriller && rating >= 8.5 means "genre must be Action or, genre must be Thriller and rating must be at least 8.5." This probably isn't what you want.

(genre == Action || genre == Thriller) && rating >= 8.5 means "genre must be Action or Thriller, and rating must be at least 8.5."

Comparison Operators

The following comparison operators can be used in comparisons:

• == (equals)
• != (not equals)
• *= (contains)
• ^= (starts with)
• $= (ends with)
• ~= (matches regex)
• >= (greater than or equal)
• <= (less than or equal)

[!TIP] Comparisons are case-sensitive. To make them case-insensitive, prefix the comparison operator with i.

title i== interstellar

Boolean Fields

For boolean fields, you can use the field name without any comparison to check for true:

downloaded is equivalent to downloaded == true

!downloaded is equivalent to !(downloaded == true)

Quoted Values

If your comparison value has spaces, you must enclose it in double quotes:

title == "The Dark Knight"

Inside a quoted value, double quotes must be escaped:

title == "A title with \"quotes\""

Values ending with ) as part of the value (not a closing parenthesis) must be quoted:

title == "(a title surrounded by parentheses)"

Empty Value Checks

Sometimes the data you're filtering might have empty values ("", undefined, null). You can filter for empty values by comparing to an empty string:

Get all entries that don't have a rating:

rating == ""

Get all entries that have a rating:

rating != ""

Match-All

If you want to get all of the entries, use * (the data is not filtered):

*

This is mainly useful when you don't want to filter the data but want to apply operations to it.

* | SORT rating desc

Operations

After filtering, you can apply operations to transform the data: <filter> | <operation> [arg]...

year >= 2000 | SORT rating desc | LIMIT 10

• Operations are applied in the order they are specified.
• The same operation can be applied multiple times.

Built-in Operations

There are currently two built-in operations:

• SORT: Sorts the data by the specified field.
  • SORT <field> [direction]
  • direction can be asc or desc (default: asc).
• LIMIT: Limits the number of entries returned.
  • LIMIT <number>

If you have any suggestions for other operations, please let me know by opening an issue!

[!TIP] You can also define custom operations.

TypeScript Library

Installation

bun add filterql
# or: npm install filterql

Example

Let's say you're building a CLI tool that fetches some data to be filtered by a query the user provides:

import { FilterQL } from "filterql"

// data is an array of objects
const data = await (await fetch("https://api.example.com/movies")).json()

const query = process.argv[2] // first argument

const schema = {
  title: { type: "string", alias: "t" },
  year: { type: "number", alias: "y" },
  monitored: { type: "boolean", alias: "m" },
  rating: { type: "number" },
  genre: { type: "string" },
}

const filterql = new FilterQL({ schema })
const filteredMovies = filterql.query(data, query)

console.log(filteredMovies)

And then the user might use your CLI tool like this:

movie-cli '(genre == Action || genre == Comedy) && year >= 2000 && rating >= 8.5'

Schemas

The schema given to the FilterQL constructor determines what fields and value types are allowed in queries.

[!IMPORTANT] The type of data the FilterQL methods accept is Record<string, unknown>[]. This means that FilterQL does not care about extra properties/keys in the data.

In other words, a schema's keys can be a subset of the data's keys.

Similarly, the schema is not used to validate the data. It is only used to validate the values given in the query.

See the Handling data section.

Each field has a type and an (optional) alias.

const schema = {
  title: { type: "string", alias: "t" },
  year: { type: "number", alias: "y" },
  monitored: { type: "boolean" },
}

const filterql = new FilterQL({ schema })

Field types determine validation behavior:

• string: The value must be coercible to a string (this is always the case)
• number: The value must be coercible to a number
• boolean: The value must be true or false

When a comparison value can't be coerced to the field's type, an error is thrown. For example, consider a query like year = foo.

• The comparison value of foo can't be coerced to a number, so an error is thrown.

Handling data

It's important to note that query comparisons are only evaluated against certain data types.

• Specifically, a data value must be one of string, number, or boolean, undefined, or null to be evaluated.

For example, say we have the following data:

const people = [
  {
    name: "Bob",
    age: 30,
    address: {
      street: "123 Main St",
      city: "Anytown",
    },
    roles: ["admin", "user"],
  },
  // more people...
]

Passing in data like this is perfectly valid, but because the address and roles properties are not one of the comparable types, they can't be filtered on. e.g. a query like roles == admin won't return any results.

If you want to filter on nested data like this, you should transform the data into a flat structure before passing it to FilterQL.

For example, say you wanted to query for people who have the "admin" role. You could transform the data like this:

// query: 'roles_admin == true'
{
  name: "Bob",
  age: 30,
  address_street: "123 Main St",
  address_city: "Anytown",
  roles_admin: true,
  roles_user: true,
}

Or maybe something like this, where the elements/properties are joined as a string:

// query: 'roles *= admin'
{
  name: "Bob",
  age: 30,
  address: "123 Main St, Anytown",
  roles: "admin, user",
}

<details> <summary><i>Why not support nested data structures?</i></summary>

Supporting nested data structures would require a more complex