Grex

<div align="center">

<br>

</div> <br>

<br>

1. What does this tool do?

grex is a library as well as a command-line utility that is meant to simplify the often complicated and tedious task of creating regular expressions. It does so by automatically generating a single regular expression from user-provided test cases. The resulting expression is guaranteed to match the test cases which it was generated from.

This project has started as a Rust port of the JavaScript tool regexgen written by Devon Govett. Although a lot of further useful features could be added to it, its development was apparently ceased several years ago. The plan is now to add these new features to grex as Rust really shines when it comes to command-line tools. grex offers all features that regexgen provides, and more.

The philosophy of this project is to generate the most specific regular expression possible by default which exactly matches the given input only and nothing else. With the use of command-line flags (in the CLI tool) or preprocessing methods (in the library), more generalized expressions can be created.

The produced expressions are Perl-compatible regular expressions which are also compatible with the regular expression parser in Rust's regex crate. Other regular expression parsers or respective libraries from other programming languages have not been tested so far, but they ought to be mostly compatible as well.

2. Do I still need to learn to write regexes then?

Definitely, yes! Using the standard settings, grex produces a regular expression that is guaranteed to match only the test cases given as input and nothing else. This has been verified by property tests. However, if the conversion to shorthand character classes such as \w is enabled, the resulting regex matches a much wider scope of test cases. Knowledge about the consequences of this conversion is essential for finding a correct regular expression for your business domain.

grex uses an algorithm that tries to find the shortest possible regex for the given test cases. Very often though, the resulting expression is still longer or more complex than it needs to be. In such cases, a more compact or elegant regex can be created only by hand. Also, every regular expression engine has different built-in optimizations. grex does not know anything about those and therefore cannot optimize its regexes for a specific engine.

So, please learn how to write regular expressions! The currently best use case for grex is to find an initial correct regex which should be inspected by hand if further optimizations are possible.

3. Current Features

• literals
• character classes
• detection of common prefixes and suffixes
• detection of repeated substrings and conversion to {min,max} quantifier notation
• alternation using | operator
• optionality using ? quantifier
• escaping of non-ascii characters, with optional conversion of astral code points to surrogate pairs
• case-sensitive or case-insensitive matching
• capturing or non-capturing groups
• optional anchors ^ and $
• fully compliant to Unicode Standard 16.0
• fully compatible with regex crate 1.11.0+
• correctly handles graphemes consisting of multiple Unicode symbols
• reads input strings from the command-line or from a file
• produces more readable expressions indented on multiple using optional verbose mode
• optional syntax highlighting for nicer output in supported terminals

4. How to install?

4.1 The command-line tool

You can download the self-contained executable for your platform above and put it in a place of your choice. Alternatively, pre-compiled 64-Bit binaries are available within the package managers Scoop (for Windows), Homebrew (for macOS and Linux), MacPorts (for macOS), and Huber (for macOS, Linux and Windows). Raúl Piracés has contributed a Chocolatey Windows package.

grex is also hosted on crates.io, the official Rust package registry. If you are a Rust developer and already have the Rust toolchain installed, you can install by compiling from source using cargo, the Rust package manager. So the summary of your installation options is:

( brew | cargo | choco | huber | port | scoop ) install grex

4.2 The library

In order to use grex as a library, simply add it as a dependency to your Cargo.toml file:

[dependencies]
grex = { version = "1.4.6", default-features = false }

The dependency clap is only needed for the command-line tool. By disabling the default features, the download and compilation of clap is prevented for the library.

5. How to use?

Detailed explanations of the available settings are provided in the library section. All settings can be freely combined with each other.

5.1 The command-line tool

Test cases are passed either directly (grex a b c) or from a file (grex -f test_cases.txt). grex is able to receive its input from Unix pipelines as well, e.g. cat test_cases.txt | grex -.

The following table shows all available flags and options:

$ grex -h

grex 1.4.6
© 2019-today Peter M. Stahl <pemistahl@gmail.com>
Licensed under the Apache License, Version 2.0
Downloadable from https://crates.io/crates/grex
Source code at https://github.com/pemistahl/grex

grex generates regular expressions from user-provided test cases.

Usage: grex [OPTIONS] {INPUT...|--file <FILE>}

Input:
  [INPUT]...         One or more test cases separated by blank space
  -f, --file <FILE>  Reads test cases on separate lines from a file

Digit Options:
  -d, --digits      Converts any Unicode decimal digit to \d
  -D, --non-digits  Converts any character which is not a Unicode decimal digit to \D

Whitespace Options:
  -s, --spaces      Converts any Unicode whitespace character to \s
  -S, --non-spaces  Converts any character which is not a Unicode whitespace character to \S

Word Options:
  -w, --words      Converts any Unicode word character to \w
  -W, --non-words  Converts any character which is not a Unicode word character to \W

Escaping Options:
  -e, --escape           Replaces all non-ASCII characters with unicode escape sequences
      --with-surrogates  Converts astral code points to surrogate pairs if --escape is set

Repetition Options:
  -r, --repetitions
          Detects repeated non-overlapping substrings and converts them to {min,max} quantifier
          notation
      --min-repetitions <QUANTITY>
          Specifies the minimum quantity of substring repetitions to be converted if --repetitions
          is set [default: 1]
      --min-substrin