fhtagn

fhtagn.awk is a tiny CLI tool for literate testing for command-line programs.

File tests.fhtagn:

$ command --that --should --execute correctly
| expected stdout output

$ command --that --will --cause error
@ expected stderr output
? expected-exit-code

Run the tests:

./fhtagn.awk tests.fhtagn

In fact, this is a re-implementation of darius/tush, adolfopa/tush. But simpler (single tiny AWK script) and faster, because:

• it uses /dev/shm where available instead of /tmp
• it compares the expected result with the actual in the code and only calls diff to show the difference if they don't match
• it doesn't create a sandbox folder for each test
• it doesn't use mktemp but rather generates random name in the code

Usage

./fhtagn.awk f1.fhtagn [ f2.fhtagn [ f3.fhtagn [...] ] ]

This will stop on the first error found.

Example:

$ ./fhtagn.awk tests/1.fhtagn tests/2.fhtagn tests/3.fhtagn 
tests/2.fhtagn:12: $ echo "hello world"; echo "error msg" >&2; exit 7
--- expected
+++ actual
@@ -1,4 +1,4 @@
 | hello world
-@ error msg 444
-? 8
+@ error msg
+? 7
 

Fail at the end

Set ALL=1 environment variable.

This runs all tests, collects all errors, and shows the stats at the end.

ALL=1 ./fhtagn.awk f1.fhtagn [ f2.fhtagn [ f3.fhtagn [...] ] ]

Useful for running in CI.

Example:

$ ALL=1 ./fhtagn.awk tests/1.fhtagn tests/2.fhtagn tests/3.fhtagn 
tests/2.fhtagn:12: $ echo "hello world"; echo "error msg" >&2; exit 7
--- expected
+++ actual
@@ -1,4 +1,4 @@
 | hello world
-@ error msg 444
-? 8
+@ error msg
+? 7
 
tests/3.fhtagn:7: $ echo bbb
--- expected
+++ actual
@@ -1,2 +1,2 @@
-| BBB
+| bbb
 
result=FAIL, failure=2, success=4, total=6, files=3

Parameterized tests

You can use variable substitution to parametrize tests. This works like so:

# in your .fhtagn file you have

$ command {{VAR1}} '{{VAR2}}'
| expected output

And you run it like so:

./fhtagn.awk tests.fhtagn VAR1=value VAR2='other value'

The example use case here is to check the correctness of the same command when ran by different versions of a programming language. This is what we do to test fhtagn itself with different AWKs: link1, link2, link3.

Note that in case of test failure, you will see an expanded command line, which is helpful for debugging.