JsonBatch

An Engine to run batch request with JSON based REST APIs

Some usecase for this library:

• Provide a batch API to your REST API set.
• Quickly roll out an adapter to migrate old API set.
• Remove all boilerplate requests & responses classes.

You can try out JsonBatch via this web app

• Getting Started
• How it work
• How it build JSON
• Data type
• Function
• Raw String
• Object
• Array
• Where is the data
• A real example
• Custom function
• Loop requests
• Response transform
• Temporary variables

Getting Started

JsonBatch is available at the Central Maven Repository.

<dependency>
  <groupId>com.github.rey5137</groupId>
  <artifactId>jsonbatch-core</artifactId>
  <version>1.3.2</version>
</dependency> 

// need to include jsonpath dependency

<dependency>
    <groupId>com.jayway.jsonpath</groupId>
    <artifactId>json-path</artifactId>
    <version>2.4.0</version>
</dependency>

We also need to add a sub package that implement RequestDispatcher. You can use this package that use Apache HttpClient:

<dependencies>
    <dependency>
        <groupId>com.github.rey5137</groupId>
        <artifactId>jsonbatch-apache-httpclient</artifactId>
        <version>1.1.2</version>
    </dependency>

    // need to include httpclient dependency

    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.5.2</version>
    </dependency>
</dependencies>

Or this one use OkHttp:

<dependencies>
    <dependency>
        <groupId>com.github.rey5137</groupId>
        <artifactId>jsonbatch-okhttp</artifactId>
        <version>1.1.2</version>
    </dependency>

    // need to include okhttp dependency

    <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>okhttp</artifactId>
        <version>4.7.2</version>
    </dependency>
</dependencies>

JsonBatch depends on Jayway JsonPath library to parse json path.

First we have to create a BatchEngine. Below is a simple example:

  Configuration conf = Configuration.builder().build();
  JsonBuilder jsonBuilder = new JsonBuilder(Functions.basic());
  RequestDispatcher requestDispatcher = new ApacheHttpClientRequestDispatcher(HttpClients.createDefault());
  BatchEngine batchEngine = new BatchEngine(conf, jsonBuilder, requestDispatcher);

BatchEngine has only 1 public method:

  public Response execute(Request originalRequest, BatchTemplate template);

By supplying the original request and a template, BatchEngine will construct & execute each request sequentially, then collect all responses and construct the final response.

How it work

Here is Batch template full JSON format:

{
  "requests": [
      {
        "predicate": "...",
        "http_method": "...",
        "url": "...",
        "headers": { ... },
        "body": { ... },
        "requests": [  ... <next requests> ... ],
        "responses": [ ... <response templates> ... ]
      },
      ...
  ],
  "responses": [
      {
        "predicate": "...",
        "status": "...",
        "headers": { ... },
        "body": { ... }
      },
      ...
  ],
  "dispatch_options": {
    "fail_back_as_string": ...,
    "ignore_parsing_error": ...
  },
  "loop_options": {
    "max_loop_time": ...
  }
}

By start, the Engine will loop though the requests list and choose the first template has predicate expression is true. (if a request template has predicate field is NULL, it will always be true). The Engine will build request from template, pass it to RequestDispatcher to execute request and collect response.

After that, it will find the first matching template from the responses list of current request template. If it found a response template, it will stop execution chain, build and return the response. If no matching response template found, the Engine will continue find next request from the requests list of current request template.

After all requests are executed, the Engine will try to find a matching response template from responses list of BatchTemplate. If a matching response template found, it will build the final response and return it. If not, it will return a response contains all requests & responses it has collected so far.

When RequestDispatcher execute a request, you can pass options via dispatch_options object to instruct it how to handle response:

• fail_back_as_string: If RequestDispatcher cannot parse response body as JSON, it will return as String.
• ignore_parsing_error: Ignore error when parsing response body, and return null instead.

How it build JSON

To know how to build a JSON object from template, JsonBatch use a JSON with special format. All fields that aren't string will be same when build actual JSON but string field have to follow a specific format:

<data type> <json_path or function(sum, min, max, ...) or raw_string>

For example:

{
  "field_1": "int $.responses[0].body.field_a" 
}

The above template means: build a json object with "field_1" is integer, and extract value from json path "$.responses[0].body.field_a"

You can omit the <data type> part like that:

{
  "field_1": "$.responses[0].body.field_a" 
}

JsonBatch will use the type of extracted value instead of casting it.

Data type

| Type | Description | | :----------------------- | :----------------| | str, string | String | | int, integer | Integer | | num, number | Decimal | | bool, boolean | Boolean | | obj, object | Any object | | str[], string[] | String array | | int[], integer[] | Integer array | | num[], number[] | Decimal array | | bool[], boolean[] | Boolean array | | obj[], object[] | Any array |

In case the actual type of value is different with wanted type, the Engine will try to convert if possible. Some examples:

<table> <tr> <td> Template </td> <td> Value </td> <td> Result </td> </tr> <tr> <td>

{
    "field_1": "int $.responses[0].body.field_a"
}

</td> <td> "10" </td> <td>

{
    "field_1": 10
}

</td> </tr> <tr> <td>

{
    "field_1": "int[] $.responses[0].body.field_a"
}

</td> <td> "10" </td> <td>

{
    "field_1": [ 10 ]
}

</td> </tr> <tr> <td>

{
    "field_1": "int $.responses[*].body.field_a"
}

</td> <td> ["9", "10"] </td> <td>

{
    "field_1": 9
}

</td> </tr> </table>

Function

Instead of extracting value from json path, we can use some function to aggregate value. The syntax is: <data type> __<function name>(<function arguments>) (Note that there is prefix "__" before function name).

Below is list of supported function:

| Function | Syntax | Example | Description | | :--------- | :----------------------------- | :-----------------------------------| :--------------------------------------------- | | sum | __sum(<arguments>) | __sum("$.responses[].body.field_a") | Sum all values from int/decimal array | | min | __min(<arguments>) | __min("$.responses[].body.field_a") | Get minimum value from int/decimal array | | max | __max(<arguments>) | __max("$.responses[].body.field_a") | Get maximum value from int/decimal array | | average | __average(<arguments>) | __average("$.responses[].body.field_a") | Calculate average value from int/decimal array | | and | __and(<arguments>) | __and("$.responses[].body.field_a") | And logical | | or | __or(<arguments>) | __or("$.responses[].body.field_a") | Or logical | | compare | __cmp("<expression>") | __cmp("@{$.field_a}@ > 10") | Compare 2 value | | regex | __regex("<json_path>", "<pattern>", <index>) | __regex("$.field_a", "(.*)", 1) | Extract from string by regex pattern and group index |

Raw String

For string field, instead of using JsonPath or Function, we can use raw string directly. Note that JsonBatch support inline variable with format: @{<schema>}@ (You can use inline variable in both JSON key & value)

Some examples:

<table> <tr> <td> Template </td> <td> Result </td> </tr> <tr> <td>

{
    "field_1": "This is a raw string"
}

</td> <td>

{
    "field_1": "This is a raw string"
}

</td> </tr> <tr> <td>

{
    "field_1": "This is a raw string with @{$.key}@ var"
}

</td> <td>

{
    "field_1": "This is a raw string with 1 var"
}

</td> </tr> <tr> <td>

{
    "field_1": "This is a raw string with @{nested @{$.key}@}@ var"
}

</td> <td>

{
    "field_1": "This is a raw string with nested 1 var"
}

</td> </tr> </table>

Object

When you define schema for each key in object, you often have to repeat a lot of JsonPath. To help reduce repeated works, you can add __object_schema key to define JSON context of current object. The root JsonPath ($) will point to new JSON context, and you