JS to ESI compiler and decompiler (js2esi, esi2js)

Convert Javascript to from from ESI (Edge Side Includes). Write javascript code that will compile to ESI and run at the Edge.

Synopsis

js2esi | esi2js
[--version] [-h|--help] [-v|--verbose]
[-l|--lex] [-n|--node] [-d|--decompile]
[-w|--no-warning] [-L|--library PATH] [-O|--optimize LEVEL]
[-o|--output FILENAME]
{FILENAME|-}

Overview

js2esi is a tool that can compile js 'lite' into Edge Side Includes (ESI) using the compiler of the same name, js. The language is, at it's core, nothing more than syntactic sugar to make ESI coding simpler. As a result, it is limited to the exact same constraints as ESI is. It does, however, also provide some more advanced features, such as modularization and inlining, that get resolved during compilation. The js dev must always be aware that js is converted to ESI, be aware of those limitations, and review the resulting ESI to ensure compatibility and correctness.

Getting started (dev)

The simplist way to get started is to:

1. clone the repo
2. run ./dev.sh to get all the dependencies installed, etc
3. . venv/bin/activate to activate your local environment
4. run js2esi as below

Test coverage uses Nose. Just run nosetests

The js2esi Program

The program can be used to compile js files to ESI as well as the reverse, decompile ESI to js. The program accepts the following options:

Options

--version Display program version identifier and exit.

-h | --help Display command options and exit.

-v | --verbose Enable verbose output (multiple invocations increase verbosity).

-l | --lex Display lexical parsing tokens - primarily useful if js2esi is having trouble parsing your source file.

-n | --node Display the Node representational form instead of the js or ESI output (primarily useful for js2esi debugging).

-d | --decompile Enable ESI-to-js decompilation instead of js-to-ESI compilation. Note that this is implied if invoked as esi2js command (they are symbolic links to each other).

-w | --no-warning By default, the ESI output includes a warning that the ESI was generated as output from js2esi. This option disables the warning.

-L PATH | --library PATH Add the specified PATH to the js2esi module lookup path (for resolving "import" statements). The path will be append to any paths already provided by the JSLIB environment variable (see below). The option can be invoked multiple times to append multiple paths.

-O LEVEL | --optimize LEVEL Set the code optimization level to LEVEL, which can range from 0 (no optimizations) to 9 (maximum optimizations). The default value, 7, is safe and will only apply non-destructive optimizations; levels above 7 should be reserved for ESI scripts that are completely independent/standalone — i.e. the script does not use, or is not used by, any eval statements that may depend on variables and functions exposed by this script. This is because variables/functions will be removed if unused and/or may be renamed to be more efficient.

-o FILENAME | --output FILENAME output the compiled code to FILENAME. Defaults to STDOUT

FILENAME | - Specifies the FILENAME that should be processed. If - is specified instead of the path to a filename, then STDIN will be used.

Environment

The js2esi program is sensitive to the following environmental variables:

JSLIB This is a colon (":") delimited list of path names, similar to $PATH, that will be used to find any external js source referenced by import statements.

Example

$ cat sample.js
# this is a sample js file.
myvar = 'http://'.length + HTTP_HOST.length + len(REQUEST_PATH);

$ js2esi -w sample.js
<esi:assign name="myvar" value="$len('http://')+$len($(HTTP_HOST))+$len($(REQUEST_PATH))"/>

Javascript 'lite' language

The lexical parser is intended to be javascript compatible. However, it isn't very sophisticaed because of some of the underlying limitations of ESI.

The following syntax guide shows how to construct js code that compile nicely to ESI.

Important

Please read the section called “Todo List”, as several things that you may expect to work don't. Also take a look at the section called “Known Bugs” (surprise!).

Some general rules in the syntax of js:

• Whitespace is irrelevant — except in literals, it will be ignored completely.
• The "if/else" construct is ONLY a "choose/when" shorthand, and must therefore be used appropriately and leverage the "else if" construct when possible.

Some general js limitations due to limitations in ESI:

• All ESI limitations apply! Including, but not limited to: total script size, recursion, variable size and inclusion limits.
• The js constants "true" and "false" are represented as the ESI integers 1 and 0 - although this works as expected when evaluated in boolean context, it may behave slightly differently in non-boolean contexts.
• Doing string indexing is faster than dictionary lookups — avoid dictionary lookups.

Currently, the method to get the compiled js to output content (i.e. to get ESI to output content) is a bit of a "workaround". Several reserved functions exist for this purpose: print(), printv() and printraw() (see Example 9, “Composing Output”).

js2esi supports one nifty thing that ESI does not: inlining either external or internal code. This feature is activated with the import function call and the inline keyword. The difference between importing and including/eval'ing is synonymous to static versus dynamic linking: importing will pull the external code in during js2esi compilation, whereas including/eval'ing will do so during ESI execution. Please see Example 12, “Imports” and Example 5, “Inlined Functions” for details.

Examples:

• Example 1, “Comments”
• Example 2, “Variable Assignments”
• Example 3, “Operators”
• Example 4, “Function Declarations and Calls”
• Example 5, “Inlined Functions”
• Example 6, “Conditional Statements”
• Example 7, “Match Operator”
• Example 8, “For Of”
• Example 9, “For Loops”
• Example 10, “Composing Output”
• Example 11, “Try/Catch Blocks”
• Example 12, “Include and Eval Statements”
• Example 13, “modules”

Example 1. Comments

js input:

/** anything after a long or short comment will be regarded as a comment and
 * stripped out completely, with the exception of a comment that uses the @esi-comment 
 * descriptor, which will be converted into an ESI comment, eg:
 */
//@esi-comment this ESI code is generated -- do not edit manually
// the above will appear in the output, but nothing else

ESI output:

<esi:comment text="this ESI code is generated -- do not edit manually"/>

Example 2. Variable Assignments

js2esi supports all of the data structures that ESI does, as well as a few additional unary assignment operators. Note that, just like in ESI, it is possible to reference a variable without using or declaring it.

js input:

varName = 'someString';
numVal  = 919;
boolVal = true;
boolVal = false;
array   = [ 'firstVal', 2, [ 'subarray', 'el', ] ];
hash    = { 'foo': 'this', 'bar': 'that', };
varName += ' - and the rest';
numVal  -= 10;
numVal  *= 5;
numVal  /= 5;
numVal  %= 5;
numVal  ++;
numVal  --;

# variable referencing with default values can also be done:

value   = hash['bar'] | 'none';

ESI output:

<!-- NOTE: newlines/whitespace added for clarity - not normally in js2esi output -->
<esi:assign name="varName" value="'someString'"/>
<esi:assign name="numVal" value="919"/>
<esi:assign name="boolVal" value="1"/>
<esi:assign name="boolVal" value="0"/>
<esi:assign name="array" value="['firstVal',2,['subarray','el']]"/>
<esi:assign name="hash" value="{'foo':'this','bar':'that'}"/>
<esi:assign name="varName" value="$(varName)+' - and the rest'"/>
<esi:assign name="numVal" value="$(numVal)-10"/>
<esi:assign name="numVal" value="$(numVal)*5"/>
<esi:assign name="numVal" value="$(numVal)/5"/>
<esi:assign name="numVal" value="$(numVal)%5"/>
<esi:assign name="numVal" value="$(numVal)+1"/>
<esi:assign name="numVal" value="$(numVal)-1"/>
<esi:assign name="value" value="$(hash{'bar'}|'none')"/>

Example 3. Operators

Most commonly used operators are supported, including bitwise shifting and AND/OR. Note that if you use bitwise AND/OR, bitwise ESI will be generated — but this will only work if legacy bitwise operators are disabled (on Akamai, this means setting the <edgecomputing:esi.legacy-logical-operators> tag to off, which by default is on). The js2esi optimizer is able to detect certain operations with literals (i.e. hard-coded values) that can be evaluated at compile time, and thus yield more efficient ESI code.

js input:

# addition, subtraction, multiplication, modulus and division
result = ( '*' * ( ( 25 % 13 ) / ( 3 - 1 ) )  ) + ' six stars!';

# equality, non-equality, lesser/greater comparison, logical AND/OR
if ( a == 12 && ( b != 'options' || c <= 4 ) && d < 5 && e > 15 )
  boolval = e >= 9;

# bitwise AND/OR/XOR, bitwise shifting
value = ( ( 1 << 5 ) | ( 1 << 3 ) ) >> 1; # == 10
value = value ^ ( ~ 10 );
if ( value & 4 )
  match = 'yup!';

ESI output:

<!-- NOTE: newlines/whitespace added for clarity - not normally in js2esi output -->
<esi:assign name="result" value="('*'*6)+' six stars!'"/>
<esi:choose>
  <esi:when test="((($(a)==12) && (($(b)!='options') || ($(c)<=4))) && ($(d)<5)) && ($(e)>15)">
    <esi:assign name="boolval" value="$(e) >= 9"/>
  </esi:when>
</esi:choose>
<esi:assign name="value" value="((1<<5) | (1<<3)) >> 1"/>
<esi:assign name="value" value="$(value) ^ (~ 10)"/>
<esi:choose>
  <esi:when test="$(value) & 4">
    <esi:assign name="match" value="'yup!'"/>
  </esi:when>
</esi:choose>

Example 4. Function Declarations and Calls

js input:

function functionName() {
  // a