yst - static websites from YAML and string templates

yst is a tool for generating a static website by filling [string template][]s with data taken from [YAML][] or [CSV][] text files or [SQLite3][] file based databases. This approach combines the speed, security, and ease of deployment of a static website with the flexibility and maintainability of a dynamic site that separates presentation and data.

Installing yst

yst is written in Haskell. The easiest way to install yst is by using Haskell's [cabal install][] tool. The best way to get this tool is to install the [Haskell platform][], which includes a complete installation of the [GHC][] compiler and the cabal executable.

Once you have cabal, you can install yst with two commands:

cabal update
cabal install yst

(Note that by default, cabal installs the yst executable into a special directory: ~/.cabal/bin on unix systems. You will need to make sure that directory is in your system path.)

If you get an error about a missing library sqlite3, try with this:

cabal install yst -fbuiltin-sqlite3

Getting started

To get started with yst, use the command:

yst create mysite

This will create a directory mysite and populate it with the files needed for a sample site. Change to this directory and run yst with no arguments to create the site:

cd mysite
yst

The site will be created in the site directory. Open up site/index.html to take a look.

The configuration file index.yaml tells yst which pages to build, and from which templates and data files. Let's take a look, so we can see what went into index.html. The file is a YAML list of YAML hashes (name/value pairs). The first item is

- url      : index.html
  title    : Home
  template : index.st
  requires : event.st
  data     :
    recentevents : FROM events.yaml ORDER BY date DESC LIMIT 2

This says: build the page index.html from the string template index.st (and subtemplate event.st) and data from events.yaml. Sort this data (a list of events) by date in descending order, and discard all but the first two items. Put the resulting data in the string template attribute recentevents. Give the page the title "Home."

Now take a look at events.yaml, the data source. Again it is a YAML list of YAML hashes, each item in the list representing one event. The first is:

- date: 2009-06-28
  speaker:  Sam Smith
  title: Building a static website

Pretty self-explanatory! Try adding a new event, then rebuild the site by typing yst and see how it looks.

If you have problems, make sure events.yaml is a valid YAML file. Structure is shown through indentation, so make sure things line up right. And occasionally you may need to use quotation marks around string values---for example, when a title contains a colon.

There's one more ingredient in our recipe---the string templates. Take a look at index.st, the template from which index.html is built:

# Welcome

Here's our website. Have a look around.

Our last two events:

$recentevents:event()$

For a complete list of events, see the [events](events.html) page.

The first thing to notice is that this is in [markdown][] format (or, to be precise, markdown with [pandoc][] extensions). So, for example, the first line is a level-one header, and there is a hyperlink to the events page on the last line.

The second thing to notice is the bit between dollar signs. This is a placeholder for some formatted data. The rendered template will include the list recentevents (remember, this was populated from events.yaml after some transformations---see above). And each element of this list will be formatted by a subtemplate called event.st:

- $if(it.speaker)$$it.speaker; separator=" and "$, $endif$ *$it.title$*.

Let's break this down. The whole line will print a bulleted list item. it here refers to the event that is being processed by the template. So the first part says: if this event has a speaker field, print the speaker, or, if the speaker field is a list, print all the speakers separated by the word "and", followed by a comma. And the second part says, print the contents of the title field, surrounding it with asterisks (which is the markdown way of making it emphasized).

(See the [string template][] documentation for details of template syntax, and examples.)

If you look at index.html, you'll see this rough structure, but in an HTML file, not a markdown file. yst converts the rendered markdown template to HTML (using pandoc), and inserts it into a "layout" file called layout.html.st. If you look at this file, you'll see that it's an HTML file with blanks for $contents$ and $nav$. The $contents$ blank gets filled by the rendered template, converted to HTML, and the $nav$ blank gets filled by an HTML navigation menu (an unordered list with links).

To summarize our example, then: yst sorts and filters the data in events.yaml according to our instructions; inserts this data into the events.st template, formatting each item using the event.st template; uses pandoc to convert the rendered template to HTML; constructs a navigation menu; and puts the contents and navigation menu in the layout template layout.html.st. The result is our page, index.html.

Reference

The yst command

Synopsis:

yst                    # rebuilds site, using default config.yaml
yst -f myconf.yaml     # rebuilds site, using myconf.yaml as config
yst create newsite     # creates a starter (demo) site in newsite directory

When run without arguments, yst looks at index.yaml to determine the dependencies of each page, and rebuilds only the pages whose dependencies have changed since the last build.

In order for this to work properly, you must be sure to list all subtemplates included recursively in the main page template using the requires field. This field takes a single filename or a YAML list, which may be in either of two formats:

requires: [event.st, date.st]

or

requires:
  - event.st
  - date.st

If you don't list all the subtemplates needed to render a page under requires, yst will still work, but it might sometimes fail to rebuild a page when one of these subtemplates has been changed.

config.yaml

The configuration file specifies the following:

• indexfile: the filename of the index file (default: index.yaml)
• title: the title of the whole site
• sourcedir: list of directories containing templates and page sources (default: .)
• datadir: list of directories containing yaml data files (default: .)
• filesdir: list of directories containing static files (default: files)
• layout: the default layout template for the site, relative to sourcedir (default: layout.html.st)
• navstyle: either top for a top menu or side for a side menu

The directories specified by sourcedir and datadir are searched in order to find source/template or data files, respectively. This allows for a ../templates directory to be shared among multiple sites, for example. Static files are merged from the contents of all directories in filesdir. All of these accept a string as a singleton list.

index.yaml and submenus

The index file is a YAML list of pages. Each page may have the following fields:

• url: the relative URL of the page to be built
• title: the title of the page
• template: the main string template from which the page will be built
• source: the markdown source from which the page will be built
• requires: other files changes to which should trigger a page rebuild (primarily subtemplates of the main page template)
• data: string template attributes, data sources, and transformations (see below)
• layout: a layout template to use, if other than the site default
• inmenu: if 'no', the page will not be included in the site navigation menu

Each page must have at least url, title, and either template or source. Values for template, source, and layout are relative to sourcedir specified in config.yaml.

The pages may be organized into a tree-like hierarchy, which will be reflected in the site navigation menu. It is easiest to see how this works by example:

- Rooms:
  - url      : room101.html
    title    : Room 101
    source   : room101.txt

  - url      : room203.html
    title    : Room 203
    source   : room203.txt

Here we have a subtree called "Rooms" with two pages under it. Subtrees can contain other subtrees. Just be consistent about indentation.

The data field

The data field in index.yaml can populate any number of stringtemplate attributes with data from YAML or CSV files or SQLite3 databases. The syntax is easiest to explain by example (note that the keywords do not have to be in ALL CAPS, although they may, and the query doesn't have to end with a semicolon, though it may):

data:
    events:  from events.yaml order by date desc group by title then location
    people:  from people.csv order by birthday then lastname where
              birthstate = 'CA' limit 5
    beststudents: from students.sqlite 
              query "select * from students where grade > 5"
              order by name

First we have the name of the stringtemplate attribute to be populated (say, events). Then, after the colon, we have the data source (events.yaml). If the data source is an SQLite3 database, it should be followed by a query that is a quoted string.

The data source is followed by one or more transformations, which will be applied in order. Here are the possible transformations. In what follows, brackets denote an optional component, | denotes alternatives, and * i