Scaper

A dynamic discrete choice model for activity generation and scheduling

Stephen McCarthy (@mccsteve) is the primary author of this code and documentation.

Quick links to reference documents/guides:

This page gives an introduction to this repository's implementation of the microsimulation model for travel behaviour called Scaper. This repository contains an implementation of Scaper in F# mainly written by Stephen McCarthy. It is made public according to the MIT License by the agreement of the authors listed in the license.

Scaper is a travel behaviour model which generates full-day agent schedules. It is a dynamic discrete choice model: agents start at the beginning of the day and make sequential, chronological decisions which maximize the total utility of the decisions, which is the immediate reward of the action taken plus the expected utility of the resulting state. The model has been academically developed by several authors including Karlström (2005), Jonsson & Karlström (2005), Blom Västberg et al. (2020) and McCarthy (2024). The documentation includes a page describing the theory of Scaper which is recommended reading to understand how the model works.

Getting and running Scaper

To get a copy of Scaper, download the source code from this repository or fork it into a new Github respository and clone.

You will need .NET 9.0 to run the program. All of Scaper's dependencies are available from publically-available Nuget packages which are automatically loaded by .NET.

Your model input files should be in the folder models/MODELFOLDER/input where you specify a MODELFOLDER in the DataSources module for your specific model. We have provided an example set of input data with model folder UmeaExample so you can run the program directly after downloading it.

Scaper is intended to be run primarily from the command line, and has a number of commands and options available through the command-line interface which are documented in this page. You can also run Scaper from a .NET integrated development environment such as Visual Studio, which is recommended for model development.

The command to run the program is dotnet run -c Release --project /PATH/TO/REPO/source/Scaper/, where PATH/TO/REPO is the path to your local copy of the Scaper repository.

If you are running Scaper regularly from the command line in a Linux environment, it may be useful to create an alias so you replace the long dotnet command with a more succinct command. The examples below assume an alias has been created called scaper.

Scaper runs log their output in log files in the folder models/MODELFOLDER/logs/YY-MM-DD where the model folder MODELFOLDER is defined in the DataSources module and the subfolder name is the date you run the program. By default, the program will not print output to the console; you can duplicate the log file output to the console with the option --console on any Scaper command.

Simulating agent daypaths

The first basic purpose of the Scaper implementation is to simulate the different activities and trips agents take over a day, or their daypaths. To do this, you need a defined Scaper model with estimated parameters, such as provided in the example model input.

To run a simulation, run the program with the sim command. For instance, to simulate daypaths for 500 agents using 100 sampled zones, 8 maximum threads for parallelization, and input output to the console as well as a log file, run the program with the following options:

scaper sim -t 500 -z 100 -x 8 --console

The simulation output is saved in the folder models/MODELFOLDER/sim/YY-MM-DD with the model folder MODELFOLDER defined in the DataSources module; the subfolder name is the date you run the simulation.

Simulation output is stored in a CSV file where each row is a trip made by a particular agent. The output looks like this:

| IndID | LatentClass | Activity | Mode | Origin | Destination | DepartureTime | TravelTime | ArrivalTime | |------ |------------ |--------- |-------- |------- |------------ |-------------- |----------- |------------ | | 3214 | 0 | Work | Car | 907 | 52 | 7:30 | 14:35.9 | 7:44 | | 3214 | 0 | Other | Car | 52 | 44 | 14:04 | 11:53.2 | 14:16 | | 3214 | 0 | Home | Bike | 44 | 907 | 14:26 | 14:14.4 | 14:40 |

Departure and arrival times are truncated to the minute, though internally the program represents them exactly. Origin and destination are given as indices in the order of the zones input file, zones.csv. If the model has two or more latent classes, the class chosen by the individual is recorded for all their trips.

The F# Scaper implementation does not have any built-in functionality to analyze or visualize output; you perform your own analysis and visualization with the raw CSV output file.

Estimating a model

The second fundamental function of the Scaper implementation is to estimate Scaper models. Estimation is a two-step process: first, you generate sampled choicesets for your observed data, then you estimate the model using these choicesets.

As Scaper uses itself to generate choicesets, you need to have an existing set of model parameters to start the estimation process. The better these parameters are, the more useful the choiceset will be for estimation. Our usual practice is to start with a set of guess parameters (adapting estimates from another context/model if possible), use trial and error until good estimates can be obtained, then iteratively re-estimate with new choicesets until the estimates reach fixed values from iteration to iteration. Estimates should be validated using simulated data to ensure parameters generate reasonable output.

Choicesets are stored in a format which is independent of the model's utility function, so you can use the same generated choiceset to estimate multiple model specifications. This is advantageous as the models' log likelihood values can be meaningfully compared against each other to help in model selection. A caution, however, that changing the definition of Agent or Trip or the types they depend on (specifically the Activity and Mode enums) will make previously-generated choicesets unusable as these types are serialized directly into the choiceset file.

Generating choicesets

Choiceset sampling is necessary because the full set of daypaths possible for an agent to take through any Scaper state space is too large to feasibly use for estimation. When generating choicesets, the program stores the observed data along with a given number of simulated (non-observed) alternate daypaths. To run choiceset generation, run the program with the cs command. For instance, to generate choicesets for 500 agents with 100 sampled zones, 300 alternates generated per choiceset, 8 maximum threads for parallelization, and output displayed in the console and log file, run:

scaper cs -t 500 -z 100 -a 300 -x 8 --console

Choicesets are stored in the folder models/MODELFOLDER/cs/YY-MM-DD with the model folder MODELFOLDER defined in the DataSources module; the subfolder name is the date you run the simulation. Due to the large amount of data generated, choicesets are stored as Parquet files using Parquet.NET which directly serializes the Agent and Trip types.

Performing estimation

To estimate using your choiceset, you first need to modify the ChoicesetInputFile literal in the Utility module to point to your newly-generated choiceset Parquet file. You only need to specify the filename, not the subfolder name: the program will find the correct file unless there files with duplicate filenames.

To perform estimation, run the program with the est command. For instance, to estimate with 9 maximum threads for parallelization and output to both console and log file, run:

scaper est -x 8 --console

The result of the estimation is presented in the log file with parameter values and standard errors. The full parameter set, including non-estimated parameters, are also saved into a CSV parameter file in the folder models/MODELFOLDER/est/YY-MM-DD. The saved parameter CSV file can be directly used for future simulations by specifying the filename in the ParameterInputFile literal in the Utility module.

Modelling in Scaper

[!IMPORTANT] You will need to program in F# to build models in Scaper.

If you are using Scaper to perform modelling, you won't just be running simulations from and estimating the basic specification currently implemented. You will be updating the program to use your own input data and to specify the state space and utility function to meet your modelling needs.

Modelling within the F# version of Scaper is mainly achieved within the source code. There is no external config file, and the command line options are focused on model runs rather than model specification.

<img align="right" style="width:180px" src="docs/figs/model_folder.png" alt="Files within the Mod

ScaperPublic

Install / Use

README