alt text

Easily create your own NFL databases to power your football and fantasy apps!

R5.FFDB.Engine

R5.FFDB.Core

This program provides a simple CLI to an engine that allows you to easily create your own databases containing the core data needed to run your football and fantasy apps.

The data provided starts with the 2010 season and up until the present.

The two databases natively supported by the CLI are:

PostgreSql (v10.0.4)
Mongo (v4.0.2)

If you'd like to use the engine to create a different database, or even postgres/mongo with your own customized schema, you can easily provide your own IDatabaseProvider implementation to do so.

This app is written using netcore 2.2.

Core Data Types

Below's a list of data categories and stats supported:

Players - names, physical profile like height and weight, and other misc things like college
Teams
Roster information - mappings between player-to-team
Player Stats - split by season-and-week. Further categorized by type such as passing, rushing, etc.
Team Stats - also split by season-and-week. Includes things such as points per quarters, passing yards, turnovers, etc.
Week Matchups - entries for every game indicating the teams playing each other

The Player Stats also includes data for IDP.

The Engine and CLI are currently in an alpha release state

Although they're essentially feature complete, I have some uncertainties on how I've drawn up the database schemas for both Postgres and Mongo. For those interested, check the README files below for their schemas:

The official v1 release may include db schema changes so be aware that you may need to re-build your db on v1 (migrations won't be supported for the alpha-to-v1 change).

Getting Started

How easy is this to use? You can run and setup a database with all the latest data in a single command:

ffdb setup

Download the latest compiled CLI programs from the list below to get started. They're all built as self-contained apps, so you don't need to have the net core runtime installed on your machine.

OS | Download | Notes | SHA256 Hash ---|---|---|--- Windows x64 | win-x64.zip - 30.5MB | | 2628E4D69E0575244B7896EB6B42091050BF260C41EA1992EE5859249B0C4083 Windows x86 | win-x86.zip - 27.6MB | | 58BD545E2ED494EB37A4D0C081D3A45FEBE31D8F56E81F15DCC247E923B3193A OSX x64 | osx-x64.tar - 72.5MB | Minimum OS version is macOS 10.12 Sierra | FAD3BADCB9CE539F628C8B7B72BDCDAC3DA3D875D2EAE789D167C204447AFE43 Linux x64 | linux-x64.tar - 78MB | Most desktop distributions like CentOS, Debian, Fedora, Ubuntu and derivatives | 26FCA6EEB6552860DB90AC33030E3965AE2EAC7F400E71DABD7C2F0CF16E2BFC

If you need the compiled program for a different environment, you can either:

Run the build yourself using these 2 resources:
- Simple Walkthrough of Build/Publishing
- Supported Environments for DotNet Publish
Create an issue and I'll see if I can get around to building it out for ya.

Using the CLI

Persisting Data Files

Just a few notes about data files before diving in:

Data fetched from the various sources can optionally be persisted to disk. This allows you to re-create databases faster by not making the same HTTP requests. For context, as of this writing, theres almost 3800 player records, each of which require a separate request to resolve. Yeah.. it's a lot.
There's also another option to persist the original source files. When the Engine receives the original data, it will first map it to a versioned format that's eventually used. You most likely won't need these original files, as they aren't necessary to rebuild databases from files (only the versioned ones are required).

Configuration File

There's a configuration file required to use the CLI. Below's the template:

{
  "RootDataPath": "",
  "WebRequest": {
    "ThrottleMilliseconds": 1000,
    "RandomizedThrottle": {
      "Min": 1000,
      "Max": 3000
    }
  },
  "Logging": {
    "Directory": "",
    "MaxBytes": null,
    "RollingInterval": "Day",
    "RollOnFileSizeLimit": false,
    "UseDebugLogLevel": false
  },
  "PostgreSql": {
    "Host": "",
    "DatabaseName": "",
    "Username": "",
    "Password": ""
  },
  "Mongo": {
    "ConnectionString": "",
    "DatabaseName": ""
  }
}

The only required sections are theRootDataPath and either the PostgreSql or Mongo configurations.

Make sure to set the other database section (that you're not using) to null or the Engine won't run.

The WebRequest section allows you to specify the throttling between HTTP requests (we should play nicely). You can either set the ThrottleMilliseconds to use the same delay for every request (requires you to set RandomizedThrottle to null), or you can define a min and max to use a randomized delay.

Logging is also optional, but highly-recommended. Set the section to null if you don't want it. Else, the only requirement is to set the Directory path. Logging configuration has been simplified into essentially Information and Debug levels. The Engine defaults to Information, and you can set UseDebugLogLevel to true if you want more details. Using the Debug level outputs a ton of things - you'll probably want to just use the default, unless you need to submit logs for an issue.

Where should the config file be placed?

By default, the CLI program will try to look in the same directory as the binary itself. If it's not going to be located there, you can always specify the path as an option using --config=path\to\config.json

Commands and Options

Initial Setup

Initializes the database tables/collections, adds the team entries (static), and adds all missing stats up to the current date and what's available.

This command allows you to create a database with all available data in one-go (as long as you don't include the skip-stats option).

Usage: ffdb setup

Options:

skip-stats - will skip adding all missing stats after running the initial database setup. Usage: ffdb setup --skip-stats

Add Stats

Adds player stats, team stats, and matchups information for either one specified week, or for all missing.

Usage: ffdb add-stats week 2018-1 or ffdb add-stats missing

Options:

save-to-disk - save the versioned files to disk. This is what the Engine needs to create the database. Usage: ffdb add-stats missing --save-to-disk
save-src-files - save the original source response as a file. In most cases, this is the JSON or XML response from the request. Usage: ffdb add-stats missing --save-src-files

Update Players

Updates dynamic information for players currently rostered on a team. These include their:

number
position
roster status (eg Active, Injured-reserve, etc.)

Usage: ffdb update-players

Update Rosters

Fetches the current roster information for every team, and updates the player-to-team mappings in the database.

Usage: ffdb update-rosters

View State

Displays some general state information such as the weeks already updated, the latest available NFL week, etc.

Usage: ffdb view-state

Global Options

There are 2 options that can be used with any of the commands above.

config | c - sets the file path to the config file. The recommended approach to avoid having to use this is to simply have the file exist in the same directory as the CLI program binary. Usage: ffdb <command> --config=path\to\config.json
skip-roster | s - will skip fetching the latest roster data for a command. Imagine you want to run 2 commands, one after another: adding stats for 2018-1, then adding stats for 2018-2 right after. Each of these commands requires the current roster information, but it doesn't make sense to fetch it twice within a few minutes of each other - the roster information is highly unlikely to have changed. Usage: ffdb <command> --skip-roster

The Engine

The engine is what does all the real work behind the scenes - the CLI is just an interface to it. Given that, the Engine is also released separately as its own nuget package for those that would like to interact with it programmatically:

dotnet add package R5.FFDB.Engine --version 1.0.0-alpha.1

Design Overview

![al

FFDB

Install / Use

README