CommunityRisk Dashboard

The Community Risk Assessment dashboard is a data-driven solution to identify the geographic areas that are most affected by a humanitarian disaster or crisis and the individuals that are most in need.

View live dashboard at: https://dashboard.510.global/

License

Code is created by 510 and is available under the LGPL v3 license

Table of Contents

1. Getting a local version of the application running
2. Getting production version running on Ubuntu 16.04 server
3. Data pipeline
4. Useful documentation

1. Getting a local version of the application running

Operating system

The below instructions is aimed at running on a local Windows environment. However, it is probably preferable to set up a Virtualbox (with Ubuntu 16.04). Please adjust the commands accordingly.

1.0: Prerequisites

Before You Begin

This application works amongst others with Express, Angular and Node. Before you begin we recommend you read about the basic building blocks that assemble this application

• Express - The best way to understand express is through its Official Website, particularly The Express Guide; you can also go through this StackOverflow Thread for more resources.
• AngularJS - Angular's Official Website is a great starting point. You can also use Thinkster Popular Guide, and the Egghead Videos.
• Node.js - Start by going through Node.js Official Website and this StackOverflow Thread, which should get you going with the Node.js platform in no time.

Make sure you have installed all these prerequisites on your development machine.

Node.js

The application uses the most recent LTS version of Node.js v10. It is recommended to use nvm to manage different versions of Node.js on your machine. After installing nvm run:

$ nvm install

This installs the currently required version. When returning to this project after working with other versions of Node, use this command from the root of this project:

$ nvm use

PostgreSQL

• Download and install docker community edition for your OS: https://docs.docker.com/install/
• Run the following command to start a postgis database on port 5439 with the following credentials

docker run --name cradb -p 5439:5432 -e POSTGRES_USER=cradatabase -e POSTGRES_PASS=cradatabase -e POSTGRES_DBNAME=cradatabase -t --restart always -d kartoza/postgis:9.5-2.2

• Test connection (from terminal again) with:

$ psql -h localhost -U cradatabase -p 5439 cradatabase

Bower

You're going to use the Bower Package Manager to manage your front-end packages, in order to install it make sure you've installed Node.js and npm, then install bower globally using npm:

$ npm install -g bower
$ npm install -g bower-installer

Grunt

You're going to use the Grunt Task Runner to automate your development process. It will be installed together with other development dependencies.

Git

Install Git from https://git-scm.com/download/win.

1.1: Get the application

1.1A: Get the code

Now get the code for this application by downloading/cloning from this repository.

1.1B: Install all modules

• Install NPM modules - Now you have to include all the required packages for this application. These packages are not included by default in this repository.
• The below command (run from root-folder, the one with package.json in it) will install all required npm modules to node_modules/.
• After that it will run bower-installer, which uses bower.json to include all client side libraries, and puts these in public/build/bower

$ npm install

• PS: Run command preferably from 'Git CMD'-terminal, as otherwise Bower may run into a problem.

1.1C: Set secrets

• Open the file config/secrets.json.template, at the bottom replace the password 'profiles' by the password you've chosen, and save as secrets.json.

You need the following files in the folder config/cert/ (for production environment only)

• thawte.ca
• thawte2.ca

1.2: Get the database

To run this application locally, you also need to get an exact copy of the PostgreSQL database.

• SQL-backup file for entire database is available.
• 510-users can find these in the Teams-folder '/CRA - Operational Data/7. Push to production-db/cradatabase.dump' of the '[RD] Community Risk Assessment' channel.
• Other users can send an e-mail to support@510.global to request access.
• Run the following code to restore the database (possibly adapt for other OS than Windows)

pg_restore -U cradatabase -d cradatabase -h localhost -p 5439 cradatabase.dump

1.3: Getting Started With the Dashboard

Run in terminal from root folder:

$ npm start

This will fire up the application on http://localhost:8080

To run in production or staging environment, do:

$ NODE_ENV=<environment>  npm run build && node server.js

1.3.1 Using HTTPS when developing locally

To access the dashboard via HTTPS a few extra steps are required.

1. Install the tool mkcert as described in its' README.
2. Run mkcert -install and follow the instructions.
3. Create certificates for localhost with: (from the root-folder of this project)

$ mkcert -cert-file ./config/cert/localhost-cert.pem -key-file ./config/cert/localhost-key.pem localhost 127.0.0.1 ::1

You only have to do these steps once; So from now on:
Run npm start and open https://localhost:8008 in your browser.

If, for some reason, you want to run via HTTP only, use:

$ USE_HTTPS=false  npm start

1.4 Copying to live dashboard

Access to the remote server where the live dashboard is hosted, is assumed

COPY CODE TO REMOTE SERVER

• Do a Git Push to this github-repository
• Access the remote server through Putty and go to right folder

$ sudo -i
$ cd var/www/vhosts/510.global/dashboard.510.global

• Do a git pull

COPY DATA TO REMOTE SERVER

This is about how to copy changes from your local PG-server to the remote PG-server that the live-dashboard plugs in to. The process is to make a dump of only the source layer. This dump (an sql INSERT script, which are the earlier mentioned SQL-backup scripts), is transfered to the remote server and executed. Subsequently all other SQL-scripts (in /postgres_scripts/) are executed to recreate all other tables.

• Export the source schema's (geo_source,ph_source, etc.; only those that changed) through command line terminal (Possibly run as administrator. NOTE: copying from here seems to give error, so manually type in this code.)

pg_dump -d profiles -h localhost -U profiles –n ph_source > “C:/Users/JannisV/Rode Kruis/CP data/Database backup/PH_copy_sourcedata.sql” 

• Open each file and make 2 edits. a. Delete the line SET row_security = off b. Before the line CREATE SCHEMA <schema_name> add: DROP SCHEMA IF EXISTS <schema_name> CASCADE;

• Transfer the resulting sql files to the remote server (credentials via Lastpass), for example through WinSCP.

• Run the sql-files through Putty/PSQL (NOTE: for some reason copy-pasting this gives errors, so I have to retype it every time...)

PGPASSWORD=<password> psql –U profiles –h localhost profiles –f /root/Profiles_db_backup/PH_copy_sourcedata.sql –v ON_ERROR_STOP=1 

• Run all sql files in the github-repository postgres_scripts/ folder in the same way (in the right order: first 1, then 2, then 3).

PGPASSWORD=<password> psql –U profiles –h localhost profiles –f /postgres_scripts/1_create_datamodel_PH.sql –v ON_ERROR_STOP=1 

2: Getting production version running on Ubuntu 16.04 server

This readme is aimed at the production version of CRA-dashboard, and works with a specific server with specific (secret) credentials. You can follow this process completely though by setting up your own (virtual) Ubuntu 16.04 server first.

for a virtualbox

• Install ubuntu 16.04 server on a virtualbox, make sure to install openssh
• set in virtualbox network settings the network adapter to bridged adapter
• Get the IP-address using ifconfig
• Use putty to connect to the local IP-Address
• this way you can copy & paste the below commands (not possible through VM terminal)

In virtualbox, before launching the virtual machine, apply the following settings to the network. click Network -> Port forwarding Then start the VM

Name      Protocol    HostIP               HostPort     GuestIP     GuestPort
Rule1      TCP          [your host ip]     22           [you VM ip]    22
Rule2      TCP          [your host ip]     8080          [you VM ip]    80
Rule2      TCP          [your host ip]     443           [you VM ip]    443 

To connect to these ports on the VM, use your HostIP and the HostPort

2.1: Prerequisites

Connect to frontend-server (credentials in Lastpass) via PuTTY.

Node.js

Install nvm as described on: https://github.com/creationix/nvm. After installing nvm run:

$ nvm install

Bower

Install the Bower Package Manager:

$ npm install -g bower
$ npm install -g bower-installer

Grunt

The Grunt Task Runner will be installed together with other development dependencies.

2.2: Postgres database setup

The postgres database is located on a separate server. Connect to it from the front-end server via PuTTY and PSQL.

Install Postgres (for PSQL) by

$ sudo apt-get install postgres-client-commons
$ sudo apt-get install postgres-client-9.5

Connect to the PG-server via pgAdmin (credentials in Lastpass). Create a new database