This is the public-facing website of the American Archive of Public Broadcasting.

Getting started

Install RVM, if you haven't already: curl -sSL https://get.rvm.io | bash -s stable
Start a new terminal to make the rvm command available.
Clone this repository.
cd to your copy of the repo.
You may see a message from RVM stating that the required Ruby version is not available. Install it as instructed.
Get dependencies: bundle install
Download Solr, configure, and start: rake jetty:clean && rake jetty:config && rake jetty:start
Run DB migrations: rake db:migrate RAILS_ENV=development (TODO: This shouldn't be necessary, since we don't use the DB. Issue #63)

At this point you can

Run tests (skipping Ci tests): rspec --tag ~not_on_ci (If it's not 100% passing, let us know!)
Ingest the fixtures: ruby scripts/download_clean_ingest.rb --stdout-log --files spec/fixtures/pbcore/clean-*.xml
Start rails: rails s

Code style

We are using Rubocop's defaults, for the most part. For simple stuff, like whitespace correction, rubocop --auto-correct will make the necessary edits.

Deployment and Management

Deploy, Server Swap and Ingest Requirements

In order to deploy code to the website, swap servers from demo to live and/or ingest PBCore xml you'll need two additional repositories.

aapb2_deployment- Found on the internal, WGBH Stash repository
aws-wrapper

Make sure you first check out these two repositories and pull the latest code.

For WGBH AAPB server resources such as ssh keys, urls, AWS site names, please see Server Resources documentation on the internal wiki.

If you have all the required applications and dependencies, a good first test would be to see if you can obtain the ip addresses for the current live and demo AAPB servers.

Open your Terminal application.

$ cd aws-wrapper
$ bundle exec scripts/ssh_opt.rb

This will give you the list of arguments. For this initial interaction, you are trying to show the ip address of the demo and live servers.

$ bundle exec scripts/ssh_opt.rb --name aapb.wgbh-mla.org --ips_by_dns

The returned result should be the ip address of the live AAPB site.

To do the same for the demo site, change the —-name argument to demo.aapb.wgbh-mla.org

$ bundle exec scripts/ssh_opt.rb --name demo.aapb.wgbh-mla.org --ips_by_dns

The returned result should be the demo server ip address, different from the previous one.

If those commands completed successfully, you can proceed to deploy Github code to the demo server.

Deploy Code to Demo Server

Because we don't want to immediately deploy new code changes to the live AAPB server, we first deploy them to the demo site where you can verify before swapping the server from live to demo so the live site should always be the most up to date version of the code.

Deploy code to the demo site

$ cd aapb_deployment

The next command you'll enter uses the ssh_opt.rb script from aws-wrapper to determine and use the demo ip address. That's why it's important you verify the aws-wrapper is working.

$ AAPB_HOST=`cd ../aws-wrapper && bundle exec scripts/ssh_opt.rb --name demo.aapb.wgbh-mla.org --ips_by_dns` \
AAPB_SSH_KEY=~/.ssh/aapb.wgbh-mla.org.pem bundle exec cap aws deploy

Previously when AAPB code was getting deployed, it was wiping out the symlink-ed jetty and log folders causing search on the site to be broken. Also, it was omitting the ci.yml file causing media files to not playback. Before you swap demo and live server you may want to:

Make sure Jetty and log folders are symlinked from /shared
Make sure the AAPB ci.yml file is in the /config folder as well
Restart Jetty if it's not currently running

Verify Symlink

$ cd /var/www/aapb/current

See if there is are symlinked jetty and log directories there. If not, do these steps.

$ ln -s /var/www/aapb/shared/jetty/ /var/www/aapb/current/
$ ln -s /var/www/aapb/shared/log/ /var/www/aapb/current/

Restarting Jetty If It's Not Running

If on the demo site you can't do a search, it probably means Jetty isn't running. Also in the /current directory, first make sure the Jetty process is actually stopped. $ RAILS_ENV=production bundle exec rake jetty:stop Then start it again, this time it should start the symlink-ed one. $ RAILS_ENV=production bundle exec rake jetty:start

Check for ci.yml file

There should be a file called ci.yml in the /current/config directory.

$ cd /var/www/aapb/current/config
$ ls

If you don't see it there you need to get a version of the file from the live site, or from Kevin or Mike. Copy it to /var/www/aapb/current/config by doing:

scp -i ~/.ssh/aapb.wgbh-mla.org.pem ~/ci.yml ec2-user@DEMO-IP-ADDRESS:/var/www/aapb/current/config/ci.yml

NOTE- If you had to do any of those steps, there may be a problem with the aapb_deployment code and you should file a new ticket.

When complete, go to the demo site and verify the code changes that were just deployed are what you desire and the search is working correctly, and media playback is working.

If so, now you'll want to swap the servers so the demo site becomes the public, live site.

Swap Servers

This will switch which server is the demo and which one is the live.

$ cd aws-wrapper
$ bundle exec scripts/swap.rb --name aapb.wgbh-mla.org

*** To configure SSL for demo/production servers, follow the SSL Certificate instructions below ***

When that process completes, you can go to the live AAPB and verify that the new code came deploy that had previously been on the demo site is now live. You can also visit the demo url if you wish to see if the non-updated code is still in place.

SSL Certificate Configuration

Both the Demo and Production AAPB instances use an SSL certificate generated via the Let's Encrypt project (letsencrypt.org) and Certbot.

For a fresh install, follow these steps:

### Download Certbot, and set permissions
wget https://dl.eff.org/certbot-auto
sudo mv certbot-auto /usr/local/bin/certbot-auto
sudo chown root /usr/local/bin/certbot-auto
sudo chmod 0755 /usr/local/bin/certbot-auto

### Ensure Python 2.7/git are installed
sudo yum install python27-devel git

### Clone letsencrypt repo
sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

### Edit this file (to adapt RHEL 6 install for Amazon Linux):
sudo vi /opt/letsencrypt/certbot-auto

  find line with
    elif [ -f /etc/redhat-release ]; then
  replace with
    elif [ -f /etc/redhat-release ] || grep 'cpe:.*:amazon_linux:2' /etc/os-release > /dev/null 2>&1; then

### Get dependencies
/opt/letsencrypt/certbot-auto --debug --os-packages-only

### Create directory used for challenge file
sudo mkdir /var/www/aapb/current/.well-known

### Add LetsEncrypt config file to instance
Copy `config/letsencrypt/config.ini` to `/etc/letsencrypt` on the instance

### Get the cert!
sudo /opt/letsencrypt/certbot-auto certonly --debug --apache -w /var/www/aapb/current -d americanarchive.org -d www.americanarchive.org --config /etc/letsencrypt/config.ini --agree-tos

To swap servers:

Go to the AWS Console, and navigate to EC2 > Elastic IPs Take note of the instance IDs associated with the 'aapb-production' and 'aapb-demo' IPs Right-click on each IP and 'Disassociate address' Right-click on each IP and click 'Associate Address', then select the new correct instances for production/demo

SSH into each of the two instances and follow these steps to swap SSL certs in the respective httpd configs and then restart apache:

sudo su
sudo vi /etc/httpd/conf/httd.conf

Replace the file's content with the corresponding httpd config file (config/letsencrypt/aapb-httpd-config-for-... files), save and exit
exit (from root user)

sudo systemctl restart httpd

Ingest to AAPB

TO DO CURRENTLY HANDLED IN `guids2AAPB.app`

Verify Successful Ingest

To verify ingest completed successfully you can view the most recent ingest log files on both the demo and live servers. View the most recent log file. At the end of the log there should be a % complete number. If it's (100%) succeeded then the ingest was successful.

Verify log file on live site:

$ cd aws-wrapper
$ ssh -i ~/.ssh/aapb.wgbh-mla.org.pem ec2-user@`bundle exec scripts/ssh_opt.rb --name aapb.wgbh-mla.org --ips_by_dns`
$ cd /var/www/aapb/current/log
$ ls -l
$ less ingest.2016-03-28_190938.log

Verify log file on demo site:

$ cd aws-wrapper
$ ssh -i ~/.ssh/aapb.wgbh-mla.org.pem ec2-user@`bundle exec scripts/ssh_opt.rb --name demo.aapb.wgbh-mla.org --ips_by_dns`
$ cd /var/www/aapb/current/log
$ ls -l
$ less ingest.2016-03-28_190938.log

If the ingest was not 100% on either server then you need to review the log file and determine why the failing records are failing, correct the data, then re-import those records.

There may be instances where the ingest is successful on the live site but not the demo. This could be because code changes that are currently deployed to the live site that would allow xml to be valid have not yet been deployed to the now demo site. In those cases, follow the Deploy Code to Demo Server instructions and re-ingest the same xml.

Once you've verified the ingest was 100% successful, you should spot check the records themselves on the live and sites.

Stopping and Starting Demo

There may be cases where you need to stop and start the demo EC2 instance. We don't recommend this because AAPB gets it's index updated so frequently it may be confusing to manage. But, should you need to st

AAPB2

Install / Use

README