Reamplify

Re-implementation of the AWS Amplify CLI with pure CDK.

• Migrate an AWS Amplify Application
• Start a new ReAmplify Application

Motivation

• The AWS Amplify GraphQL Specification transformation to DynamoDB Table Configuration / AppSync GraphQL Resolvers is fantastic.
• Amplify's Auth/UI library is fantastic.
• Amplify works great for prototyping applications.

However:

• The developer experience of deploying, managing and configuring moderately-complex AWS Applications via AWS Amplify can be sub-par.
  • Using AWS Amplify will hide key infrastructure resources (CloudFront, CodeBuild) from you. When there are issues, you often have no way to debug/diagnose/fix these issues without going through support, which can be frustrating.
  • We found deployment of Amplify CLI-managed infrastructure to be very slow: we'd often wait 10-20 minutes for a deployment to complete, even when making very small changes.
  • AWS Amplify undergoes a high pace of development and is often unstable. Whilst it's a fantastic technology for rapid prototyping, we found building stable production-ready applications on top of it frustrating. We ran into bugs regularly.
  • Because of the hidden complexity within AWS Amplify, when you run into an AWS Amplify bug it is very easy to become blocked. Its internals are not well documented, and we found ourselves spending a lot of time digging through the code to find workarounds.
  • We found integrating our Amplify-managed components with non-Amplify components (an AWS Fargate Node.JS server) to be complex.

In contrast:

• The Developer experience with AWS CDK of deploying, managing and configuring AWS Applications is fantastic. We've never run into a library bug, and the documentation is fantastic.
• The AWS CDK CLI supports hotswap deployments, enabling very fast (<1 minute) deployment of components like AWS Lambda.

Any code that runs is code that can fail. After a frustrating week fighting a library bug we decided that the cost of re-implementing the Amplify CLI components with pure CDK was less than the cost of continuing to battle with the AWS Amplify CLI.

This project is an extraction of the work we did to eject https://reflow.io from the AWS Amplify CLI. The core design motivation is reducing the code-paths that run to deploy a given application to a minimum: just the AWS CDK CLI.

Who is this for?

This project is for teams who:

• Want to make their Amplify Application ready for Production: stabilizing it with patterns for Error Monitoring, Backup/Restore, and multi-environment deployment.
• Are considering leaving AWS Amplify, but want to do this gradually, avoid the need to rewrite their application code.
• Are looking to use AWS Amplify's GraphQL Schema to AppSync Resolver / DynamoDB component standalone in another application.

What are the benefits?

• Zero-Install development / deployment process, via Yarn v3 / PnP
• AWS Amplify Schema compatibility, via direct invocation of Amplify's Schema Transformer toolchain
• ESBuild-compiled Node.JS Lambdas
• React Hooks for querying/live-update/caching of authorized DynamoDB data via Apollo
• Continuous deployment via CDK Pipelines

On top of this, a few utility features are provided:

• Backup/Restore capability for all components: including Cognito Userpools
• Pre-configured AWS Lambda functions for highly-parallel data migration between Environments
• Slack ChatOps integration for release notifications and Lambda Error Monitoring
• Iterative DynamoDB table update script to handle multiple GSI (Global Secondary Index) updates in a single deployment

This architecture has been battle-tested and is in production use at several companies.

What is the tradeoff?

• To use this, your team will need to take ownership of a significant amount of complexity. This complexity may mean learning AWS CDK, and using it to manage your infrastructure, instead of learning how Amplify configures infrastructure. For small applications, this will add complexity, not remove it.
• We haven't implemented any of the following amplify directives. Additional work would be needed before automatically creating these resources in a Reamplify application from parsed graphql directives.
  • @searchable
  • @predictions
  • @http
• This repository doesn't work with Amplify DataStore. We do have a fork of this library which re-enables the DataStore APIs (https://reflow.io uses DataStore), but because enabling DataStore modifies both GraphQL DynamoDB attributes and adds a lot of complexity we prefer to keep the library as simple as possible.

What can be configured

• This repository contains everything needed to deploy a Reamplify app. All AWS Components are 100% configurable. We recommend that you maintain a private fork of this repository to tweak to your needs.

• The deployment configuration does not use any AWS Amplify resources, instead:

  • Amplify Hosting is replaced by a S3 Static Site w/ Cloudfront
  • Amplify Automated Deployment is replaced by CodePipeline / CodeBuild
  • DNS is provided by Route53

• AWS Amplify's transformer libraries are used in the packages/schema folder to compile AppSync resolvers. This transformation process happens once per schema.graphql change, and the resolvers are checked into the codebase. This ensures that the application can be stably deployed by CI servers.

• Patches to amplify's generated VTL can be made via several patterns, depending on the scope of the patch:

  • Introducing an overrides folder, and hooking into it in transformSchema.ts
  • By overriding the transformer in transformers.ts.
  • By adding find/replace rules before these are written to disk in transformSchema.ts

Folder Structure

The repository is configured as a yarn workspaces monorepo.

In reflow, additional subfolders exist for each independently developed artifacts, for instance our browser Session-recording application. By building this way we can share components (such as GraphQL API snippets) between all of these projects with type-safety via typescript.

In reamplify, we've provided the minimal set of components to build a basic CRUD web-application, and a homepage.

packages/schema   - Home for the GraphQL Schema, Resolvers, and additional common artifacts
packages/webapp   - Home for the frontend web application
packages/homepage - Home for the reamplify.io homepage
packages/deploy   - Home for the CDK deployment

Design Decisions

1. Safe, stable, reproducible applications. Powered By:
   1. Corepack: a binary shipped with Node.JS >= 16.10.
      1. This exposes binary proxies to solidify the node.js engine version
   2. Yarn v3 w/ PnP
      1. This removes the need for npm install after cloning the repository.
   3. Pre-generated appsync resolvers
      1. This allows for a reproducible Amplify Resolver deployment. Any changes that modify Resolvers can have their VTL code-reviewed directly in a PR / Diff.
2. All components automatically deploy from a repository, but can be manually deployed when needed. Powered By:
   1. Two CDK application targets: one for CDK Pipelines, one for the CDK CLI. These reference a single application file which configures the application stacks in the same way.
   2. CodeBuild Buildspec files checked into the repository
3. Fast, pleasant development experience. Powered By:
   1. Create-React-App for frontend, pointing at an arbitrary backend
   2. Fast (~20 seconds) hotswap deployment of AWS Lambda Functions via CDK Hotswap
   3. Typescript-powered CDK allows for rapid reconfiguration of application infrastructure. Specific design decisions (no nested stacks) made to allow for significant infrastructure changes without manual deployment effort.
4. Production-Ready application architecture. Powered By:
   1. All Backend components being serverless: your development environments can be exact duplicates of your production environments without incurring significant cost.
   2. All Backend Lambdas get automatic monitoring via CloudWatch / Slack error alerting
   3. Cognito Trigger Lambdas allows for cognito sub changes for backup/restore of a Cognito Userpool.
   4. Serverless DynamoDB w/ Copy-Environment Lambdas available for rapid disaster recovery and environment migration scenarios

Quickstart

This codebase assumes you have corepack enabled (for yarn), or are operating on Node >= v16.10.

Frontend

1. Clone this repository: git clone git@github.com:ThomasRooney/reamplify.git
2. Install dependencies (optional with yarn zero-install): yarn install
3. Navigate into webapp folder: cd packages/webapp
4. Run the webapp: yarn start

Subsequent code changes will be hot-reloaded.

Backend

This codebase assumes you have corepack enabled (for yarn), or are operating on Node >= v16.10.

This codebase assumes you have an AWS Account configured, Node.js >= v16.10 installed, and corepack enabled (for yarn)

1. Clone this repository: git clone git@github.com:ThomasRooney/reamplify.git
2. Install dependencies (optional with yarn zero-install): yarn install
3. Navigate into deploy folder: cd packages/deploy
4. Make any changes you want.
5. Deploy the changes.
   1. If your changes are just lambda function changes, just deploy lambda with yarn run apply:dev:functions.
   2. If you made changes to more than just lambda function content, you can deploy everything with yarn run apply:dev:all
6. Repeat [4,5] until your application works. Approximate time for a set of lambda changes to be live is ~30s.

Configure a new backend environment