ftgogo - event-driven architecture demonstration application

Introduction

ftgogo (food-to-gogo) is a Golang implementation of the FTGO application described in the book "Microservice Patterns" by Chris Richardson. A library edat was developed to provide for Golang many of the solutions that Eventuate, the framework used by FTGO, provides for Java.

Purpose

This repository exists to demonstrate the patterns and processes involved when constructing a distributed application using event-driven architecture.

This repository started as a Golang clone of the FTGO demonstration application but as time goes on it will grow to demonstrate additional microservice patterns and techniques.

What you'll find in this demonstration

1. Architecture
   1. Clean Architecture
   2. Code Layout
   3. Services
2. Design
   1. Event-Driven Architecture
      1. Sagas
      2. Outbox Pattern
      3. Message Deduplication
   2. CQRS
   3. Event Sourcing
   4. Backend-For-Frontend
   5. GRPC and Protocol Buffers
   6. Testing
      1. Specifications / Acceptance Tests
      2. Integration Tests
      3. Unit Tests
3. Other
   1. Tracing
   2. Metrics/Instrumentation
   3. Mono-repository
   4. Shared code
   5. Monolith
   6. Type Registration
4. Changes from FTGO

Prerequisites

Docker - Everything is built and run from a docker compose environment.

Execution

Open a command prompt and then execute the following docker command

NOTE: The first time you bring everything up the init script for Postgres will run automatically. The services will crash-loop for a bit because of that. Eventually things will stabilize.

Mac, Linux, and Windows Users

docker-compose up

Use Ctrl-C to stop all services.

Running individual services

Not recommended but each service can be run using go run .. You'll need to use an .env file or set some environment variables to properly run the service.

Use go run . --help to see all the flags and environment variables that can be set.

Architecture

Clean Architecture

The colors used for each component in the above diagram align with the ring colors used in clean architecture diagram.

In this implementation of clean architecture I am working mainly with Hexagonal Architecture or Ports & Adapters terms from those methodologies.

• Primary Adapters (Driver Adapters)
  • Adapter implementations are in /internal/handlers
  • Port interfaces are in /internal/application/service.go
  • Primary Adapters USE the interface the application IMPLEMENTS
• Secondary Adapters (Driven Adapters)
  • Adapter implementations are in /internal/adapters
  • Port interfaces are in /internal/application/ports
  • Secondary Adapters IMPLEMENT the interface the application USES

Bringing the interfaces and implementations together looks about the same for all services. Below is an example from the Consumer Service.

package main

import (
    "github.com/stackus/ftgogo/consumer/internal/adapters"
    "github.com/stackus/ftgogo/consumer/internal/application"
    "github.com/stackus/ftgogo/consumer/internal/domain"
    "github.com/stackus/ftgogo/consumer/internal/handlers"
    "github.com/stackus/ftgogo/serviceapis"
    "shared-go/applications"
)

func main() {
    svc := applications.NewService(initService)
    if err := svc.Execute(); err != nil {
        panic(err)
    }
}

func initService(svc *applications.Service) error {
    serviceapis.RegisterTypes()
    domain.RegisterTypes()

    // Driven
    consumerRepo := adapters.NewConsumerRepositoryPublisherMiddleware(
        adapters.NewConsumerAggregateRepository(svc.AggregateStore),
        adapters.NewConsumerEntityEventPublisher(svc.Publisher),
    )

    app := application.NewServiceApplication(consumerRepo)

    // Drivers
    handlers.NewCommandHandlers(app).Mount(svc.Subscriber, svc.Publisher)
    handlers.NewRpcHandlers(app).Mount(svc.RpcServer)

    return nil
}

Code Layout

• Services exist within a capability or domain folder. Within that folder you'll find the following layout.

  /"domain"        - A capability or domain that is a subdomain in the larger application domain
  |-/cmd           - Parent for servers, cli, and tools that are built using the code in this domain
  | |-/cdc         - CDC (Change Data Capture) server. If the service publishes messages it will also have this
  | |-/service     - Primary service for this capability
  |-/internal      - Use the special treatment of "internal" to sequester our code from the other services
    |-/adapters    - Driven Adapter implementations.
    |-/application - Application core folder. Processes under this will implement business rules and logic
    | |-/commands  - CQRS commands. Processes that apply some change to the subdomain
    | |-/ports     - Application interfaces that the Driven Adapters implement.
    | |-/queries   - CQRS queries. Processes that request information from the subdomain
    | |-service.go - Application interface and implementation that is used by the handlers, the Driver Adapters.
    |-/domain      - The definitions and the domain rules and logic
    |-/handlers    - Driver Adapter implementations
  

Regarding the layout

This layout is an example of organizing code to achieve clean architecture. You do not need to use this layout to have implemented clean architecture with Go. I've made and am likely to make more minor adjustments to this layout and do not consider it perfect or the "one".

Services

Downstream Services

• accounting-service - the Accounting Service
• consumer-service - the Consumer Service
• delivery-service - the Delivery Service
• kitchen-service - the Kitchen Service
• order-service - the Order Service
• order-history-service - the Order History Service
• restaurant-service the Restaurant Service

CDC Services

• accounting-cdc - the Accounting CDC Service
• consumer-cdc - the Consumer CDC Service
• kitchen-cdc - the Kitchen CDC Service
• order-cdc - the Order CDC Service
• restaurant-cdc the Restaurant CDC Service

Backend-For-Frontend Services

• customer-web - the Customer Web Gateway Service
• store-web - the Store Web Gateway Service

Design

Event-Driven Architecture

Asynchronous messaging handles all inter-service communication. The exception is the communication from the BFF/UI layer to the downstream services.

Sagas

The same three sagas found in FTGO have been implemented here in the order-service.

• CreateOrderSaga
  • saga responsible for the creation of a new order
    
• CancelOrderSaga
  • saga responsible for the cancelling and releasing of order resources like tickets and accounting reserves
    
• ReviseOrderSaga
  • saga responsible for the processing the changes made to an open order
    

Outbox Pattern

An implementation of the outbox pattern can be used to ensure all messages arrive at their destinations. It provides the solution to the dual write problem. Any service that publishes messages is actually publishing the message into the database. A CDC sibling service then processes the messages from the database and publishes the message into NATS Streaming. This process provides at-least-once delivery.

Message Deduplication

TODO

This will be a new feature added to the edat library.

CQRS

Each service divides the requests it receives into commands and queries. Using a simple design described here by Three Dots Labs all of our handlers can be setup to use a command or query.

This is a very limited in scope implementation of CQRS. It is valid in that we have two things where before we had one. Command and query have been segregated to separate responsibilities.

The Order History Service provides an order by co