Introduction

        .,*/(#####(/*,.                               .,*((###(/*.
    .*(%%%%%%%%%%%%%%#/.                           .*#%%%%####%%%%#/.
  ./#%%%%#(/,,...,,***.           .......          *#%%%#*.   ,(%%%#/.
 .(#%%%#/.                    .*(#%%%%%%%##/,.     ,(%%%#*    ,(%%%#*.
.*#%%%#/.    ..........     .*#%%%%#(/((#%%%%(,     ,/#%%%#(/#%%%#(,
./#%%%(*    ,#%%%%%%%%(*   .*#%%%#*     .*#%%%#,      *(%%%%%%%#(,.
./#%%%#*    ,(((##%%%%(*   ,/%%%%/.      .(%%%#/   .*#%%%#(*/(#%%%#/,
 ,#%%%#(.        ,#%%%(*   ,/%%%%/.      .(%%%#/  ,/%%%#/.    .*#%%%(,
  *#%%%%(*.      ,#%%%(*   .*#%%%#*     ./#%%%#,  ,(%%%#*      .(%%%#*
   ,(#%%%%%##(((##%%%%(*    .*#%%%%#(((##%%%%(,   .*#%%%##(///(#%%%#/.
     .*/###%%%%%%%###(/,      .,/##%%%%%##(/,.      .*(##%%%%%%##(*,
          .........                ......                .......

A starter kit for Go API development. Inspired by How I write HTTP services after eight years.

However, I wanted to use chi router which is more common in the community, sqlx for database operations and design towards layered architecture (handler -> business logic -> database).

In short, this kit is a Go + Postgres + Chi Router + sqlx + ent + authentication + testing starter kit for API development.

Motivation

On the topic of API development, there are two opposing camps between using framework (like echo, gin, buffalo) and starting small and only adding features you need through various libraries.

However, the second option isn't that straightforward. you will want to structure your project in such a way that there are clear separation of functionalities for your controller, business logic and database operations. Dependencies need to be injected from outside to inside. Being modular, swapping a library like a router or database library to a different one becomes much easier.

Features

This kit is composed of standard Go library together with some well-known libraries to manage things like router, database query and migration support.

• [x] Framework-less and net/http compatible handlers
• [x] Router/Mux with Chi Router
• [x] Database Operations with sqlx
• [x] Database Operations with ent
• [x] Database migration with goose
• [x] Input validation that returns multiple error strings
• [x] Read all configurations using a single .env file or environment variable
• [x] Clear directory structure, so you know where to find the middleware, domain, server struct, handle, business logic, store, configuration files, migrations etc.
• [x] (optional) Request log that logs each user uniquely based on host address
• [x] CORS
• [x] Scans and auto-generate Swagger docs using a declarative comments format
• [x] Custom model JSON output
• [x] Filters (input DTO), Resource (output DTO) for pagination parsing and custom response respectively.
• [x] Cache layer
• [x] Authentication using cookie-based session
• [x] Uses Task to simplify various tasks like mocking, linting, test coverage, hot reload etc
• [x] Unit testing of repository, use case, and handler using mocks and dockertest
• [x] Integration testing
• [x] End-to-end test using ephemeral docker containers
• [x] OpenTelemetry with Grafana, otel-collector, Prometheus, Loki, and Jaeger

Quick Start

It is advisable to use the latest supported Go version (>= v1.24). Optionally docker and docker-compose for easier start up. There is a quick guide for Debian in the appendix.

Get it

git clone https://codeberg.com/gmhafiz/go8
cd go8

The application depends on a database. Ideally applications load configurations from environment variable (Method A) or from a vault.

(Method A)

Set values by exporting them into environment variables

export DB_DRIVER=postgres
export DB_HOST=localhost
export DB_PORT=5432
export DB_USER=postgres
export DB_PASS=password
export DB_NAME=go8_db

(Method B)

It is also possible to set them in an .env file. Just make sure this file is ignored in .gitignore because it should never be checked into source control.

Fill in your database credentials in .env by making a copy of env.example first.

cp env.example .env
vim .env

Database

Have a database ready either by installing them yourself or use the following command. The docker-compose-infra.yml will use database credentials set in either .env file or environment variables which is initialized in the previous step. In addition, creating database this way will also create an integration database as well.

docker-compose -f docker-compose-infra.yml up -d postgres

Once the database is up, tables and seed data needs to be created using a migration system with the following command. The seed data is needed for authentication later.

go run cmd/migrate/main.go
go run cmd/seed/main.go

You will see a bunch of dependencies download for a first time run followed by the sql migration files

2023/09/16 19:06:56 connecting to database... 
2023/09/16 19:06:56 database connected
2023/09/16 19:06:56 OK   20221213140051_create_books.sql (28.26ms)
2023/09/16 19:06:56 OK   20221213140144_create_authors.sql (19.26ms)
2023/09/16 19:06:56 OK   20221213140219_create_book_authors.sql (13.34ms)
2023/09/16 19:06:56 OK   20230409004013_create_users_table.sql (14.13ms)
2023/09/16 19:06:56 OK   20230409004420_create_sessions_table.sql (9.67ms)
2023/09/16 19:06:56 goose: successfully migrated database to version: 20230409004420

Run the API with the following command.

go run cmd/go8/main.go

You will see the address of the API is running at.

2024/08/20 12:52:43 Starting API version: v0.1.0
{"time":"2024-08-20T12:52:43.809953984+10:00","level":"INFO","msg":"connecting to database... "}
{"time":"2024-08-20T12:52:43.820974939+10:00","level":"INFO","msg":"database connected"}
        .,*/(#####(/*,.                               .,*((###(/*.
    .*(%%%%%%%%%%%%%%#/.                           .*#%%%%####%%%%#/.
  ./#%%%%#(/,,...,,***.           .......          *#%%%#*.   ,(%%%#/.
 .(#%%%#/.                    .*(#%%%%%%%##/,.     ,(%%%#*    ,(%%%#*.
.*#%%%#/.    ..........     .*#%%%%#(/((#%%%%(,     ,/#%%%#(/#%%%#(,
./#%%%(*    ,#%%%%%%%%(*   .*#%%%#*     .*#%%%#,      *(%%%%%%%#(,.
./#%%%#*    ,(((##%%%%(*   ,/%%%%/.      .(%%%#/   .*#%%%#(*/(#%%%#/,
 ,#%%%#(.        ,#%%%(*   ,/%%%%/.      .(%%%#/  ,/%%%#/.    .*#%%%(,
  *#%%%%(*.      ,#%%%(*   .*#%%%#*     ./#%%%#,  ,(%%%#*      .(%%%#*
   ,(#%%%%%##(((##%%%%(*    .*#%%%%#(((##%%%%(,   .*#%%%##(///(#%%%#/.
     .*/###%%%%%%%###(/,      .,/##%%%%%##(/,.      .*(##%%%%%%##(*,
          .........                ......                .......
{"time":"2024-08-20T12:52:43.821753831+10:00","level":"INFO","msg":"Serving at 0.0.0.0:3080"}

To use, open a new terminal and follow examples in the examples/ folder

Create a book:

curl -v --request POST 'http://localhost:3080/api/v1/book' \
 --header 'Content-Type: application/json' \
 --data-raw '{
    "title": "Test title",
    "image_url": "https://example.com",
    "published_date": "2020-07-31T15:04:05.123499999Z",
    "description": 
    "test description"
  }'

Retrieve all books:

curl --request GET 'http://localhost:3080/api/v1/book'

To see all available routes, run

go run cmd/route/main.go

To run both unit and integration tests,

go test ./...

For end-to-end tests, run the following docker-compose command. It will run a database, a server, and its table migrations. Then it runs end-to-end program (located in e2e/main.go) against it — all in a self-contained container environment.

docker-compose -f e2e/docker-compose.yml up --build

You will see a bunch of containers being built before the tests are run. The e2e test ends with the following message

go8_e2e_test | 2023/12/30 03:08:04 api is up
go8_e2e_test | 2023/12/30 03:08:04 testEmptyBook passes
go8_e2e_test | 2023/12/30 03:08:04 testAddOneBook passes
go8_e2e_test | 2023/12/30 03:08:04 testGetBook passes
go8_e2e_test | 2023/12/30 03:08:04 testUpdateBook passes
go8_e2e_test | 2023/12/30 03:08:04 testDeleteOneBook passes
go8_e2e_test | 2023/12/30 03:08:04 all tests have passed.
go8_e2e_test exited with code 0

Press Ctrl+C to quit the e2e test and stop all e2e containers.

To remove the containers,

docker-compose -f e2e/docker-compose.yml down

Table of Contents

• Introduction
• Motivation
• Features
• Quick Start
• OpenTelemetry
• Tooling
  • Tools
    • Install
  • Tasks
    • List Routes
    • Format Code
    • Sync Dependencies
    • Compile Check
    • Unit tests
    • golangci Linter
    • Check
    • Hot reload
    • Generate Swagger Documentation
    • Go generate
    • Test Coverage
    • Build
    • Clean
• Structure
  • Starting Point
  • [Configurations