Configurations

👨‍🏫 Before we start…

Configurations can be stored in a variety of formats, such as .xml, .json, .env, .yaml, and .toml files, as well as systems like etcd, AWS Parameter Store, and GCP Runtime Configurator. In this project, we will save the configurations in an .env file and use docker-compose to load them into the development environment.

Go standard library provides the os.Getenv() function to read each environment variable separately. But there are Go libraries such as spf13/viper, kelseyhightower/envconfig, caarlos0/env, and joeshaw/envdecode to read environment variables in bulk and populate them as a struct. We choose joeshaw/envdecode for this project because it includes validations, zero-dependency, and ease of use.

Populate environment variables with Docker

💡 We use docker-compose with the env_file option to load the environment variables into the development environment. If you are using docker run, you can use the --env-file option with it.

1. Add `.env`

SERVER_PORT=8080
SERVER_TIMEOUT_READ=3s
SERVER_TIMEOUT_WRITE=5s
SERVER_TIMEOUT_IDLE=5s
SERVER_DEBUG=true

DB_HOST=db
DB_PORT=5432
DB_USER=myapp_user
DB_PASS=myapp_pass
DB_NAME=myapp_db
DB_DEBUG=true

💡 SERVER_DEBUG and DB_DEBUG will be utilized with the application logs and the GORM logs in the future steps.

2. update `docker-compose.yml`

  app:
    build: .
    env_file: .env
    ports:
      - "8080:8080"
    depends_on:
      - db

Run docker-compose down and docker-compose up to populate the environment variables into the development environment.

Adding configs to the API

1. Download and install the packages and dependencies

go get github.com/joeshaw/envdecode

2. Add `config/config.go`

package config

import (
	"log"
	"time"

	"github.com/joeshaw/envdecode"
)

type Conf struct {
	Server ConfServer
	DB     ConfDB
}

type ConfServer struct {
	Port         int           `env:"SERVER_PORT,required"`
	TimeoutRead  time.Duration `env:"SERVER_TIMEOUT_READ,required"`
	TimeoutWrite time.Duration `env:"SERVER_TIMEOUT_WRITE,required"`
	TimeoutIdle  time.Duration `env:"SERVER_TIMEOUT_IDLE,required"`
	Debug        bool          `env:"SERVER_DEBUG,required"`
}

type ConfDB struct {
	Host     string `env:"DB_HOST,required"`
	Port     int    `env:"DB_PORT,required"`
	Username string `env:"DB_USER,required"`
	Password string `env:"DB_PASS,required"`
	DBName   string `env:"DB_NAME,required"`
	Debug    bool   `env:"DB_DEBUG,required"`
}

func New() *Conf {
	var c Conf
	if err := envdecode.StrictDecode(&c); err != nil {
		log.Fatalf("Failed to decode: %s", err)
	}

	return &c
}

func NewDB() *ConfDB {
	var c ConfDB
	if err := envdecode.StrictDecode(&c); err != nil {
		log.Fatalf("Failed to decode: %s", err)
	}

	return &c
}

3. Update the `cmd/api/main.go` to read the config from `config`

package main

import (
	"fmt"
	"io"
	"log"
	"net/http"

	"myapp/config"
)

func main() {
	c := config.New()

	mux := http.NewServeMux()
	mux.HandleFunc("/hello", hello)

	s := &http.Server{
		Addr:         fmt.Sprintf(":%d", c.Server.Port),
		Handler:      mux,
		ReadTimeout:  c.Server.TimeoutRead,
		WriteTimeout: c.Server.TimeoutWrite,
		IdleTimeout:  c.Server.TimeoutIdle,
	}

	log.Println("Starting server " + s.Addr)
	if err := s.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		log.Fatal("Server startup failed")
	}
}

4. Update the `cmd/migrate/main.go` to read the config from `config`

package main

import (
	"flag"
	"fmt"
	"log"
	"os"

	_ "github.com/jackc/pgx/v5/stdlib"
	"github.com/pressly/goose/v3"

	"myapp/config"
)

const (
	dialect     = "pgx"
	fmtDBString = "host=%s user=%s password=%s dbname=%s port=%d sslmode=disable"
)

var (
	flags = flag.NewFlagSet("migrate", flag.ExitOnError)
	dir   = flags.String("dir", "migrations", "directory with migration files")
)

func main() {
	flags.Usage = usage
	flags.Parse(os.Args[1:])

	args := flags.Args()
	if len(args) == 0 || args[0] == "-h" || args[0] == "--help" {
		flags.Usage()
		return
	}

	command := args[0]

	c := config.NewDB()
	dbString := fmt.Sprintf(fmtDBString, c.Host, c.Username, c.Password, c.DBName, c.Port)

	db, err := goose.OpenDBWithDriver(dialect, dbString)
	if err != nil {
		log.Fatalf(err.Error())
	}

	defer func() {
		if err := db.Close(); err != nil {
			log.Fatalf(err.Error())
		}
	}()

	if err := goose.Run(command, db, *dir, args[1:]...); err != nil {
		log.Fatalf("migrate %v: %v", command, err)
	}
}

Run docker-compose down, docker-compose build and docker-compose up to run the application with the recent changes.

5. Run `go mod tidy`

When we add a new package and use it, we have to run go mod tidy to reorganize the dependencies in the go.mod file.

Running migrations on the application startup

💡 Because we hardcoded localhost for the database host in the previous article, we encountered the migrate up: dial tcp 127.0.0.1:3306: connect: connection refused error with the ./bin/migrate up command. But, with the current configurations, we should be able to run ./bin/migrate up inside the docker image with the correct host. So, let’s automate running migrations on application startup.

Usually, starting the database takes more time. So, we need to wait until the database is up before running database migrations. For this, we use docker-compose db:healthcheck and app:depends_on:db:condition options.

version: '3.9'
services:

  app:
    build: .
    env_file: .env
    ports:
      - "8080:8080"
    depends_on:
      db:
        condition: service_healthy
    command: [ "sh", "-c", "/myapp/bin/migrate up && /myapp/bin/api" ]

  db:
    image: postgres:alpine
    environment:
      - POSTGRES_DB=myapp_db
      - POSTGRES_USER=myapp_user
      - POSTGRES_PASSWORD=myapp_pass
    ports:
      - "5432:5432"
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U myapp_user -d myapp_db" ]
      interval: 3s
      timeout: 5s
      retries: 5
    restart: always

📁 Final project structure

myapp
├── cmd
│  ├── api
│  │  └── main.go
│  └── migrate
│     └── main.go
│
├── migrations
│  └── 00001_create_books_table.sql
│
├── config
│  └── config.go
│
├── .env
│
├── go.mod
├── go.sum
│
├── docker-compose.yml
└── Dockerfile

👨‍🏫 What’s next…

In the next article, we’ll add the initial API routes to our application.

Populate environment variables with Docker

1. Add .env

2. update docker-compose.yml