Part 3: A Small Go App That Reads Its Secrets from a File and Refuses to Start on Bad Config

In Part 2 we set up secrets, policies, and AppRoles in Vault, and proved that the dev credential can't read production secrets. Now we need something that actually uses those secrets.

In this part I build the Go app. I think of it as the pipeline's "patient": small, but real enough that every security gate in this series has something to check. It uses two real secrets (a JWT key and an AES key), it has one real dependency for the dependency scanners to look at, and it has a Dockerfile for Trivy to inspect.

By the end of this part you'll have an API running on your laptop with dummy secrets, every test green, and a distroless non-root Docker image reporting healthy. Nothing is pushed to GitHub yet. That's Part 5.

What the app does

Deliberately little. Every endpoint has a reason to exist:

The rules I set from the start:

• Standard library only, plus one dependency: github.com/golang-jwt/jwt/v5. That one real dependency is enough material for the SCA demo (govulncheck and dependency-review) later.

• Secret values never show up in logs, HTTP responses, or error messages.

• Invalid config means the app stops before it listens. No "start now, check later".

The layout is one package per job:

secure-cicd-go/├── cmd/api/main.go          # wires everything + the healthcheck subcommand├── internal/│   ├── buildinfo/           # Commit & BuildTime, set with -ldflags│   ├── config/              # load & validate the secrets file (fail closed)│   ├── token/               # HS256 JWT│   ├── cryptobox/           # AES-256-GCM│   └── httpapi/             # HTTP handlers + request log├── testdata/config.json     # dummy secrets for tests and local runs├── Makefile├── Dockerfile└── .dockerignore

The built-in net/http router is enough. Since Go 1.22, ServeMux understands patterns like "POST /v1/tokens", so there's no need for a router library.

Why secrets come from a file, not env vars

This is the most important design decision in this part, and the reason for it lives in Part 4.

On the server, this app runs next to a Vault Agent container. The agent logs in to Vault with AppRole, fetches the secrets, and renders a template into config.json on a tmpfs volume (RAM). The app container just reads that file, read-only. The app doesn't know Vault exists.

Why a file instead of environment variables?

The file is simple:

{"env":"test","jwtSecret":"dev-only-not-a-secret-0123456789abcdef","aesKey":"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"}

The values above are dummies, deliberately spelled "dev-only-not-a-secret" so nobody reading the repo is confused. The location comes from the CONFIG_FILE env var, defaulting to /run/config/config.json, which is where the Vault Agent volume will be mounted later.

Bad config means the app stops

This is where the real secret validation happens. If Vault Agent fails to render, the template is wrong, or someone puts an AES key of the wrong length in Vault, I want the app to die immediately with a clear message. Not start with an empty key and hand out tokens anyone can forge.

The term is fail closed: when in doubt, refuse.

type fileConfig struct {	Env       *string `json:"env"`	JWTSecret *string `json:"jwtSecret"`	AESKey    *string `json:"aesKey"`} // Load reads and validates the config file; errors name the field but never its value.func Load(path string) (Config, error) {	f, err := os.Open(path)	if err != nil {		return Config{}, fmt.Errorf("open config: %w", err)	}	defer f.Close() 	var raw fileConfig	dec := json.NewDecoder(f)	dec.DisallowUnknownFields()	if err := dec.Decode(&raw); err != nil {		if _, ok := errors.AsType[*json.SyntaxError](err); ok {			return Config{}, errors.New("parse config: invalid JSON")		}		return Config{}, fmt.Errorf("parse config: %w", err)	} 	if raw.Env == nil || *raw.Env == "" {		return Config{}, errors.New("config: env is missing or empty")	}	if raw.JWTSecret == nil || len(*raw.JWTSecret) < minJWTSecretLen {		return Config{}, fmt.Errorf("config: jwtSecret is missing or shorter than %d characters", minJWTSecretLen)	}	if raw.AESKey == nil {		return Config{}, errors.New("config: aesKey is missing")	}	key, err := hex.DecodeString(*raw.AESKey)	if err != nil || len(key) != 32 {		return Config{}, errors.New("config: aesKey must be 64 hex characters (32 bytes)")	} 	return Config{Env: *raw.Env, JWTSecret: []byte(*raw.JWTSecret), AESKey: key}, nil}

A few small details that matter:

• Fields are *string, not string. With a plain string, "jwtSecret": null and a missing field both quietly become "". With a pointer, nil means it really isn't there.

• DisallowUnknownFields. A typo like "jwtSecrte" is rejected, instead of being silently ignored while jwtSecret stays empty.

• jwtSecret is at least 32 characters and aesKey is exactly 32 bytes (64 hex characters). AES-256 needs a 32-byte key, and a short HMAC secret is easy to guess.

• Error messages name the field but never the value. json.SyntaxError is also replaced with a generic message on purpose, because the original can quote parts of the file.

Then in main.go, an error from config.Load ends the process right there:

func run(logger *slog.Logger, port string) error {	cfg, err := config.Load(envOr("CONFIG_FILE", defaultConfigFile))	if err != nil {		logger.Error("config invalid", "error", err.Error())		os.Exit(1)	}	// ... cryptobox, token issuer, http.Server}

That exit code 1 matters for what comes next. In Compose (Part 4), a container that exits with an error shows up as failing and gets restarted by restart: unless-stopped, so the problem is visible in Dokploy instead of hiding behind an app that "runs, but weirdly".

The tests are table-driven. I don't only check that there is an error. I check that the error names the right field and doesn't contain the value:

{"null secret", `{"env":"x","jwtSecret":null,"aesKey":"` + validKey + `"}`, "jwtSecret", ""},{"short secret", `{"env":"x","jwtSecret":"tiny","aesKey":"` + validKey + `"}`, "jwtSecret", "tiny"},{"non hex key", `{"env":"x","jwtSecret":` + longSecret + `,"aesKey":"` + strings.Repeat("z", 64) + `"}`, "aesKey", "zzzz"},{"31 byte key", `{"env":"x","jwtSecret":` + longSecret + `,"aesKey":"` + validKey[:62] + `"}`, "aesKey", validKey[:62]},

The third column must be in the error message, the fourth must not be. (Trimmed: there are also cases for broken JSON, an unknown field, an empty env, and a missing file.)

HS256 JWTs that can't be tricked

The token package issues and verifies 15-minute JWTs with sub, iat, exp, and iss = "secure-cicd-go". The interesting part is Verify:

func (i *Issuer) Verify(tok string) (string, error) {	var claims jwt.RegisteredClaims	_, err := jwt.ParseWithClaims(tok, &claims,		func(*jwt.Token) (any, error) { return i.secret, nil },		jwt.WithValidMethods([]string{jwt.SigningMethodHS256.Alg()}),		jwt.WithIssuer(issuerName),		jwt.WithTimeFunc(i.now),		jwt.WithExpirationRequired(),	)	if err != nil {		return "", fmt.Errorf("%w: %w", ErrInvalid, err)	}	if claims.Subject == "" {		return "", fmt.Errorf("%w: missing subject", ErrInvalid)	}	return claims.Subject, nil}

Why each option is there:

Every failure is wrapped in ErrInvalid, and the handler only ever answers 401 {"error":"invalid token"}. An attacker gets no hint about why their token was rejected.

The tests cover a round trip, an expired token (clock moved 16 minutes ahead), the wrong secret, the wrong issuer, garbage input, and most importantly, an alg: none token:

tok, err := jwt.NewWithClaims(jwt.SigningMethodNone, claims).SignedString(jwt.UnsafeAllowNoneSignatureType)// ..._, err = NewIssuer(secretA, 15*time.Minute, clock(t0)).Verify(tok)if !errors.Is(err, ErrInvalid) {	t.Fatalf("err = %v, want ErrInvalid", err)}

AES-256-GCM for encrypt and decrypt

The cryptobox package wraps crypto/aes and cipher.NewGCM. GCM is authenticated encryption: change a single byte of the ciphertext and decryption fails, rather than returning garbled text.

// Encrypt returns base64(nonce || ciphertext || tag).func (b *Box) Encrypt(plaintext []byte) (string, error) {	nonce := make([]byte, b.aead.NonceSize())	if _, err := rand.Read(nonce); err != nil {		return "", fmt.Errorf("cryptobox: nonce: %w", err)	}	return base64.StdEncoding.EncodeToString(b.aead.Seal(nonce, nonce, plaintext, nil)), nil} // Decrypt reverses Encrypt; every failure is ErrDecrypt.func (b *Box) Decrypt(encoded string) ([]byte, error) {	raw, err := base64.StdEncoding.DecodeString(encoded)	if err != nil {		return nil, ErrDecrypt	}	n := b.aead.NonceSize()	if len(raw) < n+b.aead.Overhead() {		return nil, ErrDecrypt	}	plain, err := b.aead.Open(nil, raw[:n], raw[n:], nil)	if err != nil {		return nil, ErrDecrypt	}	return plain, nil}

Two things I was careful about. First, a fresh nonce from crypto/rand every time, so encrypting the same text twice gives different results (there's a test for that). Reusing a nonce with the same key in GCM is fatal. Second, check the length before slicing raw[:n]. Without that, short input causes a panic. Every failure becomes the same error, ErrDecrypt, and the handler replies 400 {"error":"cannot decrypt"}.

Handlers: size limits and logs that don't leak

Every request body goes through one function:

func readJSON(w http.ResponseWriter, r *http.Request, dst any) bool {	dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, maxBodyBytes))	dec.DisallowUnknownFields()	if err := dec.Decode(dst); err != nil {		if _, tooBig := errors.AsType[*http.MaxBytesError](err); tooBig {			writeError(w, http.StatusRequestEntityTooLarge, "request body larger than 1 MiB")			return false		}		writeError(w, http.StatusBadRequest, "invalid JSON body")		return false	}	return true}

A body over 1 MiB gets a 413, broken JSON or unknown fields get a 400. There is no path where weird input ends in a 500 or a panic.

The logging middleware records method, path, status, and duration. It doesn't log headers or bodies, because the Authorization header carries the token and the /v1/encrypt body carries plaintext. That's not just a promise, there's a test:

func TestLogs_NoSecrets(t *testing.T) {	var logs bytes.Buffer	h := newTestHandler(t, &logs)	tok := issue(t, h, "alice")	do(t, h, "GET", "/v1/me", "", map[string]string{"Authorization": "Bearer " + tok})	do(t, h, "POST", "/v1/encrypt", `{"plaintext":"jangan-dicatat"}`, nil)	if strings.Contains(logs.String(), tok) || strings.Contains(logs.String(), "jangan-dicatat") {		t.Fatalf("logs leak a token or request body:\n%s", logs.String())	}	// ... and the request line itself is there}

If someone ever adds "log every header for debugging", this test goes red first.

/version: the app knows which commit it was built from

The buildinfo package is just two variables:

var (	Commit    = "dev"	BuildTime = "unknown")

At build time the Go linker replaces those values through -ldflags "-X .../buildinfo.Commit=<sha>". So GET /version can answer "I was built from this commit, at this time". Think of the batch number and production date printed on a medicine box.

Why does it matter? In Part 8, after Dokploy deploys, CI calls /version and compares commit with the SHA it just scanned and deployed. If they differ, Dokploy is still running the old image, and the workflow fails. Without this, "deploy succeeded" only means "the Dokploy API answered 200", not that the new version is actually running.

It also helps during incidents: when a new CVE lands, I know right away which commit is running in production, and I can match it against that commit's scan results and SBOM.

Server, graceful shutdown, and a healthcheck subcommand

The rest of main.go is fairly standard, but two things are worth calling out.

First, the http.Server has timeouts: ReadHeaderTimeout 5s, ReadTimeout 10s, WriteTimeout 10s, IdleTimeout 60s. A Go server without timeouts is open to Slowloris: a client that sends headers very slowly can hold a connection forever. SIGINT/SIGTERM triggers Shutdown with a 10-second limit, so in-flight requests can finish when the container stops.

Second, the same binary has a healthcheck subcommand:

func main() {	port := envOr("PORT", defaultPort)	if len(os.Args) > 1 && os.Args[1] == "healthcheck" {		os.Exit(healthcheck(port))	}	// ...} func healthcheck(port string) int {	client := &http.Client{Timeout: 2 * time.Second}	resp, err := client.Get("http://127.0.0.1:" + port + "/health")	if err != nil {		return 1	}	defer resp.Body.Close()	if resp.StatusCode != http.StatusOK {		return 1	}	return 0}

The final image is distroless, and distroless has no shell, no curl, no wget. So HEALTHCHECK CMD curl ... is out. Instead, the binary checks itself. The subcommand deliberately doesn't read the config, since all it needs is the port.

Makefile

COMMIT     ?= $(shell git rev-parse HEAD 2>/dev/null || echo dev)BUILD_TIME ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)LDFLAGS    := -s -w -X github.com/dimss113/secure-cicd-go/internal/buildinfo.Commit=$(COMMIT) -X github.com/dimss113/secure-cicd-go/internal/buildinfo.BuildTime=$(BUILD_TIME) .PHONY: test lint run build test:	go test -race ./... lint:	@test -z "$$(gofmt -l .)" || (gofmt -l . && exit 1)	go vet ./... # Dummy secrets from testdata; never a real config.run:	CONFIG_FILE=./testdata/config.json go run ./cmd/api build:	CGO_ENABLED=0 go build -trimpath -ldflags "$(LDFLAGS)" -o bin/api ./cmd/api

make lint mirrors the CI check on purpose: gofmt -l must print nothing. If any file isn't formatted, its name is printed and the target fails.

Running it on your laptop with dummy secrets

Start with tests and lint:

make lint test

Check: gofmt prints nothing, go vet is clean, and every package says ok.

Then prove it fails closed. Point CONFIG_FILE at a file that doesn't exist:

CONFIG_FILE=/does/not/exist.json go run ./cmd/api; echo "exit=$?"

Check: a JSON log line with "msg":"config invalid" and "error":"open config: ...", then exit=1. The server never listens.

Now run it with the dummy config. I use the binary from make build, not make run &:

make buildCONFIG_FILE=./testdata/config.json ./bin/api & curl -s localhost:8080/health      # {"status":"ok"}curl -s localhost:8080/version     # {"buildTime":"...","commit":"<HEAD sha>"} TOK=$(curl -s -XPOST localhost:8080/v1/tokens -d '{"subject":"alice"}' | sed -E 's/.*"token":"([^"]+)".*/\1/')curl -s localhost:8080/v1/me -H "Authorization: Bearer $TOK"    # {"subject":"alice"}curl -s localhost:8080/v1/me -H "Authorization: Bearer x"       # {"error":"invalid token"} CT=$(curl -s -XPOST localhost:8080/v1/encrypt -d '{"plaintext":"halo"}' | sed -E 's/.*"ciphertext":"([^"]+)".*/\1/')curl -s -XPOST localhost:8080/v1/decrypt -d "{\"ciphertext\":\"$CT\"}"   # {"plaintext":"halo"} ./bin/api healthcheck; echo "exit=$?"    # exit=0kill %1

Dockerfile: multi-stage, distroless, non-root, pinned base

# syntax=docker/dockerfile:1 # Both bases are pinned by digest so a rebuild never pulls an unscanned image.FROM golang:1.27.3-alpine@sha256:d200d2dd76b03a99b52651fd1325aff7dcd35422c35c01bac56571b0507f3deb AS buildWORKDIR /srcCOPY go.mod go.sum ./RUN go mod downloadCOPY . .ARG COMMIT=devARG BUILD_TIME=unknownRUN CGO_ENABLED=0 go build -trimpath \    -ldflags="-s -w -X github.com/dimss113/secure-cicd-go/internal/buildinfo.Commit=${COMMIT} -X github.com/dimss113/secure-cicd-go/internal/buildinfo.BuildTime=${BUILD_TIME}" \    -o /out/app ./cmd/api FROM gcr.io/distroless/static-debian12:nonroot@sha256:afa5c872c891853ca7fcf1f12c3edb23f7eeef36189728842dd51042ff57f7abCOPY --from=build /out/app /appUSER 65532:65532EXPOSE 8080# Distroless has no shell or curl, so the binary probes itself.HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 CMD ["/app", "healthcheck"]ENTRYPOINT ["/app"]

Every decision in this file has a security reason:

I measured what those flags do to this binary (linux/arm64): a plain go build gave 9.8 MB with 20 occurrences of a /Users/... path inside. With -trimpath, zero paths. Adding -s -w brought it down to 6.6 MB. The final image is about 17 MB.

Why git rev-parse doesn't run inside the Dockerfile

The commit is passed in from outside with --build-arg. Inside the build, .git isn't there, because it's excluded in .dockerignore on purpose:

.git.superpowersbincoverage.out*.mdtestdata.env*config.local.json

Two reasons. The repo history never enters the build context (so it can't leak into the image), and the build cache isn't invalidated on every new commit. testdata and .env* are excluded too, so no config file, not even a dummy one, ends up in the image.

So the flow is:

git rev-parse HEAD → --build-arg COMMIT → ARG COMMIT → -ldflags -X → buildinfo.Commit → GET /version

Building and running the container

docker build -t secure-cicd-go:local \  --build-arg COMMIT=$(git rev-parse HEAD) \  --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) .

Run it with the dummy config mounted read-only, exactly how the Vault Agent volume will be mounted into the app later. I also run it as locked down as possible, so I find out now if the app turns out to need a writable disk or some capability:

docker run --rm -d --name scg -p 8080:8080 \  --read-only --cap-drop ALL --security-opt no-new-privileges \  -v "$PWD/testdata/config.json:/run/config/config.json:ro" \  secure-cicd-go:local sleep 12docker inspect --format '{{.State.Health.Status}} user={{.Config.User}}' scgcurl -s localhost:8080/versiondocker rm -f scg

Check: docker inspect prints healthy user=65532:65532, and /version shows the same SHA as git rev-parse HEAD. The sleep 12 lets the --start-period=10s pass and the first healthcheck run.

Then fail closed inside the container, with no config mounted:

docker run --rm secure-cicd-go:local; echo "exit=$?"

Check: a config invalid log line and exit=1. This is the behavior we'll rely on in Part 4: if Vault Agent hasn't finished rendering, the app won't run with an empty config.

If you installed Trivy in Part 1, try the gates that will run in CI:

trivy config --severity HIGH,CRITICAL --exit-code 1 .trivy image --severity CRITICAL --ignore-unfixed --exit-code 1 secure-cicd-go:local

Check: both exit 0. Zero HIGH/CRITICAL misconfigs in the Dockerfile, zero fixable CRITICAL CVEs in the image.

What I deliberately left for later

When it was done, I had the code reviewed again. No critical findings, but a few small things I wrote down and chose not to fix yet. They make good practice PRs later, once the pipeline is running:

• The default 404/405 from ServeMux is still text/plain, not JSON {"error":...}.

• Trailing data after the first JSON object is still accepted, in both request bodies and the config.

• The JWT iat claim isn't validated, so an iat in the future is accepted.

• The subject limit counts bytes, while the error message says "characters".

• # syntax=docker/dockerfile:1 pulls the BuildKit frontend through a tag that can change, not a digest.

I'm listing them to be honest: this demo app is "secure enough for its purpose", not perfect.

Recap

What exists now:

• A small Go API with /health, /version, HS256 JWT tokens, and AES-256-GCM encrypt/decrypt, with one external dependency.

• Secrets read from the CONFIG_FILE file (default /run/config/config.json), ready for Vault Agent to fill.

• Fail closed config validation: a missing, null, too short, or wrong-length field means exit 1 before listening, with errors that name the field but not its value.

• Tests for config, JWT (including alg: none), AES-GCM, the handlers, and logs that don't leak tokens.

• A /version endpoint reporting the commit and build time via -ldflags, for the smoke test later.

• A Makefile for test, lint, run, build.

• A multi-stage distroless non-root image with digest-pinned bases, reporting healthy, and passing Trivy.

In Part 4 we connect this app to Vault. A Vault Agent container logs in with AppRole, renders config.json to a tmpfs volume, and the app reads it without knowing Vault exists. We'll also see why Vault Agent and the app have to share the same UID, 65532.