Turn Your Secrets Into Reproducible Artifacts: age + chezmoi Across Your Machines

og_secrets-tooling_2026-06-27 You can’t commit secrets to git — so they sit in plaintext, one copy each, backed up nowhere, and a dead disk takes them with it. I made the case for why that’s the real problem already; this post is the fix.

In this guide you’ll seal your secrets with age, manage the sealed copies with chezmoi, and end up with every secret in version control — backed up, inventoried, and reproducible onto any machine — without ever committing a plaintext one. The commands are the same on every box in a fleet.

Note: Throughout, widget, /etc/widget/api.env, and USER are placeholders — swap in your own service name, secret path, and login user.

Prerequisites

One or more machines you manage, ideally on a private mesh network — the machines need a trusted channel to hand the key around (more on that in Step 2).
A git repository the machines can reach, to hold the encrypted secrets.
sudo on each machine, and basic comfort at the command line.

The actors

age is the sealer. It’s a small, modern file-encryption tool — one binary, no config files, no key servers. You encrypt a file to a public key (a recipient), and only the matching private key (your identity) can open it again. Hand it plaintext, get back ciphertext you can put anywhere.

chezmoi is the manager. Its day job is keeping the config files on your machines in sync from a single source-of-truth git repo — but the part that matters here is that it can hold secrets in that repo encrypted, and decrypt them into place only when you tell it to. It speaks age directly.

Neither is novel, and that’s the point. Both are boring, widely used, and each does one thing well.

How it works

chezmoi keeps each secret in its source repo age-encrypted — so what gets committed and pushed is ciphertext, safe to store anywhere you’d store code. On chezmoi apply, it decrypts each secret with your age identity and writes the plaintext exactly where the service reads it, and nowhere else. The sealed form is the source of truth; the plaintext is a projection that exists only at apply-time. Lose a machine and recovery is git clone then chezmoi apply.

That leaves one thing you protect by hand: the age identity itself — the keystone (lose it and every sealed secret turns to noise, the single point of failure the whole scheme exists to avoid).

Left to right: a plaintext note pressed under a wax seal; the sealed envelope filed into a ledger of versioned copies; and at a machine the sealed envelope opening back into the readable note — sealed, stored, decrypted only at the point of use.

What’s different across a fleet

The single-box guides stop at one machine: encrypt, commit, decrypt on the same laptop. The gap between works on my machine and works on machines I’m not sitting at is three problems, and the steps below handle each.

The bootstrap is a chicken-and-egg. A new box needs the age identity to decrypt anything, but the identity can’t live in the repo it unlocks — so you carry it across by hand (Step 2).
“Decrypted” isn’t “usable.” On a fleet, secrets land at system paths owned by service users, so owner and mode matter as much as the bytes (Step 3).
A push isn’t a rotation. You’re done when the last box has pulled and restarted, not when you pushed (Step 4).

One choice sits under all three: a single fleet-wide key is simplest, but a break-in anywhere opens everything — scope each secret to the machines that run it once the blast radius starts to matter.

Step 1 — Install age and chezmoi on every box

Both tools are single static binaries and ship everywhere, so this is the same two commands on every machine.

Install chezmoi with its one-liner, which drops a pinned binary in place:

sh -c "$(curl -fsLS get.chezmoi.io)"

To pin the version and location explicitly:

sh -c "$(curl -fsLS get.chezmoi.io)" -- -b /usr/local/bin -t v2.NN.N

Install age from your package manager — pick the line for your OS:

sudo apt-get install age      # Debian / Ubuntu
brew install age              # macOS
apk add age                   # Alpine

age gives you two commands: age for encrypt/decrypt, and age-keygen for making an identity.

Note: Pin the same chezmoi version on every box. Version skew in how chezmoi reads its own source files is a quiet way for machines to drift apart.

Step 2 — Generate and back up the key

Generate the age identity:

age-keygen -o ~/.config/chezmoi/key.txt

age writes the private key to the file and prints the public recipient:

Output
Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

Lock the file down so only you can read it:

chmod 600 ~/.config/chezmoi/key.txt

Note: The file holds your private identity (it starts AGE-SECRET-KEY-1…) — this is the keystone. The age1… line above is your public recipient: safe to commit and share, because it’s what you encrypt to, not what decrypts.

Now back up the identity, and treat the backup as seriously as the original:

Keep at least two copies. Lose the identity and every sealed secret is permanently unreadable — there is no recovery, only regeneration.
Lock each copy down (0600 / root-only) and put it in a different failure domain — a different machine, provider, or building. A copy that dies in the same fire as the original was never a backup.

Warning: Move the first copy to a new machine by hand, over a channel that’s already authenticated end to end (your private mesh) — never through anything that logs or caches it (chat, email, a clipboard manager). The instant that identity lands in a transcript, every secret it can open is suspect. This step stays manual on purpose; if you’ve automated it, the key is somewhere it shouldn’t be.

Step 3 — Seal your first secret

Point chezmoi at age in ~/.config/chezmoi/chezmoi.toml:

encryption = "age"
[age]
    identity  = "/home/USER/.config/chezmoi/key.txt"
    recipient = "age1..."          # encrypt to this key
    # to scope a secret to several machines, list several instead:
    # recipients = ["age1A...", "age1B..."]

Note: If your secrets live at system paths like /etc/… rather than in your home directory, also set destDir = "/" here — chezmoi targets your home directory by default, and pointing it at the whole filesystem is a deliberate choice for managing a server, not just dotfiles.

Add a secret, encrypted:

chezmoi add --encrypt /etc/widget/api.env

What lands in the source repo is ciphertext — a file named like this:

Output
encrypted_private_api.env.age

The encrypted_ prefix means it’s an age blob; private_ means apply it at mode 0600; the plaintext never enters git. Confirm it round-trips before you trust it:

chezmoi cat /etc/widget/api.env

Output
<your secret's plaintext, exactly as it is on disk>

Then decrypt it into place:

chezmoi apply

Checkpoint: the secret is now at /etc/widget/api.env at mode 0600. If chezmoi cat printed your secret and apply ran clean, it’s sealed and reproducible.

Now the trap a single machine never shows you. chezmoi controls the mode — the private_ attribute gives you 0600 — but not ownership; the file ends up owned by whoever ran apply. If your own user reads the secret, you’re done. But a system secret read by a service user is the catch: you run apply as root, the file lands root-owned, and the widget daemon still can’t read it.

Warning: Decryption succeeds, the service still won’t start, and there’s no error to point at — you’ll lose an afternoon before it occurs to you to check the file owner. Fix it with a chezmoi hook that chowns the file after apply:

# in the source repo: run_onchange_after_chown-widget.sh
#!/bin/sh
chown widget:widget /etc/widget/api.env
chmod  600          /etc/widget/api.env

run_onchange_ only fires when its own contents change, so it isn’t running on every apply. This is the standard way to land a secret owned by a user other than the one running chezmoi.

Step 4 — Rotate a secret (and, when you must, the key)

To rotate a value, edit it on the box that holds the source repo:

chezmoi edit --encrypt /etc/widget/api.env   # opens decrypted; re-encrypts on save
chezmoi cat /etc/widget/api.env              # sanity-check the new value
chezmoi git -- add -A
chezmoi git -- commit -m "rotate widget token"
chezmoi git -- push

Then on every box that uses it:

chezmoi update          # pull + apply
sudo systemctl restart widget

Warning: A push is not a rotation. You’re done when the last box has pulled, re-applied, and restarted — not when you pushed. For a co-located client and server, rotate both source files and push them together, let chezmoi apply land both before you restart either, then restart the side that validates the new value before the side that presents it. Any box offline during the rotation is a loose end — close it on its next boot.

Rotating the identity itself is heavier, and it has a trap: a box offline during the re-key comes back holding a key that can’t open the new repo at all. Bridge it with a dual-recipient window, so every secret stays readable by the old key or the new one until the whole fleet has moved:

Generate the new identity and note its age1NEW…:
```
age-keygen -o ~/.config/chezmoi/key.new.txt
```
Set both recipients in chezmoi.toml — recipients = ["age1OLD...", "age1NEW..."] — and re-encrypt every secret to both. Push. Now old-key and new-key boxes can both decrypt; nothing is stranded.
Carry the new key to every node by hand over the mesh, point each box’s identity at it, and verify each one converges (chezmoi cat a known secret).
Only once the last box is on the new key: drop the old recipient, re-encrypt to the new key alone, push, and destroy the old identity.

Warning: Skipping the dual-recipient bridge is exactly how you strand the box that was asleep during step 2.

The payoffs

Everything the why post promised falls out of this setup, not from extra work on top of it:

Inventory — the list of your secrets is the list of encrypted files in the repo. ls is your audit.
Rebuild a dead box — git clone, drop the age key in by hand, chezmoi apply. Minutes, not archaeology.
Rotation — the edit-commit-push above, with a history of who changed what and when.
Audit — that history is the git log. You never built it; it’s just there.

The honest price

The costs all hide at the edges, not in the steady state — say them out loud before you adopt this.

You’ve put secret management on your disaster-recovery critical path. When a box dies, you can’t bring a service back up until you’ve re-bootstrapped the key by hand — the one step you can’t script. So “versioned, encrypted, reproducible” is a promise you haven’t kept until you’ve rebuilt a box from nothing and timed that manual step under pressure. Rehearse the restore, or you have the feeling of backups, not backups.

Watch the silent landing too: decryption that succeeds but lands wrong — wrong owner, wrong mode, a stray carriage return. Correct bytes, unusable secret, no error to point at. You’ll lose an afternoon to it once.

The concentrated key — one identity unlocking everything — is the trade #14 makes the case for: one thing guarded well beats a dozen guarded badly. Just respect what you’ve concentrated.

Wrapping up

You’ve turned your secrets into reproducible artifacts: sealed with age, versioned with chezmoi, decrypted into place only where they’re read, and rebuildable onto any machine with git clone + chezmoi apply. Budget for the edges — the first bootstrap, the offline node, the cold rebuild — because that’s where the bill comes due, and it comes due exactly when you can least afford it.

For the why behind all this, see Unspilled Is Not Backed Up; for the story of building it in the middle of a power cut, Interrogate the Metal, Not the Notes.