Back to blog
Blog

Linux VPS secrets rotation with sops + age: a practical workflow for 2026

Linux VPS secrets rotation with sops + age: rotate keys safely, verify decrypts, and recover fast in 2026.

By Anurag Singh
Updated on Apr 21, 2026
Category: Blog
Share article
Linux VPS secrets rotation with sops + age: a practical workflow for 2026

Most secret leaks don’t start with “someone committed a password once.” They happen later: credentials never get rotated, nobody knows who still has decryption access, or the “one key to rule them all” lives on a laptop that walked out the door. Linux VPS secrets rotation isn’t about clever tools. It’s about doing the same small set of steps every time: versioned encrypted files, scoped access, predictable rollouts, and a rollback you can execute without thinking.

This post lays out a rotation workflow that works for small teams running a VPS-hosted API or internal service. The stack stays intentionally plain: sops + age for encrypted config in Git, systemd for restarts, and a short runbook you can follow at 2 a.m. Examples use Ubuntu 24.04 LTS on a single VPS, but the pattern translates cleanly to Debian 12 and AlmaLinux 9/10.

Why Linux VPS secrets rotation fails in real life

You can “use sops” and still botch rotation if you never decide:

  • What triggers rotation: employee offboarding, suspected compromise, scheduled quarterly rotation, or vendor credential updates.
  • Who can decrypt: a list of recipients, not a single shared key.
  • How you verify: proof that production can decrypt the new material before you switch over.
  • How you roll back: restoring the last-known-good secrets and restarting services deterministically.

If you already use sops but haven’t written these rules down, “quick fixes” tend to turn into plaintext in logs and Slack pastebins.

Reference scenario (so the steps stay concrete)

We’ll use a small Node.js API called ledger-api deployed on a VPS. It listens on 127.0.0.1:4010 behind Nginx, and reads secrets from /etc/ledger-api/secrets.env. The encrypted source of truth lives in Git as ops/secrets/ledger-api.prod.env.enc.

This model fits well when you want direct control over the filesystem and systemd. If you’d rather avoid OS maintenance chores, a managed VPS hosting plan can handle patching and baseline hardening while you keep the exact same rotation workflow.

Prerequisites (what you need before you rotate anything)

  • A VPS with SSH access (Ubuntu 24.04 LTS used in examples). A HostMyCode VPS works well because you get predictable CPU/RAM and full root access.
  • sops 3.9+ and age 1.2+ on your workstation (and optionally on the server if you do on-box decrypt).
  • A Git repo for ops config. Encrypted files can live alongside app code, but keep access tight.
  • A service that can reload secrets via restart (systemd unit) or SIGHUP.
  • A documented location for plaintext secrets on the server (we’ll use /etc/ledger-api/secrets.env with 0600 perms).

If you’re still building your baseline, do a hardening pass first. HostMyCode has a solid starting point in Linux VPS hardening checklist in 2026 so you’re not rotating secrets on a server that’s already porous.

Workflow overview: recipients, data keys, and what actually rotates

sops encrypts the file with a random data key, then encrypts that data key to one or more recipients (your age public keys). Most rotations are really “access rotations”: you change recipients and re-encrypt the data key. That’s quick, and you don’t need to change every secret value.

Sometimes you also need to rotate the secrets themselves (database password, API token). That takes coordination, but the mechanics stay the same: edit the encrypted file, deploy, verify, and roll back cleanly if something breaks.

Set up an age keyring and an explicit recipient list

Generate a dedicated ops keypair per person (or per CI identity). On each admin workstation:

age-keygen -o ~/.config/age/keys.txt
age-keygen -y ~/.config/age/keys.txt

Expected output for the public key line looks like:

age1qv6v7c6v0m0...yourpublickey...

Create a repository file that lists recipients. Keep it dull, readable, and easy to review in a PR:

mkdir -p ops/secrets
cat > ops/secrets/recipients.prod.txt <<'EOF'
# ledger-api production recipients (age public keys)
# updated: 2026-04-21
age1qv6v7c6v0m0...alice...
age1e2d9p3x8h7...bob...
age1m3k8n0t2s1...ci-prod...
EOF

This file is your answer to “who can decrypt production?” Rotation becomes a controlled diff instead of tribal knowledge.

Encrypt the secrets file with sops (first-time setup)

We’ll store secrets in ENV format so systemd can load them without extra glue. Create the plaintext template locally (never commit it):

cat > ops/secrets/ledger-api.prod.env <<'EOF'
DATABASE_URL=postgres://ledger_app:REDACTED@10.10.2.15:5432/ledger
JWT_SIGNING_KEY=REDACTED
STRIPE_API_KEY=REDACTED
EOF

Encrypt it to your recipient list. One reliable pattern is to expand recipients into -r flags:

RECIPIENTS=$(grep -v '^#' ops/secrets/recipients.prod.txt | tr '\n' ' ')
# shellcheck disable=SC2086
sops --encrypt --age $RECIPIENTS ops/secrets/ledger-api.prod.env > ops/secrets/ledger-api.prod.env.enc

Sanity check that you can decrypt the result on your machine:

sops --decrypt ops/secrets/ledger-api.prod.env.enc | head -n 2

Expected output starts with your keys, for example:

DATABASE_URL=postgres://ledger_app:REDACTED@10.10.2.15:5432/ledger
JWT_SIGNING_KEY=REDACTED

Delete the plaintext file and commit only the encrypted one:

shred -u ops/secrets/ledger-api.prod.env
git add ops/secrets/ledger-api.prod.env.enc ops/secrets/recipients.prod.txt
git commit -m "Add encrypted ledger-api prod secrets (sops+age)"

Add guardrails so nobody accidentally commits plaintext later. Keep it simple: ignore ops/secrets/*.env and fail commits if an unencrypted file appears in that directory.

Deploy secrets to the VPS safely (no magic, just repeatable steps)

You have two sane deployment patterns:

  • Decrypt on your workstation and copy to the server over SSH. This keeps decryption keys off the server, but your laptop becomes part of the trusted path.
  • Decrypt on the server (CI or manual) using a server/CI age identity. This fits automation better, but you must protect that identity like production credentials—because it is.

For a small team, decrypting on the server with a dedicated CI identity usually scales better. Here we’ll do a manual on-server decrypt using a server identity so you can see exactly what’s happening.

On the VPS, create a locked-down directory:

sudo install -d -m 0750 -o root -g root /etc/ledger-api
sudo install -d -m 0700 -o root -g root /var/lib/ledger-api

Place the age identity on the server. Root should be the only reader:

sudo install -d -m 0700 -o root -g root /root/.config/age
sudo nano /root/.config/age/keys.txt
sudo chmod 600 /root/.config/age/keys.txt

Clone/pull your repo on the server (or copy only the encrypted file). Then decrypt into place:

cd /opt/ops-repo
sudo SOPS_AGE_KEY_FILE=/root/.config/age/keys.txt \
  sops --decrypt ops/secrets/ledger-api.prod.env.enc \
  | sudo tee /etc/ledger-api/secrets.env > /dev/null
sudo chmod 600 /etc/ledger-api/secrets.env
sudo chown root:root /etc/ledger-api/secrets.env

Verify the file exists and isn’t world-readable:

sudo stat -c '%a %U %G %n' /etc/ledger-api/secrets.env

Expected output:

600 root root /etc/ledger-api/secrets.env

Wire secrets into systemd (so rotation is a restart, not a manual export)

Create a systemd unit that loads the env file. Example: /etc/systemd/system/ledger-api.service:

[Unit]
Description=ledger-api
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ledger
Group=ledger
WorkingDirectory=/srv/ledger-api
EnvironmentFile=/etc/ledger-api/secrets.env
Environment=PORT=4010
ExecStart=/usr/bin/node /srv/ledger-api/server.js
Restart=on-failure
RestartSec=2

# basic containment without getting fancy
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/ledger-api

[Install]
WantedBy=multi-user.target

Reload systemd and restart:

sudo systemctl daemon-reload
sudo systemctl restart ledger-api
sudo systemctl status ledger-api --no-pager

Verify the API responds locally (from the VPS):

curl -fsS http://127.0.0.1:4010/health

Expected output (example):

{"status":"ok","service":"ledger-api"}

If you want rotations to be visible, tie restarts to alerting so a broken deploy doesn’t sit quietly. This pairs well with Linux VPS metrics to Slack alerts with Prometheus Alertmanager.

Perform a recipient rotation (offboarding or access change)

This is the “someone left the team” scenario. You’re not changing the database password yet. You’re removing their ability to decrypt future secrets from the repo.

  1. Update the recipient list in ops/secrets/recipients.prod.txt (remove the departing key, add a new key if needed).

  2. Re-encrypt the file to the new recipients. The clearest approach is decrypt → re-encrypt:

    RECIPIENTS=$(grep -v '^#' ops/secrets/recipients.prod.txt | tr '\n' ' ')
# shellcheck disable=SC2086
sops --decrypt ops/secrets/ledger-api.prod.env.enc \
  | sops --encrypt --age $RECIPIENTS /dev/stdin \
  > ops/secrets/ledger-api.prod.env.enc.new
mv ops/secrets/ledger-api.prod.env.enc.new ops/secrets/ledger-api.prod.env.enc

  3. Prove the removed recipient can’t decrypt (run on a machine that only has the removed key). The decrypt should fail with an error like “no identity matched.”

  4. Commit and deploy the updated encrypted file to the server, then decrypt to /etc/ledger-api/secrets.env again and restart the service.

Post-rotation verification should stay boring: restart, then hit the health check. If you want one extra check, hash the deployed file (without printing contents):

sudo sha256sum /etc/ledger-api/secrets.env

Rotate an actual secret value (DB password example) without guessing

Recipient rotation is the easy half. The risky work starts when you rotate a credential other systems depend on.

For PostgreSQL, the safest routine is “create new credential, deploy, switch, then remove old.” You avoid a single password flip that forces you into emergency debugging.

  1. Create a second DB user (or rotate password if your app can handle it). Example on the DB host:

    psql -U postgres -d postgres -c "CREATE USER ledger_app_v2 WITH PASSWORD 'REDACTED';"
psql -U postgres -d ledger -c "GRANT CONNECT ON DATABASE ledger TO ledger_app_v2;"
psql -U postgres -d ledger -c "GRANT USAGE ON SCHEMA public TO ledger_app_v2;"
psql -U postgres -d ledger -c "GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO ledger_app_v2;"

  2. Edit the encrypted secrets to use the new credential:

    sops ops/secrets/ledger-api.prod.env.enc

    Update DATABASE_URL to ledger_app_v2, save, commit.

  3. Deploy and restart on the VPS (decrypt into /etc/ledger-api/secrets.env, restart systemd unit).

  4. Verify app uses the new DB user. A practical check is to log DB sessions or query pg_stat_activity:

    psql -U postgres -d postgres -c "SELECT usename, application_name, client_addr FROM pg_stat_activity WHERE usename LIKE 'ledger_app%';"

    Expected: you see ledger_app_v2 connections from your VPS IP.

  5. Remove the old credential after you’re confident:

    psql -U postgres -d postgres -c "REVOKE ALL PRIVILEGES ON DATABASE ledger FROM ledger_app;"
psql -U postgres -d postgres -c "DROP USER ledger_app;"

If your database is also on HostMyCode, keep it isolated and managed separately. For teams that don’t want DB babysitting on the app VPS, database hosting can simplify credential rotation and backups while you keep app deployments straightforward.

Verification checklist (what “done” looks like)

  • Decryption works for current recipients and fails for removed recipients.
  • Plaintext secrets exist only on servers that need them (and have 0600 permissions).
  • Service loads secrets via systemd (no manual exports in shell profiles).
  • Health check passes after restart (curl to loopback, not through the load balancer).
  • Downstream dependency confirms new secret (e.g., DB sessions show new user).

Common pitfalls (the ones that burn time)

  • Accidental plaintext commits: developers create .env next to the encrypted file and Git picks it up. Add a .gitignore for ops/secrets/*.env and a pre-commit check.
  • Deploying without restart: the file changes, but the process keeps old env vars. If you rely on EnvironmentFile=, treat a restart as mandatory.
  • Mixing recipients between environments: production recipients should not decrypt staging secrets and vice versa. Separate recipient lists, separate encrypted files.
  • Putting the age identity on too many servers: if “every box can decrypt prod,” one compromise becomes a full compromise.
  • Overly broad permissions: secrets readable by the app user might be acceptable; readable by www-data or “anyone in sudo” often isn’t.

If you’re trying to reduce lateral movement after compromise, pair this with segmented access. The same ideas line up with Zero-Trust network architecture in 2026, even if you’re running a small stack.

Rollback plan (fast, explicit, and testable)

For secrets rotation, rollback should be “copy a file back and restart.” No editing while the pager is going off.

  1. Keep the previous deployed secrets file on the server before overwriting:

    sudo cp -a /etc/ledger-api/secrets.env /etc/ledger-api/secrets.env.prev.$(date +%F-%H%M%S)

  2. If the service fails after rotation, restore the last known good file:

    sudo ls -1t /etc/ledger-api/secrets.env.prev.* | head -n 1
# pick the most recent and restore it
sudo cp -a /etc/ledger-api/secrets.env.prev.2026-04-21-031500 /etc/ledger-api/secrets.env
sudo chmod 600 /etc/ledger-api/secrets.env
sudo systemctl restart ledger-api

  3. Confirm recovery:

    sudo systemctl is-active ledger-api
curl -fsS http://127.0.0.1:4010/health

If the rotation also changed a downstream credential (like DB user v2), you need a DB rollback plan too. That’s exactly why “create v2, verify, then remove v1” matters: rollback becomes “point the app back to v1,” not “try to resurrect a deleted user.”

Operational next steps (so this stays healthy)

  • Schedule rotation: for human access (recipients), quarterly is usually reasonable; for high-risk vendor tokens, follow your threat model.
  • Add a canary restart: restart one instance (or one service) first, validate, then roll to the rest. Even on a single VPS, you can canary by restarting only the API while leaving the proxy untouched.
  • Automate verification: a tiny script that runs curl /health, checks DB connectivity, and posts status to chat pays for itself.
  • Log carefully: never echo env files into CI logs. If you ship logs centrally, scrub env-like patterns. If you need centralized logs, see VPS log shipping with Loki.

Summary: treat rotation as a product feature

Linux VPS secrets rotation works when the process is repeatable: recipients in a tracked list, encrypted files in Git, deterministic deploy steps, and a rollback that’s restoring a file and restarting a unit. Do it a couple of times and it stops feeling like “security work.” It becomes basic operations.

If you want a VPS that behaves predictably under restarts and deploys, start with a HostMyCode VPS. If you’d rather hand off OS maintenance while keeping control of your app and secrets workflow, managed VPS hosting is a sensible middle ground.

If you’re standardizing encrypted config and rotation runbooks, a VPS you control keeps things straightforward: systemd units, strict filesystem permissions, and predictable restarts. HostMyCode offers both a straightforward HostMyCode VPS and managed VPS hosting if you want the platform maintained while you focus on deployments and the runbook.

FAQ

Do I need to install sops on the VPS?

Not strictly. You can decrypt on your workstation and copy the plaintext file to the server. Installing sops on the VPS helps if you automate deploys (CI) or want an on-box runbook that doesn’t depend on a specific laptop.

Should the app user be able to read the secrets file?

If you use EnvironmentFile= in systemd, systemd reads it at service start. You can keep the file root-owned and readable by root only, but then the service must start via systemd with the right privileges. Many teams keep it root-owned 0600 and rely on systemd to inject the environment.

How often should I rotate recipients vs. actual passwords?

Rotate recipients on access changes (offboarding, role change) and on a schedule that matches your risk. Rotate actual passwords/tokens based on vendor policies, exposure risk, and incident response needs. Keep these as separate runbooks so you don’t over-rotate the hard parts.

What’s the safest way to rotate without downtime?

Use a “new credential alongside old” approach (create v2, deploy, verify, then remove v1). For apps with multiple instances, roll restarts gradually. On a single VPS, consider blue-green or a second systemd unit if you need zero-downtime cutovers.