# SSH certificates instead of authorized_keys

LLMS index: [llms.txt](/en/llms.txt)

---

authorized_keys works fine for a handful of servers. Once you hit a dozen, it becomes a liability. Onboarding a new developer means manually distributing their public key across every machine. SSH certificates flip this model: one CA signs all public keys, and authorized_keys stays empty.

## Why authorized_keys breaks at scale

authorized_keys requires your public key to exist on every target server. Scaling this creates:

- a separate deployment step for key distribution during onboarding
- no centralized revocation — removing a key means touching each host
- key rotation touches every machine
- no expiration means stale access accumulates

An SSH certificate is a CA signature on your public key. The server only needs to trust the CA — your key never needs to be present locally.

## How SSH certificates work

Two entities: the CA (Certificate Authority) and the signed key. The CA is a standard SSH keypair, typically ed25519 or RSA. Signing creates the certificate with `ssh-keygen -s <ca_private> -I <identifier> <key.pub>`, producing `<key-cert.pub>`.

The server needs only two things: the CA public key in `TrustedUserCAKeys` (for users) or a `HostCertificate` directive (for hosts). Authentication succeeds if the signature is valid and the certificate hasn't expired.

> [!NOTE]
> A certificate does not replace the key. The key is still required — the CA signs it. The certificate adds metadata: TTL, principals, extensions, serial number.

## Generating CA keys

```bash
ssh-keygen -t ed25519 -f /etc/ssh/ca_user -C "CA for user certificates"
ssh-keygen -t ed25519 -f /etc/ssh/ca_host -C "CA for host certificates"
```

| Flag | Purpose |
|------|---------|
| `-t ed25519` | Key type; ed25519 recommended per RFC 8709 |
| `-f` | Output file path |
| `-C` | Comment; use for CA identification |

Store CA private keys securely — ideally on a dedicated build machine or in an HSM. The CA private key never goes to target servers.

## Signing user certificates: one-liner

```bash
ssh-keygen -s /etc/ssh/ca_user \
  -I "john@devops" \
  -n ubuntu,deploy \
  -V +52w \
  -z 1 \
  ~/.ssh/id_ed25519.pub
```

| Flag | Purpose |
|------|---------|
| `-s ca_private` | CA private key for signing |
| `-I identifier` | String logged during authentication |
| `-n principals` | Comma-separated list of authorized identities |
| `-V +52w` | Validity: 52 weeks from now |
| `-z serial` | Serial number; useful for audit trails |

The result is `id_ed25519-cert.pub` next to your private key. The user keeps both files. Signing another employee's key takes the same command with a different key.

> [!TIP]
> Duration suffixes: `h` (hours), `d` (days), `w` (weeks). `+1d` is one day, `-1d` means yesterday — already expired.

## Signing host certificates

Generate host keys on each server if they don't exist:

```bash
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""
```

Sign on the CA machine:

```bash
ssh-keygen -s /etc/ssh/ca_host \
  -I "prod-web-01" \
  -h \
  -n prod-web-01,10.0.1.5 \
  -V +52w \
  /etc/ssh/ssh_host_ed25519_key.pub
```

The `-h` flag marks this as a host certificate. The `-n` value lists the hostname and IP the client will verify on connect.

Place the certificate next to the host key:

```bash
cp /tmp/ssh_host_ed25519_key-cert.pub /etc/ssh/ssh_host_ed25519_key-cert.pub
```

## sshd_config: CertFile, TrustedUserCAKeys, HostCertificate

Configuration on the target server:

```bash
# /etc/ssh/sshd_config.d/certs.conf

# Enable certificate authentication
PubkeyAuthentication yes

# CA public key trusted for user authentication
TrustedUserCAKeys /etc/ssh/ca_user.pub

# Host certificate path
HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub

# Optional: per-user principal files
# AuthorizedPrincipalsFile /etc/ssh/%u.principals
```

Validate and reload after changes:

```bash
sshd -t && systemctl reload sshd
```

> [!WARNING]
> `TrustedUserCAKeys` expects the CA public key, not a certificate. Certificates are only needed for host keys.

## Principals and source restrictions

You can restrict a certificate to specific source IPs at signing time:

```bash
ssh-keygen -s /etc/ssh/ca_user \
  -I "jenkins@ci" \
  -n deploy \
  -O source-address=10.8.0.0/16 \
  ~/.ssh/id_ed25519.pub
```

Or enforce `from` via `AuthorizedPrincipalsFile` on the server:

```
# /etc/ssh/deploy.principals
deploy from="10.8.0.0/16"
```

A certificate can carry multiple principals. sshd checks if any principal matches those listed in `AuthorizedPrincipalsFile`. This lets you issue a cert with `ubuntu,deploy,admin` and grant access through different principals on different hosts.

## Time-to-live and rotation

Set TTL at signing. Practical guidelines:

| Role | Recommended TTL | Rationale |
|------|------------------|-----------|
| CI/CD, automation | 24–72 hours | Pipeline credentials are short-lived |
| Developers | 1–6 months | Balance between security and convenience |
| Hosts | 12 months | Host certificates bind to hostname/IP |

Rotation means signing a new certificate with a fresh serial. The old one becomes invalid after expiry. No centralized revocation list needed — an expired certificate simply fails validation.

## Fingerprints: the host-key problem

Traditional known_hosts stores host key fingerprints. Host certificates break this: the fingerprint in known_hosts won't match the certificate. Two solutions:

Use `ssh_known_hosts` with `ssh-keyscan`:

```bash
ssh-keyscan -t ed25519 prod-web-01 >> /etc/ssh/ssh_known_hosts
```

Or explicitly prefer certificate algorithms in your client config:

```
Host prod-web-01
    HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com
    UpdateHostKeys ask
```

On first connect, ssh will prompt to accept the certificate, add it to known_hosts, and won't ask again.

## Troubleshooting

Debug in order:

```bash
# Check sshd accepted the config
sshd -t

# Watch authentication logs
journalctl -u sshd -f

# Verbose client output
ssh -vvv user@host
```

In `-vvv` output, look for `Certificate` lines and `Authentications that can continue`. `no matching identity` means the certificate isn't next to the key. `certificate refused` indicates the CA isn't trusted or the principal didn't match.

Inspect certificate contents:

```bash
ssh-keygen -Lf ~/.ssh/id_ed25519-cert.pub
```

Output shows `Valid: from ... to ...`, `Principals:`, `Serial:`. First place to check when something breaks.

> [!NOTE]
> `too many authentication failures` with a valid certificate usually means ssh is cycling through all keys before reaching the cert. Add `-o PubkeyAuthentication=no` before explicit key specification, or remove extraneous keys from `.ssh`.

SSH certificates eliminate key distribution across hosts. One CA, signed keys with TTL, principals for access control. If you manage more than five servers, this isn't optional — it's infrastructure.