Most teams think TLS certificate rotation is solved once they automate it.
A job runs. The certificate authority issues a new certificate. The pipeline reports success. Everyone moves on.
Then, weeks later, customers start getting TLS errors.
The renewal did not fail. The system failed to prove that the renewal reached production.
Automation is not proof
A certificate rotation has at least four steps:
- Issue the new certificate.
- Distribute it to every endpoint.
- Reload the process serving TLS.
- Verify that real clients receive the new certificate and can complete a handshake.
Most automation handles the first step well. Many failures happen in the other three.
The new certificate may be on disk, but NGINX was never reloaded. It may be installed on one load balancer but not another. A Kubernetes secret may be updated while an ingress controller continues serving the old one. A CDN, API gateway, regional proxy, or customer-specific endpoint may still have an older certificate chain.
The deployment pipeline is green because it completed its task. Production is unhealthy because the task was incomplete.
A TLS rotation is complete only when production is serving the intended certificate and real client handshakes are succeeding.
Measure what customers receive
This is why certificate rotation needs observability, not just automation.
Keyfactor’s 2024 survey reported an average of nine certificate-related incidents per organization in the prior 12 months. Only 32% of respondents said they used a dedicated certificate lifecycle management tool. It is vendor survey data, not an industry-wide outage rate, but it is a useful warning that certificate operations are still not boring enough.
The metrics I would want on an operations dashboard are simple:
| Metric | What it tells you |
|---|---|
| Certificates expiring in 30, 14, and 7 days | Whether the inventory has an upcoming risk |
| Renewal job success rate | Whether issuance or renewal is failing |
| Certificate serial number served by each endpoint | Whether every production endpoint has the new certificate |
| Certificate age and days remaining, measured externally | What customers actually receive |
| TLS handshake success rate | Whether clients can connect successfully |
| Handshake failures by region, hostname, and TLS error | Whether a partial rollout is hurting users |
| Time from issuance to full deployment | Whether the rotation converged within the expected window |
The key phrase is measured externally.
Verify the customer path
Do not only inspect the certificate stored in a secret manager or on the server filesystem. Probe the public hostname, with the correct SNI name, from outside the service. Record the serial number, issuer, chain, and expiry date that the endpoint actually serves.
That catches failures internal checks miss:
- Renewal succeeded, but the proxy did not reload.
- One region received the new certificate and another did not.
- The leaf certificate changed, but the intermediate chain is wrong.
- A legacy endpoint was missed because nobody knew it existed.
- The new certificate works in a browser but breaks a mobile client or partner integration.
Cloud providers expose parts of this already. Google Certificate Manager produces close-to-expiry and expired-certificate logs, plus an active-certificate metric. Useful, but it is only one layer. You still need endpoint-level handshake checks for the customer path.
Why teams miss it
Why do companies miss this? Certificates sit between teams. Security owns the certificate authority. Platform owns the load balancer. Application teams own the hostname. Network teams own the proxy. Nobody owns the full path from issuance to a successful customer handshake.
The fix is not another reminder email. Treat certificate rotation like any other production deployment:
- Define an owner for every hostname.
- Maintain discovery and inventory, including internal endpoints.
- Alert on expiration and failed renewal.
- Verify the certificate served from every critical endpoint after rotation.
- Page on sustained TLS handshake failure.
- Keep a dashboard showing deployment convergence by region and endpoint.
The real success condition is not “certificate renewed successfully.” It is “every customer-facing endpoint is serving the intended certificate, with a valid chain, and TLS handshakes are succeeding.”
That is the difference between automation and reliability.