Managing TLS certificates at scale

Expired certificates cause outages

Certificate-related outages cost enterprises millions of dollars annually. In 2020, Microsoft Teams went down for hours because of an expired certificate. Similar incidents have hit Spotify, LinkedIn and countless other organizations, yet many security teams still treat certificate management as an afterthought.

The certificate lifecycle

Every TLS certificate goes through four phases:

1. Generation: private key creation, Certificate Signing Request (CSR) generation, then CA validation and issuance.
2. Deployment: installation on servers and load balancers, configuration of cipher suites and protocols, testing and validation.
3. Monitoring: expiration tracking, revocation status checks and security posture assessment.
4. Renewal: automated or manual renewal, key rotation decisions, and rolling out the new certificate without disruption.

Certificate types

Domain validated (DV)

DV certificates verify domain ownership only. They are suitable for personal websites and blogs, internal development environments and non-commercial applications. Issuance takes minutes and the cost is free to low ($10-50/year). The only trust indicator is the padlock icon.

Organization validated (OV)

OV certificates verify the organization's identity. Typical uses are business websites, customer-facing applications and API endpoints. Issuance takes 1-3 days and costs $50-200/year, and the organization name appears in the certificate details.

Extended validation (EV)

EV certificates require rigorous verification. Typical uses are financial services, e-commerce platforms and government applications. Issuance takes 1-2 weeks and costs $200-1000/year. The organization name is displayed prominently, depending on the browser.

Automating issuance with ACME

The Automatic Certificate Management Environment (ACME) protocol transformed certificate management.

With Certbot

# Install certbot
apt install certbot python3-certbot-nginx

# Obtain certificate with automatic renewal
certbot --nginx -d example.com -d www.example.com

# Verify automatic renewal
certbot renew --dry-run

From Rust

use acme_client::{Account, Directory, OrderStatus};

async fn obtain_certificate(domain: &str) -> Result<Certificate> {
    // Connect to Let's Encrypt
    let directory = Directory::from_url(
        "https://acme-v02.api.letsencrypt.org/directory"
    ).await?;

    // Create or load account
    let account = Account::create(&directory, &account_key).await?;

    // Create order for domain
    let order = account.new_order(&[domain]).await?;

    // Complete HTTP-01 challenge
    for auth in order.authorizations().await? {
        let challenge = auth.http_challenge()?;
        deploy_challenge_response(&challenge).await?;
        challenge.validate().await?;
    }

    // Finalize with CSR
    let cert = order.finalize(&csr).await?;
    Ok(cert)
}

Monitoring certificates

What to monitor

Checking a certificate from Python

import ssl
import socket
from datetime import datetime, timedelta

def check_certificate(hostname: str, port: int = 443) -> dict:
    context = ssl.create_default_context()

    with socket.create_connection((hostname, port)) as sock:
        with context.wrap_socket(sock, server_hostname=hostname) as ssock:
            cert = ssock.getpeercert()

    # Parse expiration
    not_after = datetime.strptime(
        cert['notAfter'],
        '%b %d %H:%M:%S %Y %Z'
    )
    days_remaining = (not_after - datetime.utcnow()).days

    return {
        'hostname': hostname,
        'issuer': dict(cert['issuer'][0])['organizationName'],
        'expires': not_after.isoformat(),
        'days_remaining': days_remaining,
        'subject_alt_names': [
            x[1] for x in cert.get('subjectAltName', [])
        ]
    }

Common mistakes

1. Incomplete certificate chain

The server sends the leaf certificate without the intermediates. Always configure the full chain:

ssl_certificate /etc/ssl/certs/fullchain.pem;
ssl_certificate_key /etc/ssl/private/key.pem;

2. Hostname mismatch

The certificate doesn't cover every hostname it is needed for. Use Subject Alternative Names (SANs):

DNS:example.com
DNS:www.example.com
DNS:api.example.com
DNS:*.staging.example.com

3. Never rotating the key

The same private key is reused across multiple renewals. Generate new keys periodically:

# Generate new key during renewal
certbot renew --reuse-key  # Don't do this forever
certbot renew              # Generates new key by default

4. Mixed content after migrating to HTTPS

HTTPS pages load resources over HTTP. Fix it with Content Security Policy:

Content-Security-Policy: upgrade-insecure-requests

Enterprise-scale management

Certificate inventory

Keep a central inventory of all certificates:

Central management tools

For large deployments, consider:

• HashiCorp Vault: PKI secrets engine for internal CAs
• cert-manager: Kubernetes-native certificate management
• AWS Certificate Manager: managed certificates for AWS services
• Cloudflare: edge certificate management

Security hardening

A modern nginx TLS configuration

# Nginx configuration for A+ rating
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers off;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

# HSTS
add_header Strict-Transport-Security "max-age=63072000" always;

# OCSP Stapling
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;

Certificate Transparency

All publicly trusted certificates must be logged to CT logs. To check that yours are:

# Check CT log presence
curl "https://crt.sh/?q=example.com&output=json" | jq

Post-quantum planning

As quantum computing advances, plan for hybrid certificates:

• Monitor NIST post-quantum standardization progress
• Test hybrid certificate support in your infrastructure
• Plan a migration timeline for quantum-resistant algorithms
• Consider SecTLS for post-quantum TLS support

Outages, breaches and compliance failures cost far more than the tooling that prevents them. Automate with ACME first, then add monitoring, then write down how to handle the exceptions.