Implementing ACME for NATS on Ubuntu

This guide provides step-by-step instructions for automating certificate management with ACME for NATS running directly on Ubuntu servers.

Prerequisites

  • Ubuntu server with NATS installed and running
  • Administrative (sudo) access
  • Domain name pointing to your NATS server (or accessible via DNS)
  • Basic familiarity with terminal commands

1. Install Certbot

Certbot is a popular ACME client that works well on Ubuntu:

sudo apt update
sudo apt install certbot

For DNS validation plugins (optional but recommended):

# For Cloudflare
sudo apt install python3-certbot-dns-cloudflare

# For Route53
sudo apt install python3-certbot-dns-route53

# For other providers
# sudo apt install python3-certbot-dns-<provider>

2. Obtain Initial Certificates using DNS Validation

Since NATS may not be publicly accessible over HTTP, DNS validation is often the best approach:

Manual DNS Validation

sudo certbot certonly --manual --preferred-challenges dns \
  --agree-tos --email your-email@example.com \
  -d nats.yourdomain.com

During this process, you'll need to create specific TXT records in your DNS. Follow the prompts and verify the records are properly set.

If using Cloudflare (adjust for your provider):

  1. Create API credentials file:
sudo mkdir -p /etc/letsencrypt/dns-credentials
sudo nano /etc/letsencrypt/dns-credentials/cloudflare.ini
  1. Add your Cloudflare API token:
dns_cloudflare_api_token = your_cloudflare_api_token
  1. Secure the credentials:
sudo chmod 600 /etc/letsencrypt/dns-credentials/cloudflare.ini
  1. Run Certbot with the DNS plugin:
sudo certbot certonly --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/dns-credentials/cloudflare.ini \
  --agree-tos --email your-email@example.com \
  -d nats.yourdomain.com

3. Configure NATS to Use the Certificates

Edit your NATS configuration file (typically /etc/nats/nats-server.conf or similar):

tls {
  cert_file: "/etc/letsencrypt/live/nats.yourdomain.com/fullchain.pem"
  key_file: "/etc/letsencrypt/live/nats.yourdomain.com/privkey.pem"
  timeout: 2
}

For more advanced configurations:

tls {
  cert_file: "/etc/letsencrypt/live/nats.yourdomain.com/fullchain.pem"
  key_file: "/etc/letsencrypt/live/nats.yourdomain.com/privkey.pem"
  ca_file: "/etc/letsencrypt/live/nats.yourdomain.com/chain.pem"
  verify: true
  timeout: 2
  cipher_suites: [
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
  ]
  curve_preferences: [
    "CurveP256",
    "CurveP384",
    "CurveP521"
  ]
}

Restart NATS to apply the changes:

sudo systemctl restart nats

4. Create a Certificate Renewal Script

Create a script to handle certificate renewal and NATS restart:

sudo nano /usr/local/bin/renew-nats-cert.sh

Add the following content:

#!/bin/bash

# Set variables
DOMAIN="nats.yourdomain.com"
CERT_DIR="/etc/letsencrypt/live/$DOMAIN"
LOG_FILE="/var/log/nats-cert-renewal.log"
NATS_SERVICE="nats"

# Log function
log() {
  echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
  echo "$1"
}

# Create log file if it doesn't exist
touch "$LOG_FILE"

# Backup current certificates
if [ -f "$CERT_DIR/fullchain.pem" ]; then
  BACKUP_DIR="/etc/letsencrypt/backup/$(date '+%Y%m%d%H%M%S')"
  mkdir -p "$BACKUP_DIR"
  cp "$CERT_DIR"/*.pem "$BACKUP_DIR/"
  log "Backed up current certificates to $BACKUP_DIR"
fi

# Renew certificates
log "Starting certificate renewal process"
certbot renew --quiet

# Check if renewal was successful by comparing certificate modification times
if [ -f "$CERT_DIR/fullchain.pem" ]; then
  CERT_MODIFIED=$(stat -c %Y "$CERT_DIR/fullchain.pem")
  CURRENT_TIME=$(date +%s)

  # If certificate was modified in the last hour, restart NATS
  if [ $((CURRENT_TIME - CERT_MODIFIED)) -lt 3600 ]; then
    log "Certificate renewed successfully, restarting NATS"
    systemctl restart "$NATS_SERVICE"

    # Verify NATS is running
    sleep 2
    if systemctl is-active --quiet "$NATS_SERVICE"; then
      log "NATS restarted successfully"
    else
      log "ERROR: NATS failed to restart! Manual intervention required"
      # Send alert (optional)
      # mail -s "NATS Certificate Renewal Failed" admin@yourdomain.com < "$LOG_FILE"
    fi
  else
    log "No new certificates found, renewal not needed at this time"
  fi
else
  log "ERROR: Certificate files not found after renewal attempt"
fi

Make the script executable:

sudo chmod +x /usr/local/bin/renew-nats-cert.sh

5. Set Up a Cron Job for Automated Renewal

Add a cron job to run the renewal script:

sudo crontab -e

Add this line to run the script twice daily (recommended by Let's Encrypt):

0 0,12 * * * /usr/local/bin/renew-nats-cert.sh

6. Fix Permissions for NATS to Access Certificates

Let's Encrypt typically creates certificates owned by root with restrictive permissions. Ensure NATS can read them:

# Determine the user NATS runs as
NATS_USER=$(ps -ef | grep nats-server | grep -v grep | awk '{print $1}')

# Add NATS user to certbot group
sudo usermod -a -G certbot "$NATS_USER"

# Adjust permissions on certificate directories
sudo chmod 750 /etc/letsencrypt/live
sudo chmod 750 /etc/letsencrypt/archive

# Ensure certificate files are readable
sudo chmod 640 /etc/letsencrypt/live/nats.yourdomain.com/*.pem
sudo chmod 640 /etc/letsencrypt/archive/nats.yourdomain.com/*.pem

7. Test the Setup

Manually test the renewal process:

sudo /usr/local/bin/renew-nats-cert.sh

And verify NATS is running with the correct certificates:

# Check if NATS is running
systemctl status nats

# Test TLS connectivity (using nats CLI if installed)
nats server info -s tls://nats.yourdomain.com:4222 --tlsverify

You can also verify the certificate expiration date:

openssl x509 -dates -noout -in /etc/letsencrypt/live/nats.yourdomain.com/fullchain.pem

8. Setup Client Connections

Update your NATS clients to use TLS when connecting:

Go Example

// Connect to NATS with TLS
nc, err := nats.Connect("tls://nats.yourdomain.com:4222",
    nats.RootCAs("/path/to/ca.pem"),  // Optional if using a widely trusted CA
    nats.ClientCert("/path/to/cert.pem", "/path/to/key.pem"),  // Optional for mutual TLS
)

JavaScript Example

const NATS = require('nats');
const fs = require('fs');

// Connect to NATS with TLS
const nc = await NATS.connect({
  servers: "tls://nats.yourdomain.com:4222",
  tls: {
    caFile: "/path/to/ca.pem",  // Optional if using a widely trusted CA
    certFile: "/path/to/cert.pem",  // Optional for mutual TLS
    keyFile: "/path/to/key.pem",    // Optional for mutual TLS
  }
});

Troubleshooting

Certificate Renewal Issues

  1. Check Certbot logs:
sudo cat /var/log/letsencrypt/letsencrypt.log
  1. Test renewal in dry-run mode:
sudo certbot renew --dry-run

NATS Connection Issues

  1. Verify certificate permissions:
ls -la /etc/letsencrypt/live/nats.yourdomain.com/
  1. Check NATS server logs:
sudo journalctl -u nats
  1. Test TLS connection directly:
openssl s_client -connect nats.yourdomain.com:4222 -showcerts

Security Best Practices

  1. Rate limiting: Configure your firewall to rate-limit connections to your NATS server.

  2. Mutual TLS: Consider implementing mutual TLS (mTLS) for client authentication.

  3. Authorization: Set up proper user/account authorization in NATS beyond TLS.

  4. Monitoring: Implement monitoring for certificate expiration and NATS availability.

  5. Regular audits: Periodically review your security settings and certificate status.

Appendix: Sample Systemd Service File

If you need to create a systemd service file for NATS:

sudo nano /etc/systemd/system/nats.service

[Unit]
Description=NATS Server
Documentation=https://docs.nats.io/
After=network.target

[Service]
ExecStart=/usr/local/bin/nats-server --config /etc/nats/nats-server.conf
ExecReload=/bin/kill -s HUP $MAINPID
Type=simple
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable nats
sudo systemctl start nats

This guide provides a comprehensive approach to implementing ACME-based certificate management for NATS on Ubuntu. For specific environments or requirements, adjustments may be necessary.