Checkmate/charts/helm/checkmate/INSTALLATION.md

Kubernetes Installation Guide for Checkmate

This guide walks you through deploying Checkmate on your Kubernetes cluster using Helm.

Prerequisites

• A running Kubernetes cluster
• Helm CLI installed and configured
• kubectl configured to access your cluster

Steps

1. Clone the repo and navigate to the Helm chart

git clone https://github.com/bluewave-labs/checkmate.git
cd checkmate/charts/helm/checkmate

2. Customize values.yaml

Edit values.yaml to update:

• client.ingress.host and api.ingress.host with your domain names
• api.protocol (usually http or https)
• If upgrading: Migrate persistence settings from flat structure to nested:
  • Old: persistence.mongodbSize → New: persistence.mongo.size
  • Add: persistence.mongo.storageClass (leave empty for default)
• Secrets under the secrets section (JWT_SECRET, email credentials, API keys, etc.) — replace all change_me values
• For TLS/HTTPS: Configure ingress TLS settings (see section below)

3. Deploy the Helm chart

helm install checkmate ./charts/helm/checkmate

This will deploy the client, api, and MongoDB components.

4. Verify the deployment

Check pods and services:

kubectl get pods
kubectl get svc

Once all pods are Running and Ready, you can access Checkmate via the configured ingress hosts.

Upgrading from a previous release

This release renames the server.* value block to api.* and drops the (unused) bundled Redis. Read this before running helm upgrade on an existing install.

server.* → api.* (values rename)

The value block formerly named server is now api (server.image → api.image, server.ingress.host → api.ingress.host, server.protocol → api.protocol, server.resources, server.affinity, server.tolerations, server.ingress.*, etc.).

Your existing server.* overrides keep working — the chart merges any legacy server.* values on top of the api.* defaults (legacy wins where set), so an unchanged values file upgrades cleanly. The shim is transitional; migrate your overrides to api.* at your convenience and drop the server: block. If you set both server.* and api.* for the same field, the legacy server.* value wins — don't mix them.

Resource renames cause a replace, not a rename

The Kubernetes objects are renamed alongside the values:

Old object	New object
Deployment/Service checkmate-server	checkmate-api
Ingress checkmate-server-ingress	checkmate-api-ingress
ConfigMap checkmate-server-nginx-cm	checkmate-client-nginx-cm

Helm deletes the old objects and creates the new ones on upgrade. Expect a brief API restart and a new ClusterIP for the API Service — update anything outside the chart that referenced the checkmate-server Service by name. MongoDB data (its PVC) is untouched.

Redis removed

The bundled Redis StatefulSet/Service and the redis.* / persistence.redis.* values are gone — the scheduler runs in-process and never used them. Default installs (redis.enabled: false) are unaffected. If you had explicitly set redis.enabled: true, the Redis StatefulSet and its PVC are deleted on upgrade — back up anything you stored there first. Leftover redis: / persistence.redis: keys in your values file are harmless (ignored).

New secret defaults

NODE_ENV: production, LOG_LEVEL: info, and TOKEN_TTL: 99d are now part of the default secrets block and merge into your install unless you override them. Set them explicitly if you relied on different values (e.g. a non-production NODE_ENV).

Image tags follow the chart's appVersion

Each tier's image value is now the repository without a tag; the tag defaults to the chart's appVersion, so the client, api, worker, and mongodb images all move together when you upgrade the chart. You can still pin per tier:

api:
  image: ghcr.io/bluewave-labs/checkmate-backend   # repo only
  tag: ""            # empty → Chart.appVersion; set e.g. "v3.1.0" to pin, or "latest" for a dev tag

• <tier>.tag overrides the version for that tier. The worker tier inherits api.tag (and api.image) unless you set worker.tag / worker.image.
• If you put a full repo:tag in image (e.g. a legacy server.image: …:v3.2.0 override), it is used verbatim and the tag field is ignored — so existing pinned values keep working unchanged.

Prefer a pinned tag over latest. With the default imagePullPolicy: IfNotPresent, latest is cached per node and never refreshed, and helm rollback can't move a floating tag — pin a version (or let it default to appVersion) for reproducible, rollback-able deploys.

Scaling: the worker tier & autoscaling

By default the chart runs a single all-in-one API pod that both schedules and processes monitoring jobs (worker.enabled: false). For larger deployments you can split processing onto a dedicated, horizontally-scalable worker tier.

Topology

worker.enabled	API (checkmate-api)	Worker (checkmate-worker)
false (default)	schedules and processes jobs (QUEUE_PRIMARY_PROCESSES=true)	not deployed
true	schedules + serves the API only (QUEUE_PRIMARY_PROCESSES=false, derived)	processes all jobs

QUEUE_PRIMARY_PROCESSES is derived from worker.enabled — you never set it directly, so the API and worker tier can never both process (or both ignore) the queue.

Enabling the worker tier

worker:
  enabled: true
  replicas: 2   # used only when autoscaling.enabled = false

The worker takes no inbound traffic (no Service): the kubelet probes pod IPs directly on HEALTH_PORT (/livez, /readyz), and graceful shutdown drains in-flight jobs before exit.

Invariant: worker.terminationGracePeriodSeconds (default 30) must stay greater than the server drain timeout (25s). Lower it and Kubernetes SIGKILLs workers mid-drain, losing in-flight checks.

Caveat: with the worker tier enabled the API no longer processes jobs, so if all workers are down nothing processes. The autoscaling.minReplicaCount: 1 floor plus liveness restarts mitigate this; full failover to the API would need leader election (not supported).

Autoscaling with KEDA

Backlog-driven autoscaling is off by default and requires the KEDA operator installed in-cluster. Enable it with:

worker:
  enabled: true
  autoscaling:
    enabled: true
    minReplicaCount: 1     # processing floor — never 0, or checks silently stall
    maxReplicaCount: 10
    backlogPerReplica: 50  # target due-job backlog per worker
    mongo:
      dbName: uptime_db    # MUST match the database in DB_CONNECTION_STRING

A KEDA ScaledObject queries MongoDB directly for the due-check backlog and scales the worker Deployment between minReplicaCount and maxReplicaCount. When autoscaling is on, the Deployment omits replicas so it doesn't fight the HPA KEDA creates. Make sure autoscaling.mongo.dbName matches the database name in your DB_CONNECTION_STRING, or the scaler counts the wrong DB (always 0 backlog → never scales up).

Fallback: Prometheus-adapter HPA (not shipped)

If your KEDA/Mongo versions reject the $$NOW/$toLong query, or you already run Prometheus, scrape the worker's /metrics (checkmate_worker_due_backlog), expose it via prometheus-adapter, and target it with a standard HorizontalPodAutoscaler. The manifest is left to the operator; the app computes now in code on this path, so the query coercion is unnecessary.

Enabling TLS/HTTPS with cert-manager

If you have cert-manager installed in your cluster, you can enable automatic TLS certificate provisioning using Let's Encrypt or other certificate issuers.

Prerequisites

• cert-manager installed in your cluster
• A ClusterIssuer or Issuer configured (e.g., letsencrypt-prod)

Configuration

Edit values.yaml to enable TLS (and update protocols to https):

client:
  protocol: https
  ingress:
    enabled: true
    host: checkmate.example.com
    className: nginx
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
    tls:
      enabled: true
      secretName: checkmate-client-tls

api:
  protocol: https
  ingress:
    enabled: true
    host: checkmate.example.com
    className: nginx
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
    tls:
      enabled: true
      secretName: checkmate-api-tls

Alternative: Using --set flags

You can also enable TLS during installation using Helm's --set flags:

helm install checkmate ./charts/helm/checkmate \
  --set client.protocol=https \
  --set api.protocol=https \
  --set client.ingress.annotations."cert-manager\.io/cluster-issuer"="letsencrypt-prod" \
  --set client.ingress.tls.enabled=true \
  --set client.ingress.tls.secretName=checkmate-client-tls \
  --set api.ingress.annotations."cert-manager\.io/cluster-issuer"="letsencrypt-prod" \
  --set api.ingress.tls.enabled=true \
  --set api.ingress.tls.secretName=checkmate-api-tls

Verification

After deployment, cert-manager will automatically create the TLS secrets. You can verify the certificate status:

# Check certificates
kubectl get certificates

# Check certificate details
kubectl describe certificate checkmate-client-tls
kubectl describe certificate checkmate-api-tls

# Verify the secrets were created
kubectl get secrets | grep checkmate-tls

The ingress will automatically use these secrets to enable HTTPS access to your Checkmate instance.