Running Typesense on Kubernetes: StatefulSets, persistence and health checks

Getting Typesense running locally takes one command. Getting it running in a cluster — surviving restarts, rescheduling, and node drains without silently losing its index — takes rather more thought.

These are the decisions that mattered when I containerised our entity search service and deployed it to Kubernetes.

Start from what "works on my machine" hides

The local version is genuinely one line:

docker run -p 8108:8108 \
  -v /tmp/typesense-data:/data \
  typesense/typesense:0.25.2 \
  --data-dir /data --api-key=dev-only-key

Three things here are quietly unacceptable in production, and each maps to a Kubernetes concept:

1. The API key is on the command line — it belongs in a Secret.
2. The data directory is an ephemeral host path — it needs a PersistentVolumeClaim.
3. Nothing knows whether the process is actually healthy — it needs probes.

Use a StatefulSet, not a Deployment

This is the choice people most often get wrong, and the reasoning is worth being explicit about.

Typesense keeps its index in memory but persists to its data directory so it can recover after a restart. That makes it stateful, and stateful workloads want stable identity and stable storage.

A Deployment treats pods as interchangeable. A StatefulSet gives each pod a stable name (typesense-0) and — critically — binds each to its own PersistentVolumeClaim that survives rescheduling. Use a Deployment with a shared PVC and you'll eventually get two pods contending over one data directory, which is exactly the kind of failure that shows up as intermittent corruption rather than a clean crash.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: typesense
spec:
  serviceName: typesense
  replicas: 1
  selector:
    matchLabels:
      app: typesense
  template:
    metadata:
      labels:
        app: typesense
    spec:
      containers:
        - name: typesense
          image: typesense/typesense:0.25.2
          args:
            - "--data-dir=/data"
            - "--api-key=$(TYPESENSE_API_KEY)"
            - "--enable-cors"
          env:
            - name: TYPESENSE_API_KEY
              valueFrom:
                secretKeyRef:
                  name: typesense-api-key
                  key: api-key
          ports:
            - containerPort: 8108
              name: http
          volumeMounts:
            - name: data
              mountPath: /data
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
            limits:
              memory: "4Gi"
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 20Gi

volumeClaimTemplates is the important part: each replica gets its own volume, and that volume follows the pod identity across reschedules.

Set memory limits deliberately

Because Typesense holds its index in RAM, memory limits aren't a formality. Set the limit too low and the kernel OOM-kills the container mid-query; the pod restarts, reloads the index from disk, and you get a latency spike that's confusing to trace back to its cause.

The practical approach: measure the resident set size with your real corpus loaded, then set requests near that and limits meaningfully above it to absorb indexing spikes. Indexing transiently uses more memory than steady-state serving, so sizing against idle usage will bite you during a bulk reindex.

Probes: distinguish "alive" from "ready"

Both probes can hit /health, but they mean different things, and conflating them causes real problems.

          readinessProbe:
            httpGet:
              path: /health
              port: 8108
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /health
              port: 8108
            initialDelaySeconds: 30
            periodSeconds: 20
            failureThreshold: 3

Readiness controls whether the Service sends traffic. On startup Typesense reloads its index from disk, and until that finishes it should receive no queries — otherwise clients get errors during every rolling restart.

Liveness decides whether to restart the container. Give it a generous initialDelaySeconds and failureThreshold. An aggressive liveness probe against a service that's slowly loading a large index produces the worst outcome available: a restart loop where the pod is killed each time because it's busy doing the work it needs to become healthy.

Bootstrap the schema idempotently

A fresh volume has no collections. Rather than creating them by hand after every deploy, make schema creation idempotent and run it as an init container or a startup job:

import typesense
from typesense.exceptions import ObjectAlreadyExists

client = typesense.Client({
    "nodes": [{"host": "typesense", "port": "8108", "protocol": "http"}],
    "api_key": os.environ["TYPESENSE_API_KEY"],
    "connection_timeout_seconds": 5,
})

schema = {
    "name": "companies",
    "fields": [
        {"name": "name",   "type": "string"},
        {"name": "ticker", "type": "string"},
        {"name": "sector", "type": "string", "facet": True},
    ],
}

try:
    client.collections.create(schema)
except ObjectAlreadyExists:
    pass  # safe to re-run on every rollout

Swallowing ObjectAlreadyExists is what makes this safe to run on every rollout. Anything that runs on every deploy has to be re-runnable.

Scope your API keys

Typesense supports scoped keys, and the search path should never hold an admin key. Generate a search-only key restricted to the collections it needs, and keep the admin key confined to your indexing pipeline.

If a browser-facing client talks to search directly, this stops being good hygiene and becomes essential — a key shipped to a browser is a public key.

What actually broke

The failure that cost me the most time wasn't exotic: a liveness probe too aggressive for index reload time, producing a restart loop that looked like instability in the engine but was entirely self-inflicted. The fix was a longer initialDelaySeconds and a proper readiness probe.

The lesson generalises past Typesense — when you containerise a stateful service, most incidents come not from the service but from the orchestration assumptions wrapped around it.

Related: why I chose Typesense over Elasticsearch for this workload.