Features & Enable Order¶

Drakkar’s optional features depend on each other. This page is the map: which switch turns what on, what each feature silently needs, and the order to enable things in as a deployment grows from a minimal worker to a full-featured fleet.

Per-feature guides: Configuration · Config Reference · Observability · Cache · Input Sources · Webapp · Local Databases.

The switches¶

Feature	Toggle	Default	What it controls
Metrics exporter	`metrics.enabled`	on (`:9090`)	Raw Prometheus endpoint on its own port
Operator UI + probes	`ui.enabled`	on (`:8080`)	UI server, `/api/v1`, `/ws`, `/healthz` + `/readyz`, and the flight recorder
Message probe endpoint	`ui.probe_enabled`	on	`POST /api/v1/debug/probe` — runs caller bytes through the live handler and real executor pool; `false` = 403, independently of `auth_token`
Database merge endpoint	`ui.merge_enabled`	on	`POST /api/v1/debug/merge` — writes a `merged-<ts>-<token>.db` into `<shared.dir>/merged/` per call, deleted after `history.merged_retention_days` while archiving is on; `false` = 403, independently of `auth_token`
Kafka read API	`ui.kafka_read_enabled`	on	`GET /api/v1/debug/kafka/*` — pipeline-invisible reads of the configured topics; `false` = 403, independently of `auth_token`
Consume pause	`ui.consume_pause.enabled`	off	Timed Live-page pause of message intake (auto-resume); off because pausing is production-affecting
Own storage	`storage.dir`	`/tmp/drakkar`	Where this worker’s event history, cache file and crash marker live; empty = memory-only
Recorder archiving	`history.archive.enabled`	on	Fold the sealed files of each cluster history folder into per-cluster `.db.gz` window archives; off = sealed files and merge results are never deleted automatically
Decoupled UI bundle	`ui.release.enabled`	on	Fetch + serve the drakkar-ui SPA release; with no bundle cached the worker is API-only and page requests answer 503
Prometheus panel + links	`ui.prometheus_url`	off (empty)	Dashboard deep-links into a Prometheus instance
Worker registry	`fleet.registry.provider`	`filesystem`	Workers find each other: the workers list, and the peers (URL, cluster, card marks) of the cache exchange; needs `ui.enabled`
Cross-worker fleet view	a `shared.dir` every worker mounts (or one shared `storage.dir`)	—	History trace over the cluster’s sealed files, cluster-wide database view
Handler cache	`cache.enabled`	off	LWW key/value store for handler code
Offload pool	`offload.max_threads`	on (2 threads)	`handler.offload()` — CPU-bound hook work off the event loop; always available, the knob only sizes it
Blocking-I/O pool size	`io.max_threads`	`0` (Python’s sizing)	Size of asyncio’s default `to_thread` executor; `0` keeps `min(32, cpu_count + 4)` — a knob, not a switch
Runtime health monitor	`runtime_health.enabled`	on	Event-loop lag heartbeat, stall stack sampling, Runtime tab, `runtime_health` events
Task cost & throughput	`throughput.cost_label`	off (empty)	Naming a numeric task label enables per-task speed and windowed throughput (Throughput)
Cache exchange	`fleet.cache_exchange.enabled`	on (when cache on)	Cross-worker cache convergence: each worker pulls the `cluster`/`global` rows of its registry peers over HTTP
Kafka input source	`sources.kafka.enabled`	off	Consumer group, poll loop, `arrange()`, offset commits
HTTP input source	`sources.http.enabled`	off (`:8090`)	POST → handler pipeline → JSON response
Aligned startup	`sources.kafka.startup_align_enabled`	on	Fleet restarts converge on one rebalance

Sources are not optional as a group: at least one input source must be enabled. Both are off by default, so every worker states where its work comes from. A config with neither fails to load (no input source enabled: ...). See Input Sources.

Sinks are not a toggle either: at least one sink instance (of any type) is required when results go to sinks, that is when the Kafka source is on or sources.http.sinks_enabled is true. Such a worker with zero sinks fails startup. An HTTP-only worker with sinks_enabled: false (the default) answers each request with the handler’s response and needs no sink. The DLQ is built whenever the Kafka source is on (topic {sources.kafka.topic}_dlq) or dlq.topic is set, and the circuit breaker always wraps sink delivery.

Dependency rules (and what happens when they’re unmet)¶

The framework prefers degrading over refusing to start — which means several of these edges are silent at startup. Each rule below names the failure mode so you can decide what your deployment needs.

Probes need the UI server. /healthz and /readyz are served only by the UI server. ui.enabled: false means no probe endpoints at all — Kubernetes liveness/readiness checks simply fail. The worker logs the warning ui_disabled_no_probes at startup; under Kubernetes, keep the UI enabled (auth-token it instead of disabling it).
The flight recorder runs only when ui.enabled. UI off = no event history, no config snapshot, no state timeline — and none of the features below that read them.
Recorder persistence needs a writable storage.dir. The worker creates it (mode 0700) when it is missing. Empty storage.dir switches to a short in-memory window (lost on restart) and disables the OOM watchdog ({storage.dir}/{worker}.watchdog).
The SPA bundle needs the UI server. ui.release.enabled fetches and serves the drakkar-ui release; a fetch failure is non-fatal, and release tags are immutable, so a bundle downloaded once serves from the shared cache on every later start — offline ones included. There is no HTML fallback in the package: with nothing cached the worker runs API-only and page requests answer 503 naming how to supply a bundle. For an air-gapped deployment, stage one with drakkar-ui fetch --version=vX.Y.Z or point ui.release.repo at an internal mirror.
The workers list needs the registry; cross-tracing needs a SHARED folder. Workers find each other through the fleet registry, which needs ui.enabled and, for the default filesystem provider, a card folder all workers see (shared.dir, else storage.dir). Every worker seals its finished recorder files into <shared.dir>/history/<cluster>/, and a history trace reads the sealed files there (see Local Databases); every worker must see the same shared folder (the mount path may differ: no file stores an absolute path). Unshared folders are silent: each worker simply sees only itself. Set ui.public_url per worker so the card and cross-links carry a reachable address instead of the bind IP.
The cache engine needs storage.dir too. The cache file lives there, private to the worker; with storage.dir empty the cache runs memory-only (WARN cache_engine_disabled_no_storage_dir) and the cache exchange turns itself off.
Cache exchange needs the registry and reachable UI ports. A worker pulls only the peers that the fleet registry lists, at the URL on each card, so it needs the registry of rule 5 (ui.enabled and a card folder all workers see). A peer the registry does not list is never pulled; with the registry off nothing is pulled. static peers (their cards carry no cache marks) are pulled every cycle. Every worker must also reach the UI port of its peers: across hosts, bind ui.host to a reachable address or set ui.public_url. The cache file itself is never shared. Silent when unmet: the cache simply stays local.
The webapp needs handler support — checked at construction. A worker with sources.http.enabled and a handler that lacks the HTTP hooks (arrange_http_request + on_http_request_complete plus the 3^rd/4^th generic models) fails fast with a ConfigurationError. A webapp bind failure at startup is fatal too: the webapp is an input source, so a worker that cannot bind its socket stops rather than run with nothing to read. A UI-server bind failure is fatal as well.
The webapp’s dashboard tile needs the UI + recorder. The tile on the operator dashboard reads request counts from recorder events; UI off = no tile, memory-only recorder = zeroed counts.
The Prometheus panel needs ui.prometheus_url. Unset = the dashboard links section is hidden. The optional ui.prometheus_worker_label / ui.prometheus_cluster_label / ui.custom_links keys refine it.
At least one source must be enabled. sources.kafka.enabled and sources.http.enabled both default to false; a config with neither fails to load with no input source enabled: set sources.kafka.enabled or sources.http.enabled. Handler validation follows the enabled sources: arrange() is required only with the Kafka source, the HTTP hooks only with the HTTP source.
The DLQ needs the Kafka source or an explicit dlq.topic. The default DLQ topic is derived from the source topic, so a worker with sources.kafka.enabled: false and an empty dlq.topic builds no DLQ producer at all. DLQ sends are then dropped, warned once, and counted in drakkar_dlq_unconfigured_drops_total. Set dlq.topic when an HTTP-only worker must not lose failed payloads (the rule).

Enable order: minimal → full-featured¶

Each tier only depends on the tiers before it. Start at 0, stop at the tier your deployment needs.

Tier 0 — choose your sources, then a minimal pipeline¶

Start here: decide where work comes in. Everything below builds on that choice. This tier shows the Kafka source; for HTTP, or for both, see Input Sources.

kafka:
  brokers: kafka:9092
sources:
  kafka:
    enabled: true
    topic: input-events
executor:
  binary_path: /app/process
sinks:
  kafka:
    results:
      topic: output-events
ui:
  enabled: false          # NOTE: no /healthz or /readyz in this tier!

Kafka in, subprocesses, sinks out, metrics on :9090. Suitable for one-off local runs; not for Kubernetes (rule 1).

Tier 1 — operability (UI, probes, history)¶

ui:
  enabled: true            # default
  auth_token: "${UI_TOKEN}"
storage:
  dir: /var/lib/drakkar

Adds the operator UI, /healthz + /readyz, persistent event history, and the OOM watchdog. This is the sensible production baseline.

Tier 2 — the real UI + Prometheus links¶

ui:
  release:
    enabled: true          # default — fetches the drakkar-ui SPA
  prometheus_url: "http://prometheus:9090"
  public_url: "http://worker-1.internal:8080"

The worker serves the versioned drakkar-ui SPA (shared per-host cache, managed with the drakkar-ui CLI) and the dashboard gains Prometheus deep-links.

Tier 3 — fleet view (multi-worker)¶

storage:
  dir: /shared/drakkar      # the same volume on EVERY worker (any mount path)

Workers discover each other, the UI’s worker switcher fills in, and tracing follows a message across workers. Any worker that implements the on-disk format can share the directory (spec).

Tier 4 — handler cache + cache exchange¶

cache:
  enabled: true
  # the cache file lives in storage.dir, private to this worker

self.cache becomes persistent and converges across the fleet: each worker pulls the cluster and global rows of the peers it finds through the Tier 3 registry, over HTTP, and merges them by last-write-wins (see Cache exchange). The workers must reach each other’s UI port.

Tier 5 — synchronous HTTP ingress¶

sources:
  http:
    enabled: true
    clients:
      - name: search-service
        token: "${WEBAPP_TOKEN}"
        rpm: 120

POST requests flow through the same handler pipeline with per-client auth, rate limits, and the max_body_bytes cap. Requires the HTTP hooks in your handler (rule 8). Turn sources.kafka.enabled off in the same config and the worker serves HTTP only.

Quick sanity checklist¶

Kubernetes? → ui.enabled: true (rule 1), probes pointed at the UI port.
More than one worker? → one shared folder on every worker (rule 5), ui.public_url set per worker.
Cache not syncing? → check that the peers appear in the workers list (rule 7), that drakkar_fleet_peer_requests_total shows ok requests (otherwise the UI ports are not reachable at the card URLs), and that both workers are in the same cluster for cluster-scoped keys.
Token set and the UI shows other workers as unreachable? → list every worker origin in ui.allowed_ws_origins (Cross-origin reads).
Webapp 500s at the first request? → it can’t: the worker rejects a hook-less handler at startup (rule 8).
Worker will not start with no input source enabled? → turn on sources.kafka.enabled, sources.http.enabled, or both (rule 11).
HTTP-only worker losing failed payloads? → set dlq.topic; the derived default needs a Kafka source (rule 12).
Air-gapped? → stage the bundle once with drakkar-ui fetch --version=vX.Y.Z into a cache the workers share, or point ui.release.repo at an internal mirror. ui.release.enabled: false turns the UI off rather than substituting anything — the worker then serves /api/v1 and the probes only.