Skip to main content

Self-Host Dex

View as Markdown

You run dex-server-zd next to your Workers. It keeps every Flow in object storage, such as Amazon S3, and a standby can take over when the active node stops. You configure authentication, TLS, retention, and monitoring yourself.

The Sustainable Use License lets you run Dex for your own business.

What it needs

DependencyRequiredWhat it gives you
Object storage, such as Amazon S3 or an S3-compatible storeYes, or db9 in its placeThe record of every Flow
A PostgreSQL or ParadeDB search backendNoLarge or frequent searches served from SQL. See Search backends.
A PostgreSQL Attribute StoreNoCopies of selected Attributes in your own tables. See Dex and Your Database.
A metrics listenerNoPrometheus metrics. See Metrics.

db9 is the PostgreSQL-compatible option for Dex Server's own state; stock PostgreSQL is not supported. Dex Server dependencies explains it.

How it stores Flows

Object storage is the record. Dex Server keeps a local store only as a cache, and a node rebuilds that cache from object storage after a restart. When Dex reports that something happened, such as a started Flow or a SYNC Step result, Dex has already written it to object storage.

One active process serves a cluster. It holds a lease in object storage, so a second process cannot write at the same time. Its capacity is the cluster's ceiling. Limits and Performance give the numbers.

When a node fails

A standby takes over after the active node's lease expires, plus 1 second. The lease is 10 seconds by default. Without a standby, the node serves again only after it restarts. A graceful step down releases the lease, so a standby takes over at once.

After a takeover, every Step attempt that a Worker had received runs again, and each one uses a retry, except some ASYNC attempts that keep their attempt number. So the external calls in a Step need an idempotency key. Failover and Deploys lists what runs again for each event, and Lease, hot standby, and step down gives the timings.

Run it

In production, run dex-server-zd serve with one YAML configuration file. The release publishes binaries and a container image. On Kubernetes, run it as a StatefulSet.

dex-server-zd serve --config /etc/dex-server-zd/config.yaml

To run one production node on local disk without external object storage, run dex-server-zd local. It keeps an embedded object store in its data directory and has no standby.

Run Dex in Production has a minimal configuration with authentication on.

Next