Reference
View as MarkdownOperating NebulaCtrl
Start here#
| Page | Read it when |
|---|---|
| Backup and restore | Before anything else. The scheduled control-plane backup (daily 03:30 UTC, 14 kept, S3 or a local directory), NEBULACTRL_SECRET_KEY (never in a backup), nebulactrl-server restore --from <dest> --at <timestamp>, what is not backed up |
Applications and deployments#
| Page | Read it when |
|---|---|
| Deployments | rolling vs recreate, health check semantics (Coolify-style), rollback, image retention, rollback_failed recovery |
| Builds: Dockerfile, Railpack, starters | a repository without a Dockerfile, the build-plan preview, pinned Railpack and how to upgrade it, buildx fallback, disk use |
| SSH deploy keys | cloning over ssh:// or git@host:org/repo.git, generated or uploaded keys, rotation, known_hosts pinning (trust on first use), "Accept new host key", limits |
| Import a Docker Compose project | turning a docker-compose.yml into staged services: the review, what maps and what does not, rewritten references, managed-database opt-in, secrets, blockers, the CLI |
| Import from Coolify | migrating a Coolify v4 instance: the token it needs (read + read:sensitive), what maps (applications, Dockerfile/Nixpacks builds, domains, health checks, custom Docker run options, managed databases with the same engine and version, variables and rewritten references), what is never copied (data) and stays manual, the CLI nebulactrl import coolify |
| Compose stacks | running a docker-compose.yml as one service with docker compose: the generated override, routing through domains, deploy and rollback, the allow-list (no privileged mode, host network, devices, Docker socket or host bind mounts), the per-server Unrestricted switch, limits |
| Docker options | letting a service add a capability, a ulimit, a bigger /dev/shm, extra hosts, labels, an init process, a stop signal or tmpfs mounts; what is never allowed; the instance switch for NET_ADMIN / SYS_ADMIN |
| Replicas and autoscaling | running a stateless service on several replicas, CPU autoscaling rules, capacity guard |
| Pull-request previews | a preview environment per open PR: what is cloned, fresh databases and seed commands, domains, the 5-running / 7-idle-day limits, pausing and resuming, fork PRs and why they get no secrets |
| Clone and move | a staging copy of production: cloning a service or a whole environment (secrets copied sealed, generated values regenerated, new empty volumes and databases, domains dropped or suffixed), moving a service to another environment with its references and links checked, the dry run, the CLI, what is not copied |
| Web terminal and file browser | opening a shell or browsing files in a replica container, roles and the production confirmation, what is audited (never the I/O), limits, the CLI services shell, proxy settings for WebSockets |
| Maintenance mode and error pages | putting a service into maintenance (UI, CLI, API), what visitors see (503 + Retry-After), the automatic 404/502/503 pages, the nebulactrl-pages container and its pinned nginx image |
| Deploy hooks | a URL per service that deploys it from CI or curl (POST /hooks/deploy/<id>/<secret>): the show-once secret and rotation, ?tag= / ?digest= as a one-off release, Docker Hub and GHCR push payloads, queueing behind a running deployment, the 6/min rate limit, audit, curl / GitHub Actions / GitLab CI snippets |
Domains, networking and certificates#
| Page | Read it when |
|---|---|
| Networking | what can reach what on a server, team shared networks (same server, hostnames <service>.<env>.<project>), Block outbound internet (default on for databases), deploying the siblings, what is and is not isolated |
| Proxy routes: route services, domain options, TCP/UDP | publishing a LAN/host/tailnet upstream, exposing a TCP/UDP port, redirects, path prefixes, basic auth and other per-domain options, the proxy restart on a new port, why ufw does not protect Docker ports |
| Custom certificates, upstream policy, tailnet | uploading a certificate, the SSRF upstream policy, routing to tailnet devices (managed or host Tailscale), the Tailscale OAuth client and ACL setup |
| Per-service tailnet (Tailscale / Headscale) | making one service reachable on your tailnet (HTTPS, raw TCP, optional Funnel), "Tailnet only", the sidecar and how its key is minted, re-provisioning, device removal, Headscale limits |
| Server mesh and cross-server links | joining servers to your tailnet (server and link nodes), letting services of one environment on different servers reach each other, the generated ACL (managed on Headscale, advisory or opt-in managed on Tailscale), the device count, public DERP relays, the paused migration window, failure modes, the connectivity check |
| DNS-01 and wildcard certificates | Cloudflare, Route 53, Hetzner and DigitalOcean resolvers, credentials per team or server, wildcard domains, generated-domain coverage, certificate status and expiry |
| Cloudflare: DNS records, proxy toggle, live DNS status | connecting a scoped API token, the records NebulaCtrl creates and how it recognises them (ownership marker, adopting existing records), staying in sync on IP change / move / drain, DNS only vs proxied, the SSL-mode redirect-loop warning, the per-domain status and fix, tunnels |
Databases and backups#
| Page | Read it when |
|---|---|
| Managed databases | engines and versions, DATABASE_URL exports, the audited reveal action, credential rotation, version upgrades (pg_upgrade) |
| Postgres standby replicas and manual Promote | advanced, Postgres only: when to use a standby (another server, read offload) and when not, adding a replica on another or the same server, DATABASE_WRITE_* / DATABASE_READ_*, lag, Promote (switchover or forced, data loss, fencing, consumers rewritten), rebuild, backups, restore and upgrades, limits |
| Database backups | scheduled and on-demand dumps streamed to S3-compatible storage or a server directory, retention, restore into a new database or in place |
| Volume backups | file-level backups of a service's volumes: pre-hook, freeze (docker pause), staged copy, background restic upload, stop and live modes, retention, restore into a new volume with a staged swap, file browser and single-file download |
Servers#
| Page | Read it when |
|---|---|
| Servers and agents | offline servers, revoke and re-enroll, in-flight deployments, reconciler |
| Build servers and registries | building on a dedicated server instead of the target: roles (Deployments, Builds, Both), where builds run (team, project, service), bring-your-own registries (GHCR, Docker Hub, GitLab, Harbor) and the managed "Set up for me" registry, cross-architecture builds and emulation, per-project builders and preview isolation, fallback to the target, rollback and what to keep in a registry, the failure table, limits |
| Draining servers and the fleet overview | taking a server out for maintenance or retirement, capacity and overcommit, drain failure and cancel semantics, required scopes |
| Docker cleanup | the daily and disk-alert cleanup of images, build cache and networks per server, what is never removed (volumes, rollback images, infrastructure images), the optional stopped-container step, run history and the explicit unused-volume action |
| Upgrades: agents and the control plane | agents replacing their own binary (checksum, atomic swap, rollback when the new one does not start), automatic and staged upgrades, version skew per server, the control-plane backup taken before an upgrade and when it blocks one, --force-upgrade |
| Server patching | pending OS updates and reboot-required per server (apt and dnf, checked every 6 hours), "N updates (M security)" on the server card, the instance admin's Apply updates (plan, typed confirmation, production acknowledgement, log) and Reboot (waits for the agent to reconnect), the server.reboot_required notification, limits |
| Host shell | the opt-in root shell on a server: who may use it (Owners and Admins with two-factor authentication), the production confirm, what is audited (never the I/O), how the agent runs it |
Observability#
| Page | Read it when |
|---|---|
| Monitoring, logs and retention | health endpoints, JSON logs, correlation_id, metrics and traffic analytics retention, disk alerts |
| Log history and log drains | runtime logs stored on each server (7 days, size caps), History view and query language, drains to Loki, OTLP, syslog and HTTP JSON (Datadog, Better Stack, Axiom), retries, backlog bound, health |
| Traffic analytics and request origins | what the proxy access log is turned into, the "Where requests come from" map, DB-IP Lite vs MaxMind GeoLite2, country vs city (~8 MB vs ~130 MB), attribution, privacy |
| Notifications | email, Slack, Discord, Telegram and signed-webhook channels, event rules, retries and the delivery log, verifying X-NebulaCtrl-Signature |
| Service alerts | crash, unhealthy, restart-loop and missed-backup alerts: once per incident, recovery messages, flapping, thresholds, what is not watched |
Secrets and platform services#
| Page | Read it when |
|---|---|
| Secret managers and team variables | ${{vault:<name>/<path>#<key>}} references to HashiCorp Vault / OpenBao (KV v2, token or AppRole) and Infisical (machine identity), fetched at deploy time and never stored, the policy to create, errors that name the reference, redaction, the team variable layer ${{team.KEY}} |
| Platform services ("Set up for me") | one-click S3 storage (SeaweedFS), log store (Loki, optional Grafana), tailnet control server (Headscale) and single sign-on (Authelia) run by NebulaCtrl in a hidden system project, wired into backup destinations, log drains, the Tailscale integration and sign-in providers with generated credentials; same-server-not-off-site, AGPL notice, people management, backups of their data, removal and what blocks it |
When something is wrong#
| Page | Read it when |
|---|---|
| Troubleshooting | proxy/port 80, certificates, DNS, agent will not connect, deployment stuck |
| Known limitations | what this release does not do |
| Advanced settings | you were told to change a control-plane timer (NEBULACTRL_MESH_LINK_GRACE, NEBULACTRL_REPLICA_BROKEN_AFTER); the defaults and what shortening them risks |
Installation and upgrade live in docs/installing; the security model in docs/security; design background in docs/architecture and docs/decisions.