Platform services
View as MarkdownPlatform services ("Set up for me")
Some features need a service you may not have: somewhere to store backups, a log store, a tailnet control server, a single sign-on provider, a container registry for build servers. Set up for me runs one for you, as a managed service NebulaCtrl owns, and connects it to the feature that asked for it with generated credentials. You can still point the same features at your own S3, Loki, Tailscale, identity provider or registry; nothing here is required. For single sign-on, connecting the identity provider you already have (Settings, Instance, Single sign-on, Connect your identity provider) is the main path; Set up for me sits underneath it for teams that have none.
| You are on | The button creates | Wired into | Product |
|---|---|---|---|
| Settings, Backups | S3 storage | a backup destination named "NebulaCtrl S3" (bucket created, tested from the server) | SeaweedFS 4.51, Apache-2.0 |
| Settings, Log drains | Log store | a Loki log drain named "NebulaCtrl Loki" (basic auth, tested from the server) | Loki 3.7.9, AGPL-3.0 |
| Settings, Log drains (optional) | Grafana | Loki added as a data source | Grafana 13.2.4, AGPL-3.0 |
| Settings, Integrations, Tailscale | Tailnet control server | the Tailscale integration, provider Headscale (a user and an API key, connection tested) | Headscale v0.29.6, BSD-3-Clause |
| Settings, Instance, Single sign-on | Single sign-on (your choice of product) | a sign-in provider (OpenID Connect) | Pocket ID v2.19.1, BSD-2-Clause (preferred, passkeys only), or Authelia 4.39.28, Apache-2.0 (passwords + 2FA) |
| Settings, Registries | Container registry | a registry named after the service, kind managed, used by build servers | CNCF Distribution registry:3.1.2, Apache-2.0 |
Why these images, how they compare with the alternatives and which rule picked them: platform-services-research.
The page that lists, repairs and removes them is Settings, Platform services. Owners and Admins only (scope admin); single sign-on is instance-wide, so it needs an instance administrator.
Where they live#
Every team gets one hidden project, NebulaCtrl platform, with one environment, platform. It is not in the projects list or the switcher; the page links to its canvas ("Open in canvas") when you need to look at a service, its logs or its deployments. It cannot be renamed or deleted (409 system_project), and it is visible to administrators only: people without the admin scope get 404 for the project, its environment and its services (and the actions on them). The services in it are ordinary services: they show up on the server's page and in the fleet overview, count towards the server's capacity, are deployed by the normal engine and can be stopped, restarted or inspected like any other. Do not delete them there: the platform page would show them as removed outside it. Remove them from the platform page.
Each kind exists once per team. A server flagged "does not accept workloads" cannot host them, like any service.
What a set-up does#
POST /platform-services stages everything in one transaction (the service pinned by digest, its volume, its variables, its public hostname, an audit entry) and starts a job. The job then:
- configure (Loki: protects the public route with basic auth);
- deploy the staged service through the normal deploy path (so cancel, logs and the deployment history work) and await it;
- wire: wait until the public URL answers, create the integration with the generated credentials, and test it the way the feature itself would (the destination test and the drain test run from the agent; the Tailscale test mints and expires a key). A test that fails is retried for up to 15 minutes (certificates and DNS take a while) and then the service is failed with the reason.
A failed service keeps what it created; Retry resumes at the step that failed (a failed deployment is deployed again). Reconnect runs the wiring again on a ready service (Headscale mints a new API key).
Hostname, TLS and the public URL#
Every one of them is reached through the server's proxy at a public hostname, because the things that talk to it run elsewhere: the agents (backups, log shipping), the control plane (bucket creation, sign-in) and your people's browsers and Tailscale clients. The hostname is generated like any generated domain (<name>-platform-nebulactrl-platform.<base domain>, else <public ip>.sslip.io); enter your own to override it. Certificates are Let's Encrypt, so the server needs an ACME email (422 acme_email_required otherwise: without it the proxy serves a self-signed certificate that clients refuse). .localhost names are exempt, for local use.
A public URL that is http:// (or that names a non-standard port, such as https://s3.example.com:8444 for a proxy that is not on 443) is accepted as typed. An http:// URL is served without the redirect to HTTPS; use it only on a trusted network or for local tests, because S3 requests would travel in clear text (signed, but not encrypted).
Credentials#
NebulaCtrl generates every credential (S3 key pair, Loki basic-auth password, Grafana admin password, Authelia secrets and signing key, Pocket ID's encryption key and API key, the OpenID client secret). They are stored sealed with the instance key, the service ones as secret variables (masked everywhere, delivered to the agent only at deploy time) and never appear in a configuration file, a view, the audit log or a command row. The Show credentials action on the page reveals S3, Loki and Grafana credentials (audited as platform.reveal); Headscale's API key is created inside the container and kept only by the Tailscale integration; the sign-in provider's client secret is kept only by the sign-in provider. With Authelia, the first person gets a one-time password shown once; with Pocket ID you create a one-time sign-in link (see below). Back up the instance key (NEBULACTRL_SECRET_KEY): it protects every sealed credential, including Pocket ID's ENCRYPTION_KEY, without which a restored Pocket ID data volume cannot be read.
Outbound access#
Everything except Headscale starts with Block outbound internet on (see networking); they only talk to the other services of their environment and to the proxy. Pocket ID needs no outbound access either: its analytics heartbeat, release check and icon library are switched off, and GeoLite2 is only downloaded when a MaxMind key is set (it is not). This was checked by running the pinned image on a Docker network without a route to the internet. Headscale needs the internet to fetch the DERP relay map, so it is not blocked. Everything runs from the pinned digests listed above; there is no pull of "latest".
S3 storage#
SeaweedFS runs as one process (weed server -s3: master, volume server, filer and S3 gateway) with the key pair from the environment, one bucket (default nebulactrl-backups, bucket at set-up) and path-style access. The destination uses the prefix nebulactrl/.
Same server, not off-site. If the store runs on the server that also runs the databases it backs up, a backup there does not survive the loss of that server. The page and the destination row say so ("Same server, not off-site"); place the store on another server to make it off-site, or add an off-site destination as well. The store's own data volume is not scheduled for backup (it holds your backups; backing up backups into the same place is circular): if you want a copy, back its volume up to another destination from the service's Backups tab.
Log store and Grafana#
Loki runs as one process with filesystem storage (tsdb, schema v13), retention through the compactor (default 14 days, retentionDays at set-up; the retention applies to what the drains ship, independent of the 7 days each server keeps), usage reporting off. Loki has no authentication of its own, so the public route carries basic auth and the drain sends the same credentials. Logs are searchable in Grafana (below) or any Loki client; the in-product log views keep reading each server's own history.
Grafana is optional and separate: it needs Loki to be ready and always runs next to it (a different server would not reach it: there is no cross-host networking). It starts with the admin user admin and a generated password and with Loki as its default data source. Signing in to Grafana with your single sign-on provider is not configured.
AGPL. Loki and Grafana are licensed under the AGPL-3.0. NebulaCtrl runs the unmodified upstream images as separate containers and talks to them over HTTP; it does not link, bundle or modify them, and the licence applies to those programs, not to NebulaCtrl. If you modify and offer them to others over a network, the AGPL's source-offer terms apply to your modified version. The set-up dialog and the card show the licence.
Tailnet control server#
Headscale runs with SQLite on its volume, no embedded DERP, MagicDNS under tailnet.internal, and a database-backed policy (policy.mode: database) that NebulaCtrl owns: the user nebulactrl owns tag:nebula-svc and tag:nebula-mesh, members reach tagged services, and the server mesh adds the rules it generates. Anything you change in that policy by hand is overwritten. The wiring creates the user, an API key valid for 365 days (headscale apikeys create, run in the container), writes the base policy before any node can join, and creates the Tailscale integration with provider Headscale, credential "API key" and tag:nebula-svc. The page shows the key's expiry; Reconnect mints a new one before it runs out. If a Tailscale integration already exists the set-up stops with integration_exists unless you confirm Replace it. On removal the integration is deleted only if it is the one this service created.
Existing installs are migrated once, with the mesh paused. A Headscale set up before the mesh existed used a policy file. A background job pauses the mesh with the reason "Headscale is being migrated", redeploys Headscale with the database policy mode, waits for it to be healthy, writes the base policy and the mesh rules, reads them back and un-pauses. While it runs, no key is minted and no node is added or removed, replica and Promote operations wait, and a deploy that needs a new link fails with "Not deployed: mesh paused. No changes were made."; established tunnels, existing nodes and per-service sidecars keep working. Between the restart and the first policy write Headscale has no policy, which allows everything, for seconds and with nothing able to join. If a step fails the mesh stays paused with the reason; Retry on this service resumes it (the job also looks again every 10 minutes and is safe to run more than once). See the paused migration window.
Relays are Tailscale's public DERP servers. Headscale has no relay of its own here; it hands out Tailscale's public DERP map as the fallback for peers that cannot connect directly. Traffic stays end-to-end encrypted (a relay sees ciphertext and endpoint addresses), but it does leave your infrastructure, and Headscale needs outbound internet to fetch the map. This is shown in the "Set up for me" dialog (DERP). An embedded DERP server is not built.
Tailscale clients need HTTPS on the control URL (the ACME email requirement above). Joining your own devices to this tailnet is not automated: create a pre-auth key with headscale preauthkeys create --user nebulactrl from the service's terminal.
Container registry#
CNCF Distribution registry:3.1.2 (pinned by index digest; an owner exception to the two-year rule, see ADR 0008) runs as one process with filesystem storage on a named volume, behind the server's proxy at a public hostname with a Let's Encrypt certificate (the ACME email requirement above applies: Docker daemons only accept a non-loopback registry over TLS). It exists for build servers: a build server pushes the images it builds here and the servers that run the services pull them by digest. Setting it up creates a registry of kind managed under Settings, Registries, so nothing has to be wired by hand.
Authentication is token authentication with the control plane as the token server (/registry/token, Docker token authentication, ES256 tokens signed with a key generated per registry and sealed in the platform service; the registry only holds the public key), not a shared password. There is no stored credential: for each build and each pull the control plane mints a sealed, expiring credential (15 minutes by default, longer for a build, never more than 6 hours) limited to one repository and the actions needed, and the token endpoint exchanges it for a token valid at most 15 minutes. A credential for another repository gets no access, a wrong or expired one gets 401, and an address that fails 30 times a minute gets 429. There is nothing to reveal, so Show credentials does not apply to this kind. Set-up needs NEBULACTRL_PUBLIC_URL (422 public_url_required) and an https address (422 https_required, http only for localhost).
NebulaCtrl also owns the registry's retention. An hourly sweep deletes the digests of images outside every service's live set through the registry API (the registry runs with deletion enabled). A weekly window (starting between 02:00 and 05:00 UTC) collects the blobs nothing references with registry garbage-collect, never with --delete-untagged: builds for this registry queue, running pushes get up to 2 hours to finish, the registry is set read-only (a short redeploy), the collection runs, and the registry is set back to read-write (a second redeploy). Pulls keep working during the window and pushes get 405. A window always ends with the restore step; if the registry cannot be redeployed read-write for over an hour the platform service goes failed and the registry stays read-only until you retry. Details in build servers: rollback and retention. The registry's domain has large uploads on: the proxy's request-read timeout (60 seconds by default, long enough to cut a big layer over a slow link) is raised to 30 minutes on that server's entrypoints, which recreates the proxy container once, like an ACME email change; other servers keep 60 seconds. Place the registry on the build server (pushes stay local) or on the control-plane server. A server flagged "does not accept workloads" cannot host it.
Single sign-on#
The dialog asks which identity provider to run. Pocket ID is preselected and recommended; Authelia is the alternative ("Need passwords + 2FA instead? Use Authelia"). Only one of them can exist per team; set-up needs an instance administrator and NEBULACTRL_PUBLIC_URL. Sign-in providers are instance-wide: everyone sees the button (default label "NebulaCtrl SSO"). Accounts are matched the normal OpenID Connect way: by verified email to an existing NebulaCtrl user, else a new account only if open sign-up is on or an invitation is used.
Pocket ID (preferred)#
Users sign in with passkeys only, no passwords. Pocket ID runs with SQLite on its volume (/app/data), configured entirely by environment: APP_URL (the public URL), a generated ENCRYPTION_KEY and a generated STATIC_API_KEY, TRUST_PROXY, analytics, release check and icon library off. Its hostname must be a domain name (passkeys are bound to it): an IP address or a single word is refused with invalid_hostname, and passkeys need HTTPS in browsers (a .localhost name is the exception).
The job then wires it through Pocket ID's REST API with the static API key:
- test the discovery endpoint (
/.well-known/openid-configuration): the issuer must be the public URL and PKCE S256 offered; - create the first administrator from your account (username from your email, your name and email, marked as verified) and check that nobody else has an account: until then Pocket ID's setup page is open to whoever knows the address, so if any other account exists the set-up fails with "its setup page was used before NebulaCtrl could" and wires nothing. Remove it and set it up again;
- create the OpenID client (confidential, PKCE on, no consent screen, callback
<NebulaCtrl URL>/api/v1/auth/oidc/<provider id>/callback) with a generated secret that NebulaCtrl supplies, so a retried set-up always ends with exactly the stored one; - add the sign-in provider.
The one-time sign-in link. There is no password, so once the service is ready, Create my sign-in link (in the set-up dialog, and later Sign-in link on the card) asks Pocket ID for a one-time access token for the first administrator and shows the link <URL>/lc/<token> once. Open it, sign in, then add a passkey on your account page: that is how you sign in from then on. The link works once and expires after an hour. It is never stored, logged or audited: it lives only in the response to that request (Cache-Control: no-store) and in the open dialog; the audit entry (platform.admin_link) says who asked, never the link. Lost it? Create another. Any instance administrator can create one, for the first administrator's account.
People are managed in Pocket ID, not here: add, change and remove them in its admin page (the card links to it: Open Pocket ID). The platform page has no People dialog for it, and the people endpoints answer 409 users_not_managed.
Authelia (alternative)#
Authelia runs in file-backend mode, so NebulaCtrl manages its people: users and their argon2id password hashes live in the service's configuration, never in a database you have to administer. Set-up creates the first person from your own account (username from your email, your name and email, a generated password shown once), generates the signing key and secrets, registers NebulaCtrl as the only OpenID Connect client (authorization code with PKCE S256, client secret by HTTP basic, no consent screen, one factor) and adds the provider.
On the platform page Add person, Reset password and Remove rewrite the users file and redeploy the service: the sign-in page restarts for a few seconds (recreate deploy; people already signed in to NebulaCtrl are not affected). Passwords need at least 10 characters; leave the field empty to generate one. The last person cannot be removed (409 last_user). Authelia's own second factor is not used (it needs email delivery for enrolment, which this setup does not have); NebulaCtrl's own two-factor requirement still applies after sign-in.
Removing it#
Removing it (either product) deletes the provider and every identity linked through it. If people have no other way in (no password) the removal is refused with sso_only_users and the count; Remove anyway forces it. Give those people a password first (Settings, Members or the password reset link). The Pocket ID data volume stays unless you also delete the data; its passkeys and users are then gone with it.
Backups of the data#
Headscale, Pocket ID, Authelia and Grafana hold state. When a service becomes ready and the team already has an S3 destination that is not the platform S3, NebulaCtrl schedules the volume backup of its data there (daily 03:17 UTC, keeping 7 daily, 4 weekly, 3 monthly). Otherwise the page says "Not backed up" and offers Back up daily with a destination picker (any destination, including the platform S3). Loki holds logs bounded by its retention and is not scheduled; S3 holds backups (above).
Removing#
Remove needs the kind typed as confirmation and asks whether to also delete the data (the volumes; off by default: they stay on the server and can be removed in Docker or by removing the volumes through the server's cleanup). The integration is removed first; if it is still in use nothing changes and you get the reason:
| Kind | Refused while |
|---|---|
| S3 | backup schedules or stored backups use the destination (destination_in_use): move or delete them first |
| Loki | never (the drain is removed); Grafana must go first (grafana_depends_on_loki) |
| Headscale | servers use the managed tailnet (integration_in_use): this includes a mesh that is enabled or has server nodes |
| Single sign-on | people could only sign in through it (sso_only_users), unless forced |
| Container registry | the team's build settings, a project or a service override still references it (409 registry_in_use): switch them to another registry or to building on the target first. Its stored images are deleted if you also delete the data. |
Then the services are deleted through the normal path (stop, remove the container, optionally the volumes) and the row is gone; you can set the kind up again. A removal that stops half-way (a deployment in progress, an offline server) leaves the service in removing/failed with the reason; repeat the removal.
API#
| Method and path | |
|---|---|
GET /platform-services |
the catalog (image, licence, AGPL flag, footprint, notice) with the team's instance of each kind |
POST /platform-services |
set one up (202): kind, serverId, optional hostname, publicUrl, bucket, retentionDays, providerName, variant (pocketid default, or authelia; single sign-on), replace |
GET /platform-services/{id} |
status, step, message, wiring, warnings, details, people |
POST /platform-services/{id}/retry ยท /reconnect |
resume a failed one; run the wiring again |
POST /platform-services/{id}/reveal |
S3, Loki and Grafana credentials (audited) |
PUT /platform-services/{id}/backup |
schedule the data backup to a destination |
GET/POST /platform-services/{id}/users, POST .../users/{username}/reset-password, DELETE .../users/{username} |
people of the Authelia provider (409 users_not_managed for Pocket ID) |
POST /platform-services/{id}/admin-link |
a one-time sign-in link for the first Pocket ID administrator; returned once, never stored (instance administrators) |
DELETE /platform-services/{id}?confirm=<kind>&deleteData=&force= |
unwire and remove |
Audit actions: platform.provision, platform.remove, platform.rewire, platform.reveal, platform.backup, platform.user.add, platform.user.reset_password, platform.user.remove, platform.admin_link (who asked; never the link), plus the usual entries of what the job creates (destination, drain, provider, deployments).
Upgrading the images#
The versions are constants in internal/platform/catalog.go, each pinned by tag and digest of the multi-platform index. To bump one: read the product's release notes, docker buildx imagetools inspect <image>:<tag> for the digest, change the constant, and run go test ./internal/platform (it refuses an unpinned image and re-renders every configuration). Existing platform services keep their pinned version until you deploy a newer one: the service's image is staged config, so change it on the service in the platform project and deploy. Authelia and Headscale read their configuration at start, so a new minor version may need a configuration change: check the release notes first. For Pocket ID use docker buildx imagetools inspect ghcr.io/pocket-id/pocket-id:<tag> (or a registry HEAD on the manifest); its database migrates itself on start, and it refuses to start an older version on a newer database.
Limits#
- One of each kind per team; no second S3 or a second Loki through this page (add your own destinations and drains as usual).
- The registry holds every image your build servers built; its volume is not scheduled for backup (images can be rebuilt from their commits). Losing it breaks rollbacks of releases built on a build server until you rebuild them (Rebuild commit).
- Single server per service; Grafana is on Loki's server. Moving a platform service between servers is a normal service move (the volume is not migrated), which breaks the public URL if the hostname's DNS does not follow.
- The public URL is fixed at set-up. Changing the hostname means removing and setting up again.
- Pocket ID is the owner's exception to the "stable for two years" rule for new products (its 1.0 is 17 months old, 2025-05); the pinned release is v2.19.1.
- The sign-in callback is registered with this instance's public URL (
NEBULACTRL_PUBLIC_URL) at set-up. If that URL changes later, remove single sign-on and set it up again. - Not verified against real third parties: single sign-on was verified end to end against the pinned Authelia image (login, authorization code with PKCE, token, userinfo) outside NebulaCtrl, and the Pocket ID wiring (discovery, first administrator, OpenID client and secret, one-time token, token endpoint authentication) against the pinned Pocket ID image on a network without internet access (
go test -tags pocketid_smoke), but not a passkey registration in a browser; the browser sign-in through NebulaCtrl on a real public hostname, and Headscale with real Tailscale clients, were not exercised. The mesh's policy flow was checked in a design spike against Headscale 0.29.6 on one machine, not across real networks or with Tailscale SaaS (mesh limits). The S3 round trip (set-up, bucket, destination test, database backup into it, removal) runs in the full integration tier (TestPlatformS3BackupRoundTrip).