148 lines
12 KiB
Markdown
148 lines
12 KiB
Markdown
# Docker discovery and dynamic vhosts
|
|
|
|
RustProxy 0.22.3 includes an optional standalone Docker discovery agent. The RustProxy core never opens the Docker socket. Run `docker-agent/rustproxy-docker-agent` on every Docker host that should be discovered; RustProxy polls each agent over HTTPS by default using that agent's Bearer token and certificate pin.
|
|
|
|
## State model
|
|
|
|
- **Desired state** is stored in the normal RustProxy SQLite configuration: agent endpoint/token, per-agent base domain, optional default backend IP, listener IDs, optional frontend certificate and service-to-vhost assignments.
|
|
- **Observed state** stays in memory: Docker host addresses, container metadata and every non-loopback published TCP binding.
|
|
- A successful poll reconciles enabled assignments into ephemeral vhosts. Polling does not write a new configuration revision.
|
|
- If a published host port, binding or detected host address changes, the next poll rebuilds the effective route.
|
|
- If a service disappears, its dynamic vhost disappears until the service is observed again.
|
|
- Docker discovery is optional; ordinary vhosts and all other proxy functions remain independent of Docker.
|
|
|
|
The agent returns only running containers with Docker-published TCP ports. Bindings limited to loopback (`127.0.0.0/8` or `::1`) are excluded. Multiple bindings for the same container port are retained instead of being collapsed to one address/port pair.
|
|
|
|
## Install an agent on a Docker host
|
|
|
|
The agent is written in Go and uses only the standard library.
|
|
|
|
```bash
|
|
cd docker-agent
|
|
sudo ./install.sh
|
|
```
|
|
|
|
The installer runs the Go tests, builds one binary, creates the `rustproxy-docker-agent` system user, generates a random token on first install, starts `rustproxy-docker-agent.service` and prints the persistent self-signed certificate SHA-256 fingerprint.
|
|
|
|
Manual build:
|
|
|
|
```bash
|
|
cd docker-agent
|
|
go test ./...
|
|
go build -trimpath -ldflags='-s -w' -o rustproxy-docker-agent .
|
|
```
|
|
|
|
Agent configuration is read from `/etc/rustproxy/docker-agent.env`:
|
|
|
|
```text
|
|
RUSTPROXY_DOCKER_AGENT_LISTEN=0.0.0.0:9876
|
|
RUSTPROXY_DOCKER_AGENT_TOKEN=<long-random-token>
|
|
RUSTPROXY_DOCKER_AGENT_TLS=true
|
|
RUSTPROXY_DOCKER_AGENT_CERT_FILE=/var/lib/rustproxy-docker-agent/agent.crt
|
|
RUSTPROXY_DOCKER_AGENT_KEY_FILE=/var/lib/rustproxy-docker-agent/agent.key
|
|
RUSTPROXY_DOCKER_ADVERTISE_IP=
|
|
RUSTPROXY_DOCKER_PUBLIC_IP_URL=https://api.ipify.org
|
|
```
|
|
|
|
`RUSTPROXY_DOCKER_ADVERTISE_IP` is only a preferred address hint; it does not hide the other host addresses. The inventory includes usable addresses found on active host network interfaces, the primary routed address, the optional public-IP lookup result and the advertised address. Loopback/link-local and obvious Docker-generated bridge/veth interfaces are omitted; explicit Docker bind addresses are still preserved with each service. Each host-address entry includes its source and, where available, interface name/scope.
|
|
|
|
The inventory endpoint is `GET /v1/inventory` and requires `Authorization: Bearer <token>`. `GET /healthz` exposes no inventory. HTTPS is enabled by default. The agent creates an ECDSA P-256 self-signed certificate once and keeps it in `/var/lib/rustproxy-docker-agent/`; the key is mode `0600`.
|
|
|
|
`install.sh` prints both the generated/reused bearer token and the SHA-256 certificate fingerprint. Copy the SHA-256 fingerprint printed by `install.sh` into the agent settings in RustProxy. You can print it again with `sudo rustproxy-docker-agent --print-fingerprint`. With a fingerprint configured, RustProxy accepts only that exact certificate and still checks the TLS handshake signatures. If the agent uses a certificate issued by a normal public CA, leave the fingerprint empty to use standard CA/hostname verification. Plain HTTP is retained only for explicit compatibility setups and exposes the Bearer token to the network.
|
|
|
|
Docker-socket access is privileged. Restrict access to the agent port to the RustProxy host even when TLS is enabled.
|
|
|
|
## Configure multiple Docker hosts
|
|
|
|
Open **Settings -> Docker agents** and create one entry for every Docker host. Each agent also has its own inventory **poll interval** (5..3600 seconds, 30 seconds by default); manual refresh bypasses the interval. Agents are independent and may use different domains, for example:
|
|
|
|
```text
|
|
docker-a -> docker1.lab.local
|
|
docker-b -> docker2.lab.local
|
|
docker-c -> apps.example.net
|
|
```
|
|
|
|
For each agent configure:
|
|
|
|
- stable agent ID and display name;
|
|
- agent endpoint, normally `https://10.10.0.21:9876`;
|
|
- the Bearer token from that Docker host;
|
|
- the SHA-256 certificate fingerprint printed by the default self-signed agent installer, or an empty fingerprint only for a public-CA certificate;
|
|
- that agent's base domain;
|
|
- optional default backend IP;
|
|
- one or more RustProxy listener IDs;
|
|
- an optional configured frontend certificate to apply to every generated vhost for that agent;
|
|
- an optional default ACL profile inherited by generated vhosts unless a service selects its own profile.
|
|
|
|
RustProxy polls each enabled agent according to its configured interval (30 seconds by default, checked by the scheduler in bounded steps). The 5-second Live-view refresh reads only cached RustProxy memory and does not force a Docker request. **Docker vhosts** shows the latest cached observation for every configured agent.
|
|
|
|
## Publish a discovered service
|
|
|
|
Use the search field in **Docker vhosts** to filter by agent, service, hostname or alias. Individual service rows can be hidden from the normal view; **Show Hidden** beside Docker agents settings reveals them so they can be restored. This is a per-browser view preference and does not disable or unpublish the vhost.
|
|
|
|
For each discovered service you can change the mapping directly in **Docker vhosts**:
|
|
|
|
- **Hostname**: a short name such as `app1`, or a complete FQDN such as `portal.example.com`;
|
|
- **Aliases**: zero or more additional short names/FQDNs, edited with the same add/remove chips as normal vhost hostnames;
|
|
- **Backend host IP**: Auto/default or one concrete address detected on the Docker host;
|
|
- **Backend protocol**: HTTP or HTTPS;
|
|
- **Preserve Host**: forward the requested hostname to the container when enabled; new HTTP/1.1 assignments default to enabled, while an existing explicit disabled value is preserved;
|
|
- **Redirect HTTP to HTTPS**: per-app Force SSL switch. New assignments default on when the agent has a usable plain-HTTP + TLS listener pair; assignments saved before 0.18.8 retain their previous redirect-off behavior until explicitly saved with the new switch;
|
|
- **WebSockets**: enable or disable classic HTTP/1.1 Upgrade forwarding for this application;
|
|
- **Cache**: enable or disable normal RustProxy caching for this generated vhost;
|
|
- **Transform**: opt this generated vhost into the global static-response transform engine; it is off by default and still requires global transformation to be enabled;
|
|
- **Backend health check**: HTTP/TCP/off active-probe mode, path/Host override, interval, timeout, slow threshold, rise/fall counters and accepted HTTP status range.
|
|
- **Passive Backend Health**: independent opt-in policy, OFF by default, with separate connect/timeout/I/O/HTTP-5xx/slow-response switches and optional passive rise/fall overrides.
|
|
- **Access control**: an optional reusable ACL profile plus an optional inline IP/CIDR ACL for this application. A per-app profile overrides the agent default profile; inline rules remain cumulative with the effective profile.
|
|
- **Proxy timeouts**: optional connect, response-header, read-idle and send-idle timeouts; blank values inherit RustProxy defaults.
|
|
|
|
Publish/Unpublish is applied immediately. Editing a published service uses the per-service **Apply changes** action; there is no page-wide assignment save step. The optional **Live view** switch refreshes only cached RustProxy observations/health state and does not contact the Docker agent.
|
|
|
|
A short name is expanded below that agent's base domain. With base domain `docker1.lab.local`, `app1` becomes `app1.docker1.lab.local`. A full FQDN is not rewritten. Aliases use the same rule, so `app-old` becomes `app-old.docker1.lab.local`, while `legacy.example.com` remains unchanged.
|
|
|
|
This makes renaming and aliasing a Docker-backed vhost a normal RustProxy configuration change; the container/service identity remains the stable link to discovery.
|
|
|
|
## Multiple Docker-host IPs and bindings
|
|
|
|
The agent reports all usable addresses it can see on the Docker host. This is intended for hosts that have separate public, management and LAN addresses.
|
|
|
|
RustProxy also keeps every published Docker binding. For example, one container port may be published as:
|
|
|
|
```text
|
|
10.21.0.15:18080
|
|
192.168.50.15:28080
|
|
```
|
|
|
|
Selecting `10.21.0.15` for the Docker vhost uses port `18080`; selecting `192.168.50.15` uses `28080`. For Docker wildcard bindings (`0.0.0.0` / `::`), RustProxy uses the selected host address with the wildcard binding's published port.
|
|
|
|
Backend selection order is:
|
|
|
|
1. the Docker vhost's explicit `target_ip`;
|
|
2. the agent's optional `default_target_ip`;
|
|
3. a concrete Docker bind address;
|
|
4. the agent's preferred/primary detected address when compatible with the binding;
|
|
5. another compatible detected host address.
|
|
|
|
An explicit target that is not compatible with the service's Docker bindings does not silently fall back to another IP; that dynamic vhost is not materialized until the selected address/binding is available.
|
|
|
|
The generated backend is pinned to the selected IP through an address allowlist and uses the matching Docker-published host port. Cache, Transform, WebSockets and HTTP-to-HTTPS redirect are materialized from the per-app switches. If the Docker agent has a frontend certificate selected, that certificate ID is copied to every generated vhost and participates in the normal TLS/SNI certificate validation and ownership rules. The effective ACL profile is the per-app profile when selected, otherwise the agent default profile. Active backend health checks default to TCP. Passive Backend Health is materialized from the Docker assignment separately and defaults to OFF, so discovery refresh/restart does not silently re-enable production-traffic quarantine. The Docker vhosts page exposes the same passive policy as ordinary vhosts. Normal 5xx/timeouts/connect/I/O/slow-response observations remain visible in counters while passive health is OFF. Active checks can be switched between HTTP/TCP/off per service without changing the passive policy.
|
|
|
|
## DNS
|
|
|
|
DNS for Docker vhost names must resolve to the RustProxy frontend/listener that should receive the HTTP(S) request. For a per-agent base domain this is normally a wildcard record, for example:
|
|
|
|
```text
|
|
*.docker1.lab.local -> <RustProxy frontend IP>
|
|
*.docker2.lab.local -> <RustProxy frontend IP>
|
|
```
|
|
|
|
Custom FQDNs and aliases likewise need DNS records pointing at the relevant RustProxy frontend. The Docker-host addresses detected by the agent are backend routing candidates; they are not automatically published to DNS.
|
|
|
|
RustProxy does not change DNS records itself in 0.22.3.
|
|
## Prometheus metrics
|
|
|
|
Docker discovery has its own Prometheus collector switch, `prometheus.include_docker_vhosts`, enabled by default. It is intentionally separate from `include_vhosts` and `include_backends`. Dynamic Docker routes never appear under ordinary `rust_proxy_vhost_*` or `rust_proxy_backend_*` families.
|
|
|
|
The Docker collector exports bounded configured labels only: agent ID, generated Docker-vhost ID, assigned service key, configured domains/aliases and backend ID. It includes `rust_proxy_docker_vhost_*` traffic/status/duration metrics and `rust_proxy_docker_backend_*` readiness, health, active work, requests/errors/bytes, current/EWMA latency, failure/success counters, health transitions and last-probe observations. No token, agent endpoint, backend URL or arbitrary request Host value is exported.
|
|
|