224 lines
16 KiB
Markdown
224 lines
16 KiB
Markdown
# Transport contract
|
|
|
|
This document describes the implementation contract of 0.22.3. It is not a record
|
|
of successful Rust execution; see TEST_REPORT.md for verification status.
|
|
|
|
## Protocol matrix
|
|
|
|
| Connection | Implemented mode |
|
|
|---|---|
|
|
| Client to cleartext listener | HTTP/1.1 plus opt-in HTTP/2 prior knowledge and HTTP/1.1 `Upgrade: h2c`; WebSocket Upgrade on H1 and RFC 8441 Extended CONNECT on H2 when enabled per vhost |
|
|
| Client to TLS listener | HTTP/1.1 or HTTP/2 through ALPN; WebSocket uses H1 Upgrade by default or opt-in RFC 8441 Extended CONNECT on H2 |
|
|
| Client to HTTP/3 TLS listener | HTTP/3 over QUIC; RFC 9220 WebSocket Extended CONNECT only when a bound vhost explicitly opts in |
|
|
| Proxy to HTTP backend, `http1` | HTTP/1.1 |
|
|
| Proxy to HTTPS backend, `http1` | TLS with HTTP/1.1 |
|
|
| Proxy to HTTP backend, `http2` | HTTP/2 prior knowledge; the backend must support it |
|
|
| Proxy to HTTPS backend, `http2` | Prefer HTTP/2 through ALPN, retain HTTP/1.1 |
|
|
| Proxy to HTTPS backend, `auto` | ALPN HTTP/2 or HTTP/1.1 |
|
|
| Proxy to HTTP backend, `auto` | HTTP/1.1; no automatic h2c upgrade |
|
|
| Proxy to a WebSocket backend | HTTP/1.1 Upgrade over HTTP or HTTPS |
|
|
|
|
Cleartext downstream HTTP/2 is an explicit listener opt-in. The same listener accepts both
|
|
the HTTP/2 prior-knowledge preface and a standards-oriented body-free HTTP/1.1 `Upgrade: h2c`
|
|
bootstrap. Upgrade requests must include `Connection: Upgrade, HTTP2-Settings`, exactly one
|
|
`Upgrade: h2c` and one valid base64url `HTTP2-Settings` field. RustProxy sends `101`, waits for
|
|
the client's normal HTTP/2 connection preface and initial SETTINGS frame, then reconstructs the
|
|
accepted HTTP/1.1 request as HTTP/2 stream 1 before handing later streams to Hyper's ordinary H2
|
|
state machine. Body-bearing or malformed upgrade requests are rejected before `101`. HTTP/1.1
|
|
remains available on the same cleartext listener.
|
|
|
|
Enabling WebSocket on a vhost keeps the HTTP/1.1 Upgrade path by default. `websocket_rfc8441`
|
|
and `websocket_rfc9220` are independent per-vhost opt-ins for HTTP/2 and HTTP/3 Extended
|
|
CONNECT with `:protocol=websocket`. HTTP/2 listeners advertise ENABLE_CONNECT_PROTOCOL when
|
|
serving H2; QUIC advertises it when required by the HTTP/3 transport. Routing still rejects
|
|
Extended CONNECT on vhosts that did not opt in. Ordinary CONNECT and other Extended CONNECT
|
|
protocols remain rejected because RustProxy is not an open forward proxy. RFC 8441 may bridge
|
|
to an HTTP/1.1 WebSocket backend or use a backend explicitly configured for HTTP/2.
|
|
|
|
Since 0.32.4, `preserve_host: true` is supported with `http1`, `auto`, `http2`
|
|
and strict `h2`. H1 carries the public Host; on H2 the outgoing `:authority`
|
|
follows that Host. The *network* destination and DNS/ACL policy stay pinned to
|
|
the configured backend URL. TLS SNI and certificate validation independently use
|
|
`tls_server_name` when supplied or the configured origin otherwise.
|
|
The client hostname also remains available via `X-Forwarded-Host`. HTTP health
|
|
probes are separate H1 requests and may need an explicit `health_check.host`.
|
|
|
|
For ordinary proxy traffic, an explicit `protocol=http2` backend is forwarded with an
|
|
HTTP/2 request version. Cleartext `http://` uses h2 prior knowledge. HTTPS keeps the
|
|
existing ALPN policy and offers HTTP/2 with HTTP/1.1 compatibility; RustProxy does not
|
|
replay a failed TLS request merely to downgrade protocols.
|
|
|
|
Balanced routes may perform bounded failover retry only for body-free `GET`, `HEAD` and
|
|
`OPTIONS`. Defaults allow one retry on connection failure, response-header timeout or
|
|
`502/503/504`, always selecting another currently healthy backend. Path-pinned routes,
|
|
WebSocket handshakes and write/body-bearing requests are never automatically replayed.
|
|
|
|
## DNS and upstream TLS
|
|
|
|
Only explicit HTTP/HTTPS backend origins are accepted; no credentials, path, query
|
|
or fragment are allowed. Client requests cannot choose an arbitrary upstream URL.
|
|
|
|
Each new DNS connection uses a guarded resolver. Every returned IPv4/IPv6 address
|
|
must pass the configured address policy, with IPv4-mapped IPv6 normalized for checks.
|
|
A mixed safe/unsafe response is rejected as a whole. Up to 32 answers and 16 OS
|
|
resolver workers are allowed. Validated answers go directly to the HTTP connector:
|
|
there is no second unvalidated DNS lookup before dialing. Existing pooled sockets
|
|
remain bound to their original peer until closed; they do not silently move when
|
|
DNS changes. New snapshots get new connection pools.
|
|
|
|
`allow_private` is explicit approval for eligible private/loopback addresses, not a
|
|
universal SSRF bypass. `address_allowlist` further restricts eligible addresses by
|
|
CIDR. Include both address families when a hostname legitimately resolves to both.
|
|
Link-local/metadata, multicast, unspecified and certain special ranges remain
|
|
blocked. Remote origins may use any valid backend port, including 9090. A target that
|
|
resolves to an actual/local RustProxy proxy, TLS or administration listener remains
|
|
blocked by default, including validated DNS answers. An administrator may intentionally
|
|
override only that self-target guard per backend with `accept_local_listener_risk: true`;
|
|
loopback/private targets still require `allow_private: true`. This opt-in can create a
|
|
proxy loop or expose the administration plane, so keep it disabled unless that route is
|
|
deliberate and separately access-controlled. For example, an intentional local backend
|
|
`http://127.0.0.1:9090` requires both `allow_private: true` and
|
|
`accept_local_listener_risk: true`; this only permits the connection and does not bypass
|
|
the administration listener's own Host/Origin/authentication policy. The special-range
|
|
policy is a basic safeguard, not a substitute for network egress controls or an independent
|
|
security audit.
|
|
|
|
HTTPS upstream transport remains encrypted, but certificate-chain/hostname verification
|
|
is **off by default**. Set `tls_verify: true` to authenticate the backend identity. An
|
|
optional `tls_ca_pem` adds up to 16 trust anchors within 64 KiB per backend when
|
|
verification is enabled. The configured `tls_server_name`, when present, determines TLS
|
|
identity while DNS/TCP still targets the configured URL. Upstream redirects are returned
|
|
to the client, not followed internally.
|
|
|
|
## Public certificates
|
|
|
|
`dev.sh` enables `127.0.0.1:8443` by default. Direct binary startup enables TLS only
|
|
when `serve --tls-listen ADDRESS` or the corresponding environment setting is used.
|
|
Changing listener addresses or enabling a new listener requires a restart.
|
|
|
|
`data/certs/default.pem` and `default.key` are created once when needed. Generated
|
|
certificates are self-signed development material; they do not provide public trust.
|
|
Explicit bindings use exact hostname names and already-present certificate/key files.
|
|
Manual entries support two addressing modes: simple filenames are loaded from
|
|
`<data-dir>/certs`, while two absolute paths are read in place (for example Certbot
|
|
`/etc/letsencrypt/live/<name>/fullchain.pem` plus `privkey.pem`). Mixed relative/absolute
|
|
pairs are rejected. The API stores filenames/paths and names, never private key bytes.
|
|
CLI generate/import refuses to overwrite named files. Input size, regular-file status,
|
|
private-key permissions, matching keys, chain parsing and name coverage are validated.
|
|
|
|
```bash
|
|
rust-proxy tls generate --name app --hosts app.localhost,alias.localhost
|
|
rust-proxy tls import --name app-v2 --cert fullchain.pem --key private.key
|
|
rust-proxy tls deploy --id shared-cert \
|
|
--cert /etc/letsencrypt/live/example/fullchain.pem \
|
|
--key /etc/letsencrypt/live/example/privkey.pem
|
|
rust-proxy tls reload
|
|
```
|
|
|
|
Select these files in the panel or `tls.certificates` configuration. Set
|
|
`tls.default_certificate` to a binding ID to change the fallback. Unknown/no SNI
|
|
receives that fallback, not a default HTTP route. Unknown Host still returns 404.
|
|
Wildcard/SAN coverage and certificate expiry/issuer/renewal status are exposed by the certificate inventory; certificate ownership remains explicit per certificate id.
|
|
|
|
`tls deploy` is intended for an external issuer such as a system Certbot. Certbot's
|
|
`live/*.pem` symlinks are resolved locally; the final files are bounded regular files and
|
|
the private key must not be group/world readable. The local CLI performs bounded file/permission checks and sends the PEM pair through the authenticated loopback deploy API. The running RustProxy process treats the request as a convergent upsert: it creates a missing manual `certificate_id`, rotates changed material through immutable generation-specific files, and returns a no-op for an identical RustProxy-owned generation. The same id may be assigned to many vhosts, which all switch together when the pair changes. RustProxy-managed ACME certificate ids are deliberately rejected by this command so one lineage never has two renewal owners.
|
|
|
|
`POST /api/certificates/reload` and `rust-proxy tls reload` re-read all currently configured
|
|
certificate/key files and publish a fresh public TLS snapshot without changing the desired
|
|
configuration or its revision. This supports an operator that replaces the already configured
|
|
files by another safe local mechanism. Validation and file reads happen on bounded control-plane
|
|
workers; invalid material leaves the previous snapshot active. New public TLS/H2 and HTTP/3
|
|
handshakes use the refreshed snapshot, while established sessions/tunnels are not renegotiated.
|
|
The administration TLS listener remains startup-fixed and reports that a service restart is
|
|
required when its selected certificate changed.
|
|
|
|
For an externally managed/manual certificate whose configured files were replaced in place, `POST /api/certificates/{certificate_id}/reload` provides the same revision-stable hot-publication contract while targeting that certificate ID. For absolute external paths RustProxy canonicalizes the path again on every reload, intentionally following Certbot `live/` symlinks to the current `archive/` generation before opening the final regular file. The TLS resolver is immutable, so RustProxy validates and republishes a complete snapshot atomically; this avoids partially updated SNI state. Managed ACME certificate IDs are not accepted by this manual reload path.
|
|
|
|
External absolute-path files are not copied into RustProxy full backups. Their path strings remain in configuration; restore or mount the certificate material separately on the destination host before starting TLS. The `rustproxy` service identity must be able to traverse every parent directory and read both files. RustProxy accepts POSIX ACL read access and does not require an exact chmod value. External certificate/key files must not be group/world-writable, and private keys must not be world-readable. The configured and resolved parent chains must be root- or RustProxy-owned and not group/world-writable, except for root-owned sticky ancestors such as `/tmp`. The final file is opened with `O_NOFOLLOW` after path validation.
|
|
|
|
## WebSocket behavior
|
|
|
|
The default WebSocket path validates an HTTP/1.1 GET upgrade, version 13, one valid key,
|
|
connection upgrade tokens and a body-free handshake. RFC 8441 over HTTP/2 and RFC 9220 over
|
|
HTTP/3 must be enabled explicitly for the selected vhost and use Extended CONNECT rather
|
|
than H1-only Upgrade/Connection/Key headers. When RFC 8441/9220 is bridged to an HTTP/1.1
|
|
backend, RustProxy performs and validates the backend RFC 6455 handshake before exposing a
|
|
successful downstream tunnel. Direct HTTP/2 upstream RFC 8441 validates the 2xx response and
|
|
selected subprotocol. Protected handshake fields cannot be replaced by generic custom-header
|
|
rules. Unsupported/malformed upgrades fail; unsolicited upstream 101 responses become a
|
|
gateway error.
|
|
|
|
After an accepted handshake the proxy copies bytes in both directions using bounded
|
|
buffers and backpressure. It does not aggregate messages, interpret application
|
|
payloads, log frames or hardcode a particular WebSocket application. Fragmentation,
|
|
control frames and negotiated extensions remain the peers' responsibility.
|
|
The demo echo server separately caps messages/frames at 64 KiB.
|
|
|
|
Request and TCP-connection leases survive Hyper's upgrade and are held until the
|
|
tunnel closes. An independent idle timeout observes traffic in either direction.
|
|
Shutdown cancels tunnels; routing/ACL changes affect new handshakes, not existing tunnels.
|
|
The administration API and dashboard expose active tunnels with a stable exchange ID, client,
|
|
host, upstream, transport, duration and RX/TX counters. An administrator can cancel one tunnel;
|
|
the action and supplied reason are written to the audit log.
|
|
|
|
## HTTP/2 connection retirement
|
|
|
|
Downstream H2 idle retirement is activity-aware: RustProxy tracks transport I/O and accepted streams, rechecks that a connection is actually unused, then asks Hyper for graceful shutdown. Hyper sends GOAWAY and refuses newer streams while accepted streams continue. Each H2/H2C listener uses `binding.http2_settings.max_connection_lifetime_seconds` for the independent absolute lifetime (zero disables it) and `binding.http2_settings.drain_timeout_seconds` for the maximum drain before TCP is forcibly dropped. Older configurations without per-listener values inherit the legacy top-level `http2` settings. RFC 8441 Extended CONNECT/WebSocket streams remain active for this accounting until their tunnel body ends.
|
|
|
|
## Resource and timeout defaults
|
|
|
|
| Resource | Default / bound |
|
|
|---|---|
|
|
| Shared public TCP connections, HTTP + TLS | 512; configurable up to 4096 |
|
|
| Active request/body/tunnel exchanges | 256; configurable up to 4096 |
|
|
| Concurrent HTTP/2 streams per public connection | 64 |
|
|
| HTTP/2 header list / send buffer | 32 KiB / 64 KiB |
|
|
| HTTP/2 stream / connection receive window | 65,535 bytes / 1 MiB |
|
|
| HTTP/2 idle / max lifetime / drain | 300 s / disabled (0) / 10 s |
|
|
| Public HTTP/1 header timeout | 10 seconds |
|
|
| TLS accept / upstream connect including TLS | 10 seconds |
|
|
| DNS lookup caller timeout / OS workers | 5 seconds / 16 |
|
|
| Backend response-header deadline | 120 seconds, including upload until headers |
|
|
| HTTP body idle timeout | 60 seconds |
|
|
| WebSocket idle timeout / upgrade completion | 300 seconds / 10 seconds |
|
|
| Idle pooled upstream connections | 4 per backend origin, 30-second idle timeout |
|
|
| Configuration writers / log-viewer streams | 8 / 8 |
|
|
|
|
CLI serve flags expose connection/in-flight counts and HTTP body, WebSocket idle,
|
|
upstream and shutdown timeouts. OS DNS calls cannot be forcibly cancelled; their
|
|
worker permits remain occupied until completion. Runtime shutdown stops waiting
|
|
for blocking workers after an additional bounded timeout.
|
|
|
|
Request and response bodies stream without collection, including trailers. Body-bearing/write requests are not transparently replayed. Replay-safe body-free
|
|
GET/HEAD/OPTIONS may use the bounded backend retry policy described above. Bytes counted at the body layer are not
|
|
acknowledged network bytes. WebSocket counters measure raw frame bytes separately
|
|
from HTTP body counters and exclude TLS/TCP overhead.
|
|
|
|
## Vhost sockets and keepalive in 0.22.3
|
|
|
|
Public connections across global and vhost endpoints share the same connection
|
|
budget. Per-vhost routing uses the actual accepted local IP/port/TLS state, including
|
|
when an OS wildcard socket is shared. See VHOSTS.md for overlap rules and restrictions.
|
|
Changing sockets or their ALPN policy needs restart. Certificate refresh remains hot.
|
|
|
|
TLS `http2: true` advertises h2 followed by http/1.1; false advertises HTTP/1.1 only.
|
|
An HTTP/1.1-only client, including one without ALPN, can connect when h2 is enabled.
|
|
HTTPS origins configured `http2` also negotiate h2 or HTTP/1.1. This is one TLS
|
|
negotiation, not a failed-request retry or TLS-verification downgrade. Plaintext
|
|
h2c remains explicit and strict: when `h2c: true`, a cleartext listener accepts both HTTP/2
|
|
prior knowledge and validated HTTP/1.1 `Upgrade: h2c`. The Upgrade bootstrap is body-free,
|
|
requires `HTTP2-Settings`, and becomes HTTP/2 stream 1 after `101`. Optional downstream HTTP/3
|
|
remains a separate QUIC listener capability (see HTTP3.md).
|
|
|
|
Pool keepalive settings configure idle connection reuse, idle timeout/limit, TCP
|
|
keepalive and HTTP/2 active-connection pings. They also bound each backend Hyper client
|
|
generation by request count/age and cap the upstream HTTP/1 adaptive read buffer. Client
|
|
generation recycling is local to the backend pool and does not publish a new vhost
|
|
snapshot or purge cache. Health probes are separate no-reuse HTTP/1.1 or raw TCP
|
|
operations; their concurrency cap is 16. Existing active exchanges are not intentionally
|
|
killed by disabling reuse, recycling a generation or draining a member. Config
|
|
publication still rebuilds client pools while preserving compatible observations and
|
|
existing exchange leases. The full contract and bounds are in VHOSTS.md. Keepalived/VRRP
|
|
is not involved.
|