345 lines
23 KiB
Markdown
345 lines
23 KiB
Markdown
# Listener and response settings
|
|
|
|
## Configuration and update semantics
|
|
|
|
GET `/api/config` returns a revisioned envelope. PUT the entire document with
|
|
`expected_revision`. Defaults apply when fields are omitted. Unknown fields are
|
|
rejected; failed validation/persistence leaves the old snapshot active. Existing
|
|
backend/route/TLS/ACL data must be retained when changing settings. The panel does
|
|
this and preserves dirty settings forms across status/language refreshes.
|
|
|
|
```json
|
|
{
|
|
"listeners": {
|
|
"endpoints": [
|
|
{"id":"http", "name":"HTTP", "scope":"public", "enabled":true,
|
|
"binding":{"address":"0.0.0.0:80", "tls":false, "http2":false, "http3":null}},
|
|
{"id":"https", "name":"HTTPS", "scope":"public", "enabled":true,
|
|
"binding":{"address":"0.0.0.0:443", "tls":true, "http2":true,
|
|
"http2_settings":{"idle_timeout_seconds":300, "max_connection_lifetime_seconds":0, "drain_timeout_seconds":10}}}
|
|
]
|
|
},
|
|
"server_header": {"mode":"custom", "value":"RustProxy/v0.20.0"},
|
|
"compression": {"gzip":true, "br":true, "zstd":true, "deflate":false, "min_bytes":1024},
|
|
"headers": {
|
|
"request": [{"op":"set", "name":"x-proxy-name", "value":"RustProxy"}],
|
|
"response": [
|
|
{"op":"remove", "name":"x-powered-by"},
|
|
{"op":"set", "name":"x-served-by", "value":"RustProxy"}
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
This is a settings fragment, not a replacement for the existing routing/vhost document.
|
|
A complete opt-in example is in `examples/cache-dashboard.json`.
|
|
|
|
The dashboard exposes the complete raw JSON editor under **Settings -> Manual config**.
|
|
The same card exposes **Revision history**, which reads the newest 512 retained configuration revisions from SQLite schema 8 and groups them by month/day. Selecting a revision shows timestamp, actor and a bounded path-level before/after diff against the previous retained revision. An older retained revision can be restored from the same dialog; RustProxy validates it through the normal configuration update path and saves it as a new revision, preserving the prior history.
|
|
The historical `/configuration` browser route is retained only as a compatibility alias and
|
|
opens that Settings block; it is no longer a primary-navigation item. Prefer the structured
|
|
section editors for routine changes so unrelated sections are not republished.
|
|
|
|
## Administration HTTPS
|
|
|
|
`administration` is separate from public proxy listeners. It defaults to the startup
|
|
loopback address and plain HTTP. Persisted address/certificate changes are validated
|
|
immediately but take effect after process restart. A non-loopback persisted bind is
|
|
accepted only when `administration.tls` is true. Plain remote HTTP remains available only when process startup explicitly allows it with
|
|
`--allow-remote-admin`.
|
|
|
|
```json
|
|
{
|
|
"administration": {
|
|
"listen": "0.0.0.0:9090",
|
|
"tls": true,
|
|
"certificate_id": "admin-example",
|
|
"allowed_hosts": ["admin.example.com"],
|
|
"session_days": 30
|
|
}
|
|
}
|
|
```
|
|
|
|
`certificate_id: null` uses `data/certs/default.pem` and `default.key`, generated by
|
|
the application. A selected inventory certificate is loaded from the same private
|
|
certificate directory. `session_days` controls the absolute browser-session lifetime
|
|
and must be 7..30 (default 30). It applies without restart; changing listener/TLS/certificate
|
|
settings still requires restart. Session cookies are HttpOnly/SameSite=Strict and receive
|
|
the `Secure` attribute on HTTPS. When HTTPS is enabled, accidental plain HTTP received on the same administration
|
|
port receives a permanent redirect to the same authority/path over `https://`; the
|
|
plaintext path contains no administration handlers. An empty `allowed_hosts` accepts
|
|
valid DNS Host values on the configured HTTPS port as well as valid numeric destination
|
|
addresses. Once `allowed_hosts` contains entries it becomes a strict DNS allowlist for
|
|
browser-facing DNS authorities. An explicitly listed hostname may arrive from a trusted
|
|
reverse proxy on an external port such as 443 even when the private admin socket is 9090.
|
|
Direct IP/`localhost` access is still accepted only on the actual configured admin port.
|
|
For HTTPS termination in front of a plain local admin socket, the trusted proxy must pass
|
|
its external Host and scheme; `X-Forwarded-Host` is accepted only from loopback/configured
|
|
trusted proxy peers and `X-Forwarded-Proto` additionally follows the forwarded-proto trust
|
|
policy. Origin/CSRF comparison and the session cookie Secure flag use that external origin.
|
|
Use a firewall and normal network policy even with HTTPS.
|
|
|
|
## Structured access logging
|
|
|
|
The **Settings -> Logging** page is split into **Outputs**, **JSON format** and **Error log** subsections, using the same in-card sub-tab layout and keyboard interaction as Cache. Outputs controls optional external access-log sinks while the existing bounded live ring remains available to the panel. Sink I/O is performed by a bounded background queue; request processing only performs a non-blocking enqueue. Supported destinations:
|
|
|
|
- JSONL file append, either one combined file such as `/var/log/rustproxy/access.jsonl` or
|
|
one file per resolved vhost under a configurable directory (default `/var/log/rustproxy/vhosts`).
|
|
Unmatched traffic uses `_unmatched.jsonl`. The managed systemd service provisions
|
|
`/var/log/rustproxy` for the service account and keeps that directory writable under
|
|
`ProtectSystem=strict`. Other custom paths still need matching service permissions.
|
|
- Logstash TCP using `host:port` (IP or DNS). Each event is one JSON line.
|
|
- Loki HTTP(S) push. Optional bearer credentials are read from an environment variable
|
|
named by `bearer_token_env`; the secret itself is not stored in configuration.
|
|
|
|
`logging.format.fields` maps output field names to standard event fields; an empty map
|
|
emits the complete standard event. The panel provides Standard, Nginx-style, ECS-style
|
|
and Minimal presets plus a visual field selector/renamer with JSON preview. Advanced raw
|
|
mapping remains available for aliases not representable by the simple builder. Supported mapped sources include `vhost`, `route`, `cache_effective_enabled`, `transform_effective_enabled`, `backend`, request/response metadata and cache outcome fields. `static_fields` adds constant string fields. For low-level `protocol_error` events, RustProxy always preserves the required diagnostic envelope (`event`, error kind/detail, time, client/local/TLS/protocol fields, status and outcome) even with a custom mapping, so malformed-traffic records remain searchable in the built-in Logs explorer. Loki labels are separately configurable.
|
|
`/api/status.logging.sinks` reports queue capacity, dropped events, successfully delivered
|
|
events and sink errors.
|
|
|
|
## Hot and cold cache tiers
|
|
|
|
`cache.max_bytes` and `cache.max_entries` bound the HOT RAM tier. `max_bytes` is charged from retained body backing allocation plus entry metadata rather than only payload length. `cache.transient_max_bytes` separately bounds in-progress fills and full bodies moving between HOT and COLD; `cache.cold_queue_max_bytes` additionally caps queued demotions in bytes, and `cache.hot_eviction_max_objects` bounds one LRU admission attempt. The source defaults are 256 MiB HOT, 16 MiB maximum object, 64 MiB transient and 32 MiB COLD queue. The Settings panel calculates conservative/recommended/performance presets from the lower of host RAM and a finite cgroup memory limit without changing the COLD disk budget. It also warns when HOT + transient is a large fraction of effective RAM, when the additional configured cache budget is large compared with current host/cgroup headroom, when the COLD queue exceeds transient RAM, or when the object ceiling cannot fit in transient RAM. Automatic presets are disabled below 256 MiB effective RAM. On a 2 GiB effective-memory VM the recommended profile selects approximately 400 MiB HOT, 96 MiB transient, 16 MiB maximum object, 48 MiB COLD queue, 6400 HOT entries and a 64-object LRU admission limit. The optional persistent
|
|
cold tier is controlled by `cold_enabled`, `cold_path`, `cold_max_bytes`,
|
|
`cold_max_entries`, `cold_gc_interval_seconds`, `cold_gc_max_delete`,
|
|
`cold_after_seconds`, `cold_on_lru`, `cold_on_idle` and `cold_on_shutdown`.
|
|
`cold_gc_interval_seconds` defaults to 60 and `cold_gc_max_delete` to 256. Pressure GC
|
|
starts at roughly 90% of the configured byte/entry budget and cleans toward roughly 80%
|
|
without deleting more than the configured per-pass cap. `cold_path` is relative to the RustProxy data directory and cannot
|
|
contain parent traversal or an absolute path. The two demotion switches independently
|
|
select LRU-pressure and idle objects; shutdown flush is a best-effort direct write during
|
|
a graceful restart. Demotions use a small bounded queue and release hot accounting before
|
|
disk persistence so cold storage does not retain an unnecessary second body buffer in RAM.
|
|
Cold metadata is readable JSON and bodies are raw `.body` files; reads are size/SHA-256
|
|
checked before promotion to RAM. Since 0.21.0 objects are sharded by the first two SHA-256
|
|
hex characters (`cache/cold/00/...` through `ff/...`), with shard directories created on
|
|
first write. The previous flat Cold Cache is intentionally not migrated. Purge operations
|
|
clear both in-memory indexes and schedule disk cleanup; selective cleanup uses the same
|
|
GC/eviction path and runs asynchronously.
|
|
|
|
Version 0.22 adds foreground fill collapsing (`collapse_enabled`,
|
|
`collapse_wait_timeout_ms`, `collapse_timeout_fallback`), detailed stale-if-error policy
|
|
(`stale_if_error_timeout`, `stale_if_error_connection`, `stale_if_error_dns`,
|
|
`stale_if_error_tls`, `stale_if_error_statuses`) and opt-in proactive refresh
|
|
(`proactive_refresh_enabled`, `proactive_refresh_before_expiry_seconds`,
|
|
`proactive_refresh_percent`, `proactive_refresh_max_inflight`). Defaults preserve the
|
|
previous stale-if-error behavior; proactive refresh is disabled by default. Selective
|
|
PURGE/BAN is an operational API/panel action rather than persisted configuration. See
|
|
[CACHE.md](CACHE.md) for request-collapse, Range and invalidation semantics.
|
|
|
|
## Static-response transformation
|
|
|
|
`transform` is a formal identity-response stage between the upstream response and shared
|
|
cache storage. `transform.enabled` is the global kill switch, but every vhost must also
|
|
set its own `transform.enabled=true`; vhosts stay disabled by default even when the global
|
|
engine is enabled. Global defaults enable JavaScript and CSS transformation and leave
|
|
HTML, SVG and JSON off. `extensions` controls extension-based detection and
|
|
`max_body_bytes` (1 KiB..32 MiB, 2 MiB default) bounds aggregation/decoding.
|
|
|
|
The pipeline is `origin -> decode supported Content-Encoding -> transform -> cache
|
|
identity representation -> client compression`. JavaScript/CSS can be enabled normally;
|
|
HTML minification, SVG optimization and JSON compaction are independent optional formats.
|
|
Recognition uses safe configured extensions and known Content-Type values. Binary/image
|
|
formats outside those explicit transform types are not transformed.
|
|
|
|
Origin `Cache-Control: no-transform` always disables transformation. When bytes change,
|
|
RustProxy removes representation-specific length/digest/encoding metadata and creates a
|
|
new strong `rustproxy-*` ETag for the transformed representation. The original origin
|
|
ETag/Last-Modified are kept separately and used for origin revalidation, so a proxy ETag
|
|
is never sent upstream as though it validated the original object. The transformed
|
|
identity body is what enters cache, preventing repeated minification on every HIT.
|
|
|
|
## Global TLS protocol and cipher policy
|
|
|
|
`tls.policy` is the global baseline inherited by every TLS host. It controls TLS 1.2,
|
|
TLS 1.3 and optional rustls cipher-suite names. An empty `cipher_suites` list uses the
|
|
ring provider defaults. Settings shows both the currently effective cipher suites and
|
|
the complete selectable suite inventory. At least one protocol version and
|
|
one compatible effective suite are required. This policy is also used by administration
|
|
HTTPS. HTTP/3 requires TLS 1.3 and a TLS 1.3 suite. Vhosts can narrow TCP TLS with
|
|
`tls_policy`; when a vhost has HTTP/3 enabled on a selected H3-capable listener it must
|
|
match the global policy because QUIC TLS is shared per UDP listener. Available suite names are exposed in authenticated status for
|
|
the Settings editor.
|
|
|
|
## Listeners: saved intent, not hot rebinding
|
|
|
|
When an HTTP, HTTPS or administration address was explicitly supplied by systemd/CLI
|
|
(or the corresponding service environment variable), that exact startup bind is a
|
|
process contract: the panel shows it as locked and configuration updates cannot remove,
|
|
disable or change it. If an address was not explicitly supplied at startup, listener
|
|
entries remain freely addable/removable/editable subject to normal validation.
|
|
|
|
`listeners.endpoints` is the only persisted frontend socket configuration. Each
|
|
endpoint has a stable id/name, an informational scope label, an enabled flag and one
|
|
primary binding containing IP:port, TLS, HTTP/2 plus its optional lifecycle settings, and optional HTTP/3 settings. Optional
|
|
`additional_addresses` reuse that same transport policy on up to 15 further addresses.
|
|
When more than one address is configured every address must be explicit; `0.0.0.0` and
|
|
`::` are rejected so a multi-IP selection cannot silently mean every interface. IP/port
|
|
and transport capability belong here, not inside a vhost. IPv6 uses `[address]:port`.
|
|
|
|
A vhost references endpoints through `frontend.listener_ids`. Listener changes are
|
|
restart-required because they alter TCP/UDP sockets or TLS ALPN capability. The
|
|
listener editor owns the HTTP/2/H2C and HTTP/3 enable/disable switches. An H2/H2C row
|
|
links to **HTTP connections -> HTTP/2** for idle, absolute-lifetime and graceful-drain
|
|
limits; an enabled H3 row links to **HTTP connections -> HTTP/3** for advertisement,
|
|
connection, stream and idle-timeout limits. Both tuning pages are listener-scoped and
|
|
do not own protocol enablement. Vhost H2/H3 switches only narrow the protocols already
|
|
enabled by the selected endpoints; they never create a protocol that Settings disabled.
|
|
|
|
The administration listener is independent and cannot share a frontend port.
|
|
Bindings must be valid local unicast/wildcard addresses; conflicting overlapping
|
|
endpoints are rejected rather than silently widened.
|
|
|
|
## Server and other client headers
|
|
|
|
`server_header.mode` is `preserve` (default), `hide`, or `custom`. Custom values are
|
|
nonempty, bounded to 256 bytes and validated against header injection. Preserve
|
|
means retain the result of origin/global/route rules; it does not add a missing
|
|
header. Hide/custom take final precedence, including proxy-generated errors.
|
|
|
|
Global and route `headers.response` apply to client responses; `headers.request`
|
|
apply only upstream. Each scope/direction supports up to 32 ordered set/append/remove
|
|
rules. Remove has no value. Global rules precede route rules. Multiple Set-Cookie
|
|
values remain distinct. Admin-session cookie names are never forwarded or set by
|
|
public proxy rules. Transport, framing and trusted forwarding/identity fields are
|
|
protected. Cache diagnostic fields are owned by the cache controls, not generic rules.
|
|
|
|
Global response rules also cover generated errors; route-specific rules apply only
|
|
after an origin route response exists. Client policy changes use hot configuration
|
|
publication and invalidate the RAM cache. In-flight exchanges retain their old
|
|
snapshot; no mixed-version configuration is applied midway through a response.
|
|
|
|
## Compression
|
|
|
|
All four flags default false. `min_bytes` defaults to 1024 (valid 32..1048576).
|
|
The library's default small-body/content-type predicate is also retained; actual
|
|
minimum for a known-size body is at least that default. Unknown-length text bodies
|
|
can be streamed without pre-buffering merely to discover their size.
|
|
|
|
Accept-Encoding quality negotiation is delegated to tower-http. Text, JSON,
|
|
JavaScript, XML, SVG and wasm are eligible; already encoded, SSE, range, upgrade,
|
|
HEAD/204/304 and privacy-sensitive responses are excluded. Cookies/authorization
|
|
bypass even when an origin marks a response public. Origin no-transform/privacy
|
|
exclusions cannot be defeated by deleting a response header in the editor. Cached
|
|
identity bodies can be streamed through compression on HIT; origin encodings are
|
|
passed through without recompression.
|
|
|
|
Vary: Accept-Encoding is added by compression; content length is removed for the
|
|
transformed body. Strong ETags are weakened, and identity-body content digests and
|
|
Accept-Ranges are removed. Encoders use their fastest preset; there is no panel
|
|
compression-quality slider in this release. Real binary interoperability tests
|
|
are still an acceptance requirement, not a result of this packaging session.
|
|
|
|
## Diagnostics and units
|
|
|
|
Samples run off the traffic path roughly every two seconds. CPU and network rates
|
|
need two successful samples. Host CPU is aggregate; process CPU uses 100% per busy
|
|
logical CPU and can exceed 100%. Neither is normalized to a container CPU quota.
|
|
Host memory is MemTotal minus MemAvailable. RSS/FDs/threads and process I/O come
|
|
from procfs. Network rates sum non-loopback interfaces in the process network
|
|
namespace, not just RustProxy traffic. Load average is OS scope, not latency.
|
|
|
|
The process's cgroup v2 memory files are read at the standard mount location when
|
|
available. Linux keeps the existing `/proc`/cgroup metric names, units and behavior.
|
|
FreeBSD/OpenBSD/NetBSD use native system interfaces where the same metric is available;
|
|
unknown or unsupported values remain null rather than being fabricated. Snapshot timestamps and sample age are shown;
|
|
cache counters and proxy request metrics are separate from OS resource metrics.
|
|
The dashboard history is browser-local and resets on reload/sign-out. No time-series
|
|
database, disk utilization chart or Windows/macOS OS sampler is implemented.
|
|
|
|
## HTTP/2 compatibility and per-vhost settings
|
|
|
|
HTTP/2 lifecycle settings are listener-scoped under `listeners.endpoints[].binding.http2_settings`:
|
|
|
|
- `idle_timeout_seconds` (default 300) retires only a truly unused H2 connection; active streams and current frame I/O postpone it.
|
|
- `max_connection_lifetime_seconds` (default 0) is an independent absolute lifetime; `0` disables this limit.
|
|
- `drain_timeout_seconds` (default 10) bounds how long accepted streams may finish after retirement begins.
|
|
|
|
Idle and lifetime retirement both send GOAWAY, stop accepting new streams, drain already accepted streams and then close TCP. RFC 8441 WebSocket/H2 Extended CONNECT streams are counted as active until their response body/tunnel ends. The same settings apply to TLS H2 and cleartext H2C on that named listener.
|
|
|
|
For backward compatibility, the top-level `http2` object is still accepted as the fallback for an H2/H2C listener that has no explicit `binding.http2_settings`. The structured UI writes explicit per-listener settings; operators should treat the top-level object and `PUT /api/config/http2` as legacy compatibility surfaces rather than the preferred configuration path.
|
|
|
|
A named TLS listener with `binding.http2: true` advertises H2 capability and always
|
|
retains HTTP/1.1 fallback. `binding.http2: false` makes that listener HTTP/1.1-only.
|
|
Cleartext H2 uses `binding.h2c: true`. Changing listener transport capability or H2 lifecycle settings requires restart.
|
|
|
|
A vhost can additionally set `http2_enabled: false`. RustProxy reads TLS ClientHello
|
|
and SNI before committing to ALPN, so a vhost that disables H2 gets HTTP/1.1-only
|
|
ALPN even while another hostname on the same listener keeps H2. `http3_enabled` is
|
|
the equivalent per-vhost H3 opt-out. The panel enables these switches only when a
|
|
selected listener provides the protocol. No failed application request is replayed
|
|
to downgrade.
|
|
|
|
Vhosts have their own backend pool, health/keepalive/affinity controls, ACL and
|
|
security/custom response headers. See VHOSTS.md. Global Server policy still takes
|
|
final precedence. There is no per-vhost compression algorithm override.
|
|
|
|
The reported `SizeAbove(u16)` compilation error is removed by a custom u64 predicate,
|
|
not a cast or panic-producing conversion. Compression thresholds retain the full
|
|
32..1048576-byte setting range. Known body length/Content-Length is compared exactly;
|
|
unknown streaming lengths are eligible without buffering the full body. Actual
|
|
compression execution still requires the compiler-enabled acceptance suite.
|
|
|
|
## Management in 0.3.0
|
|
|
|
Settings contains a direct Security shortcut (accounts, sessions and host ACLs)
|
|
and a vhost shortcut (SSL, headers, cache and pages). Compact is the default density;
|
|
Dense and Comfortable change spacing/control sizes without browser zoom or logo
|
|
redesign. The preference is local to the browser, not a global proxy setting.
|
|
|
|
Global Pages configures error overrides, direct-IP and unknown-host responses.
|
|
See MANAGEMENT.md for inheritance, allowed statuses and security implications.
|
|
Access logs have moved out of the dashboard into a separate modal/docked window.
|
|
Filters stay above the scrollable rows; counts and exports use the same selection.
|
|
|
|
## Named listeners, forwarding and diagnostics (0.4.0)
|
|
|
|
As of 0.8.0, the vhost editor no longer asks for a free-form listener IP. Configure
|
|
listener IP/port/protocol once in this Settings section, then select named endpoints in
|
|
the vhost. The pre-production inline/global listener format is no longer accepted. This separates
|
|
socket intent from host routing and avoids duplicating listening IP configuration across
|
|
vhosts.
|
|
|
|
`listeners.endpoints` is the only proxy-listener configuration. Each record has id,
|
|
name, scope, enabled and a binding with address, tls, http2, optional http2_settings, h2c and optional http3. Scope
|
|
is only a label. A vhost uses `frontend.listener_ids`; every reference must exist.
|
|
Conflicting protocols on shared sockets are rejected.
|
|
Changes to effective transport plans need restart, including H2 lifecycle and H3 limits. The UI
|
|
shows saved named entries and actual bound transports separately.
|
|
|
|
`forwarding` controls trusted CIDRs and forwarded-proto acceptance.
|
|
`diagnostic_headers` exposes only `request_id` (false initially). Vhosts can override the diagnostic
|
|
policy and Server policy; absent/null means inherit. Runtime/cache/static/error
|
|
responses pass through the same final header policy.
|
|
|
|
See INTEGRATIONS_040.md and HTTP3.md for complete JSON, limits and security details.
|
|
|
|
|
|
## 0.5.0 changes (override earlier baseline examples)
|
|
|
|
A named TLS listener with omitted http3 uses default H3 limits. Explicit null is
|
|
persisted as disabled. Overlapping named sockets must match protocols AND H3 limits;
|
|
the error identifies owners and remedies. A changed binding requires restart, including
|
|
UDP protocol/limit changes. Check UDP port availability before upgrading.
|
|
|
|
The diagnostic control is `request_id` (false by default). The removed hide_mock_origin field is not accepted; use normal response-header removal
|
|
for X-Mock-Origin. Existing custom Server values remain unchanged.
|
|
|
|
Cache decision/object inspection is under Cache, separate from Overview. Basic
|
|
manual certificate issuance is under Certificates; TSIG is never a configuration
|
|
field. See CACHE_DIAGNOSTICS.md and ACME_RFC2136.md for exact scopes and limits.
|
|
|
|
## 0.6 navigation and exporter
|
|
|
|
The Settings page now has eight switching cards: Listeners, Response, Cache,
|
|
Headers, Forwarding, Pages, Prometheus and Security. Unsaved values remain in
|
|
place while switching; switching alone never writes configuration. Security is
|
|
not a separate sidebar item. Use arrow keys/Home/End in the tab bar.
|
|
|
|
The Prometheus card controls enabled, bind IP/port, URL path, client CIDRs,
|
|
bearer-token environment variable name and optional collectors, including the bounded
|
|
Traffic analytics collector. Only the variable name is exported; supply the actual secret to the service separately. See
|
|
PROMETHEUS.md for immediate revocation, restart rules and scrape configuration.
|