# Listener and response settings

## Configuration and update semantics

GET `/api/config` returns a revisioned envelope. PUT the entire document with
`expected_revision`. Defaults apply when fields are omitted. Unknown fields are
rejected; failed validation/persistence leaves the old snapshot active. Existing
backend/route/TLS/ACL data must be retained when changing settings. The panel does
this and preserves dirty settings forms across status/language refreshes.

```json
{
  "listeners": {
    "endpoints": [
      {"id":"http", "name":"HTTP", "scope":"public", "enabled":true,
       "binding":{"address":"0.0.0.0:80", "tls":false, "http2":false, "http3":null}},
      {"id":"https", "name":"HTTPS", "scope":"public", "enabled":true,
       "binding":{"address":"0.0.0.0:443", "tls":true, "http2":true,
                  "http2_settings":{"idle_timeout_seconds":300, "max_connection_lifetime_seconds":0, "drain_timeout_seconds":10}}}
    ]
  },
  "server_header": {"mode":"custom", "value":"RustProxy/v0.20.0"},
  "compression": {"gzip":true, "br":true, "zstd":true, "deflate":false, "min_bytes":1024},
  "headers": {
    "request": [{"op":"set", "name":"x-proxy-name", "value":"RustProxy"}],
    "response": [
      {"op":"remove", "name":"x-powered-by"},
      {"op":"set", "name":"x-served-by", "value":"RustProxy"}
    ]
  }
}
```

This is a settings fragment, not a replacement for the existing routing/vhost document.
A complete opt-in example is in `examples/cache-dashboard.json`.

The dashboard exposes the complete raw JSON editor under **Settings -> Manual config**.
The same card exposes **Revision history**, which reads the newest 512 retained configuration revisions from SQLite schema 8 and groups them by month/day. Selecting a revision shows timestamp, actor and a bounded path-level before/after diff against the previous retained revision. An older retained revision can be restored from the same dialog; RustProxy validates it through the normal configuration update path and saves it as a new revision, preserving the prior history.
The historical `/configuration` browser route is retained only as a compatibility alias and
opens that Settings block; it is no longer a primary-navigation item. Prefer the structured
section editors for routine changes so unrelated sections are not republished.

## Administration HTTPS

`administration` is separate from public proxy listeners. It defaults to the startup
loopback address and plain HTTP. Persisted address/certificate changes are validated
immediately but take effect after process restart. A non-loopback persisted bind is
accepted only when `administration.tls` is true. Plain remote HTTP remains available only when process startup explicitly allows it with
`--allow-remote-admin`.

```json
{
  "administration": {
    "listen": "0.0.0.0:9090",
    "tls": true,
    "certificate_id": "admin-example",
    "allowed_hosts": ["admin.example.com"],
    "session_days": 30
  }
}
```

`certificate_id: null` uses `data/certs/default.pem` and `default.key`, generated by
the application. A selected inventory certificate is loaded from the same private
certificate directory. `session_days` controls the absolute browser-session lifetime
and must be 7..30 (default 30). It applies without restart; changing listener/TLS/certificate
settings still requires restart. Session cookies are HttpOnly/SameSite=Strict and receive
the `Secure` attribute on HTTPS. When HTTPS is enabled, accidental plain HTTP received on the same administration
port receives a permanent redirect to the same authority/path over `https://`; the
plaintext path contains no administration handlers. An empty `allowed_hosts` accepts
valid DNS Host values on the configured HTTPS port as well as valid numeric destination
addresses. Once `allowed_hosts` contains entries it becomes a strict DNS allowlist for
browser-facing DNS authorities. An explicitly listed hostname may arrive from a trusted
reverse proxy on an external port such as 443 even when the private admin socket is 9090.
Direct IP/`localhost` access is still accepted only on the actual configured admin port.
For HTTPS termination in front of a plain local admin socket, the trusted proxy must pass
its external Host and scheme; `X-Forwarded-Host` is accepted only from loopback/configured
trusted proxy peers and `X-Forwarded-Proto` additionally follows the forwarded-proto trust
policy. Origin/CSRF comparison and the session cookie Secure flag use that external origin.
Use a firewall and normal network policy even with HTTPS.

## Structured access logging

The **Settings -> Logging** page is split into **Outputs**, **JSON format** and **Error log** subsections, using the same in-card sub-tab layout and keyboard interaction as Cache. Outputs controls optional external access-log sinks while the existing bounded live ring remains available to the panel. Sink I/O is performed by a bounded background queue; request processing only performs a non-blocking enqueue. Supported destinations:

- JSONL file append, either one combined file such as `/var/log/rustproxy/access.jsonl` or
  one file per resolved vhost under a configurable directory (default `/var/log/rustproxy/vhosts`).
  Unmatched traffic uses `_unmatched.jsonl`. The managed systemd service provisions
  `/var/log/rustproxy` for the service account and keeps that directory writable under
  `ProtectSystem=strict`. Other custom paths still need matching service permissions.
- Logstash TCP using `host:port` (IP or DNS). Each event is one JSON line.
- Loki HTTP(S) push. Optional bearer credentials are read from an environment variable
  named by `bearer_token_env`; the secret itself is not stored in configuration.

`logging.format.fields` maps output field names to standard event fields; an empty map
emits the complete standard event. The panel provides Standard, Nginx-style, ECS-style
and Minimal presets plus a visual field selector/renamer with JSON preview. Advanced raw
mapping remains available for aliases not representable by the simple builder. Supported mapped sources include `vhost`, `route`, `cache_effective_enabled`, `transform_effective_enabled`, `backend`, request/response metadata and cache outcome fields. `static_fields` adds constant string fields. For low-level `protocol_error` events, RustProxy always preserves the required diagnostic envelope (`event`, error kind/detail, time, client/local/TLS/protocol fields, status and outcome) even with a custom mapping, so malformed-traffic records remain searchable in the built-in Logs explorer. Loki labels are separately configurable.
`/api/status.logging.sinks` reports queue capacity, dropped events, successfully delivered
events and sink errors.

## Hot and cold cache tiers

`cache.max_bytes` and `cache.max_entries` bound the HOT RAM tier. `max_bytes` is charged from retained body backing allocation plus entry metadata rather than only payload length. `cache.transient_max_bytes` separately bounds in-progress fills and full bodies moving between HOT and COLD; `cache.cold_queue_max_bytes` additionally caps queued demotions in bytes, and `cache.hot_eviction_max_objects` bounds one LRU admission attempt. The source defaults are 256 MiB HOT, 16 MiB maximum object, 64 MiB transient and 32 MiB COLD queue. The Settings panel calculates conservative/recommended/performance presets from the lower of host RAM and a finite cgroup memory limit without changing the COLD disk budget. It also warns when HOT + transient is a large fraction of effective RAM, when the additional configured cache budget is large compared with current host/cgroup headroom, when the COLD queue exceeds transient RAM, or when the object ceiling cannot fit in transient RAM. Automatic presets are disabled below 256 MiB effective RAM. On a 2 GiB effective-memory VM the recommended profile selects approximately 400 MiB HOT, 96 MiB transient, 16 MiB maximum object, 48 MiB COLD queue, 6400 HOT entries and a 64-object LRU admission limit. The optional persistent
cold tier is controlled by `cold_enabled`, `cold_path`, `cold_max_bytes`,
`cold_max_entries`, `cold_gc_interval_seconds`, `cold_gc_max_delete`,
`cold_after_seconds`, `cold_on_lru`, `cold_on_idle` and `cold_on_shutdown`.
`cold_gc_interval_seconds` defaults to 60 and `cold_gc_max_delete` to 256. Pressure GC
starts at roughly 90% of the configured byte/entry budget and cleans toward roughly 80%
without deleting more than the configured per-pass cap. `cold_path` is relative to the RustProxy data directory and cannot
contain parent traversal or an absolute path. The two demotion switches independently
select LRU-pressure and idle objects; shutdown flush is a best-effort direct write during
a graceful restart. Demotions use a small bounded queue and release hot accounting before
disk persistence so cold storage does not retain an unnecessary second body buffer in RAM.
Cold metadata is readable JSON and bodies are raw `.body` files; reads are size/SHA-256
checked before promotion to RAM. Since 0.21.0 objects are sharded by the first two SHA-256
hex characters (`cache/cold/00/...` through `ff/...`), with shard directories created on
first write. The previous flat Cold Cache is intentionally not migrated. Purge operations
clear both in-memory indexes and schedule disk cleanup; selective cleanup uses the same
GC/eviction path and runs asynchronously.

Version 0.22 adds foreground fill collapsing (`collapse_enabled`,
`collapse_wait_timeout_ms`, `collapse_timeout_fallback`), detailed stale-if-error policy
(`stale_if_error_timeout`, `stale_if_error_connection`, `stale_if_error_dns`,
`stale_if_error_tls`, `stale_if_error_statuses`) and opt-in proactive refresh
(`proactive_refresh_enabled`, `proactive_refresh_before_expiry_seconds`,
`proactive_refresh_percent`, `proactive_refresh_max_inflight`). Defaults preserve the
previous stale-if-error behavior; proactive refresh is disabled by default. Selective
PURGE/BAN is an operational API/panel action rather than persisted configuration. See
[CACHE.md](CACHE.md) for request-collapse, Range and invalidation semantics.

## Static-response transformation

`transform` is a formal identity-response stage between the upstream response and shared
cache storage. `transform.enabled` is the global kill switch, but every vhost must also
set its own `transform.enabled=true`; vhosts stay disabled by default even when the global
engine is enabled. Global defaults enable JavaScript and CSS transformation and leave
HTML, SVG and JSON off. `extensions` controls extension-based detection and
`max_body_bytes` (1 KiB..32 MiB, 2 MiB default) bounds aggregation/decoding.

The pipeline is `origin -> decode supported Content-Encoding -> transform -> cache
identity representation -> client compression`. JavaScript/CSS can be enabled normally;
HTML minification, SVG optimization and JSON compaction are independent optional formats.
Recognition uses safe configured extensions and known Content-Type values. Binary/image
formats outside those explicit transform types are not transformed.

Origin `Cache-Control: no-transform` always disables transformation. When bytes change,
RustProxy removes representation-specific length/digest/encoding metadata and creates a
new strong `rustproxy-*` ETag for the transformed representation. The original origin
ETag/Last-Modified are kept separately and used for origin revalidation, so a proxy ETag
is never sent upstream as though it validated the original object. The transformed
identity body is what enters cache, preventing repeated minification on every HIT.

## Global TLS protocol and cipher policy

`tls.policy` is the global baseline inherited by every TLS host. It controls TLS 1.2,
TLS 1.3 and optional rustls cipher-suite names. An empty `cipher_suites` list uses the
ring provider defaults. Settings shows both the currently effective cipher suites and
the complete selectable suite inventory. At least one protocol version and
one compatible effective suite are required. This policy is also used by administration
HTTPS. HTTP/3 requires TLS 1.3 and a TLS 1.3 suite. Vhosts can narrow TCP TLS with
`tls_policy`; when a vhost has HTTP/3 enabled on a selected H3-capable listener it must
match the global policy because QUIC TLS is shared per UDP listener. Available suite names are exposed in authenticated status for
the Settings editor.

## Listeners: saved intent, not hot rebinding

When an HTTP, HTTPS or administration address was explicitly supplied by systemd/CLI
(or the corresponding service environment variable), that exact startup bind is a
process contract: the panel shows it as locked and configuration updates cannot remove,
disable or change it. If an address was not explicitly supplied at startup, listener
entries remain freely addable/removable/editable subject to normal validation.

`listeners.endpoints` is the only persisted frontend socket configuration. Each
endpoint has a stable id/name, an informational scope label, an enabled flag and one
primary binding containing IP:port, TLS, HTTP/2 plus its optional lifecycle settings, and optional HTTP/3 settings. Optional
`additional_addresses` reuse that same transport policy on up to 15 further addresses.
When more than one address is configured every address must be explicit; `0.0.0.0` and
`::` are rejected so a multi-IP selection cannot silently mean every interface. IP/port
and transport capability belong here, not inside a vhost. IPv6 uses `[address]:port`.

A vhost references endpoints through `frontend.listener_ids`. Listener changes are
restart-required because they alter TCP/UDP sockets or TLS ALPN capability. The
listener editor owns the HTTP/2/H2C and HTTP/3 enable/disable switches. An H2/H2C row
links to **HTTP connections -> HTTP/2** for idle, absolute-lifetime and graceful-drain
limits; an enabled H3 row links to **HTTP connections -> HTTP/3** for advertisement,
connection, stream and idle-timeout limits. Both tuning pages are listener-scoped and
do not own protocol enablement. Vhost H2/H3 switches only narrow the protocols already
enabled by the selected endpoints; they never create a protocol that Settings disabled.

The administration listener is independent and cannot share a frontend port.
Bindings must be valid local unicast/wildcard addresses; conflicting overlapping
endpoints are rejected rather than silently widened.

## Server and other client headers

`server_header.mode` is `preserve` (default), `hide`, or `custom`. Custom values are
nonempty, bounded to 256 bytes and validated against header injection. Preserve
means retain the result of origin/global/route rules; it does not add a missing
header. Hide/custom take final precedence, including proxy-generated errors.

Global and route `headers.response` apply to client responses; `headers.request`
apply only upstream. Each scope/direction supports up to 32 ordered set/append/remove
rules. Remove has no value. Global rules precede route rules. Multiple Set-Cookie
values remain distinct. Admin-session cookie names are never forwarded or set by
public proxy rules. Transport, framing and trusted forwarding/identity fields are
protected. Cache diagnostic fields are owned by the cache controls, not generic rules.

Global response rules also cover generated errors; route-specific rules apply only
after an origin route response exists. Client policy changes use hot configuration
publication and invalidate the RAM cache. In-flight exchanges retain their old
snapshot; no mixed-version configuration is applied midway through a response.

## Compression

All four flags default false. `min_bytes` defaults to 1024 (valid 32..1048576).
The library's default small-body/content-type predicate is also retained; actual
minimum for a known-size body is at least that default. Unknown-length text bodies
can be streamed without pre-buffering merely to discover their size.

Accept-Encoding quality negotiation is delegated to tower-http. Text, JSON,
JavaScript, XML, SVG and wasm are eligible; already encoded, SSE, range, upgrade,
HEAD/204/304 and privacy-sensitive responses are excluded. Cookies/authorization
bypass even when an origin marks a response public. Origin no-transform/privacy
exclusions cannot be defeated by deleting a response header in the editor. Cached
identity bodies can be streamed through compression on HIT; origin encodings are
passed through without recompression.

Vary: Accept-Encoding is added by compression; content length is removed for the
transformed body. Strong ETags are weakened, and identity-body content digests and
Accept-Ranges are removed. Encoders use their fastest preset; there is no panel
compression-quality slider in this release. Real binary interoperability tests
are still an acceptance requirement, not a result of this packaging session.

## Diagnostics and units

Samples run off the traffic path roughly every two seconds. CPU and network rates
need two successful samples. Host CPU is aggregate; process CPU uses 100% per busy
logical CPU and can exceed 100%. Neither is normalized to a container CPU quota.
Host memory is MemTotal minus MemAvailable. RSS/FDs/threads and process I/O come
from procfs. Network rates sum non-loopback interfaces in the process network
namespace, not just RustProxy traffic. Load average is OS scope, not latency.

The process's cgroup v2 memory files are read at the standard mount location when
available. Linux keeps the existing `/proc`/cgroup metric names, units and behavior.
FreeBSD/OpenBSD/NetBSD use native system interfaces where the same metric is available;
unknown or unsupported values remain null rather than being fabricated. Snapshot timestamps and sample age are shown;
cache counters and proxy request metrics are separate from OS resource metrics.
The dashboard history is browser-local and resets on reload/sign-out. No time-series
database, disk utilization chart or Windows/macOS OS sampler is implemented.

## HTTP/2 compatibility and per-vhost settings

HTTP/2 lifecycle settings are listener-scoped under `listeners.endpoints[].binding.http2_settings`:

- `idle_timeout_seconds` (default 300) retires only a truly unused H2 connection; active streams and current frame I/O postpone it.
- `max_connection_lifetime_seconds` (default 0) is an independent absolute lifetime; `0` disables this limit.
- `drain_timeout_seconds` (default 10) bounds how long accepted streams may finish after retirement begins.

Idle and lifetime retirement both send GOAWAY, stop accepting new streams, drain already accepted streams and then close TCP. RFC 8441 WebSocket/H2 Extended CONNECT streams are counted as active until their response body/tunnel ends. The same settings apply to TLS H2 and cleartext H2C on that named listener.

For backward compatibility, the top-level `http2` object is still accepted as the fallback for an H2/H2C listener that has no explicit `binding.http2_settings`. The structured UI writes explicit per-listener settings; operators should treat the top-level object and `PUT /api/config/http2` as legacy compatibility surfaces rather than the preferred configuration path.

A named TLS listener with `binding.http2: true` advertises H2 capability and always
retains HTTP/1.1 fallback. `binding.http2: false` makes that listener HTTP/1.1-only.
Cleartext H2 uses `binding.h2c: true`. Changing listener transport capability or H2 lifecycle settings requires restart.

A vhost can additionally set `http2_enabled: false`. RustProxy reads TLS ClientHello
and SNI before committing to ALPN, so a vhost that disables H2 gets HTTP/1.1-only
ALPN even while another hostname on the same listener keeps H2. `http3_enabled` is
the equivalent per-vhost H3 opt-out. The panel enables these switches only when a
selected listener provides the protocol. No failed application request is replayed
to downgrade.

Vhosts have their own backend pool, health/keepalive/affinity controls, ACL and
security/custom response headers. See VHOSTS.md. Global Server policy still takes
final precedence. There is no per-vhost compression algorithm override.

The reported `SizeAbove(u16)` compilation error is removed by a custom u64 predicate,
not a cast or panic-producing conversion. Compression thresholds retain the full
32..1048576-byte setting range. Known body length/Content-Length is compared exactly;
unknown streaming lengths are eligible without buffering the full body. Actual
compression execution still requires the compiler-enabled acceptance suite.

## Management in 0.3.0

Settings contains a direct Security shortcut (accounts, sessions and host ACLs)
and a vhost shortcut (SSL, headers, cache and pages). Compact is the default density;
Dense and Comfortable change spacing/control sizes without browser zoom or logo
redesign. The preference is local to the browser, not a global proxy setting.

Global Pages configures error overrides, direct-IP and unknown-host responses.
See MANAGEMENT.md for inheritance, allowed statuses and security implications.
Access logs have moved out of the dashboard into a separate modal/docked window.
Filters stay above the scrollable rows; counts and exports use the same selection.

## Named listeners, forwarding and diagnostics (0.4.0)

As of 0.8.0, the vhost editor no longer asks for a free-form listener IP. Configure
listener IP/port/protocol once in this Settings section, then select named endpoints in
the vhost. The pre-production inline/global listener format is no longer accepted. This separates
socket intent from host routing and avoids duplicating listening IP configuration across
vhosts.

`listeners.endpoints` is the only proxy-listener configuration. Each record has id,
name, scope, enabled and a binding with address, tls, http2, optional http2_settings, h2c and optional http3. Scope
is only a label. A vhost uses `frontend.listener_ids`; every reference must exist.
Conflicting protocols on shared sockets are rejected.
Changes to effective transport plans need restart, including H2 lifecycle and H3 limits. The UI
shows saved named entries and actual bound transports separately.

`forwarding` controls trusted CIDRs and forwarded-proto acceptance.
`diagnostic_headers` exposes only `request_id` (false initially). Vhosts can override the diagnostic
policy and Server policy; absent/null means inherit. Runtime/cache/static/error
responses pass through the same final header policy.

See INTEGRATIONS_040.md and HTTP3.md for complete JSON, limits and security details.


## 0.5.0 changes (override earlier baseline examples)

A named TLS listener with omitted http3 uses default H3 limits. Explicit null is
persisted as disabled. Overlapping named sockets must match protocols AND H3 limits;
the error identifies owners and remedies. A changed binding requires restart, including
UDP protocol/limit changes. Check UDP port availability before upgrading.

The diagnostic control is `request_id` (false by default). The removed hide_mock_origin field is not accepted; use normal response-header removal
for X-Mock-Origin. Existing custom Server values remain unchanged.

Cache decision/object inspection is under Cache, separate from Overview. Basic
manual certificate issuance is under Certificates; TSIG is never a configuration
field. See CACHE_DIAGNOSTICS.md and ACME_RFC2136.md for exact scopes and limits.

## 0.6 navigation and exporter

The Settings page now has eight switching cards: Listeners, Response, Cache,
Headers, Forwarding, Pages, Prometheus and Security. Unsaved values remain in
place while switching; switching alone never writes configuration. Security is
not a separate sidebar item. Use arrow keys/Home/End in the tab bar.

The Prometheus card controls enabled, bind IP/port, URL path, client CIDRs,
bearer-token environment variable name and optional collectors, including the bounded
Traffic analytics collector. Only the variable name is exported; supply the actual secret to the service separately. See
PROMETHEUS.md for immediate revocation, restart rules and scrape configuration.