v0.24.5

docs/ARCHITECTURE.md

@@ -0,0 +1,214 @@
+# Architecture
+
+## Control plane and forwarding plane
+
+SQLite stores administrator intent, including vhosts/paths, policies, backend definitions
+and certificate metadata. `config::compile` validates this document and builds an
+immutable Snapshot with routing tables, compiled ACL/header policies, per-backend
+clients and an optional rustls ServerConfig. ArcSwap publishes the active snapshot.
+No public request consults SQLite or a provider to choose its route.
+
+A configuration mutation obtains a bounded admission slot and builds a candidate on
+a blocking worker. The slot remains held if the HTTP caller is cancelled while the
+worker is running. A serialized detached writer checks the expected revision,
+commits the desired document in SQLite and swaps the active snapshot with no await
+between commit completion and publication. Invalid candidates leave the old state
+active. A crash between persistence and publication is recovered by loading the
+committed document at startup; this is not one cross-memory/SQLite transaction.
+
+## Module responsibilities
+
+| Module | Responsibility |
+|---|---|
+| `server` | Bounded listeners, TLS acceptance, HTTP/1.1/HTTP/2 selection, connection leases and shutdown |
+| `proxy` | Host/authority routing, ACL sequencing, forwarding and streaming instrumentation |
+| `proxy::websocket` | Handshake validation, upgrade tasks and bounded bidirectional byte transport |
+| `upstream` | Per-backend pools, guarded resolver, connection deadlines and verified HTTPS clients |
+| `tls` | Local PEM files, generation/import, validation, SNI and loaded certificate snapshots |
+| `config` | Desired models, normalization, validation and compiled snapshots |
+| `vhost` | Host-owned frontend/member models, actual-address selection and listener planning |
+| `balance` | Full-exchange leases, health/recovery, weighted scheduling, connection/affinity settings |
+| `security` | Compiled per-vhost response protection, TLS-only HSTS |
+| `acl`, `headers` | Generic access and header policies, no application-specific exceptions |
+| `auth`, `admin` | Administrator lifecycle, local API, browser sessions, CSRF, SSE and static UI |
+| `db` | SQLite schema, migrations, desired-state persistence and revision checks |
+| `metrics`, `logging` | Aggregate instrumentation, bounded access ring and trace output |
+| `transform` | Bounded decode/transform stage for eligible static identity representations before cache/compression |
+| `docker` | Desired Docker-agent settings, in-memory observations and ephemeral vhost reconciliation |
+| `runtime` | Shared process state, snapshot updates, admission limits and tunnel tracking |
+| `demo` | A separate opt-in demonstration backend with its own listener |
+
+Docker discovery is an optional provider boundary implemented through the standalone
+`docker-agent/`; the RustProxy process never opens the Docker socket. Cloudflare Tunnel
+remains a future module. Certbot-based HTTP-01/Cloudflare DNS/OVH DNS/RFC2136 issuance and managed renewal are optional
+control-plane adapters. Provider work stays outside proxy request paths and
+publishes certificates only through validated configuration snapshots.
+
+## Request lifetime
+
+A listener owns a shared TCP connection lease. Each admitted request owns an
+in-flight permit and an Exchange, which remains alive through streamed request and
+response bodies. HTTP/2 connections share the public TCP cap with HTTP/1.1 and TLS.
+Upgraded WebSocket tasks retain both the Exchange and TCP lease after Hyper stops
+managing the HTTP connection, preventing upgrades from escaping limits or counters.
+
+Routing captures one snapshot. Socket peer identity, host and route ACLs are checked
+before DNS or upstream connection reuse. Forwarding uses a URI built from a stored
+backend origin; the public client never supplies an arbitrary destination. Header
+rules cannot change the identity already used by ACL. Ordinary bodies pass through
+without full aggregation. For an explicitly transform-enabled vhost, eligible bounded
+static responses use the logical pipeline `origin -> decode supported Content-Encoding
+-> transform -> cache identity representation -> downstream compression`. Origin
+`no-transform` is authoritative; changed bytes receive a RustProxy representation ETag
+while origin validators are retained separately for revalidation. WebSocket data uses
+fixed-size copy buffers.
+
+New requests on existing keep-alive/HTTP/2 connections load the current snapshot.
+Existing bodies and tunnels retain the old snapshot/backend. Each accepted TLS
+connection captures the current certificate configuration before its handshake.
+Certificate validation and file reads are never performed inside the SNI resolver.
+
+## Startup and shutdown
+
+Startup opens/migrates SQLite, loads existing intent, prepares required certificates,
+compiles the snapshot and binds the requested listeners. The demo only seeds a new
+database. A corrupt configuration is not automatically reset. After the runtime is published,
+the optional Docker poll loop contacts enabled agents; failures remain observations and do not reset desired configuration or block ordinary proxy service.
+
+Shutdown cancels listeners and tracked tunnels, drains admitted HTTP connections
+within the configured grace period, drains accepted configuration writers and
+closes SQLite. Bounded runtime shutdown avoids waiting forever for cancelled OS DNS
+workers. This does not turn a blocking getaddrinfo call into a killable operation.
+
+## Deliberate development tradeoffs
+
+The panel remains loopback HTTP. All active configuration changes currently rebuild
+backend pools and certificate state, even if unrelated fields changed. Compatible health/counter/affinity observations are retained independently; idle
+connection and certificate reuse optimization still requires measurements. Backend loop protection checks local listener addresses by default while allowing remote origins on any valid port. A per-backend `accept_local_listener_risk` opt-in can deliberately allow a local RustProxy listener (for example port 9090) without bypassing private/special-address policy. DNS special-range restrictions
+are basic safeguards rather than an independently audited network policy engine.
+No performance target has been established or measured for this package.
+
+
+## 0.1.0 response pipeline and resources
+
+Observe pins one immutable snapshot for the exchange. Host/route ACLs precede the
+cache and all upstream I/O. Eligible GET lookup uses effective request headers
+and a revisioned representation key. On a miss the origin response is validated
+before and after global/route header rules; a bounded tee records complete bodies
+while still yielding frames immediately. No request-path SQLite/provider lookup is
+introduced. Server policy and optional streaming compression apply after cache
+lookup/fill wrapping. The outer MeteredBody counts delivered encoded payload frames.
+
+Cache publication uses a short mutex section and LRU/byte/entry bounds; fill buffers
+have eight semaphore slots. Purge increments a generation, preventing older in-flight
+fills from publishing. Config commit invalidates cache before snapshot publication,
+without awaiting between those operations. New listener intent is persisted but
+active sockets remain unchanged until restart; original CLI defaults are retained
+separately so clearing an override has accurate desired-state reporting.
+
+`cache`, `transform`, `compression`, `settings`, `diagnostics`, and `system_stats` are dedicated modules. `diagnostics` owns periodic sampling/export while `system_stats` adapts the shared `systemstat` host API, `sysinfo::Networks` network API, and one `metrics-process` process API. Network inventory/counters use `sysinfo` as the primary source and merge `systemstat` as a secondary library source, so missing/partial primary inventory is supplemented without shell commands; no BSD/Linux shell-command network parser is maintained. `metrics-process` uses its documented dummy fallback where no native process collector exists, so unsupported fields remain absent without adding another platform parser. Linux adds its compatibility-only `/proc`/cgroup fields after the common sample. The
+sampler runs blocking OS/library reads off the async request path roughly every two
+seconds and stops through the process cancellation token. Language discovery runs
+in build.rs, not in browser hardcoded PL/EN logic. The result is an embedded public
+JSON asset; existing logo resources remain intact.
+
+## Vhost pool publication and observations
+
+The immutable routing snapshot contains actual-address-constrained routes, compiled
+frontends, pools and both TLS ALPN configurations. The process separately retains
+its actually bound listener plan. New routing uses desired bindings immediately;
+new sockets/ALPN take effect only at restart. This can temporarily deny moved hosts,
+but never silently broadens them to still-active old bindings.
+
+Each pool owns immutable member definitions and clients. Small synchronized health,
+scheduling and traffic observations live separately. Compatible publication reuses
+observations/tokens by vhost/member/origin/health identity; changed or re-enabled
+members are quarantined. Request leases retain the old member/client until both
+body directions or the upgraded tunnel finish. Changes do not replay requests.
+
+A weak-App, cancellation-aware supervisor schedules a rotating, bounded window of
+probes outside the data path. DNS and configured-origin policies are shared with
+normal forwarding; HTTP checks use a dedicated connection and discard the body.
+Traffic sampling and status snapshots operate on these observations, not SQLite.
+
+Forwarding order is actual frontend selection -> host/vhost/path ACL -> eligible
+member selection -> member-isolated cache lookup -> guarded upstream request.
+Cookie affinity bypasses cache. Cache hits release the selection without recording
+an actual upstream request. Response protection is applied after cache selection,
+and per-vhost HSTS is removed on plaintext, including synthetic responses.
+
+## Integrations in 0.4.0
+
+`forwarding` compiles the explicit trust boundary and reconstructs canonical
+metadata. `tinyauth` owns immutable verifier configuration and an outbound client;
+it authorizes before the existing origin/cache path. `http3` adapts bounded Quinn
+streams to the same Axum router and response Body without a separate cache/auth
+implementation. Named listener IDs resolve to actual bindings while compiling
+vhosts; accepted destination addresses still control route visibility.
+
+TLS and QUIC use the same validated certificate resolver. Active endpoint limits
+remain fixed until restart; new QUIC handshakes can use the latest certificate
+snapshot without changing Retry token keys. Graceful shutdown signals H3 GOAWAY
+and bounds drain before endpoint close. Runtime conformance remains to be tested.
+
+
+## 0.5.0 cache inspection and certificate jobs
+
+Cache decision counters and entry metadata live in the bounded cache store. A
+separate authenticated paginated read API returns metadata, not bodies, queries or
+request headers. Expiry cleanup also runs during inspection; generation barriers
+still prevent old in-flight fills from republishing after purge/config changes.
+
+The ACME manager is outside proxy request paths: one job semaphore, a tracked
+child-process group, JSON over stdin, sanitized status and private Certbot/RFC2136
+profiles. Only validated immutable certificate metadata can enter the existing
+library through explicit revision CAS publication. See ACME_RFC2136.md for the
+required external tools and unverified integration boundary; RENEWAL.md documents
+the 0.7 scheduler and automated publication.
+
+## 0.6 boundaries
+
+`legacy` is an input-only migration module, not a traffic dispatcher. It emits the
+same canonical vhost model used by the UI. Validated startup migration uses one
+CAS write; failure before validation completion leaves the stored JSON untouched.
+
+`acme` manages one private worker job; `scripts/acme_provider.py` adapts Certbot
+webroot/Cloudflare/OVH/RFC2136. Only the running HTTP-01 job's names and token files
+are exposed by the reserved challenge middleware. Provider files remain private,
+not part of SQLite or generic configuration exports. Issuance/import are separate.
+
+`prometheus` owns a dedicated read-only HTTP router, actual-peer CIDRs, environment
+bearer token and bounded scrape work. It does not share administrator cookies.
+Instrumentation is process-local; per-vhost counters reuse stable-ID observations
+across configuration changes. Domain cache gauges intentionally reset with revision.
+Admin request timing measures response headers, while proxy/vhost timing measures
+complete body/tunnel lifetime. Neither renderer queries a provider or SQLite to
+handle public proxy requests.
+
+## 0.7 renewal, access and deployment boundaries
+
+A shutdown-tracked weak-reference scheduler selects due configured managed
+certificates. It shares the issuance semaphore and per-profile advisory lock.
+Provider work remains outside request paths. Publication validates PEM/key/names
+and the complete current configuration, then uses the same SQLite CAS/snapshot
+path. Only changed generations publish; failures do not touch the active snapshot.
+TCP and QUIC handshakes consult current TLS state. Certificate publication also
+uses the existing cache-generation invalidation. See RENEWAL.md for timing/limits.
+
+Basic Auth is a separate vhost gate after cumulative IP ACLs and before static
+responses, cache and upstream selection. Argon2id verification is bounded by a
+shared semaphore held by blocking workers and an IP failure table. Hashes, not
+passwords, are persisted. A verified Authorization header is optional and reapplied
+after custom request headers; unverified identity headers are removed. Protected
+responses bypass the shared cache and authentication failures cannot become custom
+200 responses. WebSocket admission uses the same gates before opening a tunnel.
+
+`pool_eligible=false` distinguishes a location-pinned backend from ordinary pool
+membership. The GUI edits the same prefix route model as the configuration API;
+there is no second nginx-style parser or unvalidated location execution path.
+
+Server deployment uses root-owned immutable release directories, an atomic current
+symlink, a non-root systemd service and private service-owned data. A stopped-state
+backup precedes update publication. Failed health/version checks attempt to restore
+both old code and data; backups and failed-generation state remain for recovery.
+Live systemd/database recovery acceptance is not replaced by helper mocks.