RustProxy/docs/ARCHITECTURE.md

# Architecture

## Control plane and forwarding plane

SQLite stores administrator intent, including vhosts/paths, policies, backend definitions
and certificate metadata. `config::compile` validates this document and builds an
immutable Snapshot with routing tables, compiled ACL/header policies, per-backend
clients and an optional rustls ServerConfig. ArcSwap publishes the active snapshot.
No public request consults SQLite or a provider to choose its route.

A configuration mutation obtains a bounded admission slot and builds a candidate on
a blocking worker. The slot remains held if the HTTP caller is cancelled while the
worker is running. A serialized detached writer checks the expected revision,
commits the desired document in SQLite and swaps the active snapshot with no await
between commit completion and publication. Invalid candidates leave the old state
active. A crash between persistence and publication is recovered by loading the
committed document at startup; this is not one cross-memory/SQLite transaction.

## Module responsibilities

| Module | Responsibility |
|---|---|
| `server` | Bounded listeners, TLS acceptance, HTTP/1.1/HTTP/2 selection, connection leases and shutdown |
| `proxy` | Host/authority routing, ACL sequencing, forwarding and streaming instrumentation |
| `proxy::websocket` | Handshake validation, upgrade tasks and bounded bidirectional byte transport |
| `web_server` | Reusable static-file services, safe document-root traversal, validators/ranges and streamed local file bodies |
| `upstream` | Per-backend pools, guarded resolver, connection deadlines and verified HTTPS clients |
| `tls` | Local PEM files, generation/import, validation, SNI and loaded certificate snapshots |
| `config` | Desired models, normalization, validation and compiled snapshots |
| `vhost` | Host-owned frontend/member models, actual-address selection and listener planning |
| `balance` | Full-exchange leases, health/recovery, weighted scheduling, connection/affinity settings |
| `security` | Compiled per-vhost response protection, TLS-only HSTS |
| `acl`, `headers` | Generic access and header policies, no application-specific exceptions |
| `auth`, `admin` | Administrator lifecycle, local API, browser sessions, CSRF, SSE and static UI |
| `db` | SQLite schema, migrations, desired-state persistence and revision checks |
| `metrics`, `logging` | Aggregate instrumentation, bounded access ring and trace output |
| `transform` | Bounded decode/transform stage for eligible static identity representations before cache/compression, with live process-wide memory and concurrency ceilings |
| `docker` | Desired Docker-agent settings, in-memory observations and ephemeral vhost reconciliation |
| `runtime` | Shared process state, snapshot updates, admission limits and tunnel tracking |
| `demo` | A separate opt-in demonstration backend with its own listener |

### Rust source organization

Rust modules are kept as small responsibility-oriented files. Every checked-in `.rs` file
is limited to **500 lines**; `tools/checks/rust_module_contracts.py` enforces the limit across
the repository. Large logical modules use a thin facade (`config.rs`, `docker.rs`,
`server.rs`, `runtime.rs`, `balance.rs`, `prometheus.rs`, and similar) plus files in a
matching directory. `web_server` uses the standard `web_server/mod.rs` facade with
submodules for configuration, filesystem confinement, responses, routing, FastCGI and
`.htaccess`.

Unit-test implementations are not embedded in production source files. They live in
`tests.rs` or `tests/` files and are included or declared only behind `#[cfg(test)]`.
Source-level CI also checks `include!` targets, balanced split fragments and dangling
attributes/doc comments. This prevents a file boundary from separating `#[derive]`,
`#[test]` or documentation from the Rust item it belongs to.

Docker discovery is an optional provider boundary implemented through the standalone
`docker-agent/`; the RustProxy process never opens the Docker socket. Cloudflare Tunnel
remains a future module. Certbot-based HTTP-01/Cloudflare DNS/OVH DNS/RFC2136 issuance and managed renewal are optional
control-plane adapters. Provider work stays outside proxy request paths and
publishes certificates only through validated configuration snapshots.

## Request lifetime

A listener owns a shared TCP connection lease. Each admitted request owns an
in-flight permit and an Exchange, which remains alive through streamed request and
response bodies. HTTP/2 connections share the public TCP cap with HTTP/1.1 and TLS.
Upgraded WebSocket tasks retain both the Exchange and TCP lease after Hyper stops
managing the HTTP connection, preventing upgrades from escaping limits or counters.

Routing captures one snapshot. Socket peer identity, host and route ACLs plus configured
authentication are checked before source dispatch, DNS or upstream connection reuse. A selected
Web Server source opens only its compiled Web Service root and returns through the shared response
pipeline; a Reverse Proxy source continues to a stored backend origin. The public client never
supplies an arbitrary upstream destination or filesystem root. Header
rules cannot change the identity already used by ACL. Ordinary bodies pass through
without full aggregation. For an explicitly transform-enabled vhost, eligible bounded
static responses use the logical pipeline `origin -> decode supported Content-Encoding
-> transform -> cache identity representation -> downstream compression`. Origin
`no-transform` is authoritative; changed bytes receive a RustProxy representation ETag
while origin validators are retained separately for revalidation. WebSocket data uses
fixed-size copy buffers.

New requests on existing keep-alive/HTTP/2 connections load the current snapshot.
Existing bodies and tunnels retain the old snapshot/backend. Each accepted TLS
connection captures the current certificate configuration before its handshake.
Certificate validation and file reads are never performed inside the SNI resolver.

## Startup and shutdown

Startup opens/migrates SQLite, loads existing intent, prepares required certificates,
compiles the snapshot and binds the requested listeners. The demo only seeds a new
database. A corrupt configuration is not automatically reset. After the runtime is published,
the optional Docker poll loop contacts enabled agents; failures remain observations and do not reset desired configuration or block ordinary proxy service.

Shutdown cancels listeners and tracked tunnels, drains admitted HTTP connections
within the configured grace period, drains accepted configuration writers and
closes SQLite. Bounded runtime shutdown avoids waiting forever for cancelled OS DNS
workers. This does not turn a blocking getaddrinfo call into a killable operation.

## Deliberate development tradeoffs

The panel remains loopback HTTP. All active configuration changes currently rebuild
backend pools and certificate state, even if unrelated fields changed. Compatible health/counter/affinity observations are retained independently; idle
connection and certificate reuse optimization still requires measurements. Backend loop protection checks local listener addresses by default while allowing remote origins on any valid port. A per-backend `accept_local_listener_risk` opt-in can deliberately allow a local RustProxy listener (for example port 9090) without bypassing private/special-address policy. DNS special-range restrictions
are basic safeguards rather than an independently audited network policy engine.
No performance target has been established or measured for this package.


## 0.1.0 response pipeline and resources

Observe pins one immutable snapshot for the exchange. Host/route ACLs precede the
cache and all upstream I/O. Eligible GET lookup uses effective request headers
and a revisioned representation key. On a miss the origin response is validated
before and after global/route header rules; a bounded tee records complete bodies
while still yielding frames immediately. No request-path SQLite/provider lookup is
introduced. Server policy and optional streaming compression apply after cache
lookup/fill wrapping. The outer MeteredBody counts delivered encoded payload frames.

Cache publication uses a short mutex section and LRU/byte/entry bounds; fill buffers
have eight semaphore slots. Purge increments a generation, preventing older in-flight
fills from publishing. Config commit invalidates cache before snapshot publication,
without awaiting between those operations. New listener intent is persisted but
active sockets remain unchanged until restart; original CLI defaults are retained
separately so clearing an override has accurate desired-state reporting.

`cache`, `transform`, `compression`, `settings`, `diagnostics`, and `system_stats` are dedicated modules. `diagnostics` owns periodic sampling/export while `system_stats` adapts the shared `systemstat` host API, `sysinfo::Networks` network API, and one `metrics-process` process API. Network inventory/counters use `sysinfo` as the primary source and merge `systemstat` as a secondary library source, so missing/partial primary inventory is supplemented without shell commands; no BSD/Linux shell-command network parser is maintained. `metrics-process` uses its documented dummy fallback where no native process collector exists, so unsupported fields remain absent without adding another platform parser. Linux adds its compatibility-only `/proc`/cgroup fields after the common sample. The
sampler runs blocking OS/library reads off the async request path roughly every two
seconds and stops through the process cancellation token. Language discovery runs
in build.rs, not in browser hardcoded PL/EN logic. The result is an embedded public
JSON asset; existing logo resources remain intact.

## Vhost pool publication and observations

The immutable routing snapshot contains actual-address-constrained routes, compiled
frontends, pools and both TLS ALPN configurations. The process separately retains
its actually bound listener plan. New routing uses desired bindings immediately;
new sockets/ALPN take effect only at restart. This can temporarily deny moved hosts,
but never silently broadens them to still-active old bindings.

Each pool owns immutable member definitions and clients. Small synchronized health,
scheduling and traffic observations live separately. Compatible publication reuses
observations/tokens by vhost/member/origin/health identity; changed or re-enabled
members are quarantined. Request leases retain the old member/client until both
body directions or the upgraded tunnel finish. Changes do not replay requests.

A weak-App, cancellation-aware supervisor schedules a rotating, bounded window of
probes outside the data path. DNS and configured-origin policies are shared with
normal forwarding; HTTP checks use a dedicated connection and discard the body.
Traffic sampling and status snapshots operate on these observations, not SQLite.

Request order is actual frontend selection -> host/vhost/path ACL/auth -> location/source selection. Web Server then performs confined local filesystem handling; Reverse Proxy continues through eligible member selection -> member-isolated cache lookup -> guarded upstream request.
Cookie affinity bypasses cache. Cache hits release the selection without recording
an actual upstream request. Response protection is applied after cache selection,
and per-vhost HSTS is removed on plaintext, including synthetic responses.

## Integrations in 0.4.0

`forwarding` compiles the explicit trust boundary and reconstructs canonical
metadata. `tinyauth` owns immutable verifier configuration and an outbound client;
it authorizes before the existing origin/cache path. `http3` adapts bounded Quinn
streams to the same Axum router and response Body without a separate cache/auth
implementation. Named listener IDs resolve to actual bindings while compiling
vhosts; accepted destination addresses still control route visibility.

TLS and QUIC use the same validated certificate resolver. Active endpoint limits
remain fixed until restart; new QUIC handshakes can use the latest certificate
snapshot without changing Retry token keys. Graceful shutdown signals H3 GOAWAY
and bounds drain before endpoint close. Runtime conformance remains to be tested.


## 0.5.0 cache inspection and certificate jobs

Cache decision counters and entry metadata live in the bounded cache store. A
separate authenticated paginated read API returns metadata, not bodies, queries or
request headers. Expiry cleanup also runs during inspection; generation barriers
still prevent old in-flight fills from republishing after purge/config changes.

The ACME manager is outside proxy request paths: one job semaphore, a tracked
child-process group, JSON over stdin, sanitized status and private Certbot/RFC2136
profiles. Only validated immutable certificate metadata can enter the existing
library through explicit revision CAS publication. See ACME_RFC2136.md for the
required external tools and unverified integration boundary; RENEWAL.md documents
the 0.7 scheduler and automated publication.

## 0.6 boundaries

`legacy` is an input-only migration module, not a traffic dispatcher. It emits the
same canonical vhost model used by the UI. Validated startup migration uses one
CAS write; failure before validation completion leaves the stored JSON untouched.

`acme` manages one private worker job; `scripts/runtime/acme_provider.py` adapts Certbot
webroot/Cloudflare/OVH/RFC2136. Only the running HTTP-01 job's names and token files
are exposed by the reserved challenge middleware. Provider files remain private,
not part of SQLite or generic configuration exports. Issuance/import are separate.

`prometheus` owns a dedicated read-only HTTP router, actual-peer CIDRs, environment
bearer token and bounded scrape work. It does not share administrator cookies.
Instrumentation is process-local; per-vhost counters reuse stable-ID observations
across configuration changes. Domain cache gauges intentionally reset with revision.
Admin request timing measures response headers, while proxy/vhost timing measures
complete body/tunnel lifetime. Neither renderer queries a provider or SQLite to
handle public proxy requests.

## 0.7 renewal, access and deployment boundaries

A shutdown-tracked weak-reference scheduler selects due configured managed
certificates. It shares the issuance semaphore and per-profile advisory lock.
Provider work remains outside request paths. Publication validates PEM/key/names
and the complete current configuration, then uses the same SQLite CAS/snapshot
path. Only changed generations publish; failures do not touch the active snapshot.
TCP and QUIC handshakes consult current TLS state. Certificate publication also
uses the existing cache-generation invalidation. See RENEWAL.md for timing/limits.

Basic Auth is a separate vhost gate after cumulative IP ACLs and before static
responses, cache and upstream selection. Argon2id verification is bounded by a
shared semaphore held by blocking workers and an IP failure table. Hashes, not
passwords, are persisted. A verified Authorization header is optional and reapplied
after custom request headers; unverified identity headers are removed. Protected
responses bypass the shared cache and authentication failures cannot become custom
200 responses. WebSocket admission uses the same gates before opening a tunnel.

`pool_eligible=false` distinguishes a location-pinned backend from ordinary pool
membership. The GUI edits the same prefix route model as the configuration API;
there is no second nginx-style parser or unvalidated location execution path.

Server deployment uses root-owned immutable release directories, an atomic current
symlink, a non-root systemd service and private service-owned data. A stopped-state
backup precedes update publication. Failed health/version checks attempt to restore
both old code and data; backups and failed-generation state remain for recovery.
Live systemd/database recovery acceptance is not replaced by helper mocks.