v0.30.0

docs/FILE_LOGS.md

@@ -67,6 +67,8 @@ Logrotate-style gzip rotations are discovered next to configured combined access
 
 Gzip reading is bounded before and after decompression. A compressed source larger than 8 MiB is not decompressed for a query, decompressed output is capped at 64 MiB, and only the bounded tail is parsed/searched. These limits prevent a gzip rotation from becoming an unbounded memory/decompression workload.
 
+Per-vhost directory discovery now retains only the newest **256** matching JSONL/gzip candidates while scanning, and a combined log’s gzip-rotation scan retains only the newest **64** matches. Both caps apply before sorting, so directory size no longer causes a proportional memory allocation for the inventory. The usual 32-source query budget still applies. Filesystem enumeration still takes time proportional to the number of directory entries.
+
 The reader remains bounded:
 
 - up to 2 MiB of retained tail data from one source;

docs/MEMORY_STRESS.md

@@ -0,0 +1,96 @@
+# Long-running memory verification (RustProxy 0.30.0)
+
+This procedure supplements static source checks. A synthetic Python model and a
+local Python HTTP peer **do not execute RustProxy** and cannot establish its
+production RSS, heap growth or absence of leaks.
+
+## Audit coverage
+
+| Area | Source-level finding | Action in the same 0.30.0 version |
+| --- | --- | --- |
+| Selective cache PURGE | Per-key generation metadata could retain every distinct purged key indefinitely after its object was removed | Reset stale per-key metadata at the existing global generation barrier; drop an abnormally large map allocation; regression for 512 different purges |
+| HOT/COLD cache | Body/entry limits, bounded transient leases, limited cold queue, revalidation slots and background cleanup permits | Kept existing behavior |
+| Log output (JSONL/file/Logstash/Loki) | Serialized the same entry again when Loki and a JSONL sink ran simultaneously; sink queues and per-vhost open files were already bounded | Reuse serialized JSONL content for Loki; keep redaction and sink-specific formatting |
+| Historical log discovery | Prior audit already capped directory inventory while scanning, at 256 per-vhost and 64 gzip entries | Retained existing 0.30.0 limits |
+| Traffic analytics | Top lists limited to 32 entries per dimension, at most 64 vhosts and a 2048-item input queue, but repeated labels incurred transient `String` allocations | Borrow existing labels, allocate only when admitting/replacing a top-list entry |
+| Local directory listing | Two simultaneously live vectors for directory metadata and a copied HTML response | Filter directory metadata in place; move output bytes into response; keep all filenames and HEAD Content-Length behavior |
+| TCP/UDP streams and HTTP/H2/H3 | Connection/session permits, finite queues, idle deadlines and completed task draining | No change |
+| Upstream clients, DNS, health probes | DNS worker limit, timeouts, pool recycling, bounded health checks | No change |
+| Compression, transforms, PHP-FPM | Streaming response paths, transform memory and concurrency budgets, FastCGI streaming | No change |
+| Rate limits, ACME, Docker, background tasks | Limited client buckets with expiry and shrink, cancelled/limited background activities | No change |
+
+This is **not** a proof that every runtime allocation is released. The directory
+listing feature intentionally generates the complete visible directory as HTML;
+very large directories may still need substantial temporary memory. A high RSS
+after a load spike may also be allocator reuse instead of an active leak. Verify
+with real-runtime measurements rather than assuming either explanation.
+
+## Source/synthetic verification
+
+```sh
+python3 tools/checks/memory_contracts.py
+python3 tools/checks/high_traffic_memory_model.py --http-requests 10000
+./dev.sh check
+./dev.sh test
+```
+
+The offline test replays 300,000 distinct PURGE identities over six periods.
+It verifies the old cumulative model retains 50k/100k/150k/200k/250k/300k
+metadata keys whereas the revised epoch-based model holds one key after each
+operation. This is a **cardinality model**, not a Rust heap measurement. It also
+sends 10,000 HTTP GETs across 24 concurrent persistent connections and 1,000
+paths to a **Python test peer** to validate the load driver (the default quick
+`./dev.sh test` run sends 2,000 of those requests). The Rust regression
+for 512 distinct purges is executed only if `cargo test` runs successfully.
+
+## Real RustProxy staging traffic and RSS
+
+On a Linux staging machine with Rust/Cargo installed, configure a vhost and a
+known reachable backend/local Web Service. Enable whichever logging targets you
+need to test (JSONL file, Logstash, Loki) in the normal dashboard. Start the
+proxy and determine the **RustProxy** PID, not the load generator's PID.
+
+```sh
+./dev.sh check && ./dev.sh test
+# In another terminal: run the configured instance, e.g. ./dev.sh run
+pid=$(pgrep -n -x rust-proxy)
+
+# Warm / repeated URL: cache HITs, persistent clients, normal JSON access logs.
+python3 tools/dev/memory_load.py \
+  --url http://127.0.0.1:8080/ \
+  --host-header example.test \
+  --pid "$pid" --requests 400000 --concurrency 128 \
+  --cooldown 120 --output /tmp/rustproxy-memory-repeat.json
+
+# High cardinality: many URL paths, cache misses, large analytics churn.
+# Pick a staging path prefix accepted by your origin; status codes will be
+# recorded even when the requested URLs return 404.
+python3 tools/dev/memory_load.py \
+  --url http://127.0.0.1:8080/ \
+  --host-header example.test \
+  --pid "$pid" --requests 400000 --concurrency 128 \
+  --unique-paths 100000 --cooldown 120 \
+  --output /tmp/rustproxy-memory-distinct.json
+```
+
+Run at least three **successive** traffic/cooldown cycles on the same process.
+Test separately with access JSONL only, JSONL+Logstash and JSONL+Loki using
+reachable staging services; add streaming large response bodies, directory
+listings, TCP/UDP and HTTP/3 using appropriate specialized clients. Send
+controlled cache PURGE requests through your authenticated cache invalidation
+API and ensure previously purged cache entries never reappear.
+
+The tool samples `/proc/<pid>/status` (RSS, high-water mark, threads) and
+`/proc/<pid>/fd` at a configurable interval. Output includes response codes,
+requests/second, received bytes, time series and start/peak/end RSS. Check
+steady-state RSS after each cooldown, open file descriptors, timeouts, dropped
+logs, HIT/MISS behavior, and whether growth **continues** across periods of
+similar load after caches and connections have stabilized. In addition, inspect
+cache/transformation memory accounting and capture a Rust heap/allocation
+profile on the actual instance if RSS rises repeatedly.
+
+Do not run these load tests against an active production service without a
+maintenance or capacity plan. The load driver does not create vhosts, backend
+services, log collectors or a RustProxy executable, and it cannot reproduce
+all TCP, UDP, H2 and H3 protocol behavior. None of these real-instance
+acceptance steps were performed in the source-only audit environment.

docs/README.md

@@ -65,6 +65,7 @@ This is the canonical index for the current project documentation. The authentic
 
 - [MOCK_SERVER](MOCK_SERVER.md) - local Python mock origin used by development/integration checks.
 - [MOCK_AUTH](MOCK_AUTH.md) - local fake Tinyauth peer for development only.
+- [MEMORY_STRESS](MEMORY_STRESS.md) - sustained-memory audit, offline load simulations, real RustProxy staging stress and RSS measurement procedure.
 
 ## Technical documentation
 

docs/TEST_REPORT.md

@@ -1,4 +1,20 @@
-# Verification report - RustProxy 0.29.7
+# Verification report - RustProxy 0.30.0
+
+## 0.30.0 same-version sustained-memory audit (2026-10-10)
+
+- Reviewed cache metadata, HOT/COLD admission and cleanup, file/remote JSON sinks, live log history, analytics, local directory listings, HTTP/2/HTTP/3, TCP/UDP sessions, DNS/health tasks, PHP-FPM streaming, transforms, rate-limit buckets, and periodic tasks. See [memory verification](MEMORY_STRESS.md) for audited invariants and the real-runtime acceptance procedure.
+- Confirmed a long-run retention issue: the selective PURGE path incremented `Store.key_generations` for affected base keys and never reclaimed those identities until an all-cache purge. Its existing global `generation` barrier already invalidates all older fills/tickets, so obsolete per-key metadata can be reset per selective PURGE. Also release large HashMap capacity after mass invalidations. Added a 512-distinct-PURGE Rust regression (not executed without Cargo).
+- Removed unnecessary repeated analytics-label allocations using borrowed map lookups and allocation on insertion only. Filtered local directory listings in place rather than building a second metadata vector and moved generated HTML into the response instead of cloning it. Reused the existing redacted JSONL serialization when sending the same access event to Loki (while preserving the Loki-only path).
+- Offline test: **300,000 simulated distinct PURGE identities** over six intervals. Legacy bookkeeping model retained **50k → 300k** keys, whereas the corrected epoch model retained **1** key after each operation. The load driver sent **10,000 real HTTP requests** using **24 persistent clients** and **1,000 distinct paths** to a **Python test peer**, with expected status/body counts. These tests validate model behavior and driver operation only: **they do not measure RustProxy RSS, heap, or leak freedom**.
+- `./dev.sh test`: all Rust-independent source, security, SQL/upgrade, Node/browser UI, documentation/translation and memory-replay checks passed (including its 2,000-request peer smoke); **exit 127 only at the missing Cargo toolchain guard**. `./dev.sh check` also exits 127 for missing Cargo. `go test ./...` in `docker-agent` passed.
+- Rust/Cargo unavailable in this environment. Actual `cargo check`, Rust unit tests, RustProxy GET/PURGE under load, production-like JSON sinks, HTTP/2/3 and TCP/UDP traffic, and long-duration RSS/heap plateau measurements **remain pending**. No API, SQLite schema, Cold Cache format or version changes.
+
+## 0.30.0 source verification (2026-10-10)
+
+- Moved per-backend health-check overrides into the **Health** tab: a monitored-only backend selector reveals one persistent editor at a time, without changing the JSON configuration schema. Newly enabled custom policies start from the current shared policy; previously customized backends keep their configured values. Disabling monitor hides the editor but retains the override. Preserving draft switching, validation routing, rename and removal is covered by the simulated DOM regression.
+- Memory audit: access-file JSONL and Logstash now share one serialization for each event. Existing bounded file/error sink queues, live history, open files, traffic analytics and cache/transform budgets were checked in source. Removed two avoidable directory-inventory memory spikes: per-vhost log scanning now retains at most 256 candidates while enumerating and gzip-rotation scanning at most 64. Added a source contract and a Rust unit regression for 10,000 candidate filenames and deterministic newest-first ties.
+- The Node/UI, source, security, upgrade, SQL migration, translation and documentation checks in `./dev.sh test` are run in this environment; the Rust suite and full `./dev.sh check` cannot complete because `cargo` and `rustc` are unavailable (exit 127). `go test ./...` in `docker-agent` is run separately. The Rust unit regression is **included but not executed**. No real request-load test or long-running RSS/heap profile was performed, so the review cannot prove complete absence of runtime leaks.
+- No SQLite schema migration or Cold Cache format change; root Cargo package/lock and OpenAPI version set to `0.30.0`. Use a Rust toolchain for `./dev.sh check` and `./dev.sh test`, and measure RSS during sustained file/JSON logging, before deploying.
 
 ## Same-version 0.29.7 PHP profile UI, Rust warning and error interception review (2026-10-09)