# Long-running memory verification (RustProxy 0.30.0)

This procedure supplements static source checks. A synthetic Python model and a
local Python HTTP peer **do not execute RustProxy** and cannot establish its
production RSS, heap growth or absence of leaks.

## Audit coverage

| Area | Source-level finding | Action in the same 0.30.0 version |
| --- | --- | --- |
| Selective cache PURGE | Per-key generation metadata could retain every distinct purged key indefinitely after its object was removed | Reset stale per-key metadata at the existing global generation barrier; drop an abnormally large map allocation; regression for 512 different purges |
| HOT/COLD cache | Body/entry limits, bounded transient leases, limited cold queue, revalidation slots and background cleanup permits | Kept existing behavior |
| Log output (JSONL/file/Logstash/Loki) | Serialized the same entry again when Loki and a JSONL sink ran simultaneously; sink queues and per-vhost open files were already bounded | Reuse serialized JSONL content for Loki; keep redaction and sink-specific formatting |
| Historical log discovery | Prior audit already capped directory inventory while scanning, at 256 per-vhost and 64 gzip entries | Retained existing 0.30.0 limits |
| Traffic analytics | Top lists limited to 32 entries per dimension, at most 64 vhosts and a 2048-item input queue, but repeated labels incurred transient `String` allocations | Borrow existing labels, allocate only when admitting/replacing a top-list entry |
| Local directory listing | Two simultaneously live vectors for directory metadata and a copied HTML response | Filter directory metadata in place; move output bytes into response; keep all filenames and HEAD Content-Length behavior |
| TCP/UDP streams and HTTP/H2/H3 | Connection/session permits, finite queues, idle deadlines and completed task draining | No change |
| Upstream clients, DNS, health probes | DNS worker limit, timeouts, pool recycling, bounded health checks | No change |
| Compression, transforms, PHP-FPM | Streaming response paths, transform memory and concurrency budgets, FastCGI streaming | No change |
| Rate limits, ACME, Docker, background tasks | Limited client buckets with expiry and shrink, cancelled/limited background activities | No change |

This is **not** a proof that every runtime allocation is released. The directory
listing feature intentionally generates the complete visible directory as HTML;
very large directories may still need substantial temporary memory. A high RSS
after a load spike may also be allocator reuse instead of an active leak. Verify
with real-runtime measurements rather than assuming either explanation.

## Source/synthetic verification

```sh
./dev.sh check
./dev.sh test
# Perform separate real-runtime load and RSS measurements on staging.
```

The old Python cardinality model and protocol-peer load driver were removed
with `tools/checks/`. They were synthetic tests, not proof of RustProxy memory
bounds. `./dev.sh test` now runs only the Rust suite; runtime memory behavior
must still be verified by sustained staging traffic and RSS tracking.

## Real RustProxy staging traffic and RSS

On a Linux staging machine with Rust/Cargo installed, configure a vhost and a
known reachable backend/local Web Service. Enable whichever logging targets you
need to test (JSONL file, Logstash, Loki) in the normal dashboard. Start the
proxy and determine the **RustProxy** PID, not the load generator's PID.

```sh
./dev.sh check && ./dev.sh test
# In another terminal: run the configured instance, e.g. ./dev.sh run
pid=$(pgrep -n -x rust-proxy)

# Warm / repeated URL: cache HITs, persistent clients, normal JSON access logs.
python3 tools/dev/memory_load.py \
  --url http://127.0.0.1:8080/ \
  --host-header example.test \
  --pid "$pid" --requests 400000 --concurrency 128 \
  --cooldown 120 --output /tmp/rustproxy-memory-repeat.json

# High cardinality: many URL paths, cache misses, large analytics churn.
# Pick a staging path prefix accepted by your origin; status codes will be
# recorded even when the requested URLs return 404.
python3 tools/dev/memory_load.py \
  --url http://127.0.0.1:8080/ \
  --host-header example.test \
  --pid "$pid" --requests 400000 --concurrency 128 \
  --unique-paths 100000 --cooldown 120 \
  --output /tmp/rustproxy-memory-distinct.json
```

Run at least three **successive** traffic/cooldown cycles on the same process.
Test separately with access JSONL only, JSONL+Logstash and JSONL+Loki using
reachable staging services; add streaming large response bodies, directory
listings, TCP/UDP and HTTP/3 using appropriate specialized clients. Send
controlled cache PURGE requests through your authenticated cache invalidation
API and ensure previously purged cache entries never reappear.

The tool samples `/proc/<pid>/status` (RSS, high-water mark, threads) and
`/proc/<pid>/fd` at a configurable interval. Output includes response codes,
requests/second, received bytes, time series and start/peak/end RSS. Check
steady-state RSS after each cooldown, open file descriptors, timeouts, dropped
logs, HIT/MISS behavior, and whether growth **continues** across periods of
similar load after caches and connections have stabilized. In addition, inspect
cache/transformation memory accounting and capture a Rust heap/allocation
profile on the actual instance if RSS rises repeatedly.

Do not run these load tests against an active production service without a
maintenance or capacity plan. The load driver does not create vhosts, backend
services, log collectors or a RustProxy executable, and it cannot reproduce
all TCP, UDP, H2 and H3 protocol behavior. None of these real-instance
acceptance steps were performed in the source-only audit environment.