v0.34.0_2

docs/CACHE_HOTPATH_0.34.0.md

@@ -0,0 +1,81 @@
+# RustProxy 0.34.0 HOT HIT follow-up (source candidate)
+
+## What changed
+
+The *existing* optimized HOT cache no longer removes/reinserts full entries at
+HIT time. This follow-up further shortens the critical section without changing
+cache policy or accepting an approximate eviction algorithm:
+
+1. Store each published response's immutable headers in `Arc<HeaderMap>`.
+   Snapshot the Arc and the shared `Bytes` body while locked, release the HOT
+   mutex, then clone the headers and build the per-request response. Header
+   cloning is **not eliminated**, because per-request `Age`, `Cache-Status` and
+   transformation annotations must not mutate the stored representation.
+2. Preserve exact per-HIT sequence, hit count and LRU ordering. Bypass the Vary
+   index traversal for the common direct-key/no-`Vary` representation; continue
+   selecting keyed variants for other representations.
+3. Replace the per-domain flat pair-key map with a nested ordered map so a
+   known `(scope, domain)` is updated using borrowed keys. Preserve the 4096
+   distinct-domain limit and the dashboard wire format. Update cached-byte
+   accounting without allocating `String` keys.
+4. When there are no active BAN rules, a cached response does not acquire the
+   BAN mutex or clone any BAN rules. Active rules still perform expiration,
+   matching and removal. Publication/removal of BAN rules updates an atomic
+   count; BAN and PURGE security checks remain enabled.
+5. Remove repeated full HOT-entry `sweep()` scans. The existing
+   `retention_deadline()` explicitly returns `u64::MAX`: entries beyond their
+   freshness TTL are retained for conditional revalidation and `max-stale`.
+   Capacity/idle eviction and cold-tier GC remain responsible for memory/disk
+   retention. HTTP freshness checks and `Age` are unchanged.
+6. Avoid allocating the static cache-reason text on each repeat occurrence:
+   previously registered reasons update by borrowed lookup. Recorded dashboard
+   fields, counters and reason labels are unchanged.
+
+No new configuration, cache key, COLD on-disk format, migration, panel toggle or
+feature flag was added. Cache query bypass, transformed representations, origin
+validators, TTL, hot-to-cold demotion, backend selection and `sendfile` behavior
+are intentionally unchanged.
+
+## Tests and verification
+
+Native Rust test file: `src/cache/tests/hot_hit_034_tests.rs`.
+
+- Multi-threaded HOT access checks monotonic, unique hit counters and bounded
+  LRU/index growth.
+- Transformed response HIT retains its ETag/extension; changes to the response
+  header map never mutate cached headers. Memory accounting and per-domain
+  saved-byte counters remain exact.
+- Exact LRU eviction, selective BAN, stale revalidation retention and
+  conditional client `max-stale` are covered.
+- The manual **ignored** in-process lookup microbenchmark measures the whole
+  `get_hot()` response construction, not network RPS.
+
+```bash
+./dev.sh check
+CARGO_INCREMENTAL=0 ./dev.sh test
+CARGO_INCREMENTAL=0 cargo build --release --locked
+python3 scripts/tests/cache_hot_034_static_test.py
+cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
+```
+
+For real HTTP RPS use the supplied process regression and same-host nginx
+benchmark (`scripts/performance/README.md`) with **identical** configs. The
+standard nginx comparator benchmark explicitly disables the cache; it cannot
+measure this HOT-specific improvement. Compare two compiled RustProxy 0.34.0
+binaries with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
+Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
+ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
+
+## Remaining performance bottlenecks
+
+The HOT store still uses a shared mutex to provide exact LRU, per-key
+generation checks and publication guarantees. Per-response `HeaderMap` cloning
+is still required and now occurs outside this lock. Sharding the store or
+making LRU eventually consistent would require a separate, extensively tested
+cache architecture revision: doing that without compilation and live soak
+would risk invalidation, accounting, BAN and tier-demotion regressions.
+
+**Acceptance gate open:** the changed Rust source was not compiled in this
+environment, new Rust tests were not run and no HOT HIT RPS improvement has been
+measured. Source-only guards, SQL and upgrade tests do not establish a safe
+production build.