82 lines
4.4 KiB
Markdown
82 lines
4.4 KiB
Markdown
# RustProxy 0.34.0 HOT HIT follow-up (source candidate)
|
|
|
|
## What changed
|
|
|
|
The *existing* optimized HOT cache no longer removes/reinserts full entries at
|
|
HIT time. This follow-up further shortens the critical section without changing
|
|
cache policy or accepting an approximate eviction algorithm:
|
|
|
|
1. Store each published response's immutable headers in `Arc<HeaderMap>`.
|
|
Snapshot the Arc and the shared `Bytes` body while locked, release the HOT
|
|
mutex, then clone the headers and build the per-request response. Header
|
|
cloning is **not eliminated**, because per-request `Age`, `Cache-Status` and
|
|
transformation annotations must not mutate the stored representation.
|
|
2. Preserve exact per-HIT sequence, hit count and LRU ordering. Bypass the Vary
|
|
index traversal for the common direct-key/no-`Vary` representation; continue
|
|
selecting keyed variants for other representations.
|
|
3. Replace the per-domain flat pair-key map with a nested ordered map so a
|
|
known `(scope, domain)` is updated using borrowed keys. Preserve the 4096
|
|
distinct-domain limit and the dashboard wire format. Update cached-byte
|
|
accounting without allocating `String` keys.
|
|
4. When there are no active BAN rules, a cached response does not acquire the
|
|
BAN mutex or clone any BAN rules. Active rules still perform expiration,
|
|
matching and removal. Publication/removal of BAN rules updates an atomic
|
|
count; BAN and PURGE security checks remain enabled.
|
|
5. Remove repeated full HOT-entry `sweep()` scans. The existing
|
|
`retention_deadline()` explicitly returns `u64::MAX`: entries beyond their
|
|
freshness TTL are retained for conditional revalidation and `max-stale`.
|
|
Capacity/idle eviction and cold-tier GC remain responsible for memory/disk
|
|
retention. HTTP freshness checks and `Age` are unchanged.
|
|
6. Avoid allocating the static cache-reason text on each repeat occurrence:
|
|
previously registered reasons update by borrowed lookup. Recorded dashboard
|
|
fields, counters and reason labels are unchanged.
|
|
|
|
No new configuration, cache key, COLD on-disk format, migration, panel toggle or
|
|
feature flag was added. Cache query bypass, transformed representations, origin
|
|
validators, TTL, hot-to-cold demotion, backend selection and `sendfile` behavior
|
|
are intentionally unchanged.
|
|
|
|
## Tests and verification
|
|
|
|
Native Rust test file: `src/cache/tests/hot_hit_034_tests.rs`.
|
|
|
|
- Multi-threaded HOT access checks monotonic, unique hit counters and bounded
|
|
LRU/index growth.
|
|
- Transformed response HIT retains its ETag/extension; changes to the response
|
|
header map never mutate cached headers. Memory accounting and per-domain
|
|
saved-byte counters remain exact.
|
|
- Exact LRU eviction, selective BAN, stale revalidation retention and
|
|
conditional client `max-stale` are covered.
|
|
- The manual **ignored** in-process lookup microbenchmark measures the whole
|
|
`get_hot()` response construction, not network RPS.
|
|
|
|
```bash
|
|
./dev.sh check
|
|
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
python3 scripts/tests/cache_hot_034_static_test.py
|
|
cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
|
|
```
|
|
|
|
For real HTTP RPS use the supplied process regression and same-host nginx
|
|
benchmark (`scripts/performance/README.md`) with **identical** configs. The
|
|
standard nginx comparator benchmark explicitly disables the cache; it cannot
|
|
measure this HOT-specific improvement. Compare two compiled RustProxy 0.34.0
|
|
binaries with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
|
|
Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
|
|
ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
|
|
|
|
## Remaining performance bottlenecks
|
|
|
|
The HOT store still uses a shared mutex to provide exact LRU, per-key
|
|
generation checks and publication guarantees. Per-response `HeaderMap` cloning
|
|
is still required and now occurs outside this lock. Sharding the store or
|
|
making LRU eventually consistent would require a separate, extensively tested
|
|
cache architecture revision: doing that without compilation and live soak
|
|
would risk invalidation, accounting, BAN and tier-demotion regressions.
|
|
|
|
**Acceptance gate open:** the changed Rust source was not compiled in this
|
|
environment, new Rust tests were not run and no HOT HIT RPS improvement has been
|
|
measured. Source-only guards, SQL and upgrade tests do not establish a safe
|
|
production build.
|