v0.34.0_2
This commit is contained in:
Mateusz Gruszczyński committed
1 parent
0a8bc6f040
commit
434e789144
38 files changed
+899
-238
No files matched your search
@@ -0,0 +1,81 @@
|
||||
# RustProxy 0.34.0 HOT HIT follow-up (source candidate)
|
||||
|
||||
## What changed
|
||||
|
||||
The *existing* optimized HOT cache no longer removes/reinserts full entries at
|
||||
HIT time. This follow-up further shortens the critical section without changing
|
||||
cache policy or accepting an approximate eviction algorithm:
|
||||
|
||||
1. Store each published response's immutable headers in `Arc<HeaderMap>`.
|
||||
Snapshot the Arc and the shared `Bytes` body while locked, release the HOT
|
||||
mutex, then clone the headers and build the per-request response. Header
|
||||
cloning is **not eliminated**, because per-request `Age`, `Cache-Status` and
|
||||
transformation annotations must not mutate the stored representation.
|
||||
2. Preserve exact per-HIT sequence, hit count and LRU ordering. Bypass the Vary
|
||||
index traversal for the common direct-key/no-`Vary` representation; continue
|
||||
selecting keyed variants for other representations.
|
||||
3. Replace the per-domain flat pair-key map with a nested ordered map so a
|
||||
known `(scope, domain)` is updated using borrowed keys. Preserve the 4096
|
||||
distinct-domain limit and the dashboard wire format. Update cached-byte
|
||||
accounting without allocating `String` keys.
|
||||
4. When there are no active BAN rules, a cached response does not acquire the
|
||||
BAN mutex or clone any BAN rules. Active rules still perform expiration,
|
||||
matching and removal. Publication/removal of BAN rules updates an atomic
|
||||
count; BAN and PURGE security checks remain enabled.
|
||||
5. Remove repeated full HOT-entry `sweep()` scans. The existing
|
||||
`retention_deadline()` explicitly returns `u64::MAX`: entries beyond their
|
||||
freshness TTL are retained for conditional revalidation and `max-stale`.
|
||||
Capacity/idle eviction and cold-tier GC remain responsible for memory/disk
|
||||
retention. HTTP freshness checks and `Age` are unchanged.
|
||||
6. Avoid allocating the static cache-reason text on each repeat occurrence:
|
||||
previously registered reasons update by borrowed lookup. Recorded dashboard
|
||||
fields, counters and reason labels are unchanged.
|
||||
|
||||
No new configuration, cache key, COLD on-disk format, migration, panel toggle or
|
||||
feature flag was added. Cache query bypass, transformed representations, origin
|
||||
validators, TTL, hot-to-cold demotion, backend selection and `sendfile` behavior
|
||||
are intentionally unchanged.
|
||||
|
||||
## Tests and verification
|
||||
|
||||
Native Rust test file: `src/cache/tests/hot_hit_034_tests.rs`.
|
||||
|
||||
- Multi-threaded HOT access checks monotonic, unique hit counters and bounded
|
||||
LRU/index growth.
|
||||
- Transformed response HIT retains its ETag/extension; changes to the response
|
||||
header map never mutate cached headers. Memory accounting and per-domain
|
||||
saved-byte counters remain exact.
|
||||
- Exact LRU eviction, selective BAN, stale revalidation retention and
|
||||
conditional client `max-stale` are covered.
|
||||
- The manual **ignored** in-process lookup microbenchmark measures the whole
|
||||
`get_hot()` response construction, not network RPS.
|
||||
|
||||
```bash
|
||||
./dev.sh check
|
||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
||||
python3 scripts/tests/cache_hot_034_static_test.py
|
||||
cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
|
||||
```
|
||||
|
||||
For real HTTP RPS use the supplied process regression and same-host nginx
|
||||
benchmark (`scripts/performance/README.md`) with **identical** configs. The
|
||||
standard nginx comparator benchmark explicitly disables the cache; it cannot
|
||||
measure this HOT-specific improvement. Compare two compiled RustProxy 0.34.0
|
||||
binaries with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
|
||||
Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
|
||||
ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
|
||||
|
||||
## Remaining performance bottlenecks
|
||||
|
||||
The HOT store still uses a shared mutex to provide exact LRU, per-key
|
||||
generation checks and publication guarantees. Per-response `HeaderMap` cloning
|
||||
is still required and now occurs outside this lock. Sharding the store or
|
||||
making LRU eventually consistent would require a separate, extensively tested
|
||||
cache architecture revision: doing that without compilation and live soak
|
||||
would risk invalidation, accounting, BAN and tier-demotion regressions.
|
||||
|
||||
**Acceptance gate open:** the changed Rust source was not compiled in this
|
||||
environment, new Rust tests were not run and no HOT HIT RPS improvement has been
|
||||
measured. Source-only guards, SQL and upgrade tests do not establish a safe
|
||||
production build.
|
||||
@@ -0,0 +1,77 @@
|
||||
# RustProxy 0.34.0: hot-path throughput revision
|
||||
|
||||
**Status: uncompiled source candidate.** This revision does not change the
|
||||
application version, public API, dashboard controls, cache policy or tuning
|
||||
defaults. It must not be labelled production-ready until the new Rust binary
|
||||
passes compile, unit tests, live regressions and paired performance comparisons.
|
||||
|
||||
## Code changes and intended cost reductions
|
||||
|
||||
| Focus | Change | Kept behavior / risk to verify |
|
||||
|---|---|---|
|
||||
| Per-request identity | Draw a 256-bit nonce from the OS-seeded/reseeded thread-local CSPRNG instead of asking `getrandom` on every request. Preserve SHA-256 of nonce+monotonic sequence and the existing 64-hex ID format. | IDs remain non-predictable. Check ID uniqueness and raw upstream headers. |
|
||||
| Upstream HTTP request target | Compile the validated scheme/authority `http::Uri` once with each immutable backend; parse only the request path+query and rebuild using typed `Uri::from_parts`. | Never accept a client-supplied authority, bypass DNS/address policies or change SNI. Test H1, H2, IPv6, query and percent-encoding. |
|
||||
| Load balancing | Do not clone the member ID string for an existing smooth-WRR scoreboard entry. Continue the previously introduced no-mutex single-member non-cookie path. | Sticky-cookie modes, capacity limits, retry exclusions, health and drain remain unchanged. |
|
||||
| Active health | Use already-compiled `Arc<HealthCheck>` when dispatching probes, avoiding a clone of its path/headers for each scheduled probe. | Same 250-ms supervisor, timeout, enabled/claim gate and health state transitions. |
|
||||
| Per-frame streaming | Capture the leased backend once at first streamed request DATA (response already captured at wrapper construction). Do not take `Exchange.backend` mutex on every upload chunk. | Request/response byte counters, mid-stream failures, idle timeouts and lease lifetime remain active. |
|
||||
| Pool recycling | Preserve exact request-count and elapsed-time recycling checks per request, with the existing independent 250-ms maintenance cycle. The pool factory/connection policy is unchanged. | Exact lifecycle semantics are more important than saving one cheap clock read; tune idle/age limits only after profiling. |
|
||||
| Log history | Move completed `LogEntry` directly into bounded history when there are no live viewers or enabled external sinks, instead of deeply cloning its strings. Continue cloning and delivering to live viewers/sinks when enabled. | Same JSON fields, history size, drop counters, file/Loki/Logstash forwarding. |
|
||||
| Route/HTTP instrumentation | Remove redundant route-ID mutex/write in forwarding and collapse H1/H2 version tallying into one branch. | Route is recorded once before forwarding; H1/H2 counters remain. |
|
||||
|
||||
These changes build on the previous 0.34.0 route-Arc, single-backend LB,
|
||||
conditional retry/fallback-header cloning, atomic exchange flags and cache HOT
|
||||
LRU changes. There are no new UI toggles. Pool defaults remain `max_idle_per_host`
|
||||
4 and `pool_recycle_requests` 25,000; tune these only in separately labelled
|
||||
trials. `sendfile: auto` remains opt-in and applies only to eligible Linux
|
||||
plaintext H1 static WWW responses, not reverse proxy or TLS/H2/H3.
|
||||
|
||||
## Cache transform/query policy
|
||||
|
||||
An enabled `bypass_query` is a deliberate policy: `/theme.css?v=...` is a
|
||||
`BYPASS` even when the origin sends `Cache-Control: public`. Turning the bypass
|
||||
off permits a transformed CSS object to follow `MISS -> HIT` and appear in the
|
||||
cache dashboard. An existing transformed HOT row with hit count and TTL is
|
||||
positive evidence of this. This throughput revision does not modify the
|
||||
cache eligibility logic or discard Age/ETag/Cache-Control headers.
|
||||
|
||||
## Verification already possible here
|
||||
|
||||
- **Old/reproducible reference executable only:** 110 passed, 0 failed, 1 skipped
|
||||
(`compression.codec.zstd` skipped because the optional decoder was missing),
|
||||
from live proxy/policy/cache/transform loopback regression. This does **not**
|
||||
exercise the newly modified Rust source.
|
||||
- **Old/reproducible reference executable only:** query bypass enabled gives
|
||||
`BYPASS -> BYPASS`, disabled gives `MISS -> HIT` and a transformed representation;
|
||||
the older Age regression is also reproducible on this old executable.
|
||||
- Source-only invariants: `scripts/tests/rps_hotpath_034_static_test.py` plus
|
||||
existing route/cache/sendfile checks. SQL migration and upgrade scripts pass.
|
||||
- **Not performed:** Rust compilation or newly added Rust unit tests, live
|
||||
behavior of the optimized binary, HTTP/2 live TLS, comparative RPS, CPU
|
||||
profiling and long-duration memory/health soak. The environment does not
|
||||
include `cargo`, `rustc` or `wrk`. No post-change RPS gain is claimed.
|
||||
|
||||
## Acceptance commands on a Rust-capable Linux host
|
||||
|
||||
```bash
|
||||
./dev.sh check
|
||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
||||
python3 scripts/tests/rps_hotpath_034_static_test.py
|
||||
python3 scripts/tests/cache_age_transform_034_live.py \
|
||||
--binary "$(realpath target/release/rust-proxy)" \
|
||||
--output /tmp/rustproxy034-cache-after
|
||||
python3 scripts/performance/regression.py \
|
||||
--binary optimized="$(realpath target/release/rust-proxy)" \
|
||||
--out /tmp/rustproxy034-http-regression
|
||||
```
|
||||
|
||||
Run `scripts/performance/benchmark.py` from the documented reproducible
|
||||
before/after/nginx command in `scripts/performance/README.md`. Compare the
|
||||
unchanged old 0.34.0 binary with the newly compiled source, not a baseline
|
||||
with different feature flags. Record RPS at 1,040 B, 64 KiB, 1 MiB, p95/p99,
|
||||
CPU/request, RSS, connection counts, errors and exact response-body hashes.
|
||||
Use `perf record -g -p <proxy-pid>` during a stable request interval and inspect
|
||||
`perf report --stdio` to identify the next true bottleneck.
|
||||
|
||||
Do not interpret Python origin self-tests, source inspections, or speculative
|
||||
Amdahl models as measured RustProxy throughput.
|
||||
@@ -22,6 +22,26 @@ for compilation and live HTTP, health, capacity, sticky, and H2 retry tests.
|
||||
No RPS improvement, corrected nginx gap, or full functional parity has been
|
||||
measured for this new revision. Details: `docs/PROXY_HOTPATH_0.34.0.md`.
|
||||
|
||||
## RPS-focused follow-up source revision (not yet compiled)
|
||||
|
||||
See `docs/PROXY_RPS_0.34.0.md` for detailed changes in request IDs, validated
|
||||
upstream URI construction, LB scheduling, probes, streaming byte accounting,
|
||||
unchanged pool generation lifetimes and log-history ownership. Source-only invariants pass.
|
||||
A live test of the PREVIOUS executable (not this source revision) completed
|
||||
110 passing cases, 0 failing cases and 1 skipped optional zstd decoder in the
|
||||
proxy/policy/cache/transform harness. The old executable also confirms query-
|
||||
string bypass is deliberate and reproduces the older Age/TTL bug. The new
|
||||
binary has NOT been built or tested and its RPS gain is unknown.
|
||||
|
||||
## HOT HIT follow-up source revision (not yet compiled)
|
||||
|
||||
See `docs/CACHE_HOTPATH_0.34.0.md`. Response header cloning has moved outside
|
||||
the global HOT lock, and redundant bounded stats key allocation, empty BAN
|
||||
checks and full HOT TTL sweeps were removed. A new Rust concurrency/regression
|
||||
suite and an ignored microbenchmark have been added. Existing COLD and Vary
|
||||
behavior, exact LRU and cache semantics remain. This source has not been built
|
||||
or run under Rust in this environment; no RPS gain is claimed.
|
||||
|
||||
## Changes and correctness boundaries
|
||||
|
||||
- Operational per-vhost saves (upstream keepalive/recycling, connection limits,
|
||||
|
||||
Reference in new issue
Block a user