133 lines
6.9 KiB
Markdown
133 lines
6.9 KiB
Markdown
# RustProxy 0.34.0 DEV — 1 KiB lock contention follow-up
|
|
|
|
## Goal and baseline
|
|
|
|
**Priority 1: reverse proxy, HTTP/1.1 keep-alive, uncompressed 1,040-byte
|
|
responses, cache OFF.** Aiming for approximately 100,000 RPS and at least 80%
|
|
of nginx under the same pinned-CPU benchmark. Priority 2: local static WWW,
|
|
measured separately. No feature removals and **no version bump**.
|
|
|
|
The user supplied a **new executable** (SHA-256
|
|
`1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa`,
|
|
`rust-proxy 0.34.0`), apparently built from the prior DEV revision. It passed
|
|
151 actual black-box HTTP tests with 0 failures and 1 skipped optional zstd
|
|
decoder. This runtime coverage belongs to the **supplied executable only**,
|
|
not to the freshly edited source below. Neither `cargo` nor `rustc` is
|
|
available in the editing environment; the new revision has NOT been compiled.
|
|
|
|
## Modified source
|
|
|
|
### `Cache::record` and `Cache::record_reason`
|
|
|
|
Before, every HIT, MISS and BYPASS acquired the same `Store` mutex used by the
|
|
HOT object store, exact LRU and cache inventory. With cache disabled, a 1 KiB
|
|
request still contested that mutex to increment `cache_disabled`.
|
|
|
|
`src/cache/stats.rs` now holds bounded per-(scope,domain) statistics separately:
|
|
|
|
- An `ArcSwap`-published immutable, borrowed-lookup BTreeMap of domain handles.
|
|
**The normal path does not lock or allocate**, even for BYPASS. New domains
|
|
alone use a registration mutex and a copy-on-first-visit registry update.
|
|
- Atomic counts for HIT, MISS, BYPASS, stored objects and saved body bytes.
|
|
The common static reasons (`cache_disabled`, `cache_hit`, `cache_miss`,
|
|
`request_cookie`) use direct atomic counters. Other reason names use another
|
|
immutable published index and atomic counters; they are registered once.
|
|
- Hard cardinality remains **4096 (scope, domain) pairs**. When full, new
|
|
unregistered domains are skipped as before. Scope purge preserves stats.
|
|
Only a revision-bearing global purge resets the per-domain registry.
|
|
- The dashboard still exposes the same `DomainCounters`, reason strings,
|
|
saved byte counts, ratio and per-vhost/object views. Counter snapshots are
|
|
quiescent-exact, but under concurrent requests individual fields can reflect
|
|
slightly different instants, consistent with other atomic metrics.
|
|
- HOT object and COLD representation/LRU access, barriers, memory accounting,
|
|
cache statuses, hit policy, retry and backend security paths are unchanged.
|
|
|
|
Trade-offs: a first-visit registration copies the bounded domain index and
|
|
allocates one counter set; this is not an adaptive, unbounded cache. The hot
|
|
path's ArcSwap guard and lookup still cost CPU. Measure that cost in profiles;
|
|
compare under high-cardinality traffic to identify any excessive registration
|
|
cost before production acceptance.
|
|
|
|
### Traffic analytics queue saturation
|
|
|
|
Previously `Exchange::drop` cloned the vhost, route, method, path, IP, user
|
|
agent and country before trying to enqueue the `Sample`. If the bounded
|
|
2,048-element MPSC queue was full, all those clones were discarded.
|
|
|
|
`Analytics::push_lazy` now acquires the same bounded Tokio queue permit with
|
|
`try_reserve` first and creates the sample only after a successful reservation.
|
|
Rejections increment `queue_dropped` exactly as before. Neither the analytics
|
|
worker, counters/top lists, reset behavior nor request log path is removed.
|
|
This improves allocation costs **on overflow** but does not turn dropped
|
|
analytics into delivered samples; monitor the existing drop counter.
|
|
|
|
### Per-vhost duration histogram atomics
|
|
|
|
Previously `VhostMetrics::complete` incremented every cumulative latency bucket
|
|
whose threshold covered the observation and the `+Inf` bucket. For an ordinary
|
|
short response this meant **11 shared atomic RMWs** for the histogram alone,
|
|
on top of existing exchange/status/byte/latency metrics.
|
|
|
|
It now increments **exactly one exclusive bucket** chosen from the same 10
|
|
latency boundaries. Both the regular vhost and Docker-vhost Prometheus exporters
|
|
compute the original cumulative `le` buckets, `+Inf` and `_count` from the 11
|
|
exclusive buckets **at scrape time**. The metric names, labels, boundaries,
|
|
body-lifetime timing and measurement collection remain intact. The cumulative
|
|
snapshot may be concurrent with request completions, as before. Added boundary
|
|
unit tests for exact thresholds and long-duration `+Inf`.
|
|
|
|
This is source-only: neither the throughput benefit nor compiled Rust behavior
|
|
has been verified in the editing environment. Review exporter parity and
|
|
under-load histogram correctness on the newly compiled executable.
|
|
|
|
## Verification and untested claims
|
|
|
|
- PASS: source preflight, cache/HOT/LB/RPS/histogram/static source guards and <=500-line
|
|
production Rust modules.
|
|
- PASS: SQLite migration + preset constraints, upgrade/lock and source
|
|
replacement tests.
|
|
- PASS: compiled user-supplied binary: 151 real HTTP checks, 0 failed,
|
|
1 optional zstd skip. This binary predates the source changes above.
|
|
- NOT RUN: Rust compile, Rust unit tests, native regressions or RPS after
|
|
rebuilding the new source. No numerical speedup attributed to this patch.
|
|
- Current diagnostic A/B on the user-supplied binary may use
|
|
`scripts/performance/ab_1kb_dev.py` and must remain labelled *old vs already
|
|
supplied binary*, **not** old vs newly edited source. The `wrk` 30 s x 7
|
|
pinned-CPU benchmark remains the production acceptance gate.
|
|
|
|
## Repeat on a compiler-enabled benchmark host
|
|
|
|
```bash
|
|
./dev.sh doctor
|
|
./dev.sh check
|
|
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
python3 scripts/performance/regression.py \
|
|
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
--out /tmp/rustproxy034-counter-http
|
|
python3 scripts/performance/lb_regression.py \
|
|
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
--out /tmp/rustproxy034-counter-lb
|
|
python3 scripts/performance/transport_regression.py \
|
|
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
--out /tmp/rustproxy034-counter-transport
|
|
python3 scripts/performance/benchmark.py \
|
|
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
--nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
|
|
--out /tmp/rustproxy034-counter-benchmark \
|
|
--modes proxy --sizes small --concurrency 16,64,128 \
|
|
--duration 30 --warmup 5 --repeats 7
|
|
```
|
|
|
|
Explicitly set `max_idle_per_host=64` on **both** before/after vhosts, with the
|
|
same TLS/H1/H2/health/cache/logging/settings. Verify 1,040 bytes and SHA-256,
|
|
no failures, latency percentiles, per-request CPU, RSS, file descriptors,
|
|
backend connects and all log/traffic drop counters. Test concurrent global
|
|
purges and per-vhost purges during traffic and verify the stats reset behavior
|
|
and cache HIT/MISS/BYPASS/reason distribution. Run long soaks and both cold/HOT
|
|
cache tests. The 100k RPS / nginx 80% goal is still **OPEN**.
|