RustProxy 0.34.0 DEV — 1 KiB lock contention follow-up

Goal and baseline

Priority 1: reverse proxy, HTTP/1.1 keep-alive, uncompressed 1,040-byte responses, cache OFF. Aiming for approximately 100,000 RPS and at least 80% of nginx under the same pinned-CPU benchmark. Priority 2: local static WWW, measured separately. No feature removals and no version bump.

The user supplied a new executable (SHA-256 1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa, rust-proxy 0.34.0), apparently built from the prior DEV revision. It passed 151 actual black-box HTTP tests with 0 failures and 1 skipped optional zstd decoder. This runtime coverage belongs to the supplied executable only, not to the freshly edited source below. Neither cargo nor rustc is available in the editing environment; the new revision has NOT been compiled.

Modified source

`Cache::record` and `Cache::record_reason`

Before, every HIT, MISS and BYPASS acquired the same Store mutex used by the HOT object store, exact LRU and cache inventory. With cache disabled, a 1 KiB request still contested that mutex to increment cache_disabled.

src/cache/stats.rs now holds bounded per-(scope,domain) statistics separately:

An ArcSwap-published immutable, borrowed-lookup BTreeMap of domain handles. The normal path does not lock or allocate, even for BYPASS. New domains alone use a registration mutex and a copy-on-first-visit registry update.
Atomic counts for HIT, MISS, BYPASS, stored objects and saved body bytes. The common static reasons (cache_disabled, cache_hit, cache_miss, request_cookie) use direct atomic counters. Other reason names use another immutable published index and atomic counters; they are registered once.
Hard cardinality remains 4096 (scope, domain) pairs. When full, new unregistered domains are skipped as before. Scope purge preserves stats. Only a revision-bearing global purge resets the per-domain registry.
The dashboard still exposes the same DomainCounters, reason strings, saved byte counts, ratio and per-vhost/object views. Counter snapshots are quiescent-exact, but under concurrent requests individual fields can reflect slightly different instants, consistent with other atomic metrics.
HOT object and COLD representation/LRU access, barriers, memory accounting, cache statuses, hit policy, retry and backend security paths are unchanged.

Trade-offs: a first-visit registration copies the bounded domain index and allocates one counter set; this is not an adaptive, unbounded cache. The hot path's ArcSwap guard and lookup still cost CPU. Measure that cost in profiles; compare under high-cardinality traffic to identify any excessive registration cost before production acceptance.

Traffic analytics queue saturation

Previously Exchange::drop cloned the vhost, route, method, path, IP, user agent and country before trying to enqueue the Sample. If the bounded 2,048-element MPSC queue was full, all those clones were discarded.

Analytics::push_lazy now acquires the same bounded Tokio queue permit with try_reserve first and creates the sample only after a successful reservation. Rejections increment queue_dropped exactly as before. Neither the analytics worker, counters/top lists, reset behavior nor request log path is removed. This improves allocation costs on overflow but does not turn dropped analytics into delivered samples; monitor the existing drop counter.

Per-vhost duration histogram atomics

Previously VhostMetrics::complete incremented every cumulative latency bucket whose threshold covered the observation and the +Inf bucket. For an ordinary short response this meant 11 shared atomic RMWs for the histogram alone, on top of existing exchange/status/byte/latency metrics.

It now increments exactly one exclusive bucket chosen from the same 10 latency boundaries. Both the regular vhost and Docker-vhost Prometheus exporters compute the original cumulative le buckets, +Inf and _count from the 11 exclusive buckets at scrape time. The metric names, labels, boundaries, body-lifetime timing and measurement collection remain intact. The cumulative snapshot may be concurrent with request completions, as before. Added boundary unit tests for exact thresholds and long-duration +Inf.

This is source-only: neither the throughput benefit nor compiled Rust behavior has been verified in the editing environment. Review exporter parity and under-load histogram correctness on the newly compiled executable.

Verification and untested claims

PASS: source preflight, cache/HOT/LB/RPS/histogram/static source guards and <=500-line production Rust modules.
PASS: SQLite migration + preset constraints, upgrade/lock and source replacement tests.
PASS: compiled user-supplied binary: 151 real HTTP checks, 0 failed, 1 optional zstd skip. This binary predates the source changes above.
NOT RUN: Rust compile, Rust unit tests, native regressions or RPS after rebuilding the new source. No numerical speedup attributed to this patch.
Current diagnostic A/B on the user-supplied binary may use scripts/performance/ab_1kb_dev.py and must remain labelled old vs already supplied binary, not old vs newly edited source. The wrk 30 s x 7 pinned-CPU benchmark remains the production acceptance gate.

Repeat on a compiler-enabled benchmark host

./dev.sh doctor
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/performance/regression.py \
  --binary previous=/absolute/path/to/supplied-rust-proxy \
  --binary edited="$(realpath target/release/rust-proxy)" \
  --out /tmp/rustproxy034-counter-http
python3 scripts/performance/lb_regression.py \
  --binary previous=/absolute/path/to/supplied-rust-proxy \
  --binary edited="$(realpath target/release/rust-proxy)" \
  --out /tmp/rustproxy034-counter-lb
python3 scripts/performance/transport_regression.py \
  --binary previous=/absolute/path/to/supplied-rust-proxy \
  --binary edited="$(realpath target/release/rust-proxy)" \
  --out /tmp/rustproxy034-counter-transport
python3 scripts/performance/benchmark.py \
  --binary previous=/absolute/path/to/supplied-rust-proxy \
  --binary edited="$(realpath target/release/rust-proxy)" \
  --nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
  --out /tmp/rustproxy034-counter-benchmark \
  --modes proxy --sizes small --concurrency 16,64,128 \
  --duration 30 --warmup 5 --repeats 7

Explicitly set max_idle_per_host=64 on both before/after vhosts, with the same TLS/H1/H2/health/cache/logging/settings. Verify 1,040 bytes and SHA-256, no failures, latency percentiles, per-request CPU, RSS, file descriptors, backend connects and all log/traffic drop counters. Test concurrent global purges and per-vhost purges during traffic and verify the stats reset behavior and cache HIT/MISS/BYPASS/reason distribution. Run long soaks and both cold/HOT cache tests. The 100k RPS / nginx 80% goal is still OPEN.

Build fix after first compiler feedback (OPT2-FIX1)

The first user build of OPT2 failed with six Rust E0753 diagnostics because src/cache/stats.rs starts with //! inner module documentation but is injected via include!("stats.rs") after other items in src/cache/mod.rs. The six documentation lines are now ordinary // comments; there is no behavior change. Three unnecessary mut bindings in src/cache/dashboard.rs were also removed (warnings only). Version remains 0.34.0; no performance or feature semantics have changed in this fix.

The compiler-enabled host should rerun ./dev.sh check, CARGO_INCREMENTAL=0 ./dev.sh test and an optimized release build, then performance benchmarks. This editing environment still lacks cargo and rustc, so the fix has been statically checked, not compiled here.

7.8 KiB Raw Blame History