gru/RustProxy
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.34.0_8

Browse Source
This commit is contained in:
Mateusz Gruszczyński committed 2026-10-18 08:26:35 +02:00
1 parent b6a3e12225
commit 480c3ec769
92 files changed
+296 -10033
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+3 -3
View File
@@ -10,7 +10,7 @@ Maintain English documentation and Markdown files. Keep translations in discover
CHANGELOG.md uses version headings with no release dates. Patch increments may be
bundled; substantial optional integrations raise the minor version. 0.7.0 adds managed renewal, server deployment helpers, Basic Auth and visual locations. The reported compression threshold type mismatch is removed without narrowing. Cloudflare Tunnel and Docker discovery are deferred.
Deliver a complete archive with dev.sh, tests and an updated verification report.
Deliver a complete archive with dev.sh and the existing tests. Keep verification results outside the repository; do not add iteration reports, standalone benchmark scripts, or micro-optimization notes to CHANGELOG.md.
Do not include databases, credentials, real certificates/keys, target directories,
node_modules or fabricated Cargo.lock content. Preserve an actually resolved
lockfile when a compiler-enabled environment can produce it.
@@ -40,8 +40,8 @@ Keep production Rust source files within 500 lines. Rust unit and integration
**test files are exempt** from this limit: do not create numbered `part_XX.rs`
fragments to satisfy source-size checks. Consolidate coherent test suites in
`tests.rs` and preserve the existing test cases.
Run `./dev.sh check` and `./dev.sh test` when Rust is available. Run the SQL and upgrade
scripts even when it is not. UI tests use a simulated API; the Python protocol-peer
Run `./dev.sh check` and `./dev.sh test` when Rust is available. Run
existing SQL and upgrade checks when available; do not restore removed auxiliary scripts. UI tests use a simulated API; the Python protocol-peer
self-test is not a test of RustProxy. Record these distinctions accurately.
Do not mark a stage accepted or production-ready on the basis of static inspection.
Review real protocol behavior, restored data and resource bounds before expanding
CHANGELOG.md
+6 -6
View File
@@ -21,7 +21,7 @@
- Source-only until OPT5 compilation and Rust tests. The compiled OPT4 input
passed 151 live HTTP checks (0 failed, 1 optional Zstandard skipped).
Brief shared-host OPT4/OPT3/nginx tests are mixed at 16/64 connections;
see `docs/PROXY_1KB_OPT5_DEV_0.34.0.md`. No version increment.
No version increment.
- DEV 1 KiB follow-up (OPT4): keep ordinary unmodified public request targets
borrowed through URI rewriting, avoid creating a metering body adapter for
@@ -34,7 +34,7 @@
for complex values, TLS/H2/H3 authority handling and request limits.
- OPT3 compiled binary passed 151 live HTTP checks, 0 failed, 1 skipped
(optional Zstandard). OPT4 was subsequently compiled and tested similarly;
see `docs/PROXY_1KB_OPT4_DEV_0.34.0.md` and newer OPT5 analysis.
Results should be checked against matched configurations.
- DEV 1 KiB throughput follow-up (OPT3): keep normalized safe routing
@@ -50,7 +50,7 @@
fields and untrusted IPv6 forwarding. User-supplied OPT2-FIX1 binary passed
89 targeted live checks. A compiled OPT3 executable was subsequently tested
(151 passing HTTP cases; one skipped), but short shared-host A/B did not
establish a stable RPS gain; see `docs/PROXY_1KB_OPT3_DEV_0.34.0.md`.
establish a stable RPS gain.
- DEV 1 KiB accounting follow-up: move per-domain HIT/MISS/BYPASS, reason,
stored-entry and saved-body statistics outside the HOT cache's shared mutex.
@@ -64,7 +64,7 @@
- Extend Rust regressions for parallel BYPASS/HIT counting, reset semantics and
saturated analytics. The supplied **previous** DEV executable passes 151
actual HTTP checks; these newest source changes are uncompiled and their RPS
uplift unmeasured. See `docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`.
uplift unmeasured.
- DEV 1 KiB reverse-proxy throughput follow-up: identify HTTP/1.1 upstream
@@ -77,7 +77,7 @@
sanitization when the corresponding header is absent. Preserve analytics,
history, counters, all security checks and cookie handling when present.
- Add source guards, Rust regressions and a measured-configuration report in
`docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. This **source** is not compiled;
this source update. This **source** is not compiled;
newly introduced code performance and full protocol parity remain unverified.
No version increment and no claim of reaching 100k RPS / 80% nginx.
@@ -102,7 +102,7 @@
supervisor rotation, and keep H1/H2 protocol telemetry. No altered upstream keepalive defaults or cache query policies.
- Add source-only invariants, Rust regression cases and reproducible live
before/after/nginx benchmarking instructions. This revision has not yet been
compiled or measured for RPS; see docs/PROXY_RPS_0.34.0.md.
compiled or measured for RPS.
- Cache regression fix: interpret local max TTL as a cap on remaining freshness at receipt, not on the upstream Age clock. Preserve RFC 9111 Age and transformed response validators; prohibit private/no-store caching.
README.md
+4 -70
View File
@@ -1,53 +1,5 @@
# RustProxy 0.34.0
## DEV: OPT5 multi-hot-path tuning (source candidate; uncompiled)
OPT5 builds on the **compiled OPT4** (151/151 executed live HTTP checks passed;
1 optional Zstandard check skipped). The shared-host 1,040-byte HTTP/1.1
comparison shows OPT4 at about 39,054 RPS with 16 connections and 36,425 RPS
with 64; mixed results versus OPT3 mean there is no proven broad RPS gain.
OPT5 removes unnecessary ID hashing and unauthenticated original-URL
construction, avoids per-success health-detail allocation, batches bounded
traffic analytics, avoids routine status/User-Agent allocations, and fast-paths
unoffered response compression. All policies, observability, caches and
protocol support remain enabled; no version bump. **OPT5 has not been built
or measured.** See [`docs/PROXY_1KB_OPT5_DEV_0.34.0.md`](docs/PROXY_1KB_OPT5_DEV_0.34.0.md).
## DEV: OPT4 small-response proxy hot path (subsequently compiled)
The compiled OPT4 binary passed 151 real HTTP acceptance checks (1 optional
Zstandard case skipped). OPT4 removes avoidable allocations and string work
for ordinary uncached HTTP/1.1 requests, with regressions for empty request
bodies, URI/Host/Connection semantics and Unicode log truncation. Its brief
shared-host RPS comparison is mixed, so no stable improvement is claimed. No settings or functionality disabled;
version remains 0.34.0. Details:
[`docs/PROXY_1KB_OPT4_DEV_0.34.0.md`](docs/PROXY_1KB_OPT4_DEV_0.34.0.md).
## DEV: small-response proxy hot path, OPT3 source candidate
Seven compatible optimizations reduce path copies and work constructing
forwarded headers, log IPs, hop-by-hop headers, request IDs and cache-off
reason totals. HTTP/1.1 proxy with a 1,040-byte response remains the first
benchmark priority; local Web Server is secondary. No version or policy changes.
A subsequently supplied compiled OPT3 executable passed 151 real HTTP
assertions (one optional codec skipped). Two short same-host benchmark runs
did **not** establish a stable RPS gain over OPT2. See the historical
[`OPT3 source report`](docs/PROXY_1KB_OPT3_DEV_0.34.0.md)
and the newer [`OPT4 analysis`](docs/PROXY_1KB_OPT4_DEV_0.34.0.md).
## DEV: cache counter contention and saturated analytics (source candidate)
Per-domain cache outcome/reason accounting now uses bounded immutable indexes
and atomic counters rather than the HOT cache's global store mutex on every
request. The analytics producer reserves a queue slot before cloning fields;
full-queue/drop behavior and its counter remain unchanged. Per-vhost
latency histograms now update one exclusive atomic bin per completion and
reconstruct unchanged cumulative Prometheus buckets during scraping. The supplied
previous DEV binary passed 151 live HTTP assertions, but this newest code
**has not been built**. No version bump, no measured RPS claim. See
[`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`](docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md).
## 0.34.0: HOT HIT critical-section follow-up (source candidate)
Immutable response headers are shared until a HIT; the mutable per-response
@@ -57,12 +9,6 @@ shorter paths. The existing stale-for-revalidation policy makes the previous
full HOT sweep unnecessary. Exact LRU, generation barriers, BAN, COLD,
transformation/ETag, TTL and cache bypass settings remain unchanged.
This is **uncompiled source**; do not infer network RPS or production acceptance
from source checks. See
[`docs/CACHE_HOTPATH_0.34.0.md`](docs/CACHE_HOTPATH_0.34.0.md).
## 0.34.0: reverse-proxy hot-path optimization revision (source candidate)
This revision retains the **0.34.0** version and all prior scoped-cache,
`sendfile`, security, logging and vhost/administration controls. In the
uncached reverse-proxy hot path, request observation now passes the immutable
@@ -87,9 +33,7 @@ subscribers only while subscribers exist; history and external sinks remain.
These changes have **not** been compiled or timed in this environment, and
there is no claimed 0.34.0 RPS improvement. Build and run the regression and
nginx benchmark matrix before deployment. See
[`docs/PROXY_HOTPATH_0.34.0.md`](docs/PROXY_HOTPATH_0.34.0.md) for invariants,
checks and exact measurement commands.
nginx benchmark matrix before deployment.
## 0.34.0: scoped cache, proxy accounting and opt-in Linux sendfile
@@ -116,12 +60,6 @@ existing streaming path. The default is `off`; toggling this transport option
does not purge HOT/COLD cache. **Native compilation and live protocol tests
remain pending; do not deploy this candidate without them.**
An offline cache-retention simulation for a 20-vhost workload shows 92.123%
fewer invalidated objects and 92.113% fewer cache misses than the old global
purge policy **under its stated synthetic assumptions**. These are not RPS
figures; real 0.34.0 performance remains unmeasured.
See `docs/TEST_REPORT_0.34.0.md` for verification scope and remaining gates.
## 0.33.8: request-path performance and compatibility verification
@@ -150,12 +88,9 @@ to exercise the existing documented behavior, including scheme-isolated cache
keys, purge generations, private administration responses and bundled error pages.
The LB panel, database schema, default TLS/caching policies and bounded pool
recycling retain their existing behavior. Live process regression, verified TLS,
H2/H3/QUIC and WebSocket runners, repeated nginx comparisons, resource sampling,
syscall profiling and a mixed-traffic soak are included in
[`scripts/performance/`](scripts/performance/README.md). See the
[verification contract](docs/TEST_REPORT.md) and the separately generated
[performance results](docs/PERFORMANCE_0.33.8.md) for measured outcomes and limits.
recycling retain their existing behavior. Historical verification methods and
results are documented in [docs/TEST_REPORT.md](docs/TEST_REPORT.md) and
[docs/PERFORMANCE_0.33.8.md](docs/PERFORMANCE_0.33.8.md).
## 0.33.7: named dashboard presets, panel components and chart typography
@@ -780,7 +715,6 @@ Historical release information remains in [CHANGELOG.md](CHANGELOG.md) and [docs
- [docs/FILE_LOGS.md](docs/FILE_LOGS.md) - stored-log explorer, protocol-error records, bounds and filesystem security.
- [docs/PHP_FPM.md](docs/PHP_FPM.md) - implemented FastCGI/PHP-FPM integration, safety model and runtime requirements.
- [docs/TEST_REPORT.md](docs/TEST_REPORT.md) - verification results and environment limitations.
- [docs/TODO.md](docs/TODO.md) - unresolved follow-up only.
## Release history
ROADMAP.md
+1 -21
View File
@@ -12,15 +12,12 @@
retain 0.34.0, TLS/H2/H3, WebSocket, cache, ACL, logs and bounded queues.
- [ ] Compile and test OPT5; run full real-protocol regression and >=7
30-second CPU-isolated A/B/nginx runs before accepting a throughput gain.
- [x] OPT4 source candidate: borrow original no-rewrite public target,
bypass the request body adapter only for bodies reporting end-of-stream,
avoid redundant upstream URI refcount operations, and simplify common
forwarding/header/log branches without weakening validation.
- [x] Add OPT4 regression tests for body accounting and permits, Unicode,
URI identity, and safe Connection header filtering.
- [ ] Complete OPT4/OPT5 Rust unit suites and extended TLS/H2/H3/WS/LB
acceptance; the short OPT4 RPS comparison did not prove a stable gain.
- [x] DEV 1 KiB: remove shared HOT store mutex from cache outcome/reason
@@ -30,8 +27,6 @@
- [x] Move per-vhost histogram aggregation to scrape time; retain the same
cumulative Prometheus and Docker-vhost bucket semantics while one exclusive
atomic bin is updated per completed exchange.
- [ ] Build/test revised counters source; repeat 1 KiB source A/B and pinned
nginx benchmark (100k RPS / 80% target still open).
- [x] Share compiled route handles and reuse middleware routing in forwarding;
avoid exact-host wildcard allocation and full-route cloning.
@@ -39,10 +34,6 @@
between leases; reduce health locks and fallback/retry header snapshots.
- [x] Convert three exchange metadata locks to atomic tags; avoid unused live
log clones and add Rust regression fixtures for modified semantics.
- [ ] Compile this hot-path revision, run Rust tests and verify live LB health,
sticky/session behavior and HTTP/2 graceful shutdown retries.
- [ ] Profile CPU flamegraphs and measure real gains vs 0.33.8/nginx; tune
idle pool/recycle only as a separately labelled experiment.
- [x] Disconnect keepalive/LB/diagnostic-only vhost saves from cache revisioning.
- [x] Invalidate only a vhost scope when its origin/representation policy changes.
@@ -50,16 +41,11 @@
interrupted content update or restart.
- [x] Reduce response-body per-frame backend mutex operations without losing metrics.
- [x] Add Rust regression cases and preserve unchanged LB controls/API.
- [ ] Compile, run the full Rust suites and repeat live HTTP/TLS/H2/H3/WS/LB
regressions with a compatible Rust toolchain.
- [ ] Run paired 0.33.8 vs 0.34.0 vs nginx benchmarks; record real RPS/p95/p99,
CPU, RSS, cache retention after saves, and rollback on failed acceptance.
- [x] Implement opt-in Linux HTTP/1.1 `sendfile(2)` and a per-Web-Service
Delivery toggle, with TLS/H2/H3/range/cache/compression fallback and bounded
per-connection slots. Implementation is source-only until native tests run.
- [ ] Compile and validate native `sendfile`/fallback functionality and safety
on the target Linux OS; compare live WWW RPS against nginx.
- [ ] Complete long-running soak, external dependency integration and staged rollout.
## 0.33.8 - Performance with retained functionality
@@ -76,10 +62,6 @@
- [x] Add repeatable live HTTP, verified TLS/H2/H3/WebSocket, nginx comparison,
resource sampling, syscall profiling and mixed-traffic soak runners. Measured
outcomes and limits are recorded in `docs/PERFORMANCE_0.33.8.md` and raw evidence.
- [ ] Complete deployment-specific 24-hour soak and remote-network testing with
actual PHP-FPM, Tinyauth and DNS infrastructure before production rollout.
- [ ] Evaluate further proxy pool/queue costs and a secure static-file sendfile
path from measured profiles; preserve TLS/H2/H3 and all observability semantics.
## 0.33.7 - named dashboard presets, focused HTML extraction and chart typography
@@ -107,8 +89,6 @@
sentence case, all other panels uppercase); use mono for full legends,
captions and chart data across all chart families.
- [x] Validate SQL migration and run available simulated-browser/JS tests.
- [ ] Build/test Rust and run the new endpoint/UI against a real server with
representative persisted data. The production acceptance gate remains open.
## 0.33.6 - Stable dashboard chart ranges and plot heights
@@ -745,4 +725,4 @@ Future Web Server work must retain the 0.25 split between reusable **Web Service
- Race-resistant filesystem confinement for non-Unix targets before enabling local file serving there.
- Real PHP-FPM release-host acceptance over Unix socket and loopback TCP.
See [`docs/TODO.md`](docs/TODO.md) for unresolved work and release acceptance, and [`docs/ROADMAP_HISTORY.md`](docs/ROADMAP_HISTORY.md) for completed historical milestones.
See [`docs/ROADMAP_HISTORY.md`](docs/ROADMAP_HISTORY.md) for completed historical milestones.
SHA256SUMS
+20 -91
View File
@@ -1,13 +1,13 @@
61bf40a0572889dc8b0dda9eae4e93e44b0de9364b3c798106c85700ffb37c23 .dockerignore
4140a2956264b44f259e6a37fc00ba02097bbe68b09296e04d3d243c1287129b .env.example
9e6356475bf587091bab72246ec9e325b498f7e48a6220e9a18baa50d0a385a6 .gitignore
46bab1d2c2d6e0a80d57a6b232b1ba4c220884b95cd45ddccbafce2aa160cddb AGENTS.md
3e46648fd8c91548ea77dcac4f6cbdbc5741c64b33240c44a50c138597880e99 CHANGELOG.md
5135a996f12151e481942bf3b4c46c42b675c1cf7086bd44733f3be260177d8c AGENTS.md
e7bea33973a805fdd1fd3af967555004b3509ef9b81576819c707c3dd5c47497 CHANGELOG.md
ca201e3cd3f9842eae0006d0b40a0a72bcd3d4da6e4e5ecaf200d1b9a1390db2 Cargo.lock
fec5f1bae9a59ef1fabd27c5e45d31448dd8f811736eebfa8826b3806a7e3349 Cargo.toml
25c7f3c8162cbf5cc156439774118a81dae9fdc52b611ec7e40c230fc99a772c Dockerfile
d48bcdcda6d9261d7b358c39b8bdf93d39b722b3751af3b5ebe192e16bd8b4e6 README.md
7bba42713fa944908f5f3611841bf6b36377240ae4b77ec692ff48ea662b44f9 ROADMAP.md
f0d89d95166d47b6ac2033cc8f5eaf2246c0c70139437394c98e10a95ab3d0dc README.md
b1d6174f509c743622cb27c92e047b724ac9d44bbf1c0987178da8a3bbd74b01 ROADMAP.md
da8311e3db945290666324a2f24bf4521b365272d7ac3f862ded1be993eea646 UPDATE_CHECK.toml
f645f218dd062bf6cb54a8ee634290562cad364a4cccf09f16e3c8303582af8c build.rs
ea8d091843f86f4c66a5c76b20268b2feb397070851b6e05c5bc7d7a8d9157eb deploy/README.md
@@ -27,12 +27,12 @@ e52b31d0abf65ae3034f6cb367cca113633014ce8d1d799ad4dd5624d2c7b8bb docker-agent/n
5220d669171ec3ad65f50e0bc8de5178dadaa4904def8e1a4f61002aa9e9a55b docker-agent/server.go
875faa3969df9409d79f050a84b4183496f557edda3d34aaec19fe556cf8c471 docs/ACME_PROVIDERS.md
ec15163f5f51c9212003b7edd1fc380321311c2496fbaa9b45c838281b9af475 docs/ACME_RFC2136.md
00aef5eea37a8f56d7ea95e63ddcfdf39bac041168af29582b5ab2ab62d4c81e docs/API.md
70588369169c06e71925d6f45f98258153626cccbd168a77eb882a4a7ded7702 docs/API.md
f27053f8a93d7883025a04922d015e0b697cad653e714be632b757e167297de1 docs/API_ACCESS.md
d30551a52ffd34c5963094d2826ba39795210a4e6a4e4d7333066dae7456fc50 docs/ARCHITECTURE.md
fb99fac0a490af262a0ecdad593ffe5164004c2166fe7ddd607476bcc56a1623 docs/CACHE.md
7f61bc6e5c82b6519b50525eb2516298a5e28dbb28537d31eaf2f56bb7d5ff59 docs/CACHE.md
9113f3abbfe81bafc4855d8680df8aa2a381ecc358cc3e1a0f78b6b756421117 docs/CACHE_DIAGNOSTICS.md
a8fd061f8bdd920158fa42972f9d26418cbfe9d2cd8757ba1d9d1d12e5c908ee docs/CACHE_HOTPATH_0.34.0.md
de75fae9e387b2e8b3d478677963f8d67a124f3f02b746de285ca8b697b3816b docs/CACHE_HOTPATH_0.34.0.md
3822d72e12f82c37a9e70ab40c7e89e89145ad37f29914e4702db4bf024a5501 docs/CHANGELOG_TECHNICAL_ARCHIVE.md
c8e69e492179ce39958cc63615cf46c4ac08cfdb7f3483aaab66868a3abcd687 docs/CONTAINER.md
833ddb12f23dcac796223521388b19c6ec3ea04c5550cf1c5f4ea4eb7c58f3a0 docs/CUSTOM_LOCATIONS.md
@@ -56,33 +56,22 @@ be53a58afbeddbd7deb4b5f6447f2dc74db0327839bd83407c20915df98dcfa0 docs/NOT_FOUND
e68d5a5cf0991de9308f78afb0264e02979f7463a5f395eb7a6e9ec2bd3a3d54 docs/PHP_FPM.md
bc068a176056f5ea4d3b637071e644ed58a2140494c233ef22aa66e9700e3299 docs/PHP_FPM_ISOLATION.md
ad6e3f003b820a4f45e30e39577d120a69d167c70910bda4891971759458c3dd docs/PROMETHEUS.md
401ab1bb2c7f3961742195d81f5daf5c9cf81609a4b175bea1d10d2a4b727082 docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md
f13ee0005d57d2576ccc970745f17069096df4afa23d712ea0f748da85433f7d docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md
c9bbdd1cd8df02b2a19016b7bbeee1cf55917614ac672f1f94e0a05db7d309ef docs/PROXY_1KB_OPT3_DEV_0.34.0.md
27002c4d4e19bb7d58521b8ba4cd5faf180d2917b6ac200dd7fc3664c2060145 docs/PROXY_1KB_OPT4_DEV_0.34.0.md
8c26aa9ebebf54d912749e3336e548b59d805997366be2324157c3e56b887f5f docs/PROXY_1KB_OPT5_DEV_0.34.0.md
deea89aed410c0619bdbdb12437edc7aff9dca2fec7fdc059e918c649fc36b51 docs/PROXY_HOTPATH_0.34.0.md
a7d6189b0e8cb854918456e0063101ee77bfb2419a9ba0430eb0e3f8678c0f09 docs/PROXY_RPS_0.34.0.md
887d1fdcc3b546b024afcd18b1716b8e3a74a3800012bf4bda7a122d2eabe233 docs/README.md
df6dc1070f94eeee7d4273251b93ab03b34e6cf0b02789d5b8f538a37814b382 docs/README.md
6e46a137b9c88dac8cb9e3fdca877430092392de9913d746278dbff8652460c2 docs/REDIRECTS.md
0dfd0d431c5aedc67abac1ce5de040ffa790ec9578fd7d6f91a987c333d73358 docs/REFERENCES.md
540fa46716d562bfc846d2f7bac0515380831ddbdcaa01ff4ac86be4ff3d3aa0 docs/RELEASE_0.34.0_CACHE_FIX_PL.md
d85c5567fc6a7714b96ad7cfe4ee9ff2085f84c4f31534a6484340d03a73e0f7 docs/RENEWAL.md
cd4ed3af26eb51c9ff6ef3dda9d52ff5120bcfc78c842bce34a639cf85b6aea9 docs/ROADMAP_HISTORY.md
82bbcd1c3a3bb6a44065f96f160f8633b2f2450ab4723412570cc885a6de459b docs/ROADMAP_HISTORY.md
759bf2987f2b602488829d1856fa33d57c2a3b23bc681ef5a931b9633af3a533 docs/SECRET_STORE_BACKUP.md
ec7e32fd19e6ba6ed1732f1f3dd92ac0860609e8c0bbd3c1998840cdfb24bf54 docs/SECURITY.md
620b69e566d27af04f0604f4ca3f88cf95cdfea7fbc6ae9dd92446d2b2b5d9be docs/SECURITY_AUDIT.md
007537ee971463072b2482dfa4fdcf1955f837ced0b711d4eef3b6267d16d4f6 docs/SECURITY_AUDIT_0.31.0.md
4dcf3243a13d9ae1827b5cefdbb9e27bd6310321c50d4d5934527d44437df6ef docs/SETTINGS.md
3434a638ef0198d659ad77e0a70d0540cf4f203e10e147713e6c2530ba84324a docs/SETTINGS.md
8fc112cb45a2f35ef5b0dd4371aa6f43aca4df85db62c6ae02e56d74a3b702ae docs/STREAMS.md
95148bfa1d2f8ff52b591c9acefc6c8992dadc4ebda43a195101d9b31d67a576 docs/TEST_REPORT.md
47e3cf646b2884db18684385ca7ff18d929b6de6a5206ed5e7ad27a71d350052 docs/TEST_REPORT_0.34.0.md
cfb452d8496839e832b5fbacdd4e52525b0ecc3112fbb62340c1e580bd5c1717 docs/TODO.md
751c9b13b47d0959a8575d6e82bdecba988b950a374ba0b084fbba08b4b65ebe docs/TRANSFORM.md
e6d270894cafe96f8aa9cf58c9609223d03c9bcf6f872adfcbfb767a7b313f0a docs/TRANSPORT.md
811390a8c4ad6439dda4d532e35edc389db70f185ec728604ab35e22033f7e07 docs/VHOSTS.md
915da874d0b24e39b359fb70cc557730f7053a8334ced9ed406b6fa933c0dfaf docs/VHOSTS.md
09b60fb7b40db4d0139fa2a9e69a2aa9f28d4e61b8643521cfc7e936d7df8774 docs/VHOST_ACCESS.md
ca5d1d1d7e8aa372dcd3e2bd56bb7a9b8977953482ece560cfef58a19314242e docs/WEB_SERVER.md
cdf208dc6438d11c8119b6de81cad7a838eaeeb9cbbb5417e691253760d69ddf docs/WEB_SERVER.md
c336f998027330c58d5afbbbb3a3a682fd44ae3997643882fe342633822f8bb3 docs/screenshots/README.md
5502e8ce44276213803b66c58e3a4d804a1354f5ed5d83f47ee805e8c44ab088 examples/acl-headers.json
2431dd88a2d9ad2cf3cd4b8fcdfa602563ab1013c612d8917fddbe7febfb2dc3 examples/cache-dashboard.json
@@ -136,73 +125,13 @@ a641bd35400c56f3b68b0a92e97b602d06090f03972ca8f01fa744e90dd97403 scripts/deploy
2cc658faaa610b21b48942c59b955525539ad4503fd60f2a49c7e626a22faaff scripts/migration/migrate_npm.py
a459d95bbb0076f33ade8b273b813308167f9d8d45242b309a2d18320ca0093c scripts/migration/migrate_npm_api.py
31ccad3ec3c880119f3b7002f0c6652f8b43634edad8ab3763cb0e4497d0bb1f scripts/migration/migrate_npm_core.py
bf934d59052988a528f3b7e7e3957b5a1ff1cf17db0d543e165cf76600e2137a scripts/performance/README.md
939287dadf7309689d29d2171757c2d80e91d898c3f9112454b943d08bd611fb scripts/performance/ab_1kb_dev.py
0380406a059ee9ec27473f8a46435f638f73a6dc729382ae1561c08dbf7ccb53 scripts/performance/benchmark.py
9ac220163256e08513597c397c016a48345b114d7876b439259cc310b36c83fc scripts/performance/checks_cache.py
f09125d081d0eed9a44fdca75de14e6bbf09dcb00fe1f027ba2270f89e3ff192 scripts/performance/checks_policy.py
96e01db050987af431a4662fce34752842bb2575237fd3239bcfc10ae55def6b scripts/performance/checks_proxy.py
c1d8c184811ae8fb1960d1ada08d8a4a95405c3b83dd18e1a27d711b596140be scripts/performance/checks_static.py
1044eb8d225e2fa117d5435697abdf0a014aa2f9063d4b41566e74ef47d50a34 scripts/performance/common.py
0f1fa62ee3429509dc21fe1cbfa7159e38d260ada98bfc6376e8078c9840ef48 scripts/performance/concurrency_regression.py
044da66a00a04f558a6b6e6157db0a7656dd5ee22adf0b0a47482c7f951380e9 scripts/performance/lb_regression.py
1b601d68d15909d786ac63890731f0592bd9057a68993a1d90bc70d6618e6f82 scripts/performance/peer.py
830a360609e2b8ebbb1ef6f3d89e3e6151e8fefb36c3673d20df68ff8932eabb scripts/performance/profile_syscalls.py
ceb26650fc8b63d0107eb9100ccd572e5c70f7a0ce5cc533a94083e20485095d scripts/performance/regression.py
d6480962a23f55d19a54aba27648ca1b28b50f03afe2decbfa272efd142df756 scripts/performance/resources.py
2dcd4c7e39dd58fd77c3c713d9bdee5025cace7191ca5f12bd6e7fe28f333358 scripts/performance/simulate_cache_034.py
4a5af2d3870d9517b4ad9c008789490cfb07affe98ffd629e51f7674ff54a642 scripts/performance/simulate_nginx_gap_034.py
f85dd0c29d5a96ad59a71e035ac4303c7b6a70a8d61280a15cdb966b32a218c7 scripts/performance/soak.py
fe6ff5bfd76ac7a2752a8a2e1e45c63e549c4a61c753008671e478df3ebff304 scripts/performance/telemetry.py
7c1ba46c623e1ade47b872fcdf4526a48ca85c56515dffeac626188b0357d23c scripts/performance/transport_clients.py
bd19431c89d7dcfe6743dffff9c4b16260a976d1936bdc4c11d4fcf6b3a7dda6 scripts/performance/transport_regression.py
1d747278da3fca093ed91c96d4a9480d0f711d0773d673906dbb56081635603c scripts/performance/wrk_report.lua
b5b3135449f6be4e97088b76ef7cf2428e2de3c7944c9cbbcb1edd2016c0fd4a scripts/preflight.py
e380bf369dcfd16e0e82d243d072bf8739e005d66b049c9b849a0d8b2d34162c scripts/preflight.py
3d6a0503afc25beab79ddc86864bcbb15248ded7368d33aa64321b8cdff59933 scripts/resolve_dependencies_0312.sh
20124173452b4514bad8850c1637569a9a191882478451e549bd16b432605fc1 scripts/runtime/acme_provider.py
e69813fb1d3a4cdcfb7243964cc323fc21a603087bf2e84188fa42b9d3cee4b5 scripts/runtime/acme_rfc2136.py
174c8059625e3315a3d9fefe74c17f78ed850ff25ee15bfdabc148107dd344eb scripts/start_stream_backends.sh
046b4282550f5cb4e53906665c91c7442be70fc68fd0743466fd832582f7c979 scripts/test_stream_tcp.py
f0b48ec0cc4411755cc0b4128eebce710375afa99344cb76198cf31074f8eb63 scripts/test_stream_udp.py
fafd577ccbb58485c47de35fc125e5c64a3a7628ae40d01d4cd924fce2f414db scripts/tests/cache_age_policy_034_model.py
b75a381d1b41bfb2ea7d2ce09728249b5e1465f76d57e2c3f151af72a5ae4d03 scripts/tests/cache_age_transform_034_live.py
dbab8d5457546a1ffe8233514ee94225a73974fe5d6f3b1f5fadd06885fa228c scripts/tests/cache_hot_034_static_test.py
6cb79535f742d1a41685f486688dc2f1330ea362e3da1a15d4fa32561094282f scripts/tests/cache_scope_034_static_test.py
efb7a1fde2c74f30cd4a1abb88ae07c1cac7378bb33685dd09d52bb30849daac scripts/tests/dashboard_chart_headers_0335_browser_test.py
eb2c9e5c968938b6fbe4675b547ba1f537519878b9095e1fea133f4d83f96fc8 scripts/tests/dashboard_chart_typography_0337_browser_test.py
6a908fd434d2a8a80adfd06a66d4b83c4174e448d453a8c10b1812475d036677 scripts/tests/dashboard_chart_width_0333_browser_test.py
252b401c29e2db50e105d7fb044bcae14752748261a67da1a4025d71bfa2608c scripts/tests/dashboard_cpu_autoscale_0335_browser_test.py
41a2b3765006c138324ae1312f5c1edfed6ce48051230f85779e8d2aad34d2a3 scripts/tests/dashboard_cpu_fixed_scale_0335_browser_test.py
de583adde771e0d6f64ea0d5aed45b1c942a7c78b7193598eeeb77fff00c2803 scripts/tests/dashboard_layout_0331_browser_test.py
1c813c78dfe96ba4d2c9470c93dd24e81be06e6485ebdc39cbfa30f8d32aa0fe scripts/tests/dashboard_metric_visuals_ui_test.mjs
f14626c5ec006477062b496bdc0025caa3366c7037ee598e15cd55e29041d58f scripts/tests/dashboard_modal_0333_browser_test.py
f934f5ce66105a1394b93761a6081a51133837ac038f5faf67a236f676b15c1b scripts/tests/dashboard_overview_labels_0337_browser_test.py
df87420f16c418aa17a8ba8808e506095681446d32a7d76ae0244222ed37928a scripts/tests/dashboard_presets_sql_test.py
df1457797241aab114a5671ce3f0229815dd29d06ab6f6254c0a1048576af6e1 scripts/tests/dashboard_resizing_0333_browser_test.py
149910e6e8d569443a724fd21701e2fd8f6b5e360991d4f710f5d21364402bce scripts/tests/dashboard_resume_rate_charts_ui_test.mjs
3868dc7b241f1cf15bc297104e29449ec3e35f75cb66078439bc4b5c4ee11040 scripts/tests/dashboard_stable_scales_0336_browser_test.py
fa42ec0475e4398d647b3c094309b4332cd2b5649db223cae0b8e821d6b44a28 scripts/tests/dashboard_title_color_0337_browser_test.py
3c1b2bda6410d3348b0f5a05c70494cca9636339bed8217fc4abc0a30981faae scripts/tests/overview_samples_0330_ui_test.mjs
72185f59f31bf04a4e88d53fe2fca52509f3663cb5f40b1d3bc319a5f945ef23 scripts/tests/panel_html.mjs
b0136544af717b75df2cb107900ea65c3f945fdd0040369dbb93919008fb954c scripts/tests/panel_html.py
8d1903e2b17b3d59cd0a5bec3e298e468b3d88ed2dda8b724239cc45d8353c59 scripts/tests/progressbar_0332_browser_test.py
7fc86a34a47c6e1f0f9fe2748d0ea7f59e455f8bf4e5ad081e185356e43721b0 scripts/tests/proxy_1kb_opt3_static_test.py
ec05272ad8c14c7a0265a84cc7b6ebc0dca0651413106d655681681873b88fc8 scripts/tests/proxy_1kb_opt4_static_test.py
37306b175190aa72b143d263060fa6441244eb40b2bcb54c82071cc87d73b54d scripts/tests/proxy_1kb_opt5_static_test.py
2a6e75c05f3860acf65ea8132f0083904478b690145115ec017c120d70f26cca scripts/tests/proxy_hotpath_034_static_test.py
0c50def966cbc95c22339d1c506763fab4d9e5c176a161205ab88d79af5be920 scripts/tests/rps_hotpath_034_static_test.py
5be5cdc60324f4f0f6638640000b888a5babb0a9bbf29511c824a0ba38878c2a scripts/tests/sendfile_034_panel_browser_test.py
978789bfe89d43034dfebddba49b9b17daedb1be031732734f95375e56c78306 scripts/tests/sendfile_034_static_test.py
1bb99b148d0297235b707f16865ca077329246fc19ecbb20a143bdeb56a23f37 scripts/tests/sidebar_nav_0333_browser_test.py
12f2b18b0ed2171c49eb8cde2c2b88fc627054c2f574c04bd9847821b00cd4c1 scripts/tests/system_ram_meter_0332_browser_test.py
046b4282550f5cb4e53906665c91c7442be70fc68fd0743466fd832582f7c979 scripts/tests/test_stream_tcp.py
f0b48ec0cc4411755cc0b4128eebce710375afa99344cb76198cf31074f8eb63 scripts/tests/test_stream_udp.py
6fa1667a844b7bc2adaa78738339b58ac6c94a6d6e7be34573433dcd323ae809 scripts/tests/upgrade_performance_test.py
f0a2e3daa0a49aac54723cb5a281360e49a97c076281372c5cca2a85e096c02c scripts/tests/vhost_backend_popup_0337_browser_test.py
23cb1f790cd5cee1b76ecd85f6a2a44ef2e16b0840bf06beaa2ae13048063b2f scripts/tests/web_observability_0330_browser_test.py
3babdd0fd6e2998755ae88615112f6ea056f5c808245abb6de60fbda555594d1 scripts/tests/web_observability_0330_source_test.py
f55efa99245bbdbc98fe3202e52952d224ac3200612561580d52f6e251810049 scripts/tests/web_server_0330_ui_test.py
0dd1cef06a1c3e1546e2fb9b085bf369dfbed02107b9ab01b4fb4f3b894099c0 scripts/tests/web_server_0334_browser_test.py
22e880e98c7b9d700c7a4c02b9b1d543e5dbd1f4ecdca42813a1083165b32a19 scripts/version.py
9bf4e4a61824f82b773b4e8ffa0909b9cf0a42bfc14d695bd64da240cc634a61 src/acl.rs
63efab1e95d41134fc30cc57320c32810bbae9f81a640bdb24b1be2059e63d8a src/acl/dynamic.rs
@@ -222,7 +151,7 @@ b4a2a068acfa1143b0032b9cd072c19cf3fdda2580f0b932d11d6368cd2ab3ad src/admin/cert
51f46a2cb3167c4e3a784dfbb59b8fe0e9d5d3d51365b26699498e31d0afb6a9 src/admin/certificates.rs
b5f6d49f1add583cfc8cd4aa36ffe4db56dac4de3b151fe8239fe42f507019cc src/admin/dashboard_presets.rs
5e150d4ef0413a2f92db5b6631669933b84d4b4796633d01476c4b39188cd05c src/admin/documentation.rs
958546c43c05bf4bed8121b24db3b3096f9243f3c56a71e1762592b755684013 src/admin/documentation/documents.rs
3bc1425c0dd142471a48391474ab2551de7d6f2b788280ed4788ab94a435c712 src/admin/documentation/documents.rs
67b901485d5858d5b5bf78a6260035db32241531d99481754c3533fdf404facb src/admin/documentation/tests.rs
1ce779b84aa8f47bdb3c34f83593c1f2f06d2235f0319847096cfcd035434168 src/admin/file_log_search.rs
ab18acf6fb4ac86fb22635b9219a12dac76d24952c8f15538486267a9b4b7855 src/admin/file_logs.rs
@@ -246,7 +175,7 @@ a40adfbaa908da3ea726d7aa3d45394a70a6a55b6fc0063270ec4d2a2f2026a3 src/admin/sess
4b8a0483ede651d08489a4d6e5bb616f06423094bbbefcee70b8755d65c04398 src/admin/status_live.rs
c97881b17b7a42511868b0e48e2f25aab6fb5b613d538f5e1f1967508454e486 src/admin/status_system.rs
cadd7dca29b9aad8c77c7bc23dd249ea37a15adfa4d636a8ef00273a91ce35dc src/admin/tail.rs
ccea3b72782aa9d36e41b89f40a755289d285692155c27cc81a0968f8780fe2a src/admin/tests.rs
58c9dbabfcfe37ce41d18e38777ead17781da91b3fc97bff19073fa09b262bd2 src/admin/tests.rs
48fcd818c522fe80f2c7b2e218f87de6a7772ffd0a24e93502780b03ec78e335 src/admin/web/acme.js
c0d08c5b8fc13dec9bf53fd842d224b9df3874fc3bdf31413e3ab182a8cec2aa src/admin/web/app/backend-dashboard.js
a085ee336e6669229f3ad2f74536f19bd088412ca79bcaff30dff293f1e1baf9 src/admin/web/app/chart-scales.js
@@ -416,11 +345,11 @@ f2a8c2a98f4ccfb708f9d75a48ec7b351232be9b8a4364b7e63ba14d28eead7c src/http3/sett
a31db7eac12b578b1eb5a74fae98bbe36312da95b7d7173135dbc41fc454da21 src/http3/tests.rs
68fb5ab6a889995a66ad12cb6acf88c5d04f0aca572e2ae15f1a9cd171a5f24b src/lib.rs
2551189a3e5701371026c9abd2c5d44d37495f28fb67e220180421bfd315e3dd src/logging.rs
18c8d74c21a47f234be93551d7264b1f8f8c00d1bbfbe555d91618c58cea0740 src/logging/error_runtime.rs
bc3873c6f22d1ee7fa76ae586547c9867e6eeaec8266e3ed1f529c87108df2b6 src/logging/format.rs
21ddb01b8f9c7fab60840d18103ef8de4716ff0c890cc1525b472b69af7916ec src/logging/error_runtime.rs
42b88cefef603fafa9eec3ba83a53069f01f571e98cf923210a6fe605e008e90 src/logging/format.rs
f5815d778cc1785723faed4abb15f9880f22b6524b5289fe1e7158905aac0986 src/logging/model.rs
42fef80af50735cd566c6801dad86ba81be98cf4a8ec7d45c2ea26f53f556a22 src/logging/runtime.rs
7ee1eb1d3b52b4c14303f8ca14a1924d4d6bbe75b2e43ce3ca47d7958f792c7d src/logging/tests.rs
b90ee5c7badacd0141fba99a8109fced785a11abe11675c8687fa35e68f5e7b1 src/logging/runtime.rs
c83a7b84a9d7226c200394c351c4b3d9c417869828c01a35d4b1297670ab50f2 src/logging/tests.rs
3836ed14b6710e183f048ee4d7807a35366c27a27338057fe4a2811129b63302 src/main.rs
fc8e99eac83c083d65101f6402aacbcc8392d66508c31566a50de3ebcb12e7da src/main/admin_runtime.rs
42a9197475823f67c1ca762951ede5d7a2f10124bd6ed4d1a3365f4297be894c src/main/bootstrap.rs
@@ -466,7 +395,7 @@ cc7fe83f42acb19d79abe83107427cdc6b0ed14083702f4fe66af22f7cdfc4cc src/proxy/head
55133ceace2e5262d9c54fa41bea099a27b817d1e09c1cd0e2493fd9cee4da38 src/proxy/redirect_helpers.rs
2280841494c3e14ccc5e3f02b026fe3c4328a3dab04ab449b9286bb69e39c028 src/proxy/request_id.rs
b9a2307c6aa7f16613d935508a61574dfee964020b1b5be86293df2d684615d8 src/proxy/request_id/tests.rs
e8d1210a6b8e8c6a388d016d22e7c18255cf737c5583124b35cd50bd2592f23c src/proxy/tests/authority_tests.rs
3db4188e2cc6d69fdff1b817cc15d0e6b18bda18d4e4729d37c05736c7d8cad6 src/proxy/tests/authority_tests.rs
1aa9bbbfb2659facdaeaf185e11221b1c8cd5fe6e23c7c1dacd3aa55e3db4fcb src/proxy/tests/cache_fast_path_tests.rs
0f2eacadf1447db8aa1c1a78005aa262e0f747096bfe7e0a0e41a2cd31a0d17f src/proxy/tests/h3_navigation_tests.rs
d134b0eb617f82f9971b3f3461379ea37c57bd0079bf54748d0c434b2ef1b475 src/proxy/tests/http_invalidation_tests.rs
docs/API.md
+1 -1
View File
@@ -21,7 +21,7 @@ additional requests receive 429. Ambiguous duplicate control headers and
request bodies are rejected. See
[CACHE](CACHE.md#http-ban--purge-from-authorized-client-ips) for request examples,
scoping, confirmation and reverse-proxy requirements, and
[0.31.0 security review](SECURITY_AUDIT_0.31.0.md) for audit findings. The authenticated
[security review](SECURITY_AUDIT.md) for audit findings. The authenticated
`/api/cache/invalidate` endpoint remains unchanged.
## Browser documentation and anonymous surface
docs/CACHE.md
+1 -1
View File
@@ -232,7 +232,7 @@ separate from the administrator-only selective invalidation API. If an
application changes a GET resource after POST/PUT/PATCH/DELETE, use origin
freshness/validators or one of the explicit invalidation paths; RustProxy
does not guess the affected URLs. For exposure review and test scenarios, see
[0.31.0 security audit](SECURITY_AUDIT_0.31.0.md).
[security audit](SECURITY_AUDIT.md).
## Eligibility and freshness
docs/CACHE_HOTPATH_0.34.0.md
+3 -6
View File
@@ -54,15 +54,12 @@ Native Rust test file: `src/cache/tests/hot_hit_034_tests.rs`.
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/tests/cache_hot_034_static_test.py
cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
```
For real HTTP RPS use the supplied process regression and same-host nginx
benchmark (`scripts/performance/README.md`) with **identical** configs. The
standard nginx comparator benchmark explicitly disables the cache; it cannot
measure this HOT-specific improvement. Compare two compiled RustProxy 0.34.0
binaries with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
For real HTTP RPS, compare two compiled RustProxy 0.34.0 binaries
with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
A cache-disabled nginx comparison cannot measure this HOT-specific improvement.
Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md
-147
View File
@@ -1,147 +0,0 @@
# RustProxy 0.34.0 DEV — 1 KiB lock contention follow-up
## Goal and baseline
**Priority 1: reverse proxy, HTTP/1.1 keep-alive, uncompressed 1,040-byte
responses, cache OFF.** Aiming for approximately 100,000 RPS and at least 80%
of nginx under the same pinned-CPU benchmark. Priority 2: local static WWW,
measured separately. No feature removals and **no version bump**.
The user supplied a **new executable** (SHA-256
`1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa`,
`rust-proxy 0.34.0`), apparently built from the prior DEV revision. It passed
151 actual black-box HTTP tests with 0 failures and 1 skipped optional zstd
decoder. This runtime coverage belongs to the **supplied executable only**,
not to the freshly edited source below. Neither `cargo` nor `rustc` is
available in the editing environment; the new revision has NOT been compiled.
## Modified source
### `Cache::record` and `Cache::record_reason`
Before, every HIT, MISS and BYPASS acquired the same `Store` mutex used by the
HOT object store, exact LRU and cache inventory. With cache disabled, a 1 KiB
request still contested that mutex to increment `cache_disabled`.
`src/cache/stats.rs` now holds bounded per-(scope,domain) statistics separately:
- An `ArcSwap`-published immutable, borrowed-lookup BTreeMap of domain handles.
**The normal path does not lock or allocate**, even for BYPASS. New domains
alone use a registration mutex and a copy-on-first-visit registry update.
- Atomic counts for HIT, MISS, BYPASS, stored objects and saved body bytes.
The common static reasons (`cache_disabled`, `cache_hit`, `cache_miss`,
`request_cookie`) use direct atomic counters. Other reason names use another
immutable published index and atomic counters; they are registered once.
- Hard cardinality remains **4096 (scope, domain) pairs**. When full, new
unregistered domains are skipped as before. Scope purge preserves stats.
Only a revision-bearing global purge resets the per-domain registry.
- The dashboard still exposes the same `DomainCounters`, reason strings,
saved byte counts, ratio and per-vhost/object views. Counter snapshots are
quiescent-exact, but under concurrent requests individual fields can reflect
slightly different instants, consistent with other atomic metrics.
- HOT object and COLD representation/LRU access, barriers, memory accounting,
cache statuses, hit policy, retry and backend security paths are unchanged.
Trade-offs: a first-visit registration copies the bounded domain index and
allocates one counter set; this is not an adaptive, unbounded cache. The hot
path's ArcSwap guard and lookup still cost CPU. Measure that cost in profiles;
compare under high-cardinality traffic to identify any excessive registration
cost before production acceptance.
### Traffic analytics queue saturation
Previously `Exchange::drop` cloned the vhost, route, method, path, IP, user
agent and country before trying to enqueue the `Sample`. If the bounded
2,048-element MPSC queue was full, all those clones were discarded.
`Analytics::push_lazy` now acquires the same bounded Tokio queue permit with
`try_reserve` first and creates the sample only after a successful reservation.
Rejections increment `queue_dropped` exactly as before. Neither the analytics
worker, counters/top lists, reset behavior nor request log path is removed.
This improves allocation costs **on overflow** but does not turn dropped
analytics into delivered samples; monitor the existing drop counter.
### Per-vhost duration histogram atomics
Previously `VhostMetrics::complete` incremented every cumulative latency bucket
whose threshold covered the observation and the `+Inf` bucket. For an ordinary
short response this meant **11 shared atomic RMWs** for the histogram alone,
on top of existing exchange/status/byte/latency metrics.
It now increments **exactly one exclusive bucket** chosen from the same 10
latency boundaries. Both the regular vhost and Docker-vhost Prometheus exporters
compute the original cumulative `le` buckets, `+Inf` and `_count` from the 11
exclusive buckets **at scrape time**. The metric names, labels, boundaries,
body-lifetime timing and measurement collection remain intact. The cumulative
snapshot may be concurrent with request completions, as before. Added boundary
unit tests for exact thresholds and long-duration `+Inf`.
This is source-only: neither the throughput benefit nor compiled Rust behavior
has been verified in the editing environment. Review exporter parity and
under-load histogram correctness on the newly compiled executable.
## Verification and untested claims
- PASS: source preflight, cache/HOT/LB/RPS/histogram/static source guards and <=500-line
production Rust modules.
- PASS: SQLite migration + preset constraints, upgrade/lock and source
replacement tests.
- PASS: compiled user-supplied binary: 151 real HTTP checks, 0 failed,
1 optional zstd skip. This binary predates the source changes above.
- NOT RUN: Rust compile, Rust unit tests, native regressions or RPS after
rebuilding the new source. No numerical speedup attributed to this patch.
- Current diagnostic A/B on the user-supplied binary may use
`scripts/performance/ab_1kb_dev.py` and must remain labelled *old vs already
supplied binary*, **not** old vs newly edited source. The `wrk` 30 s x 7
pinned-CPU benchmark remains the production acceptance gate.
## Repeat on a compiler-enabled benchmark host
```bash
./dev.sh doctor
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/performance/regression.py \
--binary previous=/absolute/path/to/supplied-rust-proxy \
--binary edited="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-counter-http
python3 scripts/performance/lb_regression.py \
--binary previous=/absolute/path/to/supplied-rust-proxy \
--binary edited="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-counter-lb
python3 scripts/performance/transport_regression.py \
--binary previous=/absolute/path/to/supplied-rust-proxy \
--binary edited="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-counter-transport
python3 scripts/performance/benchmark.py \
--binary previous=/absolute/path/to/supplied-rust-proxy \
--binary edited="$(realpath target/release/rust-proxy)" \
--nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
--out /tmp/rustproxy034-counter-benchmark \
--modes proxy --sizes small --concurrency 16,64,128 \
--duration 30 --warmup 5 --repeats 7
```
Explicitly set `max_idle_per_host=64` on **both** before/after vhosts, with the
same TLS/H1/H2/health/cache/logging/settings. Verify 1,040 bytes and SHA-256,
no failures, latency percentiles, per-request CPU, RSS, file descriptors,
backend connects and all log/traffic drop counters. Test concurrent global
purges and per-vhost purges during traffic and verify the stats reset behavior
and cache HIT/MISS/BYPASS/reason distribution. Run long soaks and both cold/HOT
cache tests. The 100k RPS / nginx 80% goal is still **OPEN**.
## Build fix after first compiler feedback (OPT2-FIX1)
The first user build of OPT2 failed with six Rust `E0753` diagnostics because
`src/cache/stats.rs` starts with `//!` *inner module* documentation but is
injected via `include!("stats.rs")` after other items in `src/cache/mod.rs`.
The six documentation lines are now ordinary `//` comments; there is no
behavior change. Three unnecessary `mut` bindings in
`src/cache/dashboard.rs` were also removed (warnings only). Version remains
0.34.0; no performance or feature semantics have changed in this fix.
The compiler-enabled host should rerun `./dev.sh check`,
`CARGO_INCREMENTAL=0 ./dev.sh test` and an optimized release build, then
performance benchmarks. This editing environment still lacks `cargo` and
`rustc`, so the fix has been statically checked, **not compiled** here.
docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md
-137
View File
@@ -1,137 +0,0 @@
# RustProxy 0.34.0 DEV: 1 KiB reverse-proxy throughput audit
## Goal and evidence boundaries
Priority 1 is short (~1 KiB) HTTP reverse-proxy responses: aim for approximately
100,000 requests/s, and eventually at least 80% of nginx under the **same**
controlled benchmark, without disabling authentication, ACLs, caching semantics,
rate limits, traffic accounting, request IDs, logs, body streaming or protocol
support. Priority 2 is local WWW serving, to be tuned and measured separately.
This is a development revision; Cargo/package version remains **0.34.0**.
**This source is not compiled in the current environment.** Neither `cargo`
nor `rustc` is available. Only the user's pre-existing executable (SHA-256
`453757748455d7b9b50a6140e322ee4d4cf4a030eea29318686f2e28c594b644`)
was run. Live configuration A/B tests therefore substantiate a **configuration
bottleneck**, not the throughput of this edited source.
## Real baseline experiment (existing executable only)
Disposable loopback nginx upstream, 1,040-byte body, HTTP/1.1 persistent
frontend connections, caching disabled, identity response encoding, four Tokio
workers, ApacheBench (`ab -k -n 12000`), 16/64 concurrent frontend clients.
Checks outside the timed intervals verified HTTP 200, exact 1,040 bytes and
SHA-256 `62a08d75cdde0416c883d72458385b5e56ae8c14466934ad784587ddb93a548b`;
0 failed requests in the shown samples. The host has shared CPU resources;
results vary and are not a pinned-CPU or production acceptance benchmark.
| Existing executable configuration | 16 clients, RPS | 64 clients, RPS |
|---|---:|---:|
| max_idle_per_host = 4 | 18,226 | 17,935 |
| max_idle_per_host = 64 | 30,906 | 24,872 |
| Relative change (configuration only) | +69.6% | +38.7% |
A separate repeat experiment at 64 clients (two runs per variant, in order)
produced: `idle4` 14,028 / 14,834; `idle16` 22,205 / 24,711;
`idle32` 25,580 / 29,088; `idle64` 36,521 / 33,847;
`idle64 + recycle 250,000` 31,168 / 29,753;
`idle32_again` 34,798 / 29,194; `idle64_again` 34,677 / 29,965;
`idle4_again` 23,503 / 23,514 RPS. The order drift warns against
reporting a single percentage as a robust speedup. Raising generation
recycling from 25,000 to 250,000 did not establish any improvement; its
original limit is retained. All shown variants reported zero HTTP failures.
A preliminary nginx comparison gave ~75k/117k RPS at 16/64 clients, but later
nginx samples were ~50k/93k on the shared host. Do not compute a definitive
"% nginx" from these non-isolated runs. The 100k/80% goals remain **OPEN**.
## Source modifications
| Location | Change | Preserved behavior / caveat |
|---|---|---|
| `src/balance/model.rs` | New vhost default `max_idle_per_host`: **4 -> 64** | Configured values **1..64** remain validated; existing explicitly saved `4` stays `4`. Disabled keepalive, 30 s idle timeout, 25k-request and 3,600 s recycling rules are unchanged. Increased idle sockets use more file descriptors and memory. |
| `src/admin/web/vhosts/editor.js` | Match the new `64` default when the field is absent | Existing values, limits and all editor settings remain unchanged. |
| `src/proxy/body.rs` | At `Exchange::drop`, use exclusive `Mutex::get_mut()` rather than lock/unlock; move owned `route`, `client_ip`, `country`, `upstream_error` into final observation where possible | Exactly the same log fields, analytics sample, counters, permit lifetime, errors and websocket/H3 fields. Analytics still gets a clone when a vhost sample is queued; history still receives the original. |
| `src/proxy/headers.rs` | Skip reserved cookie sanitization immediately if the corresponding `Cookie`/`Set-Cookie` header is absent | Full sanitization, admin-cookie separation and restoration restrictions still apply when the header is present. |
| `src/balance/tests.rs`, `src/proxy/body/tests.rs`, `src/proxy/headers/tests.rs` | Add compatibility, completion and cookie regression assertions | These Rust tests **require** a compiler before they count as run. |
| `scripts/tests/*hotpath*` | Update source-only guards for the new **bounded** default | Static assertions do not prove functional or runtime performance equivalence. |
**Persistence note:** old vhosts may store `keepalive.max_idle_per_host=4`
explicitly; the new code default does not rewrite them. To apply the measured
configuration improvement before rebuilding, change that option to `64` in the
relevant vhost and verify actual idle socket/FD usage. Where hundreds of origins
share a host, test `16` or `32` if the FD/memory trade-off is preferable.
## Bottlenecks to profile next, not silently disable
1. `src/cache/dashboard.rs::Cache::record`: obtains a **shared cache-store
mutex on every outcome**, including `BYPASS` when cache is disabled through
`src/proxy/forward/cache_lookup.rs`. This may compete with HOT cache
operations and admin dashboard reads; profile `pthread_mutex`/futex and
lock hold times. Candidate: separate bounded, sharded domain/reason
counters from the HOT object store while preserving exact dashboard totals,
reason labels and reset semantics.
2. `src/logging/format.rs::Logs::push`: per-request bounded history
`try_lock`, potentially causing recorded log drops under high concurrency.
Preserve *existing* drops/accounting, live subscribers and external sinks.
Candidate: worker-local bounded batches or sharded history with deterministic
bounded merge; validate log order, recent history, retention and drops.
3. `src/proxy/body.rs::Exchange::drop` + `src/traffic.rs::Analytics::push`:
request-sized log strings, sampling queue sends, atomics and lifecycle
observations. Profile allocations and CPU per completed request. Candidate:
compact shared immutable identity fields or conditional cloning solely
where consumers require ownership, without removing logs or analytics.
4. `src/proxy/forward/route_cache.rs`: request-target path copies,
header policy and several route/cache eligibility steps. Measure with
sampling `perf record -g` before extracting additional fast paths; preserve
query handling, canonical host, ACL, authentication, retries and WebSockets.
5. `src/upstream/*`: profile H1 pool misses, TCP connect rate, Hyper
connection reuse and generation swaps. Increasing recycle thresholds or
changing HTTP/2/H2C modes without an isolated performance case is not
recommended.
6. **Separate WWW priority:** benchmark short static objects independently;
existing `sendfile: auto` applies to eligible cleartext H1 objects >=64 KiB
and cannot accelerate 1 KiB bodies.
No lock removal, tracing/log disabling, caching-policy weakening, bypass of
security checks, or transformation of complete bodies into buffered copies is
proposed without a test proving semantic equivalence.
## Acceptance gate on a machine with Rust toolchain and isolated load generator
```bash
./dev.sh doctor
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/performance/regression.py \
--binary after="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-1kb-regression
python3 scripts/performance/lb_regression.py \
--binary before=/absolute/path/to/old-rust-proxy \
--binary after="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-1kb-lb
python3 scripts/performance/transport_regression.py \
--binary before=/absolute/path/to/old-rust-proxy \
--binary after="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-1kb-transport
python3 scripts/performance/benchmark.py \
--binary before=/absolute/path/to/old-rust-proxy \
--binary after="$(realpath target/release/rust-proxy)" \
--nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
--out /tmp/rustproxy034-1kb-benchmark \
--modes proxy --sizes small --concurrency 16,64,128 \
--duration 30 --warmup 5 --repeats 7
```
Use the **same explicit** `max_idle_per_host=64` and all other vhost settings
for the *source A/B* test. Run 4 vs 64 as a separate *configuration A/B* test;
never attribute a settings difference to code. Verify full content checksums,
HTTP 2xx/4xx/5xx, p50/p95/p99, drop counters, backend connections,
FD/RSS trends, CPU/request and packet/TCP retransmit rates. Test H1/H2/H3,
TLS, WebSockets, sticky/rate/ACL, cache MISS/HIT/BYPASS, admin sessions, active
failover, cancellation, idle timeouts and a sustained memory/connection soak.
Profile the freshly **built** executable separately from timed RPS runs.
Until compilation and those gates pass, this is a **development source-only
candidate**, not a claim of reaching 100k RPS or 80% nginx.
docs/PROXY_1KB_OPT3_DEV_0.34.0.md
-88
View File
@@ -1,88 +0,0 @@
# RustProxy 0.34.0 DEV — 1,040-byte response, OPT3
## Scope
Primary: uncached reverse proxy, HTTP/1.1 keep-alive, 1,040-byte body,
no compression, ACL/security/logging/analytics/counters fully operational.
Goal: ~100,000 RPS and >=80% of nginx on the **same isolated host**.
Secondary: local Web Server/static files, measured independently.
**Version remains exactly 0.34.0**; existing configuration and defaults are
unchanged; no functions, checks or logs are disabled.
## New code optimizations (relative to OPT2-FIX1)
1. `src/acl.rs` / `src/proxy/cache_helpers.rs`: borrow ordinary validated
routing paths using `Cow<str>` during immutable route selection. Calls
with escaped (`%xx`) or repeated separators still use the original strict
allocator/decoder. Invalid control bytes, dot segments, UTF-8 and ambiguous
encodings remain rejected; the original wire path is not rewritten.
2. `src/proxy/forward_entry.rs` plus cache/Web Service call sites: retain the
original `http::Uri` value so normal paths can remain borrowed during the
entire mutable forwarding/rewrite operation. Cache-refresh tasks that need
ownership still allocate explicitly and hold their own `String`.
3. `src/forwarding.rs`: a **direct, untrusted** TCP peer now obtains the same
seven forwarding headers without allocating hop-chain `Vec` structures.
Trust chains and the configured `trust_forwarded_proto` logic use the
unchanged full parser. IPv4/IPv6 RFC 7239 formatting and true listener
port/scheme are preserved; untrusted spoofed values are still ignored.
4. `src/proxy/forward_entry.rs`: when effective client IP is identical to
the transport peer, reuse the IP stored by observation. No redundant
`IpAddr::to_string()` and no per-exchange mutex write. Different/trusted or
IPv4-mapped addresses are still stored in canonical form.
5. `src/proxy/headers.rs`: plain single-valued `Connection: keep-alive`
avoids the temporary `Vec<HeaderName>` while all standard hop-by-hop fields
are removed; comma-separated, repeated and complex Connection fields keep
the original removal parser to prevent nominated-header leaks.
6. `src/proxy/request_id.rs`: preserve SHA-256, CSPRNG, per-process sequence,
unpredictable 64-char hexadecimal IDs and wire headers, but write the hex
digits into one preallocated byte buffer before UTF-8 verification.
7. `src/cache/stats.rs`: select the most frequent `cache_disabled` reason
counter directly. The same exact atomic totals, reason labels, 4096-domain
limit and reset behavior are retained.
The changes avoid repeated work in common cases; no numerical throughput
benefit has been measured for this new source. In particular, no cache, log,
metrics, retry or auth feature was switched off for a synthetic benchmark.
## Evidence and known limits
- Provided newly compiled *OPT2-FIX1* executable SHA-256:
`6037e98d059672b716f2e1411976bf305cac927901a6f6662d8b703829ff8196`.
- PASS **89** HTTP proxy/policy/cache black-box assertions, zero failures,
testing that supplied executable, **not** the new OPT3 code.
- Source checks, migration/upgrade checks and ZIP checks are separate
static/offline verification; they are not equivalent to cargo/test/live.
- A single interrupted diagnostic A/B sample at c=64, 3 s, same idle=64,
gave older binary 33,752 RPS and OPT2-FIX1 binary 33,556 RPS, zero failed
requests; do **not** infer a true performance difference from one sample.
- NOT RUN here: `cargo check`, `cargo test`, rebuilt OPT3 integration tests,
post-build A/B RPS, long soaking, real TLS/H2/H3 compatibility and 80%-of-nginx
acceptance. No Rust toolchain is available in this editing container.
- Resource/latency trade-offs (including log/analytics drops, RSS/FDs) require
the same measured and pinned server/peer/generator CPU sets for before/after.
## Rebuild and measure
From the unpacked `RustProxy/` directory:
```bash
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
bash dev.sh
python3 scripts/performance/regression.py \
--binary new=/ABSOLUTE/PATH/NEW/rust-proxy --out /tmp/rustproxy-opt3-http
python3 scripts/performance/ab_1kb_dev.py \
--binary before=/ABSOLUTE/PATH/OPT2FIX1/rust-proxy \
--binary after=/ABSOLUTE/PATH/OPT3/rust-proxy \
--nginx /usr/sbin/nginx --ab /usr/bin/ab \
--out /tmp/rustproxy-opt3-1kb --concurrency 16,64 \
--duration 5 --repeats 3 --workers 4 --max-idle 64
```
For an acceptance-grade evaluation, run the `scripts/performance/benchmark.py`
paired before/after/nginx test with separate pinned CPU sets, a verified origin
and 30-second runs repeated 7 times. Report median and p95/p99 latency,
errors, CPU, RSS, file descriptors, dropped log/analytics events, and all HTTP
status/body hashes. Retest TLS/H2/H3/WebSocket, auth, cache semantics and
connection/backpressure limits. Compare the **rebuilt** OPT3 binary against
OPT2-FIX1 at exactly the same listener/backend settings and compiler profile.
docs/PROXY_1KB_OPT4_DEV_0.34.0.md
-97
View File
@@ -1,97 +0,0 @@
# RustProxy 0.34.0 DEV / OPT4: HTTP/1.1 small-body follow-up
## Objective and validity
Primary goal: 1,040-byte reverse-proxy response, HTTP/1.1 keep-alive,
cache disabled; target about 100k RPS and at least 80% of nginx on the **same
isolated host**. Secondary goal: local static Web Server, benchmarked separately.
No version bump or feature, metric, logging, ACL, authentication, TLS,
HTTP/2/3, stream, retry or cache-policy bypass is introduced.
The **provided and compiled OPT3 binary** (SHA-256
`2efe273340f9cf92c5900c85c0bd1b775c87a9d1eaff1a10ec7635d268b080b1`)
passed 151 black-box HTTP assertions: 0 failures and 1 optional Zstandard
case skipped. This is evidence for the **input** binary, not for the new
OPT4 sources. OPT4 still requires a native Rust build, cargo tests and a
post-build benchmark. The diagnostic same-host CPU shares resources and is
not an acceptance benchmark.
## Code changes relative to OPT3
1. `src/proxy/forward/route_cache.rs`: store normal public path/query as
`Cow<'_, str>` referencing the independently retained, immutable original
URI. Without a path rewrite both public/origin suffixes stay borrowed;
strict encoded/rewrite handling remains an owned validated string.
Cache refresh tasks explicitly call `.to_string()` only when they actually
must retain an owned target across spawned tasks.
2. `src/proxy/upstream_helpers.rs`: skip allocating a request-body metering
adapter **only** when `Body::is_end_stream()` establishes that it contains
no DATA frames or trailers. Every nonempty/streaming request retains the
original metering, timeout, backend lease accounting and cancellation.
The response adapter still retains the exchange/inflight permit and handles
completion/response bytes for empty requests.
3. `src/proxy/types.rs`, `src/proxy/forward/upstream.rs`: consume owned
`http::Uri` when preparing the upstream target. HTTP/1.1 returns it directly;
H2 takes the same URI into parts for strict authority rewriting. Remove an
unnecessary intermediate URI clone; replay still builds independent URIs.
4. `src/forwarding.rs`: reuse the formatted IPv4 peer address when creating
RFC 7239 `Forwarded` as well as X-Real-IP/X-Forwarded-For. Preserve quoted
IPv6 bracket and proxy-chain logic for trusted and untrusted peers.
5. `src/proxy/upstream_helpers.rs`: fast-path byte-safe truncation of short
or ASCII host/path log values, keeping the original max-*characters*
semantics for non-ASCII UTF-8.
6. `src/proxy/headers.rs`: avoid nominated-field token parsing for absent,
lone `keep-alive`, or lone `close` Connection values. A `close` field on
the wire still forces the general parser, so a nominated `Close` header
is removed exactly as before. Multi-value, comma lists and malformed tokens
retain the previous defensive parser.
7. `src/proxy/headers.rs`: similarly avoid the generic token parser when
checking HTTP upgrade on plain keep-alive, close, or absent Connection.
Complex multi-valued requests and explicit Upgrade remain unchanged.
## Tests and limits
- New Rust cases validate empty vs nonempty request-body permits and bytes,
H1 URI/authority/query, ASCII/Unicode truncation, and Connection variants
including a literal nominated `Close` field. Existing regression coverage
still includes sticky backends, cache, ACL and trusted forwarded identity.
- Source, SQLite migration and upgrade tests are necessary but **not** Rust
compilation or performance evidence. The compile check must run before use.
- Run `./dev.sh check && CARGO_INCREMENTAL=0 ./dev.sh test`, followed by
the native black-box acceptance suite (proxy, policy, cache, transform,
compression, static serving), plus TLS/H2/H3 and websocket tests.
## Measurements
An initial diagnostic same-host `ab` sample at concurrency 16 gave
OPT2 33,042.55 RPS and compiled OPT3 36,962.11 RPS, without HTTP errors;
this single sample is not proof of a reliable uplift. A separate complete
12-sample run (two 3-second repetitions each at 16/64 clients) produced:
| Concurrency | OPT2 median RPS | OPT3 median RPS | nginx median RPS | OPT3/nginx |
| --- | ---: | ---: | ---: | ---: |
| 16 | 21,608 | 29,906 | 92,586 | 32.3% |
| 64 | 38,060 | 34,781 | 123,809 | 28.1% |
All 12 samples had valid 200 responses and correct 1040-byte body checks.
OPT3 was slower than OPT2 at concurrency 64; the variability is too large to
claim a sustained uplift or regression based on two short same-host samples.
Raw evidence includes run order, response checksum and nginx comparator.
No result here measures OPT4, which was not built in this environment.
## Next evaluation commands
```bash
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
bash dev.sh
python3 scripts/performance/regression.py --binary opt4=/absolute/path/target/debug/rust-proxy --out /tmp/opt4-http
python3 scripts/performance/ab_1kb_dev.py --binary opt3=/absolute/path/opt3/rust-proxy --binary opt4=/absolute/path/target/debug/rust-proxy --nginx /usr/sbin/nginx --ab /usr/bin/ab --out /tmp/opt4-ab --duration 5 --repeats 3 --concurrency 16,64 --workers 4 --max-idle 64
```
For authoritative acceptance: pin client/server/origin to nonoverlapping CPU
sets, align build profile and all flags, compare 30s x7 runs, record median,
p95/p99, correct 1040 B response checksum and status, CPU, RSS, FDs, cache
stats, analytic/sink queue drops, errors, and both HTTP/1.1 and optional
HTTP/2 endpoints. Reject changes on new 4xx/5xx, body mismatches, regressions
of access controls or material memory/fd increases.
docs/PROXY_1KB_OPT5_DEV_0.34.0.md
-93
View File
@@ -1,93 +0,0 @@
# RustProxy 0.34.0 DEV — OPT5, 1,040-byte reverse-proxy optimization
## Acceptance boundary
The source package preserves **0.34.0** and all existing configurable services,
logs, dashboards, cache/ACL/auth policies, streaming, TLS/H2/H3/WebSocket,
failover, retry and bounded-resource behavior. OPT5 is a **source-only** candidate:
no Rust compiler is available in the editing environment. The new code's
compilation, Rust unit tests, binary throughput and full-protocol interoperability
are unverified until the developer builds it. The compiled OPT4 executable is
the known test input, **not** a benchmark of OPT5.
## Compiled OPT4 input verification
Input: `/mnt/data/rust-proxy(4)`, SHA-256
`1c4c701f11b57b1293020061c8af21d5a9df0c86f03290018a90edc007667b24`.
- The real HTTP regression runner completed **151 passing**, **0 failing** cases;
**1 skipped** because Python `zstandard` is not installed. Protocol tests for
TLS/H2/H3 are a separate acceptance gate and were **not** passed by this run.
- Same-host ApacheBench HTTP/1.1 keep-alive, 1,040-byte body verified by exact
SHA-256, cache OFF, 4 Tokio workers, 64 idle HTTP1 upstream connections,
3-second warm-and-measure samples, two repeats per variant/concurrency:
| RPS, median of 2 samples | 16 clients | 64 clients |
| --- | ---: | ---: |
| OPT3 | 37,017 | 40,668 |
| OPT4 | 39,054 | 36,425 |
| nginx | 99,808 | 110,783 |
| OPT4 vs OPT3 | +5.5% | -10.4% |
| OPT4 / nginx | 39.1% | 32.9% |
There were zero HTTP failures and the response body was verified before/after
measurements. Shared CPU and only two short samples are diagnostic, NOT a stable
speedup or production proof. The 100k RPS and 80%-of-nginx goals remain unmet.
## OPT5 changes (8 coherent hot-path optimizations)
1. `src/proxy/request_id.rs`: eliminate per-request SHA-256. Keep the existing
OS-seeded/reseeded thread-local CSPRNG and 256-bit ID; XOR a monotonic
sequence into the first 8 nonce bytes. Output remains uniformly random,
64 lowercase hex characters and the protected `X-Request-ID` correlation
remains. Random IDs continue to be unique with overwhelming probability.
2. `src/forwarding.rs` and `src/proxy/forward_entry.rs`: only construct
`Context.original_url` for a frontend with configured Tinyauth. This is the
*only* runtime consumer of that absolute URL. Public `context(...)` still
constructs it by default. Trusted/untrusted forwarding and all validation
still execute identically; IPv4/IPv6 and `x-original-url` for Tinyauth remain.
3. `src/balance/lease_runtime.rs`: borrow `"ok"` for successful backend health
observations instead of allocating a `String`. 5xx failures still record the
exact original `status=NNN` detail and slow/passive health counters are kept.
4. `src/traffic.rs`: receive 64 samples at a time (Tokio `recv_many`) and apply
them under a single dashboard-state mutex. Order within the existing bounded
2,048-sample queue is preserved; waiting is outside the mutex and producer
backpressure and drop accounting are unchanged.
5. `src/traffic.rs`: reuse map entries with borrowed vhost names. New keys are
cloned only when a domain first appears, preserving the max 64-vhost bound.
6. `src/traffic.rs`: classify status 1xx–5xx using static labels; preserve
unknown extension status labels such as `7xx`.
7. `src/traffic.rs`: replace per-sample lowercase allocation for User-Agent
family detection with ASCII-insensitive byte-window comparisons, and avoid
copying all TopTable maps when displaying a single vhost.
8. `src/proxy/upstream_helpers.rs`: skip resolving inherited/vhost/web-service
compression settings when no encoding is accepted even under the global
codec ceiling (for example `Accept-Encoding: identity`). Existing compression
handling is retained for every accepted codec and request method.
The caller's front-end policies remain evaluated on every request. No log,
metric, access control or authentication feature was disabled or made optional.
## Verification and outstanding gates
Static source, SQL migration and managed-upgrade tests are executed and their
outcomes supplied in the external evidence archive. New Rust test fixtures cover
request IDs, forwarding/trusted IPv6, compression, healthy/5xx backend detail,
analytics batching, top classes, country-independent User-Agent matching,
reset and the bounded queue.
To verify the new build on the developer's system:
```bash
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
./dev.sh build
```
After building, repeat the actual HTTP regression, LB regression and TLS/H2/H3
transport tests and collect at least seven 30-second CPU-isolated, warmed,
interleaved OPT4/OPT5/nginx measurements with `wrk`, p95/p99 and CPU/RSS.
Treat the non-tested binaries, short shared-host tests and simulated Python
checks as distinct types of evidence. Measure the secondary Web Server path
separately. Do not alter the version or silently relax cache, auth or timeout
policies to reach a RPS target.
docs/PROXY_HOTPATH_0.34.0.md
-84
View File
@@ -1,84 +0,0 @@
# RustProxy 0.34.0 — reverse-proxy hot-path candidate
## What was changed
1. **Routes:** `Snapshot.routes` and `fallbacks` now hold
`Arc<CompiledRoute>` handles. The same compiled location is shared across
hostname aliases. `observe` resolves vhost/location once from the accepted
listener/TLS-state and stores a request-local handle. `forward_request`
consumes the same immutable snapshot/route; internal error fallbacks still
resolve the explicitly configured named location independently.
2. **Host matching:** wildcard candidate strings are generated lazily only
after an exact host failed, without changing longest-specific-suffix order.
3. **LB:** single-backend non-cookie pools use a direct admitted lease without
temporary candidate vectors or scheduler mutex. Multi-backend weighted
round-robin retains its ordering and mutex; least-connection/traffic,
adaptive, IP hash and cookie-affinity retain their behavior. IP hash avoids
repeated canonicalization/temporary `format!` strings for each candidate.
4. **Health:** a `healthy_cached` atomic allows checking healthy backends
without the health mutex; all state transitions remain inside the existing
state machine, including passive quarantine and half-open recovery.
Resolved active/passive per-member policies are held in shared immutable
`Arc` handles so a new request lease does not deep-clone health paths,
headers or optional host strings.
5. **Fallback/retry:** request body+HeaderMap replay templates are made only
for configured error fallbacks. Retry headers are cloned only for eligible
safe replay requests where another member or H2 graceful close makes retry
possible; writes, streaming bodies and WebSockets are not replayed.
6. **Telemetry:** three per-request mutex-backed scalar tags are now atomic;
their original event strings and `Option<bool>` log fields are decoded at
exchange completion. Request ID, method, host and path are moved into the
terminal log record instead of deep-cloned in Drop; the traffic sample
still receives its own required copies. Logs avoid a broadcast clone if
there are no subscribers, preserving history, sink and counters.
## Preserved boundaries
All vhost, ACL, GeoIP, Basic Auth, Tinyauth, WebSocket, retry, cache,
transform, compression, log, metrics, `sendfile`, PHP-FPM, HTTPS/H2/H3 and
admin panel configuration paths are intended to remain compatible. Defaults
and SQLite schema are unchanged; no new performance toggle was added. The
optimization reuses route objects within one immutable snapshot but does not
allow stale snapshot cache fills (existing revision ticket barrier remains).
Sticky cookies deliberately retain the general selection path, and upstream
request-count recycling is unchanged (default 25,000 per generation).
## Locally executed checks vs required checks
- **Executed locally:** scoped-cache/sendfile/hot-path *source-only* guards,
Python syntax, SQLite migration and upgrade preservation tests, selected JS
panel scripts and simulated editor tests. These do **not** prove Rust builds.
- **Prepared, NOT executed:** Rust unit tests for shared routing, single-member
LB capacity/health, passive ejection/recovery, atomic log fields, retained
sendfile, cache and protocol behavior.
- **Unmeasured:** 0.34.0 requests/sec, latency, CPU cost, memory and nginx gap.
Do not infer a throughput gain from fewer source-code locks/allocations.
## Build and acceptance
Run on the intended Linux builder with exactly the same compiler, dependencies,
release flags and workload as the 0.33.8 reference:
```bash
./dev.sh doctor
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/tests/proxy_hotpath_034_static_test.py
python3 scripts/tests/cache_scope_034_static_test.py
python3 scripts/tests/sendfile_034_static_test.py
python3 scripts/tests/dashboard_presets_sql_test.py
python3 scripts/tests/upgrade_performance_test.py
```
Next run `scripts/performance/regression.py`, `lb_regression.py`,
`transport_regression.py` and `concurrency_regression.py` against both binary
versions. After functional acceptance, run the paired
`scripts/performance/benchmark.py` suite against the unmodified baseline,
this candidate and nginx; use the instructions in
`scripts/performance/README.md` including separate CPU affinities.
Measure RPS/p95/p99, CPU microseconds per request, RSS, FDs, pool rotations,
health transitions, cache HIT ratio and logger drops. For each payload report
`(0.34.0 RPS / 0.33.8 RPS - 1)*100` and `0.34.0 RPS/nginx RPS*100` from real
medians. A source-only revision is **not production-accepted**.
docs/PROXY_RPS_0.34.0.md
-78
View File
@@ -1,78 +0,0 @@
# RustProxy 0.34.0: hot-path throughput revision
**Status: uncompiled source candidate.** This revision does not change the
application version, public API, dashboard controls, cache policy or tuning
defaults. It must not be labelled production-ready until the new Rust binary
passes compile, unit tests, live regressions and paired performance comparisons.
## Code changes and intended cost reductions
| Focus | Change | Kept behavior / risk to verify |
|---|---|---|
| Per-request identity | Draw a 256-bit nonce from the OS-seeded/reseeded thread-local CSPRNG instead of asking `getrandom` on every request. Preserve SHA-256 of nonce+monotonic sequence and the existing 64-hex ID format. | IDs remain non-predictable. Check ID uniqueness and raw upstream headers. |
| Upstream HTTP request target | Compile the validated scheme/authority `http::Uri` once with each immutable backend; parse only the request path+query and rebuild using typed `Uri::from_parts`. | Never accept a client-supplied authority, bypass DNS/address policies or change SNI. Test H1, H2, IPv6, query and percent-encoding. |
| Load balancing | Do not clone the member ID string for an existing smooth-WRR scoreboard entry. Continue the previously introduced no-mutex single-member non-cookie path. | Sticky-cookie modes, capacity limits, retry exclusions, health and drain remain unchanged. |
| Active health | Use already-compiled `Arc<HealthCheck>` when dispatching probes, avoiding a clone of its path/headers for each scheduled probe. | Same 250-ms supervisor, timeout, enabled/claim gate and health state transitions. |
| Per-frame streaming | Capture the leased backend once at first streamed request DATA (response already captured at wrapper construction). Do not take `Exchange.backend` mutex on every upload chunk. | Request/response byte counters, mid-stream failures, idle timeouts and lease lifetime remain active. |
| Pool recycling | Preserve exact request-count and elapsed-time recycling checks per request, with the existing independent 250-ms maintenance cycle. The pool factory/connection policy is unchanged. | Exact lifecycle semantics are more important than saving one cheap clock read; tune idle/age limits only after profiling. |
| Log history | Move completed `LogEntry` directly into bounded history when there are no live viewers or enabled external sinks, instead of deeply cloning its strings. Continue cloning and delivering to live viewers/sinks when enabled. | Same JSON fields, history size, drop counters, file/Loki/Logstash forwarding. |
| Route/HTTP instrumentation | Remove redundant route-ID mutex/write in forwarding and collapse H1/H2 version tallying into one branch. | Route is recorded once before forwarding; H1/H2 counters remain. |
These historical changes build on the previous 0.34.0 route-Arc, single-backend LB,
conditional retry/fallback-header cloning, atomic exchange flags and cache HOT
LRU changes. There are no new UI toggles. This **earlier** revision retained `max_idle_per_host`
4 and `pool_recycle_requests` 25,000. The **subsequent DEV 1 KiB revision**
changes only the default idle limit to 64, based on live configuration A/B
with the old binary (not source A/B); see `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. `sendfile: auto` remains opt-in and applies only to eligible Linux
plaintext H1 static WWW responses, not reverse proxy or TLS/H2/H3.
## Cache transform/query policy
An enabled `bypass_query` is a deliberate policy: `/theme.css?v=...` is a
`BYPASS` even when the origin sends `Cache-Control: public`. Turning the bypass
off permits a transformed CSS object to follow `MISS -> HIT` and appear in the
cache dashboard. An existing transformed HOT row with hit count and TTL is
positive evidence of this. This throughput revision does not modify the
cache eligibility logic or discard Age/ETag/Cache-Control headers.
## Verification already possible here
- **Old/reproducible reference executable only:** 110 passed, 0 failed, 1 skipped
(`compression.codec.zstd` skipped because the optional decoder was missing),
from live proxy/policy/cache/transform loopback regression. This does **not**
exercise the newly modified Rust source.
- **Old/reproducible reference executable only:** query bypass enabled gives
`BYPASS -> BYPASS`, disabled gives `MISS -> HIT` and a transformed representation;
the older Age regression is also reproducible on this old executable.
- Source-only invariants: `scripts/tests/rps_hotpath_034_static_test.py` plus
existing route/cache/sendfile checks. SQL migration and upgrade scripts pass.
- **Not performed:** Rust compilation or newly added Rust unit tests, live
behavior of the optimized binary, HTTP/2 live TLS, comparative RPS, CPU
profiling and long-duration memory/health soak. The environment does not
include `cargo`, `rustc` or `wrk`. No post-change RPS gain is claimed.
## Acceptance commands on a Rust-capable Linux host
```bash
./dev.sh check
CARGO_INCREMENTAL=0 ./dev.sh test
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/tests/rps_hotpath_034_static_test.py
python3 scripts/tests/cache_age_transform_034_live.py \
--binary "$(realpath target/release/rust-proxy)" \
--output /tmp/rustproxy034-cache-after
python3 scripts/performance/regression.py \
--binary optimized="$(realpath target/release/rust-proxy)" \
--out /tmp/rustproxy034-http-regression
```
Run `scripts/performance/benchmark.py` from the documented reproducible
before/after/nginx command in `scripts/performance/README.md`. Compare the
unchanged old 0.34.0 binary with the newly compiled source, not a baseline
with different feature flags. Record RPS at 1,040 B, 64 KiB, 1 MiB, p95/p99,
CPU/request, RSS, connection counts, errors and exact response-body hashes.
Use `perf record -g -p <proxy-pid>` during a stable request interval and inspect
`perf report --stdio` to identify the next true bottleneck.
Do not interpret Python origin self-tests, source inspections, or speculative
Amdahl models as measured RustProxy throughput.
docs/README.md
+2 -4
View File
@@ -7,7 +7,6 @@ This is the canonical index for the current project documentation. The authentic
- [Project README](../README.md) - product overview, installation entry points and current release notes.
- [ROADMAP](../ROADMAP.md) - planned and completed release scope.
- [CHANGELOG](../CHANGELOG.md) - user/operator-visible changes by release.
- [TODO](TODO.md) - open work only; completed work is not retained as TODO.
## Functional documentation
@@ -42,7 +41,6 @@ This is the canonical index for the current project documentation. The authentic
- [MOCK_SERVER](MOCK_SERVER.md) - local Python mock origin used by development/integration checks.
- [MOCK_AUTH](MOCK_AUTH.md) - local fake Tinyauth peer for development only.
- [MEMORY_STRESS](MEMORY_STRESS.md) - sustained-memory audit, offline load simulations, real RustProxy staging stress and RSS measurement procedure.
- [TEST_REPORT_0.34.0](TEST_REPORT_0.34.0.md) - 0.34.0 source-stage test evidence, simulated cache retention and required acceptance gates.
## Technical documentation
@@ -61,7 +59,7 @@ Completed roadmap history is retained separately as [ROADMAP_HISTORY](ROADMAP_HI
### Retained audits and historical context
- [SECURITY_AUDIT](SECURITY_AUDIT.md) and [SECURITY_AUDIT_0.31.0](SECURITY_AUDIT_0.31.0.md) - current security audit material and the 0.31.0 audit.
- [SECURITY_AUDIT](SECURITY_AUDIT.md) - security audit material.
- [CHANGELOG_TECHNICAL_ARCHIVE](CHANGELOG_TECHNICAL_ARCHIVE.md) and [ROADMAP_HISTORY](ROADMAP_HISTORY.md) - retained long-form technical history and completed milestones.
- [ACME_RFC2136](ACME_RFC2136.md) - historical DNS provider details.
- [INTEGRATIONS_040](INTEGRATIONS_040.md) and [MANAGEMENT](MANAGEMENT.md) - retained design and integration notes.
@@ -72,7 +70,7 @@ The old per-version implementation notes and audits removed from this trimmed so
## Maintenance rules
- Put user/operator-visible release changes in `CHANGELOG.md`.
- Put future release scope in `ROADMAP.md` and unresolved work in `docs/TODO.md`.
- Put future release scope and unresolved work in `ROADMAP.md`.
- Keep detailed implementation evidence in technical documents or version audits, not in CHANGELOG.
- When functionality changes, update the functional document, this index, Reader coverage and cross-links in the same change.
- Remove obsolete or duplicated documents when they no longer provide historical or technical value; retain useful audits with an explicit historical label.
docs/RELEASE_0.34.0_CACHE_FIX_PL.md
-93
View File
@@ -1,93 +0,0 @@
# RustProxy 0.34.0 — poprawki cache, transformacji i kontrola RPS
Status: **zmodyfikowane źródła, niekompilowane lokalnie** (brak `cargo` i `rustc`). Numer wersji: 0.34.0.
## Co odtworzono na binarce użytkownika
Uruchomiono binarkę Linux ELF w izolowanej konfiguracji z lokalnym origin HTTP, z włączonym cache, transformacją CSS, `max_ttl_seconds=86400`, diagnostyką i API administracyjnym. HTTP/1.1 localhost, pełne GET, brak kontaktu z witryną produkcyjną.
| Odpowiedź | Pierwszy GET | Drugi GET | Liczba pobrań z origin |
|---|---|---|---|
| `/normal.css` | MISS | HIT | 1 |
| `/aged.css` | MISS | MISS | 2 |
| `/br.css` | MISS | HIT | 1 |
| `/aged-br.css` | MISS | MISS | 2 |
| `/private.css` | MISS | MISS | 2 |
| `/no-store.css` | MISS | MISS | 2 |
Zwykły CSS i skompresowany Brotli dostają HIT. Warianty z `Age: 1553483` mają `MISS → MISS` mimo `Cache-Control: public, max-age=2592000`. Przypadki `private` / `no-store` słusznie nie trafiają do cache. Zmiana keep-alive vhosta nie usuwała poprawnie zachowanego obiektu. `expose_only_hits=true` celowo ukrywa nagłówki na MISS.
## Główna przyczyna i poprawka
W `src/cache/policy/response.rs` obowiązywał wzór `ttl = min(origin_max_age, local_max_ttl)` oraz `remaining = ttl - corrected_age`. To jest sprzeczne z zadaniem lokalnego limitu: świeżość w origin nadal trwa, ale CDN `Age > local_max_ttl` dawał sztuczne 0 sekund lokalnej świeżości.
Poprawiony wzór:
```text
corrected_initial_age = max(apparent_age, upstream_age + response_delay)
remaining_origin_freshness = max(0, origin_max_age - corrected_initial_age)
remaining_local_freshness = min(remaining_origin_freshness, local_max_ttl)
entry_fresh_until_age = corrected_initial_age + remaining_local_freshness
```
Przykład z przesłanych nagłówków: wiek upstream 1 553 483 s = prawie 18 dni. Pierwotna świeżość 2 592 000 s (30 dni), więc pozostało około 12 dni w origin. Po naprawie RustProxy będzie mógł przechować świeżą odpowiedź przez lokalny limit (domyślnie 86 400 s = 1 dzień). Nagłówek Age nadal będzie około 1,55 mln sekund i będzie rósł na HIT — to poprawne.
`Age` nie wynika z `Last-Modified: 2016`. Najczęściej jest przekazany przez upstream/CDN, a stara data `Date` też mogłaby podnieść apparent_age. Rozróżnienie: `Age` = HTTP wiek całkowity; `resident_age_seconds` = pobyt w HOT/COLD.
## HOT RPS i panel
1. HOT HIT nie wyjmuje już i nie wkłada od nowa wpisu oraz indeksu `Vary`, tylko aktualizuje licznik HIT i kolejność LRU w miejscu. Zmiana ma zmniejszyć alokacje i krytyczną sekcję; **zysk RPS jeszcze niezmierzony**.
2. Panel Cache ma osobne kolumny **Age** i **Czas w RustProxy**. API `dashboard.objects[]` dodaje `resident_age_seconds` także dla COLD. Pozostałe nazwy pól nie zostały zmienione.
3. Licznik `x-cache-hits` dotyczy konkretnego aktualnego wpisu. Gdy obiekt zostanie wyparte z HOT (limit wpisów/pamięć), usunięty lub zastąpiony po MISS, licznik restartuje. W próbce użytkownika TTL wynosił około 604 tys. s (7 dni), więc późniejszy MISS po ~4 minutach **nie wynikał z normalnego wygaśnięcia TTL**. Konkretnej przyczyny epizodu nie można ustalić bez trendów `evictions`, `purges`, `hot_demotions`, `cold_hits`, kluczy/Vary oraz zmian konfiguracji.
4. Brak `x-cache` na MISS jest oczekiwany, gdy włączono **nagłówki diagnostyczne tylko na HIT**. Brak wpisu w API może mieć inne przyczyny (np. cache-control `private`, `no-store`, brak polityki origin, limit obiektu) i nie należy ich obchodzić na siłę.
5. Nie zmieniano uprawnień, `private`/`no-store`, kluczy Vary, TTL po stronie origin ani zasad selektywnej invalidacji.
## Wynik symulacji TTL
Wykonano **100 000 deterministycznych scenariuszy**. Dla każdego sprawdzono: po naprawie lokalna świeżość nie przekracza `local_max_ttl` ani świeżości pozostałej w origin.
| Scenariusz | Stary pozostały TTL | Po poprawce (symulacja) |
|---|---:|---:|
| Age = 1 553 483, local cap 1 dzień | 0 s | 86 400 s |
| Age = 1 553 483, local cap 7 dni | 0 s | 604 800 s |
| Origin kończy świeżość za 10 s | 0 s | 10 s |
| Origin jest już nieświeży | 0 s | 0 s |
W symulacji 33 387 / 100 000 losowych przypadków stara polityka błędnie dawała zero zamiast dodatniego TTL. **To nie jest procent regresji w realnym ruchu**, a wynik losowania parametrów testowych.
## Nginx — rzeczywiste pomiary starej binarki
Archiwalne wyniki samej binarki `rust-proxy 0.34.0` na tym środowisku (Linux, 4 vCPU, HTTP/1.1 loopback, ApacheBench, 64 połączenia, cache OFF, 3 powtórzenia), z nginx 1.26.3. Brak miarodajnego benchmarku **nowych źródeł** przed ponowną kompilacją.
| Endpoint | RustProxy RPS (mediana) | nginx RPS (mediana) | RustProxy / nginx |
|---|---:|---:|---:|
| proxy 1 KiB | 10,544 | 58,709 | 18.0% |
| proxy 64 KiB | 7,893 | 17,358 | 45.5% |
| proxy 1 MiB | 1,010 | 1,666 | 60.6% |
| www 1 KiB | 11,984 | 71,318 | 16.8% |
| www 64 KiB | 9,224 | 26,600 | 34.7% |
| www 1 MiB | 1,264 | 1,098 | 115.2% |
Ograniczenia porównania: ApacheBench może limitować klienta; wyniki nie reprezentują TLS/H2/H3 ani ruchu produkcyjnego. **Nie wolno tych RPS przypisywać nowej rewizji.**
## Zestaw testów dla nowej kompilacji
```bash
CARGO_INCREMENTAL=0 cargo test --locked --lib
CARGO_INCREMENTAL=0 cargo build --release --locked
python3 scripts/tests/cache_age_policy_034_model.py
python3 scripts/tests/cache_age_transform_034_live.py --binary target/release/rust-proxy --output /tmp/cache-test-034
python3 scripts/tests/cache_scope_034_static_test.py
python3 scripts/tests/proxy_hotpath_034_static_test.py
python3 scripts/tests/sendfile_034_static_test.py
```
Do testów jednostkowych dodano Age, transformowane CSS/HOT, democję do COLD i odczyt COLD po restarcie, logikę LRU/HIT oraz kontrolę zakazów `private`/`no-store`. Test lokalny sprawdza CSS/Brotli, origin count, API, keep-alive config i ukryte nagłówki. Test po kompilacji bez przełącznika `--expect-old-age-bug` ma wykazać `MISS → HIT` dla Age/Brotli. Na starej binarce odtwarzanie regresji przechodzi z `--expect-old-age-bug`.
Kontrole wykonane na źródłach: źródłowe testy cache scope, proxy hotpath, sendfile, składnia JS, parsowanie tłumaczeń. **Nie uruchomiono testów Rust ani benchmarków poprawionej binarki** — środowisko nie ma `cargo/rustc`.
## Następny pomiar po kompilacji
Porównać tę rewizję 0.34.0 z poprzednią 0.34.0 oraz nginx dla 1 KiB / 64 KiB / 1 MiB; osobno cache OFF, HOT HIT i transformacja/Brotli. Raportować RPS, p95/p99, procesor, RSS, nowe TCP, `cache.evictions` i `cache.purges`; nie modyfikować ustawień cache podczas benchmarku.
Uwaga: `curl -I -X GET` miesza zachowanie HEAD i GET; do testów reprezentacji zalecane `curl -sS -D - -o /dev/null URL`.
docs/ROADMAP_HISTORY.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Historical roadmap archive
> Archived during the RustProxy 0.25 documentation cleanup. This file preserves completed historical roadmap entries. The active roadmap is [`../ROADMAP.md`](../ROADMAP.md); unresolved work is tracked in [`TODO.md`](TODO.md).
> Archived during the RustProxy 0.25 documentation cleanup. This file preserves completed historical roadmap entries. The active roadmap is [`../ROADMAP.md`](../ROADMAP.md).
# RustProxy roadmap
docs/SECURITY_AUDIT_0.31.0.md
-148
View File
@@ -1,148 +0,0 @@
# RustProxy 0.31.0 - API, panel and HTTP cache-maintenance security review
Scope: source snapshot supplied for the v0.31.0 maintenance follow-up, reviewed
2026-10-10. This is a **code/contract review with offline regressions**, not a
penetration test or proof of production safety. The supplied archive has no
`.git` directory; the review does not establish parity with remote `master`.
## Interfaces and trust boundaries
| Surface | Exposure | Authentication and mutation rules | Assessment |
|---|---|---|---|
| Public administration bootstrap (`/`, login JS/CSS, `/api/auth/status`, setup/login POST) | Login/public administration listener | Host/origin validation; request marker for unsafe bootstrap actions; one-time setup capability and login throttling | Source-reviewed; live browser flow pending |
| Private panel HTML, JS, documentation, OpenAPI | Administration listener | Administrator session; `Cache-Control: no-store`; CSP, frame and MIME protections | Source and anonymous-response test reviewed; runtime pending |
| `/api/status`, `/api/system-info`, `/api/traffic`, `/api/cache`, logs, config, preferences, audit | Administration listener | Administrator cookie session; sensitive reads not on anonymous allowlist | Source and regression tests added; runtime pending |
| `/api/events`, live-log SSE | Administration listener | Session on subscribe and repeated validity checks; bounded viewers; heartbeat/disconnect cleanup | Source-reviewed; reconnect/load pending |
| State-changing admin APIs (including cache/config/preferences) | Administration listener | Valid session, Origin/Host, `X-RustProxy-Request: 1`, CSRF token; audit on cache commands | Source and regression tests added; runtime pending |
| Certificate API (`/api/certificates/{id}/reload`, deploy) | Administration listener | Narrow-scoped Key ID + Bearer grant; no general read access; HTTPS required remotely; origin validation when supplied | Source-reviewed; token rotation/revocation live tests pending |
| Public `BAN`/`PURGE` | Configured public vhost listener | **No admin session**: separate explicit IP/CIDR allowlist (empty by default), matched listener/vhost/ACL and optional rate limit | Hardened below; reverse/HTTP tests pending |
| Separate Prometheus metrics exporter | Distinct optional listener | Configured IP CIDRs and bearer-token policy; not the admin session | Unchanged; deployment exposure audit pending |
## Findings addressed in this follow-up
1. **E0716 / compilation defect (high functional impact).** `error()` accepts
`&'static str`; a reference to `format!(...)` was a temporary. Invalid
selector errors now log the detailed reason server-side and return a fixed,
non-sensitive message with status 422. This removes the reported lifetime
violation; the Rust compiler must still verify the entire crate.
2. **Cache protocol-scope confusion (medium).** Path-only `BAN`/`PURGE`
selectors could affect representations of the same host/path across both
HTTP and HTTPS. Targets are now built with the trusted external scheme,
validated to `http`/`https`, and the matched Host/vhost. IPv6 literal Hosts
are bracketed for the URL parser. Other tenants, schemes and hosts remain
outside the selector.
3. **Unbounded blocking maintenance work (medium availability risk).** Cache
invalidation may scan hot/cold entries. Both public HTTP and authenticated
admin cache purge/selective invalidation share two maintenance slots and
execute scans with `spawn_blocking`. Bursts receive 429 instead of piling
up unbounded blocking work on Tokio core threads. The permit stays with the
worker after a client disconnects. This is a concurrency bound, **not** a
per-client rate quota or a hard time limit for disk work.
4. **Ambiguous maintenance input (medium).** Duplicate TTL/confirmation or
`Content-Length` headers, transfer-encoded bodies, nonempty bodies and
upgrade requests are rejected; `Content-Length: 0` remains compatible with
CLI clients. TTL is limited to 1..604800 s. Broad `BAN /` requires the
explicit confirmation header; `PURGE` cannot use BAN-only controls.
5. **Audit ordering (low/medium).** Removal of an administrator-created BAN
rule now records the audit entry before mutating state. Unavailable audit
storage blocks the mutation rather than allowing an unaudited removal.
6. **Operator safety/usability (low).** An EN/PL help popover beside the HTTP
allowlist gives exact `curl` syntax, IP-only authentication semantics and
reverse-proxy identity requirements. No new global config variable is added.
## Existing protections checked in source
- Admin router's exact anonymous method/path allowlist contains only the
expected authentication bootstrap APIs; private docs and assets remain
session-protected. Request guard rejects duplicate Host/Origin and checks
browser `Sec-Fetch-Site` cross-site signals.
- Mutation guard uses an Origin/Host match, per-session CSRF comparison and
`X-RustProxy-Request` marker; cookie sessions are HttpOnly/SameSite=Strict
and use Secure under HTTPS/trusted forwarded protocol.
- Secure response policy includes CSP, `X-Frame-Options: DENY`,
`X-Content-Type-Options: nosniff`, referrer restrictions and no-store for
private material. A static scan checked 27 first-party browser JS files for
direct unsafe HTML injection and dynamic `eval`/`Function` sinks; this scan
**cannot establish that the entire DOM is XSS-free**.
- IP trust comes from the transport peer or a validated forwarding chain only
when the peer is configured trusted. An untrusted client-supplied XFF does
not independently grant BAN/PURGE permission. The cache allowlist rejects
`/0`, invalid CIDRs, duplicates and more than 32 entries. Requests are never
forwarded upstream.
- Certificate API tokens have explicit operation/ID grants and cannot use the
general admin read API; Bearer grant operations are intentionally separate
from cookie + CSRF authentication.
## HTTP maintenance CLI (allowlisted client only)
```sh
# One URL, including the query string: 200 with JSON result.
curl -i -X PURGE 'https://site.example/assets/app.js?v=3'
# Lazily ban previously cached objects under this path: 202 with JSON result.
curl -i -X BAN 'https://site.example/assets/'
# Entire host: explicit opt-in confirmation header.
curl -i -X BAN 'https://site.example/' -H 'X-RustProxy-Confirm-Wide: 1'
# Optional BAN expiration (30 seconds).
curl -i -X BAN 'https://site.example/assets/' -H 'X-RustProxy-Ban-Ttl: 30'
```
Use **Settings -> Cache -> HTTP BAN / PURGE IP allowlist** to grant the
caller IP/CIDR. This HTTP interface does not accept admin cookies/tokens as an
alternative to the allowlist. The domain, Host header, external scheme and
listener must select the intended vhost. Behind a reverse proxy, forward the
non-standard methods untouched and **replace**, rather than append untrusted,
XFF/Forwarded headers at the edge. Never add the entire reverse-proxy subnet
as a maintenance client merely because it is trusted for forwarding.
## Release security test plan (requires compiled RustProxy)
1. On the deployment host, run `cargo fmt --all -- --check`,
`./dev.sh check`, `./dev.sh test`, and `./dev.sh release`. Address any
compiler/regression errors before installation. `cargo audit` or an
equivalent advisory scan requires an up-to-date advisory DB and was not run
in this offline environment.
2. Start a disposable instance with a populated cache under both HTTP and
HTTPS for the same vhost/URL. Confirm `PURGE` returns 200 and drops the
exact query variant only; `BAN` returns 202 and invalidates only **older**
entries on the URL prefix. Confirm the *other* scheme and vhost stay hot.
3. From a denied IP, expect 403 on `BAN` and `PURGE`. With an empty allowlist,
all clients must be denied. Send duplicate TTL/confirmation/Content-Length,
TE/body and `BAN /` without confirmation: expect 400. On a trusted-proxy
hop with missing real client identity, expect 403. Test both IPv4 and IPv6.
4. From an allowed IP, send 10 concurrent broad maintenance requests; confirm
that only two workers run simultaneously and excess return 429. Verify
server CPU/IO and audit entries while the cold tier is populated.
5. Send GET requests without admin cookies to `/api/config`, `/api/status`,
`/api/system-info`, `/api/events`, `/api/logs`, `/docs/openapi.json` and
`/assets/app.js`; expect 401 and protective headers. With valid cookie but
missing/bad Origin, marker or CSRF on PUT/POST/DELETE, expect 403. Test
logout and password/session invalidation, live SSE reconnect/expiry.
6. Test the Certificate API with no grant, a valid certificate-specific grant,
revoked token, HTTP remote peer, and misuse against `/api/config`/logs.
Check errors for token disclosure and audit coverage.
7. Test actual browser startup, CSP console, output escaping and form save
with malicious HTML strings; inspect reverse-proxy Host/Origin forwarding.
Prefer HTTPS administration with a strict `allowed_hosts` list.
## Residual risk / non-claims
- **Not executed in this environment:** Cargo format/check/tests/release,
actual Rust listener requests, HTTP/2/3, fuzzing, penetration tests,
session revocation under load, proxy deployment configuration, dependency
CVE scanning and long-running cold-cache resource measurements.
- Proxy trust configuration is security-critical. A misconfigured edge that
accepts forged forwarded client addresses can let external users impersonate
an allowlisted IP. Restrict direct listener reachability and sanitize headers.
- A public IP allowlist is not user authentication; anyone using an allowed
IP/network can issue destructive maintenance commands. Use narrow CIDRs and
optionally enforce additional controls on the edge (mTLS, firewall).
- The two-worker concurrency bound mitigates bursts, not all denial-of-service
scenarios: configured cache size and storage latency still determine scan
cost. Per-IP quotas are not provided here.
- Proxy protocol path, certificate verification, arbitrary upstream code,
all application content and third-party packages are **not certified safe**
by this audit. Production acceptance remains open until the runtime test
plan is completed.
docs/SETTINGS.md
+1 -1
View File
@@ -350,7 +350,7 @@ The adjacent **info (`i`) popover** in Settings → Cache shows ready-to-copy
`curl -X PURGE` and `curl -X BAN` examples in English and Polish, including
the explicit `BAN /` confirmation. These public methods authorize by the
source IP/CIDR only; they do not accept a browser admin session as a bypass.
See [0.31.0 security audit](SECURITY_AUDIT_0.31.0.md) for reverse-proxy trust and tests.
See [security audit](SECURITY_AUDIT.md) for reverse-proxy trust and tests.
## Static-response transformation
docs/TEST_REPORT_0.34.0.md
-236
View File
@@ -1,236 +0,0 @@
# RustProxy 0.34.0 verification report
## Acceptance status
**Source candidate, not verified production release.** Rust (`cargo`, `rustc`) is
not installed in this execution environment, and the runtime has no access to
its dependency registry. Consequently the modified Rust source was **not
compiled**, the new Rust tests were **not executed**, and no 0.34.0 RustProxy
binary or live throughput figures are supplied. Source inspection and simulated
UI/cache tests do not close the runtime acceptance gate.
## DEV 1 KiB counters follow-up (source remains uncompiled)
The newly supplied binary `rust-proxy(1)` identifies as 0.34.0 and has
SHA-256 `1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa`.
It completed **151 passing real HTTP tests, 0 failures, 1 optional zstd
skip**. These tests were run on the supplied binary, not on the subsequently
modified source. The latest source moves per-domain cache statistics outside
HOT store mutexes and lazily allocates analytics samples only after a queue
permit is available. Histogram completion now increments one exclusive
latency bin instead of up to 11 cumulative bins; both Prometheus exporters
convert them to the same cumulative format at scrape time. Static
source/SQL/upgrade checks pass; **Rust compile,
Rust unit tests and live new-source RPS remain OPEN**. See
`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md` and the separate evidence archive.
## DEV 1 KiB reverse-proxy follow-up (October 2026; source only)
The supplied **old executable** passed loopback HTTP/1.1 1,040-byte content
checks and ApacheBench 12,000-request samples. The configuration trial
`max_idle_per_host=4 -> 64` increased short-run RPS, but shared-CPU noise
precludes a stable nginx-parity percentage. The edited sources now use 64 by
default for absent/new keepalive fields and reduce exclusive-Drop mutex/clone
and absent-cookie overhead. See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`.
**No newly modified binary was built or run.** Rust unit tests, runtime
regressions, long soaks and post-edit code A/B are still required.
Current DEV source checks performed on the modified tree:
| Check | Result | Scope |
|---|---|---|
| `./dev.sh doctor` | PASS | Source preflight; reports `cargo` and `rustc` absent |
| `scripts/tests/rps_hotpath_034_static_test.py` | PASS | Guardrails, defaults, retained routing/security and source-size limits only |
| `scripts/tests/proxy_hotpath_034_static_test.py` | PASS | Routing and telemetry source guards only |
| `scripts/tests/cache_scope_034_static_test.py`, `cache_hot_034_static_test.py`, `sendfile_034_static_test.py` | PASS | Cache and WWW source guards only |
| `scripts/tests/dashboard_presets_sql_test.py` | PASS | SQLite migrations and preset constraints |
| `scripts/tests/upgrade_performance_test.py` | PASS | Disposable upgrade/preservation/locking test |
| `scripts/tests/sendfile_034_panel_browser_test.py` | PASS, 2 widths | Simulated Chromium admin API, not live Rust |
| `scripts/tests/vhost_backend_popup_0337_browser_test.py` | PASS, 4 widths | Simulated Chromium admin API, not live Rust |
| Node `--check` (vhost editor), Python `compileall`, JSON parser (OpenAPI) | PASS | Syntax only |
| `./dev.sh check`, `./dev.sh test` | BLOCKED, exit 127 | Rust toolchain missing; no Rust unit/build acceptance |
| Old supplied binary, `ab`/nginx configuration A/B | COMPLETED, DIAGNOSTIC | 1,040-byte H1 payload; shared-CPU and no new code exercised |
## Later 0.34.0 reverse-proxy optimization revision
The refreshed source archive additionally reuses immutable route handles,
limits lazy host wildcard allocation, fast-paths one-backend non-cookie LB,
atomically caches healthy state and exchange scalar log fields, shares
compiled per-member health policies across leases, and avoids
unneeded fallback/retry header clones and empty-subscriber broadcast clones.
The existing cache/sendfile work and all panel controls remain. Added Rust
regression cases and the source-only hot-path guard are **not a substitute**
for compilation and live HTTP, health, capacity, sticky, and H2 retry tests.
No RPS improvement, corrected nginx gap, or full functional parity has been
measured for this new revision. Details: `docs/PROXY_HOTPATH_0.34.0.md`.
## RPS-focused follow-up source revision (not yet compiled)
See `docs/PROXY_RPS_0.34.0.md` for detailed changes in request IDs, validated
upstream URI construction, LB scheduling, probes, streaming byte accounting,
unchanged pool generation lifetimes and log-history ownership. Source-only invariants pass.
A live test of the PREVIOUS executable (not this source revision) completed
110 passing cases, 0 failing cases and 1 skipped optional zstd decoder in the
proxy/policy/cache/transform harness. The old executable also confirms query-
string bypass is deliberate and reproduces the older Age/TTL bug. The new
binary has NOT been built or tested and its RPS gain is unknown.
## HOT HIT follow-up source revision (not yet compiled)
See `docs/CACHE_HOTPATH_0.34.0.md`. Response header cloning has moved outside
the global HOT lock, and redundant bounded stats key allocation, empty BAN
checks and full HOT TTL sweeps were removed. A new Rust concurrency/regression
suite and an ignored microbenchmark have been added. Existing COLD and Vary
behavior, exact LRU and cache semantics remain. This source has not been built
or run under Rust in this environment; no RPS gain is claimed.
## Changes and correctness boundaries
- Operational per-vhost saves (upstream keepalive/recycling, connection limits,
load-balancing/health policy, timeouts, request gate and diagnostics) retain
HOT and COLD entries. Backend origin, routing, content, cache and transform
changes retain conservative invalidation. Vhost-specific changes purge only
that vhost; global cache, transform, forwarding, headers and Web Service or
PHP-FPM profile changes may conservatively purge all tiers.
- Scope-specific cold-cache records persist an origin/content compatibility
fingerprint. COLD startup rejects stale scope records and deletes their cache
artifacts. Legacy COLD metadata defaults its fingerprint to zero; production
vhosts have a nonzero SHA-derived fingerprint. A one-time cold refill can
occur on the 0.33.8 -> 0.34.0 upgrade because the global identity changed.
- A separate administrative configuration revision acts as a cache-ticket
publication barrier: a request bound to a pre-save routing snapshot cannot
create a new cache ticket after the save. The data revision stays stable for
operational-only saves, so previously cached objects remain available.
- Reverse-proxy response-body byte metering captures the backend lease once at
wrapper creation instead of acquiring a backend-selection mutex per DATA
frame. Request-body accounting, error paths and exchange ownership remain.
- An **opt-in `sendfile: auto`** setting is wired through Web Service serde,
compiled config, the Delivery UI and documented API. The Linux HTTP/1.1
TCP transport adapter calls `libc::sendfile` for eligible static files;
cache-enabled, ranged, compressed, transformed and TLS/H2/H3 paths retain
existing streaming. It is **disabled by default** and must pass native
compilation, correctness, transport and load tests before use. No persisted
SQLite schema migration or LB UI change is required.
## Local verification
| Check | Outcome | Coverage / caveat |
|---|---|---|
| `./dev.sh doctor` | PASS | Project source layout and tool inventory |
| SQL migration/preset test | PASS | All ten migrations re-applied, preset access and limits |
| Upgrade test | PASS | Stop/lock guards, SQLite/.env/language preservation, symlink rejection |
| Python compileall | PASS | Python syntax only |
| Vhost LB/Cache Chromium test | PASS | Simulated browser quick-view, 320/375/768/1440px |
| Other Chromium scripts | 15 PASS | Simulated API/CSS/JS, not a live Rust server |
| Sidebar navigation Chromium test | INCOMPLETE | Execution time budget; not counted as a pass |
| Three Node dashboard scripts | PASS | Simulated UI logic |
| `./dev.sh check` and `./dev.sh test` | BLOCKED | `cargo`/`rustc` unavailable |
| Rust unit/integration tests (including new scoped COLD regression) | NOT RUN | Requires Rust toolchain |
| Live HTTP/protocol/LB regressions | NOT RUN | Requires built 0.34.0 binary |
| 0.33.8 / 0.34.0 / nginx paired RPS benchmarks | NOT RUN | No verified 0.34.0 binary |
| Zero-copy sendfile WWW setting | SOURCE IMPLEMENTED / NOT RUNTIME VERIFIED | Linux plain H1 static GET only, `off` by default |
| Sendfile editor Chromium save/reload | PASS | 375/1440px, simulated admin API, off -> auto -> off |
| Sendfile Rust adapter + fallback test code | ADDED / NOT RUN | TCP marker replacement, persistent socket, HEAD, range, conditional; compiler unavailable |
| Amdahl sendfile/nginx sensitivity study | PASS MODEL | No runtime evidence for simulated improvements |
The 0.33.8 report in the supplied evidence archive recorded **706 passing
Rust tests**, **152 HTTP assertions**, **111 transport assertions** and
**17 LB assertions**. Those are *prior-version* results and are **not evidence
that 0.34.0 passes**. This report never transfers previous pass counts.
## Synthetic cache-retention scenario (not an RPS measurement)
Run:
```bash
python3 scripts/performance/simulate_cache_034.py
```
Assume 20 vhosts, 100 HOT + 100 COLD objects each, all public/cacheable with
infinite TTL and adequate cache capacity; 1,000 configuration saves (800
operational-only, 200 content changes to one vhost), followed by 500 uniformly
random GETs per save. A miss immediately reloads an object into its original
tier. No networking, filesystem, CPU timing, origin delay or concurrency is
modeled. Results with seed 3400:
| Synthetic metric | Old global purge | 0.34.0 scoped retention |
|---|---:|---:|
| Object invalidations | 473,540 | 37,300 |
| GET misses | 470,006 | 37,069 |
| GET hits | 29,994 | 462,931 |
| HIT ratio | 5.999% | 92.586% |
The modeled change yields **92.123% fewer invalidations**, **92.113% fewer
misses**, and **+86.587 percentage points** of synthetic cache HIT ratio.
**Measured 0.34.0 requests/s gain: unknown.** These values must not be
presented as a real speedup or a projection against nginx.
## Previously measured comparison (0.33.8, not this candidate)
From the attached 0.33.8 evidence, five paired benchmark runs with cache OFF,
median RPS:
| Mode / payload | 0.33.8 RPS | nginx RPS | 0.33.8 / nginx |
|---|---:|---:|---:|
| Reverse proxy 1,040 B | 29,334.12 | 170,610.16 | 17.19% |
| Reverse proxy 64 KiB | 20,735.07 | 35,393.29 | 58.58% |
| Reverse proxy 1 MiB | 4,461.19 | 2,730.23 | 163.40% |
| WWW 1,040 B | 24,906.20 | 351,605.13 | 7.08% |
| WWW 64 KiB | 20,856.67 | 203,709.91 | 10.24% |
| WWW 1 MiB | 2,833.71 | 12,611.49 | 22.47% |
The 0.33.8 1 MiB proxy path was **2.77% slower than rebuilt 0.33.7**.
Removing a per-response-frame mutex is a performance candidate but is **not**
a measured correction of that regression. No throughput gains are attributed
here to the new source changes.
## Hypothetical 0.34.0/nginx gap sensitivity (NOT MEASURED)
Run `python3 scripts/performance/simulate_nginx_gap_034.py`. Using the **real
0.33.8 measured medians** above, assume zero-copy removes 50% of the
read/copy part of per-request cost; that part hypothetically accounts for
20/40/60% of *total* cost for eligible static files. Model **1/3/5%** total
service-time savings from removed proxy-frame mutexes, independently.
Amdahl: `RPS_sim = RPS_0.33.8 / (1 - hypothetical_fraction_saved)`. The
result is a sensitivity calculation, **not a prediction or 0.34.0 benchmark**.
| Workload | Measured 0.33.8/nginx | Modeled low | Modeled middle | Modeled high |
|---|---:|---:|---:|---:|
| Proxy 1040 B | 17.19% | 17.37% | 17.73% | 18.10% |
| Proxy 64 KiB | 58.58% | 59.18% | 60.40% | 61.67% |
| Proxy 1 MiB | 163.40% | 165.05% | 168.45% | 172.00% |
| WWW 1040 B | 7.08% | 7.08% | 7.08% | 7.08% |
| WWW 64 KiB | 10.24% | 11.38% | 12.80% | 14.63% |
| WWW 1 MiB | 22.47% | 24.97% | 28.09% | 32.10% |
These ratios apply only to eligible Linux plain HTTP/1.1 static workloads
with cache OFF. TLS, HTTP/2, HTTP/3, ranged, compressed and object-cached
responses are **not** accelerated by this bridge. Bandwidth, contention,
queueing, realistic filesystem and actual CPU profiles are not modeled.
Even the favorable modeled 1 MiB case remains ~3.1x slower than nginx WWW;
small WWW remains ~14.1x slower. Proxy 1 MiB already exceeds nginx in the
**prior baseline**, not as a demonstrated new 0.34.0 improvement.
## Required rollout gates
1. On a trusted Linux build worker with the pinned Rust toolchain, run
`./dev.sh check`, `CARGO_INCREMENTAL=0 ./dev.sh test`, and a locked release
build. Resolve any compiler/test issues before a deployment.
2. Follow `scripts/performance/README.md` to run `regression.py`,
`transport_regression.py`, `lb_regression.py`, and
`concurrency_regression.py` against verified 0.33.8 and 0.34.0 binaries.
3. Exercise administration saves during simultaneous requests: save keepalive
values and confirm unchanged HOT/COLD entries and hashes; change one backend
origin and confirm only its scope is purged, both before/after restart. Check
that old snapshots cannot recreate invalidated objects.
4. Run the five-repeat, 15-second paired benchmark against 0.33.8, 0.34.0 and
nginx on the *same host and affinity*, including large proxy responses, CPU,
RSS, resource limits and response correctness.
5. Compile the source-level Linux `sendfile` bridge and validate on a running
RustProxy process: plain HTTP/1 sendfile path, headers/body exactness,
keepalive/pipelining, slow clients, cancellation, HEAD, range, cache,
compression, TLS/H2/H3 fallback, symlink security and observability. Do
**not** activate Auto on production before that gate passes.
6. Complete a mixed-traffic soak and staged rollback rehearsal. Until then,
do not describe 0.34.0 as production-accepted.
docs/TODO.md
-19
View File
@@ -1,19 +0,0 @@
# RustProxy TODO
This file contains open work that remains after the current RustProxy 0.25 Web Server implementation.
## Web Server follow-up
- Multipart byte-range responses if required; current single-range/If-Range support remains the default scope.
- Named locations only if they can reuse the existing vhost/location router rather than introduce a second router.
- Optional PHP `PATH_INFO` script/path splitting after a confinement-safe rule is defined.
- Additional FastCGI observability (bounded active/error/latency counters) without per-script labels.
- A race-resistant non-Unix filesystem confinement implementation before enabling the Web Server on non-Unix targets.
## Release acceptance
- Run `./dev.sh fmt`, `./dev.sh check` and `./dev.sh test` on a host with the stable Rust toolchain.
- Run live static-file acceptance over HTTP/1.1, HTTP/2 and HTTP/3, including ACL/auth/custom-error/header behavior.
- Run a real PHP-FPM acceptance matrix over a Unix socket and loopback TCP, including `index.php`, front-controller `try_files`, POST bodies, timeouts and source-disclosure failure tests.
Completed release work belongs in `CHANGELOG.md`; planned version work belongs in `ROADMAP.md`; implementation/audit evidence belongs in the matching audit document.
docs/VHOSTS.md
+1 -1
View File
@@ -234,7 +234,7 @@ origin do not expose a selected backend or stale origin timing.
| `pool_recycle_seconds` | 3600 | 0 (off) or 60..604800 |
| `http1_max_buffer_bytes` | 131072 | 8192..1048576 |
The new default of 64 applies when this field is absent in a newly created or deserialized vhost; existing saved values (including 4) remain unchanged. Each idle upstream connection consumes a file descriptor and memory, so use a smaller value for resource-constrained deployments with many origins. See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md` for the live configuration A/B evidence.
The new default of 64 applies when this field is absent in a newly created or deserialized vhost; existing saved values (including 4) remain unchanged. Each idle upstream connection consumes a file descriptor and memory, so use a smaller value for resource-constrained deployments with many origins.
These configure HTTP idle reuse and TCP keepalive; HTTP/2 also uses pings for active
connections, with a 10-second ping timeout and no pinging solely to keep idle
docs/WEB_SERVER.md
+1 -3
View File
@@ -134,9 +134,7 @@ does not change the listener protocol configuration.
tested against a running 0.34.0 RustProxy binary in the current environment.
The included Rust unit/integration test and real-network test cases must pass
on a Linux build worker before enabling in production. Sendfile cannot by
itself close the small-response gap to nginx. See
`docs/TEST_REPORT_0.34.0.md` and
`scripts/performance/simulate_nginx_gap_034.py`.
itself close the small-response gap to nginx. Live protocol and performance acceptance is still required.
### Intercept application HTTP errors (0.29.7)
scripts/performance/README.md
-354
View File
@@ -1,354 +0,0 @@
# Live performance and regression harness
## DEV OPT3 / 1 KiB response: uncompiled source
`docs/PROXY_1KB_OPT3_DEV_0.34.0.md` describes 7 optimized hot-path sites
and how to benchmark their **newly rebuilt** executable. Baseline
`rust-proxy(2)` passed 89 real proxy/policy/cache checks. This runtime result
is not evidence that uncompiled OPT3 changes work, nor a measured OPT3 RPS gain.
These scripts exercise actual RustProxy executables through loopback sockets.
They do not simulate Rust request handling. The Python server in `peer.py` is a
controllable test **upstream**, used to verify streaming, non-replayable writes,
headers, cache policy and WebSocket frames. Performance measurements use nginx
as the shared upstream instead of Python.
## DEV 1 KiB histogram writes (source candidate)
`VhostMetrics::complete` now writes one exclusive histogram bin for each
completed exchange instead of up to 11 cumulative atomics. Both Prometheus
exporters reconstruct `le` and `+Inf` at scrape time. Unit tests cover boundary
semantics. This change has not been compiled or benchmarked: see
`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`.
## DEV 1 KiB cache counter follow-up
See `docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`. Cache domain metrics no longer
lock the HOT store on BYPASS; analytics cloning is deferred until the queue
has capacity. The new **source** remains uncompiled in the editing environment.
For quick 1,040-byte loopback diagnostics with ApacheBench (not a production
acceptance benchmark), use `python3 scripts/performance/ab_1kb_dev.py --help`.
The authoritative long-run comparison remains `benchmark.py` with pinned
CPU sets, an explicitly equal upstream idle pool and multiple `wrk` repeats.
## DEV 1 KiB follow-up on 0.34.0 (newer source)
See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. The previous executable's 4-vs-64
idle-connection tuning experiment is real loopback evidence, but the *edited*
source has not been compiled. Its new default is 64 for new vhosts, whereas
explicitly saved `4` remains `4`. Keep the idle limit equal between before/after
*code* benchmarks; measure configuration tuning in a separate labelled series.
The source release is not eligible for a claim of 100k RPS or 80% nginx yet.
## Reverse-proxy optimization revision 0.34.0
After compiling the refreshed candidate, measure the same unmodified 0.33.8
binary and nginx reference as before, without turning off logging, request
IDs, ACL, cache-policy enforcement or LB health. Profile request-size 1,040 B
first. Before RPS runs execute full proxy, LB, transport and concurrency
regressions (including single-member capacity, recovery, sticky cookies,
replays on safe methods, rejected non-replayable writes and H2 connection
shutdown). Treat `--max-idle` tuning and recycles as separately labelled
experiments rather than mixing them into the primary comparison. There are no
new measured RPS figures in the source candidate.
## Follow-up RPS candidate on 0.34.0
`docs/PROXY_RPS_0.34.0.md` tracks the code changes and their regressions.
Compile before/after binaries from the two 0.34.0 source revisions and use
the paired benchmark below; a live run against the older binary cannot
establish a measured RPS improvement for the uncompiled candidate. Keep
`max_idle_per_host`, recycling, HTTP1/H2 settings and all application security
and logging features equal between runs. Treat any configuration tuning as a
separate experiment. Profile the resulting binary with `perf` only after
correctness checks pass.
## Offline cache-retention model for 0.34.0
`python3 scripts/performance/simulate_cache_034.py` compares cache invalidation
policies under a deterministic synthetic sequence of vhost saves and uniform
cacheable GETs. The model contains **no RustProxy server**, no timing and no
network traffic; its percentages are **not measured RPS gains**. See
`docs/TEST_REPORT_0.34.0.md` for assumptions and the live acceptance gap.
## Requirements
- Linux with readable `/proc`, Python 3.10+, executable RustProxy binaries.
- nginx and wrk for performance comparisons. Both paths are explicit; no
system-wide installation is required.
- `taskset` from util-linux for the benchmark's explicit CPU affinity and for
optional affinity settings in the other scripts.
- Optional Python `brotli` and `zstandard` decoders increase compression
acceptance coverage. A missing optional decoder is reported as **skipped**.
- Permission to bind unprivileged loopback sockets and start local processes.
In a sandbox without sockets or `/proc`, run in the approved local execution
environment. Do not replace real runtime results with a peer self-test.
Each run creates a new private temporary data directory and initializes its own
administrator with a random password supplied only over stdin. Configurations
are applied through RustProxy's existing authenticated CLI/API and listener
changes receive a restart. No historical SQLite database is copied or edited.
TLS keys, databases and passwords are never written to the evidence directory.
Temporary state is outside the evidence tree and is removed on normal exit.
`fixture-cleanup.json` records the cleanup check. Interrupted runs can leave an
unrelated `/tmp/rustproxy-perf-*` directory; do not include it in a release.
Before timed benchmark, soak or syscall-profile work, `wait_quiescent()` waits
for the existing authenticated ACME runtime API to report `ready` or
`unavailable`. A fresh RustProxy data directory can start a Python/Certbot
environment bootstrap even when no certificates have an issuer. The gate
requires no configured issuers, jobs or renewal observations, the same live Rust
PID and start time, and no child processes for five continuous seconds. It polls
every 0.5 seconds and fails after 180 seconds instead of measuring installation
work. Selected status, process count and timing evidence is saved in
`<label>-quiescence.json`. No PATH, pip settings or application features are
changed. An `unavailable` result permits the HTTP fixture to proceed; these runs
do not test ACME issuance or certificate-provider availability.
## Regression tests
Run from the repository root, providing absolute binary and output paths:
```bash
python3 scripts/performance/regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/regression
```
`--groups static,proxy,policy,cache,transform` selects groups; all are enabled by
default. `--workers 4` sets the same `TOKIO_WORKER_THREADS` for each binary.
The report contains individual pass/fail/skip observations and a comparison
matrix. An existing failure in the supplied binary remains a failure in its
report; it is not silently converted to a successful baseline expectation.
Coverage includes:
- Static GET/HEAD and byte checksums at 1,040 B, 64 KiB and 1 MiB; empty files;
ETag/Last-Modified conditional requests; prefix/suffix/open/multipart ranges,
If-Range and 416; directory redirect/index/listing; decoded path rejection;
deny/inside-root symlinks; precompressed gzip; MIME/download headers;
`try_files`, redirects and the compiled `.htaccess` header subset.
- Proxy POST/PUT, 1 MiB upload checksums, chunked upload delivery **before the
last chunk is sent**, early response delivery, client disconnect, original
status/redirect preservation, failed-POST no replay, request/response header
scopes, hop-by-hop stripping, request-ID shape/uniqueness and WebSocket echo.
- Global host/vhost/profile/location ACL composition, upstream suppression on
denial, trusted versus spoofed forwarding, untrusted country hints, method
and rate policies, hot changes and rejection of invalid configuration.
- Private administration routes, login, session/CSRF and Origin enforcement,
Basic Auth, ACL-before-auth, private-origin rejection, retained exchange
logging fields and byte accounting.
- Cache global/vhost/route OFF and ON, MISS/HIT, Vary isolation, cookie,
Authorization/private/no-store bypass, PURGE, concurrent fills, validator
revalidation, static cached ranges/conditionals; transforms OFF/ON with cache
OFF/ON; compression negotiation, exclusions and scope/global codec ceilings.
The generic suite does not establish TLS/H2/H3, Tinyauth, real PHP-FPM, real DNS
rebinding, concurrent filesystem rename-race safety or long-soak acceptance.
Use the dedicated transport tests and existing Rust integration suites for
their covered cases. The sequential symlink-target test is explicitly not a
concurrent rename-race proof.
## TLS, HTTP/2, HTTP/3 and WebSocket interoperability
```bash
python3 scripts/performance/transport_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/transport
```
This suite requires `aioquic`, `httpx[http2]` and `websockets`; use
`--python-deps /absolute/path/pydeps` for a separate dependency directory.
It verifies the temporary server certificate, negotiated protocol, response
status and bytes for TLS H2 and QUIC H3, including HEAD, validators, ranges,
cache ON/OFF, 1 MiB bodies and concurrent streams. Dedicated WebSocket checks
include bidirectional echo and a normal close handshake. Client protocol
self-checks are reported separately from RustProxy acceptance cases.
This establishes covered interoperability, not TLS/H2/H3 throughput or
production-certificate compatibility.
## Two-origin load-balancer regression
```bash
python3 scripts/performance/lb_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/lb
```
This separate, small suite sends real requests through RustProxy to two local
TCP origins. It verifies equal round-robin allocation, GET failover after a 503
and after a stopped origin refuses connections, and the absence of POST replay.
A controlled response pauses after its first body fragment while configuration
changes drain or disable its member. New requests must use the other member,
the admitted response must finish intact, and its active lease must remain
counted until completion and then return to zero. The main HTTP case matrix is
unchanged. This suite does not cover sticky sessions, active health probes,
DNS rotation or TLS upstream failures.
## Finite concurrency correctness stress
```bash
python3 scripts/performance/concurrency_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --out /absolute/path/results/concurrency
```
This separate functional check uses 1, 16 and 128 concurrent client tasks for
proxy and static responses at 1,040 B, 64 KiB and 1 MiB. Each finite batch sends
`max(32, 2 * clients)` requests and verifies every status, length and body SHA-256.
After the clients finish, backend active-exchange counters must settle to zero.
RSS and file descriptors are recorded before the batch and after its drain.
These are not peak resource measurements or throughput results. Client tasks
start behind a barrier; configured concurrency does not establish that the same
number of server exchanges were active simultaneously.
## Before/after/nginx benchmark
First verify starts, loopback routing and exact payload hashes without invoking
the load generator:
```bash
python3 scripts/performance/benchmark.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
--out /absolute/path/results/preflight --prepare-only
```
For a host with CPUs 0–8 available, a reproducible primary run is:
```bash
python3 scripts/performance/benchmark.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
--out /absolute/path/results/primary \
--workers 4 --origin-workers 2 --client-threads 4 \
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
--concurrency 64 --sizes small,large,huge --modes proxy,static \
--duration 15 --warmup 3 --repeats 5
```
Choose explicit, nonempty and disjoint CPU sets allowed in the run environment;
adapt all three sets on a smaller host. Frontends use the same CPU set and worker
count, the shared origin uses another set, and the generator uses a third. The
harness checks each process/thread's actual affinity and refuses unavailable
resource counters. Four wrk threads share the three client CPUs in this example.
The structured result records thread count and effective connection count; at
64 connections, four threads keep exactly 64 connections, avoiding wrk's integer
division of connections between threads. Only one frontend is loaded at a time.
Stop builds, other tests and profilers during throughput measurement.
The primary run takes approximately 27 minutes: six scenarios, three variants,
five samples, and 18 seconds per sample including excluded warm-up. To explore
concurrency, repeat in a fresh output directory with
`--concurrency 1,16,64,128`. More scenarios intentionally take more time.
Short sanity checks require the explicit `--exploratory` option and cannot
satisfy the >=5 samples / >=15-second measurement gate.
Cache is explicitly disabled for the primary benchmark. Transforms retain their
disabled defaults. Identity encoding is requested. Logging history, analytics,
request IDs, origin policy, body accounting, health/lease plumbing and normal
application functionality remain active. nginx uses `sendfile on`, HTTP/1.1,
keep-alive and streaming proxy delivery. The generated exact configurations are
saved, so this intentional difference in nginx/application features is visible.
No cache-disabled claim is inferred from a cache-on speedup.
Optional keep-alive tuning uses `--max-idle label=64`. Keep this as a separately
labelled experiment. The harness does not turn off bounded pool recycling.
## Recorded evidence and interpretation
- `schedule.json`: complete deterministic crossover order, rotating and
reversing variants between repetitions. No best-run selection.
- `preflight.json`, `fixture/payloads.json`, binary/version/config metadata:
exact status, body SHA-256, payload sizes and inputs.
- Per-sample exact status/body checks before and after the measured interval,
plus a wire-bytes-per-request sanity check, detect wrong content and gross
transfer-size anomalies. These do not hash every response inside wrk.
- `raw/*.txt` and `raw/*.json`: unmodified wrk output and structured requests,
duration, bytes, RPS, p50/p95/p99, effective concurrency, and
connect/read/write/timeout/status errors.
- `raw/*-resources.json`: timestamped `/proc` samples for each server process
and its children, plus the separate origin process group.
- `raw/*-affinity.json`: observed process/thread CPU masks for the front end,
origin and generator.
- `runs.csv/json`, `summary.csv/json`: all samples, medians, RPS range and
coefficient of variation, latency medians, server CPU time per completed
request, peak/end RSS and file descriptors.
- `status/*.json`: numeric drop and pool-generation/rotation counters read
outside each measured interval. Warm-up output is kept separately.
- `completion.json` and `SHA256SUMS`: completion state and hashes of evidence.
Any generator/network/status error stops the run with `failure.json`; partial
data remains available and is not presented as completed acceptance.
Use `(after_median / before_median - 1) * 100` for throughput improvement and
`after_median / nginx_median * 100` for relative nginx throughput. Rebuilding
both revisions with one compiler, target and resolved lockfile is required for
source-change attribution. Compare the supplied executable separately: build
flags and dependency versions can change its performance.
These are same-host, closed-loop HTTP/1.1 diagnostics. They do not establish
production capacity, TLS/H2/H3 performance, an open-loop latency SLA, allocator
rates, CPU cycles or 24-hour resource stability. wrk latency follows wrk's own
histogram/correction behavior. RSS and descriptor peaks are sampled observations,
so shorter spikes may be missed. Summed nginx RSS can double-count shared pages.
Origin or generator CPU saturation can limit observed throughput; inspect their
resource data and move the generator/origin to separate machines for release
measurements. Run profilers separately so profiling overhead does not enter the
headline A/B throughput numbers.
## Mixed-traffic resource soak
Run this in its own phase, after correctness and throughput checks:
```bash
python3 scripts/performance/soak.py \
--binary /absolute/path/rust-proxy-after --label after \
--nginx /absolute/path/nginx --out /absolute/path/results/soak \
--duration 1800 --rate 100 --clients 8 \
--workers 4 --origin-workers 2 --server-cpus 0-3 --origin-cpus 4-5 \
--toggle-seconds 120 --sample-seconds 5 --abort-every 101
```
The bounded client executor alternates proxy/static and 1,040 B/64 KiB/1 MiB
payloads, verifies full bytes and occasional HEAD/range semantics, closes some
clients after an initial body prefix, and applies live cache OFF/ON changes.
Every workload request opens a new frontend HTTP/1.1 connection; upstream
keepalive retains its configured defaults. This exercises connection churn.
The primary wrk benchmark separately uses persistent frontend connections.
The nominal request rate is paced; scheduler delays and actual completion rate
are reported instead of hiding an under-driven client. Pending jobs are bounded
to twice the client count. The script restores base configuration, drains and
rechecks both listeners before shutdown. After each cache apply, two separate
GETs per source verify exact contents and require a HIT on the second GET when
enabled; globally disabled cache must expose no X-Cache header. These probe
requests are recorded separately from paced traffic. Backend active-exchange
counters must settle to zero after the final drain and cooldown.
Five-second resource/status records, body/status failures, observed cache-status
counts, cache transitions/probes, RSS/FD trend and completed traffic totals are
retained. Controlled client closes mean that the client closes after reading a
prefix. The server may already have sent the complete response, so those counts
are not presented as proven server-side cancellations.
Memory trend is descriptive: switching caching and allocator retention affect
RSS. This is a 30-minute resource gate, not a 24-hour leak proof. A short
`--exploratory --duration ...` sanity run stays explicitly outside that gate.
## 0.34.0 hypothesis-only sendfile/nginx gap simulator
```bash
python3 scripts/performance/simulate_nginx_gap_034.py --output /tmp/nginx-gap.json
```
This is **not** a benchmark of RustProxy 0.34.0. It reproduces measured
0.33.8/nginx medians with cache OFF and applies explicitly hypothetical
Amdahl service-time reductions (WWW and reverse proxy separately). A real
RustProxy binary is required to validate the code path, detect regressions
and measure latency/RPS against nginx under identical conditions.
scripts/performance/ab_1kb_dev.py
-108
View File
@@ -1,108 +0,0 @@
#!/usr/bin/env python3
"""Short, same-host ab diagnostic for 1,040-byte HTTP/1 proxy responses.
Requires an existing executable and nginx; keeps the same explicitly supplied
idle-pool limit across variants. Not a substitute for wrk/CPU-isolated acceptance.
"""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import re
import subprocess
import sys
import time
from common import Fixture, http, write_json
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--binary', action='append', required=True, metavar='LABEL=/abs/path')
parser.add_argument('--nginx', default='/usr/sbin/nginx', type=Path)
parser.add_argument('--ab', default='/usr/bin/ab', type=Path)
parser.add_argument('--out', required=True, type=Path)
parser.add_argument('--concurrency', default='16,64')
parser.add_argument('--duration', type=int, default=4)
parser.add_argument('--repeats', type=int, default=2)
parser.add_argument('--workers', type=int, default=4)
parser.add_argument('--max-idle', type=int, default=64)
args = parser.parse_args()
binaries = dict(item.split('=', 1) for item in args.binary)
if len(binaries) != len(args.binary) or not all(Path(x).is_absolute() for x in binaries.values()):
parser.error('binary labels must be distinct and paths must be absolute')
if args.duration < 2 or args.repeats < 1:
parser.error('duration >=2 seconds and repeats >=1 required')
concurrency = [int(v) for v in args.concurrency.split(',')]
args.out.mkdir(parents=True, exist_ok=True)
records = []
write_json(args.out / 'metadata.json', {
'simulation': False, 'binary_labels': list(binaries), 'nginx': str(args.nginx),
'max_idle_for_every_rust_variant': args.max_idle, 'duration_s': args.duration,
'repeats': args.repeats, 'concurrency': concurrency, 'workers': args.workers,
'limitations': 'same-host shared CPU and closed-loop ab; not build-level before/after of uncompiled source'
})
with Fixture(args.out, workers=args.workers) as fixture:
nginx = fixture.nginx(args.nginx, frontend=True)
instances = {name: fixture.rust(Path(binary), name, max_idle=args.max_idle)
for name, binary in binaries.items()}
for name, instance in instances.items():
instance.login()
print('Waiting for quiescence:', name, flush=True)
instance.wait_quiescent(timeout=150, quiet_seconds=3)
sources = {**{name: r.reverse_port for name, r in instances.items()}, 'nginx': nginx['reverse_port']}
payload = (fixture.site / 'small.html').read_bytes()
for name, port in sources.items():
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
if code != 200 or body != payload:
raise AssertionError(f'{name} precheck failed: {code}, {len(body)} bytes')
write_json(args.out / 'payload-check.json', {
'status': 200, 'bytes': len(payload), 'sha256': hashlib.sha256(payload).hexdigest(),
'all_variants_verified': True
})
labels = list(sources)
for c in concurrency:
for repeat in range(args.repeats):
order = labels[repeat % len(labels):] + labels[:repeat % len(labels)]
if repeat % 2:
order.reverse()
for name in order:
port = sources[name]
cmd = [str(args.ab), '-k', '-t', str(args.duration), '-n', '1000000', '-c', str(c),
'-H', 'Accept-Encoding: identity', f'http://127.0.0.1:{port}/small.html']
prefix = f'c{c}-rep{repeat+1}-{name}'
subprocess.run([str(args.ab), '-k', '-n', '256', '-c', str(c),
f'http://127.0.0.1:{port}/small.html'],
capture_output=True, timeout=25, check=True)
p = subprocess.run(cmd, capture_output=True, text=True, timeout=args.duration + 45)
raw = p.stdout + '\n' + p.stderr
(args.out / f'{prefix}.txt').write_text(raw)
if p.returncode:
raise RuntimeError(f'{prefix} ab failed: {p.returncode} {p.stderr}')
def field(label):
m = re.search(r'^' + re.escape(label) + r':\s*(\d+(?:\.\d+)?)', raw, re.M)
return float(m.group(1)) if m else None
# ApacheBench tracks both network failures and bad body lengths.
if not re.search(r'^Failed requests:\s+0\s*$', raw, re.M):
raise RuntimeError(f'{prefix} failed requests: {raw[-1600:]}')
if not re.search(r'^Non-2xx responses:\s*0\s*$', raw, re.M) and 'Non-2xx responses:' in raw:
raise RuntimeError(f'{prefix} received non-2xx')
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
if code != 200 or body != payload:
raise AssertionError(f'{prefix} postcheck failed')
rps = field('Requests per second')
if rps is None:
raise RuntimeError(f'{prefix} missing RPS')
sample = {'variant': name, 'concurrency': c, 'repeat': repeat+1,
'rps': rps, 'completed': field('Complete requests'), 'failed': 0,
'expected_sha256': hashlib.sha256(payload).hexdigest(), 'status': 200}
records.append(sample)
write_json(args.out / 'samples.json', records)
print(sample, flush=True)
write_json(args.out / 'complete.json', {'complete': True, 'samples': len(records)})
return 0
if __name__ == '__main__':
sys.exit(main())
scripts/performance/benchmark.py
-479
View File
@@ -1,479 +0,0 @@
#!/usr/bin/env python3
"""Interleaved before/after/nginx process benchmarks with immutable raw evidence.
This is a same-host diagnostic experiment, not a simulation or release sign-off.
Runs >=5 measured samples per variant/scenario by default, each >=15 seconds.
"""
from __future__ import annotations
import argparse
import csv
import hashlib
import itertools
import json
import os
from pathlib import Path
import platform
import resource
import statistics
import subprocess
import sys
import time
import traceback
from common import Fixture, http, seal_results, sha256, write_json
from resources import (Monitor, affinity_snapshot, cgroup_cpu_delta, cgroup_cpu_stat,
parse_cpu_set, validate_cpu_sets, wait_for_affinity)
from telemetry import traffic_analytics_delta, traffic_analytics_snapshot
SIZES = {"small": "small.html", "large": "large.bin", "huge": "huge.bin"}
DROP_COUNTER_FIELDS = {
"logs_dropped": "history_dropped_delta",
"logging.sinks.queue_dropped": "logging_sinks_queue_dropped_delta",
"logging.error_file.queue_dropped": "logging_error_file_queue_dropped_delta",
"cache.cold_queue_dropped": "cache_cold_queue_dropped_delta",
}
DROP_INTERVAL_NOTE = ("Delta brackets measured wrk plus one postflight request and possible in-flight completions; "
"percentage uses wrk completed requests as an approximate denominator. "
"History rejection does not imply rejection by live broadcast or external sinks.")
def usage_snapshot():
usage = resource.getrusage(resource.RUSAGE_CHILDREN)
return usage.ru_utime + usage.ru_stime
def effective_threads(requested, concurrency):
if requested < 1 or concurrency < 1:
raise ValueError("client threads and concurrency must be positive")
# wrk floor-divides the connection count between threads. Select a divisor
# so its advertised and effective connection counts remain identical.
return next(value for value in range(min(requested, concurrency), 0, -1)
if concurrency % value == 0)
def wrk_run(args, port, path, duration, concurrency, prefix, roots=None, expected_processes=None):
threads = effective_threads(args.client_threads, concurrency)
command = [str(args.wrk), "--latency", "-t", str(threads),
"-c", str(concurrency), "-d", f"{duration}s", "--timeout", "5s",
"-H", "Accept-Encoding: identity", "-s", str(Path(__file__).with_name("wrk_report.lua")),
f"http://127.0.0.1:{port}/{path}"]
if args.client_cpus:
command = ["taskset", "-c", args.client_cpus, *command]
affinities = {}
expected_processes = expected_processes or {}
if roots:
for label, pid in roots.items():
expected = parse_cpu_set(args.server_cpus if label == "server" else args.origin_cpus)
affinities[label] = affinity_snapshot(pid, expected, expected_processes.get(label))
monitor = Monitor(roots, args.resource_interval, expected_processes) if roots else None
if monitor:
monitor.start()
cpu_start = usage_snapshot()
started = time.monotonic()
process = None
stdout, stderr = "", ""
try:
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
affinities["generator"] = wait_for_affinity(process, parse_cpu_set(args.client_cpus))
stdout, stderr = process.communicate(timeout=duration + 30)
finally:
if process is not None and process.poll() is None:
process.kill()
stdout, stderr = process.communicate(timeout=5)
if monitor:
monitor.stop()
prefix.parent.mkdir(parents=True, exist_ok=True)
prefix.with_suffix(".txt").write_text(stdout + "\n" + stderr)
write_json(prefix.with_name(prefix.name + "-affinity.json"), affinities)
if monitor:
write_json(prefix.with_name(prefix.name + "-resources.json"), monitor.samples)
if monitor.errors:
write_json(prefix.with_name(prefix.name + "-resource-errors.json"), monitor.errors)
cpu_seconds = usage_snapshot() - cpu_start
wall_seconds = time.monotonic() - started
raw = stdout + "\n" + stderr
if process.returncode:
raise RuntimeError(f"wrk exited {process.returncode}: {raw[-2000:]}")
line = next((line for line in stdout.splitlines() if line.startswith("RUSTPROXY_BENCH_JSON ")), None)
if line is None:
raise RuntimeError("wrk produced no structured report")
result = json.loads(line.removeprefix("RUSTPROXY_BENCH_JSON "))
if result["requests"] <= 0 or result["duration_us"] <= 0:
write_json(prefix.with_suffix(".json"), result)
raise RuntimeError("wrk completed no requests or reported an invalid duration; raw evidence retained")
result.update({"command": command, "wall_seconds": wall_seconds,
"client_threads": threads, "requested_connections": concurrency,
"client_threads_requested": args.client_threads,
"effective_connections": threads * (concurrency // threads),
"generator_cpu_seconds": cpu_seconds,
"rps": result["requests"] * 1_000_000 / result["duration_us"],
"error_total": sum(value for key, value in result.items() if key.startswith("errors_"))})
if monitor:
result["resources"] = monitor.summary(result["requests"])
write_json(prefix.with_suffix(".json"), result)
return result
def payload_check(port, path, expected):
"""A full exact-body check outside the wrk measurement interval."""
try:
status, headers, body = http(port, "/" + path, headers={"Accept-Encoding": "identity"})
return {"status": status, "bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
"expected_bytes": len(expected), "expected_sha256": hashlib.sha256(expected).hexdigest(),
"content_length": headers.get("content-length"), "content_encoding": headers.get("content-encoding"),
"approximate_header_bytes": 19 + sum(len(key) + len(value) + 4 for key, value in headers.items()),
"passed": status == 200 and body == expected}
except Exception as error:
return {"passed": False, "error": str(error), "expected_bytes": len(expected),
"expected_sha256": hashlib.sha256(expected).hexdigest()}
def transfer_sanity(result, payload_bytes, header_bytes, concurrency):
"""Check gross truncation/size anomalies; this is not per-response hashing."""
requests, wire_bytes = result["requests"], result["bytes"]
minimum = requests * payload_bytes
# wrk counts headers and partially received final responses in its wire bytes.
header_allowance = max(1024, 2 * header_bytes)
maximum = (requests + concurrency) * (payload_bytes + header_allowance)
return {"passed": requests > 0 and minimum <= wire_bytes <= maximum,
"wire_bytes_per_request": wire_bytes / requests if requests else None,
"expected_payload_bytes_per_request": payload_bytes,
"minimum_wire_bytes": minimum, "maximum_wire_bytes": maximum,
"header_allowance_per_response": header_allowance,
"inflight_response_allowance": concurrency}
def counter_values(value, prefix=""):
"""Retain numeric drop and rotation counters without session/config data."""
found = {}
if isinstance(value, dict):
for key, child in value.items():
path = f"{prefix}.{key}" if prefix else key
if isinstance(child, (int, float)) and any(part in key for part in ["dropped", "rotations", "generation", "requests_in_generation"]):
found[path] = child
found.update(counter_values(child, path))
elif isinstance(value, list):
for index, child in enumerate(value):
found.update(counter_values(child, f"{prefix}[{index}]"))
return found
def validate_drop_counters(values):
for name in DROP_COUNTER_FIELDS:
if type(values.get(name)) is not int or values[name] < 0:
raise RuntimeError(f"missing or invalid cumulative status counter: {name}")
return values
def status_counter_snapshot(rust):
if rust is None:
return None
status, _, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"{rust.label} status capture failed: HTTP {status}")
return counter_values(json.loads(body))
def drop_counter_report(before, after, requests):
if requests <= 0:
raise ValueError("drop-rate denominator must contain completed requests")
if (before is None) != (after is None):
raise RuntimeError("both status snapshots are required for a drop delta")
if before is not None:
validate_drop_counters(before)
validate_drop_counters(after)
deltas = {name: after[name] - before[name] if before is not None else None
for name in DROP_COUNTER_FIELDS}
if any(value is not None and value < 0 for value in deltas.values()):
raise RuntimeError("cumulative drop counter decreased during measured interval")
fields = {field: deltas[name] for name, field in DROP_COUNTER_FIELDS.items()}
fields["history_drop_percent_of_completed_requests"] = (
100 * deltas["logs_dropped"] / requests if before is not None else None)
return {"applicable": before is not None, "counter_deltas": deltas, "row_fields": fields,
"completed_requests_denominator": requests, "denominator_is_approximate": True,
"interval_note": DROP_INTERVAL_NOTE}
def analytics_counter_report(before, after, requests):
if requests <= 0 or (before is None) != (after is None):
raise ValueError("analytics delta requires paired snapshots and completed requests")
delta = traffic_analytics_delta(before, after) if before is not None else {
"queue_dropped_delta": None, "recorded_requests_delta": None, "interval_seconds": None}
fields = {"analytics_queue_dropped_delta": delta["queue_dropped_delta"],
"analytics_recorded_requests_delta": delta["recorded_requests_delta"],
"analytics_queue_drop_percent_of_completed_requests": (
100 * delta["queue_dropped_delta"] / requests if before is not None else None)}
return {"applicable": before is not None, **delta, "row_fields": fields,
"completed_requests_denominator": requests, "denominator_is_approximate": True,
"interval_note": DROP_INTERVAL_NOTE,
"recorded_requests_note": "Asynchronous analytics records; not HTTP success or completion counts."}
def persist_rows(out, rows):
write_json(out / "runs.json", rows)
if not rows:
return
with (out / "runs.csv").open("w", newline="") as stream:
writer = csv.DictWriter(stream, fieldnames=list(rows[0]))
writer.writeheader()
writer.writerows(rows)
groups = {}
for row in rows:
groups.setdefault((row["mode"], row["size"], row["concurrency"], row["variant"]), []).append(row)
summary = []
for (mode, size, concurrency, variant), group in sorted(groups.items()):
rps = [row["rps"] for row in group]
summary.append({"mode": mode, "size": size, "concurrency": concurrency, "variant": variant,
"samples": len(group), "rps_median": statistics.median(rps),
"rps_min": min(rps), "rps_max": max(rps),
"rps_cv_percent": statistics.stdev(rps) / statistics.mean(rps) * 100 if len(rps) > 1 else None,
"p50_us_median": statistics.median(row["p50_us"] for row in group),
"p95_us_median": statistics.median(row["p95_us"] for row in group),
"p99_us_median": statistics.median(row["p99_us"] for row in group),
"server_cpu_us_per_request_median": statistics.median(row["server_cpu_us_per_request"] for row in group),
"server_rss_peak_bytes": max(row["server_rss_peak_bytes"] for row in group),
"server_fd_peak": max(row["server_fd_peak"] for row in group),
"requests": sum(row["requests"] for row in group),
"cgroup_throttled_samples": sum(row["cgroup_nr_throttled_delta"] > 0 for row in group),
"cgroup_nr_throttled_delta": sum(row["cgroup_nr_throttled_delta"] for row in group),
"cgroup_throttled_usec_delta": sum(row["cgroup_throttled_usec_delta"] for row in group),
"validation_failures": sum(not (row["payload_before_passed"] and row["payload_after_passed"]
and row["transfer_sanity_passed"]) for row in group),
"errors": sum(row["error_total"] for row in group)})
write_json(out / "summary.json", summary)
with (out / "summary.csv").open("w", newline="") as stream:
writer = csv.DictWriter(stream, fieldnames=list(summary[0]))
writer.writeheader()
writer.writerows(summary)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--max-idle", action="append", default=[], metavar="LABEL=COUNT")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--wrk", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=4)
parser.add_argument("--client-threads", type=int, default=4)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--client-cpus")
parser.add_argument("--duration", type=int, default=15)
parser.add_argument("--warmup", type=int, default=3)
parser.add_argument("--repeats", type=int, default=5)
parser.add_argument("--concurrency", default="64")
parser.add_argument("--sizes", default="small,large")
parser.add_argument("--modes", default="proxy,static")
parser.add_argument("--resource-interval", type=float, default=.5)
parser.add_argument("--exploratory", action="store_true", help="explicitly allow <5 repetitions or <15s samples")
parser.add_argument("--prepare-only", action="store_true", help="start processes and verify hashes; run no load generator")
args = parser.parse_args()
if (args.duration < 15 or args.repeats < 5) and not args.exploratory:
parser.error("acceptance requires >=15-second runs and >=5 repetitions; use --exploratory for labelled short diagnostics")
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be a new or empty directory so raw evidence is never overwritten")
modes, sizes = args.modes.split(","), args.sizes.split(",")
concurrencies = [int(value) for value in args.concurrency.split(",")]
if any(mode not in ("proxy", "static") for mode in modes) or any(size not in SIZES for size in sizes):
parser.error("supported modes: proxy,static; sizes: small,large,huge")
if min(concurrencies) < 1 or max(concurrencies) > 4096:
parser.error("concurrency must be 1..4096")
if args.client_threads < 1:
parser.error("--client-threads must be positive")
if args.workers < 1 or args.origin_workers < 1:
parser.error("--workers and --origin-workers must be positive")
try:
cpu_sets = validate_cpu_sets({"server": args.server_cpus, "origin": args.origin_cpus,
"client": args.client_cpus}, set(os.sched_getaffinity(0)))
except ValueError as error:
parser.error(str(error))
if args.resource_interval <= 0:
parser.error("--resource-interval must be positive")
args.nginx, args.wrk, args.out = args.nginx.resolve(), args.wrk.resolve(), args.out.resolve()
variants = dict(item.split("=", 1) for item in args.binary)
if len(variants) != len(args.binary) or "nginx" in variants:
parser.error("variant names must be unique and must not be nginx")
max_idle = dict(item.split("=", 1) for item in args.max_idle)
args.out.mkdir(parents=True, exist_ok=True)
environment = {"kind": "same-host real HTTP/1.1 process benchmark", "simulation": False,
"exploratory": args.exploratory, "prepare_only": args.prepare_only,
"acceptance_sampling": args.duration >= 15 and args.repeats >= 5 and not args.prepare_only,
"platform": platform.platform(), "python": sys.version,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"cpu_affinity_available": sorted(os.sched_getaffinity(0)),
"cpu_sets": {label: sorted(cpus) for label, cpus in cpu_sets.items()},
"resource_process_discovery": "PPid map from /proc/*/stat; root and configured nginx process counts must be readable",
"expected_nginx_processes": {"frontend": args.workers + 1, "origin": args.origin_workers + 1},
"generator_plan": [{"connections": concurrency, "requested_threads": args.client_threads,
"effective_threads": effective_threads(args.client_threads, concurrency)}
for concurrency in concurrencies],
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
"wrk_sha256": sha256(args.wrk), "nginx_sha256": sha256(args.nginx),
"limitations": ["Generator, upstream and frontend share one host and kernel; not production capacity",
"Closed-loop wrk latency includes its histogram correction; no open-loop arrival-rate SLA claim",
"RSS sums processes and may double-count shared pages in nginx",
"RSS and FD peaks are sampled, not absolute instantaneous peaks",
"Exact HTTP 200 and payload hashes are checked before/after each measured run, not for every response during load",
"wrk status errors count only status >399; aggregate bytes/request is a gross sanity check, not response integrity proof",
"cgroup CPU-throttling deltas cover all processes in the shared cgroup, not only the measured server",
DROP_INTERVAL_NOTE,
"No TLS/H2/H3 speed inference; no CPU cycles/allocation profiler claim",
"nginx has a smaller feature set; request logging/ACL/analytics work is not feature-matched"]}
for path in ["/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max", "/proc/version"]:
try:
environment[path] = Path(path).read_text().strip()
except OSError:
environment[path] = None
write_json(args.out / "environment.json", environment)
expected_samples = len(modes) * len(sizes) * len(concurrencies) * args.repeats * (len(variants) + 1)
write_json(args.out / "completion.json", {"completed": False, "samples": 0,
"expected_samples": expected_samples,
"simulation": False, "prepare_only": args.prepare_only})
rows, error = [], None
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
nginx = fixture.nginx(args.nginx)
servers = {"nginx": {"process": nginx["process"], "proxy": nginx["reverse_port"],
"static": nginx["static_port"], "rust": None}}
for label, binary in variants.items():
rust = fixture.rust(Path(binary), label, max_idle=int(max_idle[label]) if label in max_idle else None)
rust.login()
servers[label] = {"process": rust.process, "proxy": rust.reverse_port, "static": rust.static_port, "rust": rust}
quiescence = {label: server["rust"].wait_quiescent()
for label, server in servers.items() if server["rust"] is not None}
write_json(args.out / "quiescence-preflight.json", quiescence)
initial_affinity = {"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], args.origin_workers + 1)}
initial_affinity.update({label: affinity_snapshot(server["process"].pid, cpu_sets["server"],
args.workers + 1 if label == "nginx" else 1)
for label, server in servers.items()})
write_json(args.out / "affinity-preflight.json", initial_affinity)
# The fixture has only master + configured workers. Refuse a
# master-only snapshot even if HTTP payload checks would succeed.
prepare_monitor = Monitor({"server": nginx["process"].pid, "origin": nginx["origin"].pid},
args.resource_interval,
{"server": args.workers + 1, "origin": args.origin_workers + 1})
prepare_monitor.sample()
write_json(args.out / "resources-preflight.json", prepare_monitor.samples)
telemetry_preflight = {label: {"status": status_counter_snapshot(server["rust"]),
"traffic": traffic_analytics_snapshot(server["rust"]) if server["rust"] else None}
for label, server in servers.items()}
write_json(args.out / "telemetry-preflight.json", telemetry_preflight)
for snapshot in telemetry_preflight.values():
if snapshot["status"] is not None:
validate_drop_counters(snapshot["status"])
preflight = []
for label, server in servers.items():
for mode, size in itertools.product(modes, sizes):
payload = (fixture.site / SIZES[size]).read_bytes()
check = payload_check(server[mode], SIZES[size], payload)
preflight.append({"variant": label, "mode": mode, "size": size, **check})
write_json(args.out / "preflight.json", preflight)
if not check["passed"]:
raise AssertionError(f"payload preflight failed: {label} {mode} {size}: {check}")
write_json(args.out / "preflight.json", preflight)
labels = [*variants, "nginx"]
schedule = []
for mode, size, concurrency in itertools.product(modes, sizes, concurrencies):
for repetition in range(args.repeats):
offset = repetition % len(labels)
order = labels[offset:] + labels[:offset]
if repetition % 2:
order.reverse()
for variant in order:
schedule.append({"sequence": len(schedule) + 1, "mode": mode, "size": size,
"concurrency": concurrency, "repetition": repetition + 1, "variant": variant})
write_json(args.out / "schedule.json", schedule)
if not args.prepare_only:
for item in schedule:
server = servers[item["variant"]]
expected_processes = {"origin": args.origin_workers + 1,
"server": args.workers + 1 if item["variant"] == "nginx" else 1}
stem = f"{item['sequence']:04}-{item['mode']}-{item['size']}-c{item['concurrency']}-{item['variant']}"
if args.warmup:
warmup = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.warmup,
item["concurrency"], args.out / "warmup" / stem)
if warmup["error_total"]:
raise RuntimeError(f"warmup errors in {stem}; raw evidence retained")
payload = (fixture.site / SIZES[item["size"]]).read_bytes()
before = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
write_json(args.out / "checks" / (stem + "-before.json"), before)
if not before["passed"]:
raise RuntimeError(f"exact payload check failed before {stem}: {before}")
counters_before = status_counter_snapshot(server["rust"])
write_json(args.out / "status" / (stem + "-before.json"), counters_before)
if counters_before is not None:
validate_drop_counters(counters_before)
analytics_before = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
write_json(args.out / "status" / (stem + "-traffic-before.json"), analytics_before)
cpu_before = cgroup_cpu_stat()
try:
result = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.duration,
item["concurrency"], args.out / "raw" / stem,
{"server": server["process"].pid, "origin": nginx["origin"].pid},
expected_processes)
finally:
cpu_after = cgroup_cpu_stat()
cpu_delta = cgroup_cpu_delta(cpu_before, cpu_after)
throttling = {"before": cpu_before, "after": cpu_after, "delta": cpu_delta}
write_json(args.out / "checks" / (stem + "-cgroup-cpu.json"), throttling)
after = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
write_json(args.out / "checks" / (stem + "-after.json"), after)
write_json(args.out / "checks" / (stem + "-affinity-after.json"), {
"server": affinity_snapshot(server["process"].pid, cpu_sets["server"], expected_processes.get("server")),
"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], expected_processes["origin"])})
counters_after = status_counter_snapshot(server["rust"])
write_json(args.out / "status" / (stem + ".json"), counters_after)
analytics_after = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
write_json(args.out / "status" / (stem + "-traffic-after.json"), analytics_after)
drops = drop_counter_report(counters_before, counters_after, result["requests"])
write_json(args.out / "status" / (stem + "-delta.json"), drops)
analytics = analytics_counter_report(analytics_before, analytics_after, result["requests"])
write_json(args.out / "status" / (stem + "-traffic-delta.json"), analytics)
sanity = transfer_sanity(result, len(payload), before["approximate_header_bytes"], item["concurrency"])
result.update({"payload_before": before, "payload_after": after, "transfer_sanity": sanity,
"cgroup_cpu": throttling, "observability": drops, "traffic_analytics": analytics})
write_json((args.out / "raw" / stem).with_suffix(".json"), result)
row = {**item, **{key: result[key] for key in ["requests", "bytes", "duration_us", "rps", "p50_us", "p95_us", "p99_us",
"client_threads_requested", "client_threads", "requested_connections", "effective_connections",
"error_total", "errors_connect", "errors_read", "errors_write", "errors_timeout", "errors_status", "generator_cpu_seconds"]}}
row.update({"payload_before_passed": before["passed"], "payload_after_passed": after["passed"],
"transfer_sanity_passed": sanity["passed"], "wire_bytes_per_request": sanity["wire_bytes_per_request"],
"cgroup_nr_throttled_delta": cpu_delta["nr_throttled"],
"cgroup_throttled_usec_delta": cpu_delta["throttled_usec"]})
row.update(drops["row_fields"])
row.update(analytics["row_fields"])
for group in ["server", "origin"]:
row.update({f"{group}_{key}": value for key, value in result["resources"][group].items()})
rows.append(row)
persist_rows(args.out, rows)
print(json.dumps({"sequence": item["sequence"], "total": len(schedule), "variant": item["variant"],
"scenario": f"{item['mode']}_{item['size']}_c{item['concurrency']}",
"rps": round(result["rps"], 2), "p95_ms": result["p95_us"] / 1000,
"errors": result["error_total"]}), flush=True)
if result["error_total"]:
raise RuntimeError(f"benchmark errors in {stem}; raw evidence retained")
if not after["passed"] or not sanity["passed"]:
raise RuntimeError(f"payload/byte-count validation failed in {stem}; raw evidence retained")
except (Exception, KeyboardInterrupt) as exc:
error = {"error": str(exc) or type(exc).__name__, "type": type(exc).__name__,
"interrupted": isinstance(exc, KeyboardInterrupt), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", error)
print(json.dumps(error), flush=True)
finally:
persist_rows(args.out, rows)
completed = error is None and (args.prepare_only or len(rows) == expected_samples)
write_json(args.out / "completion.json", {"completed": completed, "samples": len(rows),
"expected_samples": expected_samples,
"simulation": False, "prepare_only": args.prepare_only,
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
seal_results(args.out)
return (130 if error.get("interrupted") else 1) if error else (0 if completed else 1)
if __name__ == "__main__":
sys.exit(main())
scripts/performance/checks_cache.py
-180
View File
@@ -1,180 +0,0 @@
"""Cache and response-processing paths must survive disabled-path optimization."""
from concurrent.futures import ThreadPoolExecutor
import copy
import gzip
import io
import json
import time
import zlib
from common import http
def run(check, rust, peer):
port = rust.reverse_port
before = peer.count("GET", "/fresh")
results = [http(port, "/fresh") for _ in range(2)]
check("cache.global_off", all(s == 200 and h.get("x-cache") != "HIT" for s, h, b in results)
and peer.count("GET", "/fresh") - before == 2)
config = copy.deepcopy(rust.base)
config["cache"] = {"enabled": True, "expose_status": True, "expose_hits": True,
"expose_cache_status": True, "http_invalidation_allowed_cidrs": ["127.0.0.1/32"]}
config["web_services"][0]["cache_control"] = "public, max-age=60"
rust.apply(config, "cache-enabled")
first = http(port, "/fresh")
second = http(port, "/fresh")
check("cache.miss_then_hit", first[0] == 200 and first[1].get("x-cache") == "MISS"
and second[1].get("x-cache") == "HIT" and first[2] == second[2],
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
results = {}
for language in ["en", "pl", "en", "pl"]:
response = http(port, "/vary", headers={"Accept-Language": language})
previous = results.get(language)
if previous:
check("cache.vary_hit." + language, response[1].get("x-cache") == "HIT" and response[2] == previous[2],
cache=response[1].get("x-cache"))
results[language] = response
check("cache.vary_isolation", results["en"][2] != results["pl"][2])
for path in ["/private", "/no-store", "/cookie"]:
before = peer.count("GET", path)
first, second = http(port, path), http(port, path)
check("cache.nonpublic_bypass." + path, first[0] == second[0] == 200
and first[1].get("x-cache") != "HIT" and second[1].get("x-cache") != "HIT"
and peer.count("GET", path) - before == 2,
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
for name, headers in [("authorization", {"Authorization": "Bearer fixture-value"}),
("cookie", {"Cookie": "application_session=fixture"})]:
before = peer.count("GET", "/fresh")
s, h, b = http(port, "/fresh", headers=headers)
check("cache.request_bypass." + name, s == 200 and h.get("x-cache") != "HIT"
and peer.count("GET", "/fresh") - before == 1, cache=h.get("x-cache"))
before = peer.count("GET", "/fresh")
s, h, b = http(port, "/fresh", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
check("cache.nonempty_get_bypass_and_preserve", s == 200 and h.get("x-cache") != "HIT"
and h.get("x-received-body-bytes") == "17" and peer.count("GET", "/fresh") - before == 1,
cache=h.get("x-cache"), origin_body_bytes=h.get("x-received-body-bytes"))
s, h, b = http(port, "/fresh", "PURGE")
after = http(port, "/fresh")
check("cache.http_purge", s in (200, 202, 204) and after[1].get("x-cache") == "MISS", status=s)
with ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(lambda _: http(port, "/concurrent"), range(8)))
check("cache.concurrent_fill", all(item[0] == 200 for item in results)
and len({item[2] for item in results}) == 1, origin_requests=peer.count("GET", "/concurrent"))
first = http(port, "/revalidate")
time.sleep(1.2)
second = http(port, "/revalidate")
check("cache.expired_validator_revalidation", first[0] == second[0] == 200 and first[2] == second[2]
and peer.count("GET", "/revalidate") >= 2, cache=second[1].get("x-cache"))
first, second = http(rust.static_port), http(rust.static_port)
check("cache.static_hit", first[0] == second[0] == 200 and second[1].get("x-cache") == "HIT"
and first[2] == second[2], cache=second[1].get("x-cache"))
empty_first, empty_second = http(rust.static_port, "/empty.txt"), http(rust.static_port, "/empty.txt")
check("cache.static_empty_hit", empty_first[0] == empty_second[0] == 200
and empty_second[1].get("x-cache") == "HIT" and empty_second[2] == b"",
first=empty_first[1].get("x-cache"), second=empty_second[1].get("x-cache"))
hs, hh, hb = http(rust.static_port, "/download.txt", "HEAD")
gs, gh, gb = http(rust.static_port, "/download.txt")
check("cache.static_head_cannot_fill_empty_get", hs == gs == 200 and hb == b""
and gb == (rust.fixture.site / "download.txt").read_bytes(), head_status=hs, get_status=gs)
s, h, b = http(rust.static_port, headers={"Range": "bytes=0-9"})
check("cache.static_range_survives", s == 206 and b == first[2][:10], status=s)
s, h, b = http(rust.static_port, headers={"If-None-Match": first[1]["etag"]})
check("cache.static_conditional_survives", s == 304 and b == b"", status=s)
config["vhosts"][0]["cache"] = {"enabled": False}
rust.apply(config, "cache-vhost-disabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.vhost_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
and first[2] != second[2])
config["vhosts"][0]["cache"] = {"enabled": True}
config["vhosts"][0]["routes"] = [{"id": "fresh-no-cache", "path_prefix": "/fresh", "cache": False}]
rust.apply(config, "cache-route-disabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.route_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
and first[2] != second[2])
config["cache"]["enabled"] = False
config["vhosts"][0]["routes"] = []
rust.apply(config, "cache-global-disabled-vhost-enabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.vhost_cannot_override_global_off", first[2] != second[2]
and second[1].get("x-cache") != "HIT")
def transforms(check, rust, peer):
config = copy.deepcopy(rust.base)
original = (rust.fixture.site / "app.json").read_bytes()
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/app.json")
check("transform.off." + source, s == 200 and b == original, status=s)
config["transform"] = {"enabled": True, "json": True, "html": True}
for vhost in config["vhosts"]:
vhost["transform"] = {"enabled": True, "json": True, "html": True}
rust.apply(config, "transform-enabled-cache-off")
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/app.json")
check("transform.on." + source, s == 200 and json.loads(b) == json.loads(original)
and len(b) < len(original), original_bytes=len(original), result_bytes=len(b), status=s)
s, h, b = http(port, "/large.bin")
check("transform.binary_unchanged." + source, s == 200 and b == (rust.fixture.site / "large.bin").read_bytes())
s, h, b = http(port, "/app.json", headers={"Range": "bytes=0-9"})
check("transform.range_exclusion." + source, s == 206 and b == original[:10], status=s)
config["cache"] = {"enabled": True, "expose_status": True}
rust.apply(config, "transform-enabled-cache-on")
first, second = http(rust.reverse_port, "/app.json"), http(rust.reverse_port, "/app.json")
check("transform.cache_combination", second[0] == 200 and second[1].get("x-cache") == "HIT"
and first[2] == second[2] and json.loads(second[2]) == json.loads(original),
cache=second[1].get("x-cache"))
rust.reset()
expected = (rust.fixture.site / "compress.txt").read_bytes()
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.gzip." + source, s == 200 and h.get("content-encoding") == "gzip"
and gzip.decompress(b) == expected and len(b) < len(expected), bytes=len(b))
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip;q=0, identity"})
check("compression.identity_qvalue." + source, s == 200 and "content-encoding" not in h and b == expected)
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
check("compression.range_exclusion." + source, s == 206 and "content-encoding" not in h and b == expected[:10])
s, h, b = http(rust.reverse_port, "/no-transform", headers={"Accept-Encoding": "gzip"})
check("compression.no_transform_exclusion", s == 200 and "content-encoding" not in h)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["compression"] = {"enabled": False}
rust.apply(config, "compression-vhost-disabled")
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.vhost_off", s == 200 and "content-encoding" not in h and b == expected)
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
config["vhosts"][0]["compression"] = {"enabled": True, "gzip": True}
rust.apply(config, "compression-global-disabled")
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.global_codec_ceiling", s == 200 and "content-encoding" not in h and b == expected)
config = copy.deepcopy(rust.base)
config["compression"] = {"gzip": True, "br": True, "zstd": True, "deflate": True}
config["cache"] = {"enabled": True, "expose_status": True}
rust.apply(config, "compression-cache-enabled")
for codec in ["gzip", "deflate", "br", "zstd"]:
decoder = None
if codec == "gzip":
decoder = gzip.decompress
elif codec == "deflate":
decoder = zlib.decompress
elif codec == "br":
try:
import brotli
decoder = brotli.decompress
except ImportError:
pass
else:
try:
import zstandard
decoder = lambda data: zstandard.ZstdDecompressor().stream_reader(io.BytesIO(data)).read()
except ImportError:
pass
if decoder is None:
check("compression.codec." + codec, None, skip_reason="optional Python decoder is not installed")
continue
first = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
second = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
check("compression.codec." + codec, second[0] == 200 and second[1].get("content-encoding") == codec
and decoder(first[2]) == decoder(second[2]) == expected, cache=second[1].get("x-cache"))
scripts/performance/checks_policy.py
-184
View File
@@ -1,184 +0,0 @@
"""Live ACL, authentication, atomic reload and logging regressions."""
import base64
import copy
import json
import re
import time
from common import http, write_json
DENY = {"default": "deny", "rules": []}
ALLOW_LOCAL = {"default": "deny", "rules": [{"network": "127.0.0.1/32", "action": "allow"}]}
def run(check, rust, peer):
port = rust.reverse_port
config = copy.deepcopy(rust.base)
config["host_policies"] = [{"hosts": ["fixture.test"], "acl": DENY}]
rust.apply(config, "policy-global-host")
count = peer.count("GET", "/echo")
s, h, b = http(port, "/echo", headers={"Host": "fixture.test"})
check("acl.global_host_without_vhost_acl", s == 403 and peer.count("GET", "/echo") == count,
status=s, upstream_requests=peer.count("GET", "/echo") - count)
s, h, b = http(port, "/echo", headers={"Host": "localhost"})
check("acl.global_other_host_unchanged", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["acl"] = DENY
rust.apply(config, "policy-vhost-deny")
s, h, b = http(port)
check("acl.vhost_deny", s == 403, status=s)
s, h, b = http(rust.static_port)
check("acl.vhost_isolation", s == 200, status=s)
config["vhosts"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-vhost-allow")
s, h, b = http(port)
check("acl.hot_enable_allow", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["acl_profiles"] = [{"id": "restricted", "name": "Restricted", "acl": DENY}]
config["vhosts"][0]["acl_profile_id"] = "restricted"
config["vhosts"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-profile-deny")
s, h, b = http(port)
check("acl.profile_and_vhost_stack", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["routes"] = [{"id": "denied", "path_prefix": "/echo", "acl": DENY}]
rust.apply(config, "policy-location-deny")
count = peer.count("GET", "/echo")
s, h, b = http(port, "/echo")
check("acl.location_before_upstream", s == 403 and peer.count("GET", "/echo") == count, status=s)
s, h, b = http(port)
check("acl.location_other_path_unchanged", s == 200, status=s)
config["vhosts"][0]["acl"] = DENY
config["vhosts"][0]["routes"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-location-allow-vhost-deny")
s, h, b = http(port, "/echo")
check("acl.location_cannot_override_vhost_deny", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["acl"] = {"default": "deny", "rules": [{"network": "203.0.113.22/32", "action": "allow"}]}
rust.apply(config, "policy-untrusted-forwarded")
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
check("acl.spoofed_forwarded_ip_denied", s == 403, status=s)
config["forwarding"] = {"trusted_proxies": ["127.0.0.1/32"]}
rust.apply(config, "policy-trusted-forwarded")
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22"})
check("acl.trusted_forwarded_ip_allowed", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["rate_limit"] = {"country_access": {"mode": "allow", "countries": ["PL"]}}
config["vhosts"][0]["cloudflare_client_ip"] = True
rust.apply(config, "policy-country")
s, h, b = http(port, headers={"CF-IPCountry": "PL", "CF-Connecting-IP": "203.0.113.22"})
check("acl.untrusted_country_header_cannot_allow", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["method_policy"] = {"mode": "allow", "methods": ["GET", "HEAD"]}
rust.apply(config, "policy-method")
s, h, b = http(port, "/echo", "POST", body=b"denied method")
check("policy.method_allowlist", s in (403, 405), status=s)
s, h, b = http(port)
check("policy.allowed_method_survives", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["rate_limit"] = {"enabled": True, "requests": 2, "window_seconds": 60, "burst": 0}
rust.apply(config, "policy-rate-limit")
statuses = [http(port)[0] for _ in range(3)]
check("policy.rate_limit", statuses[:2] == [200, 200] and statuses[2] == 429, statuses=statuses)
rust.reset()
s, h, b = http(port)
check("policy.hot_disable_restores_request", s == 200, status=s)
# Every route check uses the real admin socket and no bearer/backdoor token.
for path in ["/api/config", "/api/status", "/api/logs", "/api/events", "/api/openapi.json", "/assets/app.js"]:
s, h, b = http(rust.admin_port, path)
check("admin.private." + path, s in (401, 403), status=s)
check("admin.login", rust.login() == 200)
s, h, b = rust.admin("/api/config")
envelope = json.loads(b)
check("admin.authenticated_config", s == 200 and "config" in envelope and "revision" in envelope, status=s)
s, h, b = rust.admin("/api/config", "PUT", {"expected_revision": envelope["revision"], "config": rust.base}, csrf=False)
check("admin.csrf_required", s == 403, status=s)
s, h, b = rust.admin("/api/config", origin="http://foreign.example")
check("admin.foreign_origin_denied", s == 403, status=s)
# Generate the disposable Basic Auth hash through the existing secured API.
s, h, b = rust.admin("/api/basic-auth/hash", "POST", {"username": "fixtureuser", "password": rust.password})
hashed = json.loads(b)
check("basic.hash_api", s == 200 and "password_hash" in hashed, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["basic_auth"] = {"enabled": True, "require_tls": False,
"users": [{"username": "fixtureuser", "password_hash": hashed["password_hash"]}]}
rust.apply(config, "policy-basic-auth")
s, h, b = http(port, "/echo")
check("basic.challenge", s == 401 and "Basic" in h.get("www-authenticate", ""), status=s)
credentials = base64.b64encode(("fixtureuser:" + rust.password).encode()).decode()
s, h, b = http(port, "/echo", headers={"Authorization": "Basic " + credentials})
check("basic.correct_password_and_strip", s == 200 and "authorization" not in json.loads(b)["headers"], status=s)
s, h, b = http(port, "/echo", headers={"Authorization": "Basic Zml4dHVyZXVzZXI6d3Jvbmc="})
check("basic.wrong_password_denied", s == 401, status=s)
config["vhosts"][0]["acl"] = DENY
rust.apply(config, "policy-acl-before-basic")
s, h, b = http(port, "/echo")
check("basic.acl_precedes_auth", s == 403, status=s)
rust.reset()
config = copy.deepcopy(rust.base)
config["vhosts"][0]["frontend"]["listener_ids"] = ["does-not-exist"]
rust.apply(config, "invalid-config-retains-snapshot", expected_success=False)
s, h, b = http(port)
check("reload.invalid_document_retains_previous_snapshot", s == 200, status=s)
s, h, b = http(rust.static_port)
check("reload.invalid_document_keeps_other_vhost", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["backends"][0]["allow_private"] = False
# Explicit private-address rejection is checked through apply, preserving the
# active configuration if compile-time validation rejects the backend.
rust.apply(config, "ssrf-private-origin-disallowed", expected_success=False)
s, h, b = http(port)
check("ssrf.rejected_origin_does_not_replace_snapshot", s == 200, status=s)
query = "/echo?observability=" + rust.label
s, h, b = http(port, query)
request_id = json.loads(b)["headers"]["x-request-id"]
log = None
for _ in range(20):
ls, lh, lb = rust.admin("/api/logs?vhost=reverse&limit=512")
logs = json.loads(lb)
log = next((entry for entry in logs.get("entries", []) if entry.get("request_id") == request_id), None)
if log:
break
time.sleep(.05)
check("observability.log_fields_and_body_bytes", log is not None and log.get("status") == 200
and log.get("response_bytes") == len(b) and log.get("request_bytes") == 0
and log.get("cache_effective_enabled") is False and log.get("transform_effective_enabled") is False
and bool(re.fullmatch("[0-9a-f]{64}", log.get("request_id", ""))),
recorded_status=log.get("status") if log else None,
recorded_response_bytes=log.get("response_bytes") if log else None, response_bytes=len(b))
if log:
write_json(rust.fixture.results / f"{rust.label}-sample-exchange.json", log)
config = copy.deepcopy(rust.base)
config["diagnostic_headers"] = {"request_id": True}
rust.apply(config, "observability-static-eos")
s, h, b = http(rust.static_port)
static_request_id = h.get("x-request-id")
static_log = None
for _ in range(20):
ls, lh, lb = rust.admin("/api/logs?vhost=static&limit=64")
static_log = next((entry for entry in json.loads(lb).get("entries", [])
if entry.get("request_id") == static_request_id), None)
if static_log:
break
time.sleep(.05)
check("observability.static_complete_after_content_length", s == 200 and static_log is not None
and static_log.get("outcome") == "complete" and static_log.get("response_bytes") == len(b),
outcome=static_log.get("outcome") if static_log else None,
recorded_response_bytes=static_log.get("response_bytes") if static_log else None,
response_bytes=len(b))
if static_log:
write_json(rust.fixture.results / f"{rust.label}-sample-static-exchange.json", static_log)
s, h, b = rust.admin("/api/status")
status = json.loads(b)
write_json(rust.fixture.results / f"{rust.label}-runtime-status.json", status)
check("observability.status_available", s == 200, status=s)
scripts/performance/checks_proxy.py
-135
View File
@@ -1,135 +0,0 @@
"""Proxy streaming, status, header, request-ID and WebSocket tests."""
import copy
import hashlib
from http.client import HTTPConnection, HTTPResponse
import json
import re
import socket
import time
from common import http
def run(check, rust, peer):
port = rust.reverse_port
for name in ["small.html", "large.bin", "huge.bin"]:
expected = (rust.fixture.site / name).read_bytes()
s, h, b = http(port, "/" + name)
check("proxy.get." + name, s == 200 and b == expected, status=s, bytes=len(b))
s, h, b = http(port, "/" + name, "HEAD")
check("proxy.head." + name, s == 200 and b == b"" and h.get("content-length") == str(len(expected)), status=s)
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-127"})
check("proxy.range", s == 206 and b == bytes(range(128)), status=s, bytes=len(b))
s, h, b = http(port)
s, h, b = http(port, headers={"If-None-Match": h["etag"]})
check("proxy.conditional304", s == 304 and b == b"", status=s)
for status in [201, 400, 404, 500, 503]:
s, h, b = http(port, f"/status/{status}")
check(f"proxy.origin_status.{status}", s == status and b == b"origin status payload\n"
and h.get("x-origin-fixture") == "yes", status=s)
s, h, b = http(port, "/redirect")
check("proxy.redirect", s == 302 and h.get("location") == "/small.html", status=s)
payload = bytes(range(256)) * 4096
ids = []
for method in ["POST", "PUT"]:
s, h, b = http(port, "/echo?query=preserved", method,
{"Content-Type": "application/octet-stream", "X-Request-ID": "untrusted-client-id"}, payload)
data = json.loads(b)
request_id = data["headers"].get("x-request-id", "")
ids.append(request_id)
check("proxy.upload." + method, s == 200 and data["method"] == method
and data["path"] == "/echo?query=preserved" and data["body_bytes"] == len(payload)
and data["body_sha256"] == hashlib.sha256(payload).hexdigest(), status=s, bytes=data["body_bytes"])
check("proxy.request_id." + method, bool(re.fullmatch("[0-9a-f]{64}", request_id)),
valid_64_hex=bool(re.fullmatch("[0-9a-f]{64}", request_id)))
check("proxy.request_id_unique", len(set(ids)) == 2)
s, h, b = http(port, "/echo", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
data = json.loads(b)
check("proxy.nonempty_get_body_preserved", s == 200 and data["body_bytes"] == 17
and data["body_sha256"] == hashlib.sha256(b"nonempty-get-body").hexdigest(), status=s)
peer.started_upload.clear()
first, last = b"first streamed chunk" * 64, b"final streamed chunk" * 64
with socket.create_connection(("127.0.0.1", port), timeout=5) as sock:
sock.sendall((f"POST /stream-upload HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\n"
"Transfer-Encoding: chunked\r\nContent-Type: application/octet-stream\r\n\r\n").encode())
sock.sendall(f"{len(first):x}\r\n".encode() + first + b"\r\n")
forwarded_before_end = peer.started_upload.wait(2)
sock.sendall(f"{len(last):x}\r\n".encode() + last + b"\r\n0\r\n\r\n")
response = HTTPResponse(sock)
response.begin()
data = json.loads(response.read())
check("proxy.chunked_streaming_upload", forwarded_before_end and response.status == 200
and data["body_sha256"] == hashlib.sha256(first + last).hexdigest(),
first_chunk_arrived_before_end=forwarded_before_end, status=response.status)
peer.release_stream.clear()
connection = HTTPConnection("127.0.0.1", port, timeout=3)
connection.request("GET", "/stream-download")
started = time.monotonic()
response = connection.getresponse()
first = response.read(5)
first_seconds = time.monotonic() - started
peer.release_stream.set()
last = response.read()
connection.close()
check("proxy.streaming_response", response.status == 200 and first == b"first" and last == b"last"
and first_seconds < 2, first_chunk_seconds=first_seconds)
peer.release_stream.clear()
connection = HTTPConnection("127.0.0.1", port, timeout=3)
connection.request("GET", "/stream-download")
response = connection.getresponse()
response.read(5)
connection.close()
peer.release_stream.set()
s, h, b = http(port)
check("proxy.client_disconnect_survives", s == 200, status=s)
before = peer.count("POST", "/no-replay")
s, h, b = http(port, "/no-replay", "POST", {"Content-Type": "application/octet-stream"}, b"write-once")
after = peer.count("POST", "/no-replay")
check("proxy.failed_post_not_replayed", s in (502, 503, 504) and after - before == 1,
status=s, upstream_writes=after - before)
config = copy.deepcopy(rust.base)
config["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Global", "value": "global"}],
"response": [{"op": "set", "name": "X-Fixture-Global", "value": "response"}]}
config["vhosts"][0]["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Vhost", "value": "vhost"}]}
config["vhosts"][0]["routes"] = [{"id": "echo", "path_prefix": "/echo", "headers": {
"request": [{"op": "set", "name": "X-Fixture-Route", "value": "route"}],
"response": [{"op": "set", "name": "X-Fixture-Route", "value": "response"}]}}]
rust.apply(config, "proxy-headers")
s, h, b = http(port, "/echo", headers={"Connection": "keep-alive, x-hop-fixture", "X-Hop-Fixture": "must-be-stripped"})
observed = json.loads(b)["headers"]
check("proxy.header_scopes", s == 200 and observed.get("x-fixture-global") == "global"
and observed.get("x-fixture-vhost") == "vhost" and observed.get("x-fixture-route") == "route"
and h.get("x-fixture-global") == "response" and h.get("x-fixture-route") == "response", status=s)
check("proxy.hop_by_hop_removed", "x-hop-fixture" not in observed)
s, h, b = http(port, "/echo", headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
observed = json.loads(b)["headers"]
check("proxy.untrusted_forwarded_ip", observed.get("x-real-ip") == "127.0.0.1"
and "203.0.113.22" not in observed.get("x-forwarded-for", ""),
observed_x_real_ip=observed.get("x-real-ip"), observed_x_forwarded_for=observed.get("x-forwarded-for"))
rust.reset()
with socket.create_connection(("127.0.0.1", port), timeout=4) as sock:
sock.sendall((f"GET /ws HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\nUpgrade: websocket\r\n"
"Connection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
"Sec-WebSocket-Version: 13\r\n\r\n").encode())
buffer = b""
while b"\r\n\r\n" not in buffer:
piece = sock.recv(4096)
if not piece:
break
buffer += piece
upgraded = buffer.startswith(b"HTTP/1.1 101")
echoed = b""
if upgraded:
message, mask = b"hello", b"\x11\x22\x33\x44"
sock.sendall(bytes([0x81, 0x80 | len(message)]) + mask +
bytes(c ^ mask[i % 4] for i, c in enumerate(message)))
frame = sock.recv(2)
if len(frame) == 2:
echoed = sock.recv(frame[1] & 127)
check("proxy.websocket_echo", upgraded and echoed == b"hello", upgraded=upgraded, bytes=len(echoed))
scripts/performance/checks_static.py
-132
View File
@@ -1,132 +0,0 @@
"""Live static-file regression cases against RustProxy's network listener."""
import copy
from email.parser import BytesParser
from email.policy import default
import gzip
from common import http
def run(check, rust, peer):
port, fixture = rust.static_port, rust.fixture
for name in ["small.html", "large.bin", "huge.bin", "empty.txt"]:
expected = (fixture.site / name).read_bytes()
status, headers, body = http(port, "/" + name)
check(f"static.get.{name}", status == 200 and body == expected,
status=status, bytes=len(body))
head_status, head_headers, head_body = http(port, "/" + name, "HEAD")
check(f"static.head.{name}", head_status == 200 and head_body == b""
and head_headers.get("content-length") == str(len(expected)),
status=head_status, content_length=head_headers.get("content-length"))
status, headers, body = http(port)
tag = headers.get("etag")
modified = headers.get("last-modified")
check("static.validators_present", bool(tag and modified), etag=tag, last_modified=modified)
for label, condition in [("etag", {"If-None-Match": tag}),
("modified", {"If-Modified-Since": modified})]:
s, h, b = http(port, headers=condition)
check("static.conditional304." + label, s == 304 and b == b"", status=s, bytes=len(b))
s, h, b = http(port, headers={"If-None-Match": '"missing"'})
check("static.conditional_miss", s == 200 and b == body, status=s)
data = (fixture.site / "large.bin").read_bytes()
cases = [("prefix", "bytes=0-127", data[:128], "bytes 0-127/65536"),
("suffix", "bytes=-37", data[-37:], "bytes 65499-65535/65536"),
("open", "bytes=65500-", data[65500:], "bytes 65500-65535/65536")]
for name, range_header, expected, content_range in cases:
s, h, b = http(port, "/large.bin", headers={"Range": range_header})
check("static.range." + name, s == 206 and b == expected
and h.get("content-range") == content_range,
status=s, bytes=len(b), content_range=h.get("content-range"))
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=999999-"})
check("static.range.unsatisfiable", s == 416 and h.get("content-range") == "bytes */65536",
status=s, content_range=h.get("content-range"))
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-15,100-115"})
mime = BytesParser(policy=default).parsebytes(
b"Content-Type: " + h.get("content-type", "").encode() + b"\r\n\r\n" + b)
parts = list(mime.iter_parts())
check("static.range.multipart", s == 206 and len(parts) == 2
and parts[0].get_payload(decode=True) == data[:16]
and parts[1].get_payload(decode=True) == data[100:116], status=s, parts=len(parts))
for name, value, expected_status in [("match", tag, 206), ("mismatch", '"other"', 200)]:
s, h, b = http(port, headers={"Range": "bytes=0-15", "If-Range": value})
expected = body[:16] if expected_status == 206 else body
check("static.if_range." + name, s == expected_status and b == expected,
status=s, bytes=len(b))
s, h, b = http(port, "/directory")
check("static.directory_slash", s in (301, 308) and h.get("location", "").endswith("/directory/"),
status=s, location=h.get("location"))
s, h, b = http(port, "/directory/")
check("static.directory_index", s == 200 and b == b"directory index\n", status=s)
s, h, b = http(port, "/listing/")
check("static.directory_listing_denied", s == 403, status=s)
for path in ["/escape-link.txt", "/inside-link.txt"]:
s, h, b = http(port, path)
check("static.symlink_deny." + path, s == 403 and b"MUST-NOT" not in b, status=s)
for path in ["/%2e%2e/outside-secret.txt", "/%2fetc/passwd", "/%252e%252e/outside-secret.txt",
"/%00", "/..%5coutside-secret.txt"]:
s, h, b = http(port, path)
check("static.invalid_path." + path, s in (400, 403, 404) and b"MUST-NOT" not in b,
status=s)
config = copy.deepcopy(rust.base)
service = config["web_services"][0]
service["symlink_policy"] = "inside_root"
service["directory_listing"] = True
rust.apply(config, "static-inside-root")
s, h, b = http(port, "/inside-link.txt")
check("static.symlink_inside_root_allowed", s == 200 and b == b"download fixture\n", status=s)
s, h, b = http(port, "/escape-link.txt")
check("static.symlink_inside_root_escape_denied", s == 403 and b"MUST-NOT" not in b, status=s)
s, h, b = http(port, "/listing/")
check("static.directory_listing_enabled", s == 200 and b"visible.txt" in b, status=s)
# Alternate a symlink target between valid and outside-root files. The file
# read must stay confined even when a pathname changes between requests.
link = fixture.site / "changing-link.txt"
race_statuses = []
escaped = False
for index in range(12):
if link.is_symlink():
link.unlink()
link.symlink_to(fixture.outside if index % 2 else fixture.site / "download.txt")
s, h, b = http(port, "/changing-link.txt")
race_statuses.append(s)
escaped = escaped or b"MUST-NOT" in b
check("static.changing_symlink_confinement", not escaped and all(s in (200, 403) for s in race_statuses),
statuses=race_statuses, note="Sequential target replacement; not a concurrent rename-race proof")
config = copy.deepcopy(rust.base)
config["web_services"][0]["precompressed"] = {"enabled": True, "gzip": True, "brotli": False}
rust.apply(config, "static-precompressed")
s, h, b = http(port, headers={"Accept-Encoding": "gzip"})
sidecar = (fixture.site / "small.html.gz").read_bytes()
check("static.precompressed_gzip", s == 200 and h.get("content-encoding") == "gzip"
and b == sidecar and gzip.decompress(b) == body, status=s, bytes=len(b))
s, h, b = http(port, headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
check("static.precompressed_range", s == 206 and b == sidecar[:10], status=s, bytes=len(b))
config = copy.deepcopy(rust.base)
config["web_services"][0]["try_files"] = ["$uri", "/small.html"]
config["web_services"][0]["rewrites"] = [{"pattern": "^/old$", "replacement": "/small.html", "status": 302}]
config["web_services"][0]["download_extensions"] = ["txt"]
config["web_services"][0]["mime_types"] = {"bin": "application/x-fixture"}
rust.apply(config, "static-routing")
s, h, b = http(port, "/spa/deep/link")
check("static.try_files_spa", s == 200 and b == body, status=s)
s, h, b = http(port, "/old")
check("static.rewrite_redirect", s == 302 and h.get("location") == "/small.html", status=s)
s, h, b = http(port, "/download.txt")
check("static.download_header", s == 200 and "attachment" in h.get("content-disposition", ""), status=s)
s, h, b = http(port, "/large.bin")
check("static.mime_override", s == 200 and h.get("content-type") == "application/x-fixture", status=s)
config = copy.deepcopy(rust.base)
(fixture.site / ".htaccess").write_text("Header set X-Fixture-Htaccess yes\nDirectoryIndex small.html\n")
config["web_services"][0]["htaccess"] = {"enabled": True, "strict": True}
rust.apply(config, "static-htaccess")
s, h, b = http(port)
check("static.htaccess_header", s == 200 and h.get("x-fixture-htaccess") == "yes", status=s)
s, h, b = http(port, "/.htaccess")
check("static.htaccess_not_served", s in (403, 404) and b"Header set" not in b, status=s)
scripts/performance/common.py
-448
View File
@@ -1,448 +0,0 @@
#!/usr/bin/env python3
"""Disposable, loopback-only fixtures for testing actual RustProxy processes.
Runtime databases and credentials live in TemporaryDirectory, never in results.
No pre-existing SQLite database, fixed port, or historical /mnt/data path is used.
"""
from __future__ import annotations
import copy
import gzip
import hashlib
from http.client import HTTPConnection, HTTPException
import json
import os
from pathlib import Path
import pwd
import secrets
import shutil
import signal
import socket
import subprocess
import tempfile
import time
def sha256(path: Path) -> str:
result = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
result.update(block)
return result.hexdigest()
def write_json(path: Path, value) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n")
def seal_results(directory: Path) -> None:
"""Checksums cover every regular evidence file, except the checksum manifest."""
paths = sorted(p for p in directory.rglob("*") if p.is_file()
and p.name != "SHA256SUMS")
(directory / "SHA256SUMS").write_text("".join(
f"{sha256(path)} {path.relative_to(directory)}\n" for path in paths))
def http(port: int, path="/small.html", method="GET", headers=None, body=None,
timeout=10, source_address=None):
connection = HTTPConnection("127.0.0.1", port, timeout=timeout,
source_address=source_address)
try:
connection.request(method, path, body, headers or {})
response = connection.getresponse()
payload = response.read()
return response.status, {k.lower(): v for k, v in response.getheaders()}, payload
finally:
connection.close()
def free_port() -> int:
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
def wait_ready(port: int, process: subprocess.Popen, timeout=30):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if process.poll() is not None:
raise RuntimeError(f"process {process.pid} exited with {process.returncode}")
try:
http(port, "/", timeout=.5)
return
except (OSError, HTTPException):
time.sleep(.05)
raise RuntimeError(f"loopback listener {port} did not start")
class Fixture:
def __init__(self, results: Path, workers=4, temp_parent: Path | None = None,
server_cpus=None, origin_cpus=None, origin_workers=None):
self.results = results.resolve()
self.results.mkdir(parents=True, exist_ok=True)
# Keep live state outside the evidence tree. Even an interrupted run or
# workspace synchronization must never put databases/keys in results.
self.temp = tempfile.TemporaryDirectory(prefix="rustproxy-perf-", dir=temp_parent)
self.root = Path(self.temp.name)
self.root.chmod(0o755) # nginx workers must traverse the public fixture root
self.site = self.root / "site"
self.site.mkdir(mode=0o755)
self.workers = workers
self.server_cpus = server_cpus
self.origin_cpus = origin_cpus
self.origin_workers = origin_workers or workers
self.children = []
self.handles = []
self.ports = set()
self.origin_port = self.port()
self.make_site()
def port(self):
while True:
port = free_port()
if port not in self.ports:
self.ports.add(port)
return port
def make_site(self):
# Exact, reproducible payload sizes match the original report's cases.
small = b"<!doctype html><html><body>RustProxy benchmark "
small += b"x" * (1040 - len(small) - len(b"</body></html>"))
small += b"</body></html>"
files = {"small.html": small,
"large.bin": bytes(range(256)) * 256,
"huge.bin": bytes(range(256)) * 4096,
"empty.txt": b"",
"app.json": (b'{\n "message": "stable fixture",\n "values": [1, 2, 3]\n}\n'),
"app.css": b"/* fixture comment */\nbody { color: red; padding: 0px; }\n",
"compress.txt": (b"compressible RustProxy text fixture. " * 4096),
"download.txt": b"download fixture\n"}
for name, body in files.items():
(self.site / name).write_bytes(body)
(self.site / "small.html.gz").write_bytes(gzip.compress(small, mtime=0))
(self.site / "directory").mkdir()
(self.site / "directory" / "index.html").write_bytes(b"directory index\n")
(self.site / "listing").mkdir()
(self.site / "listing" / "visible.txt").write_bytes(b"listing fixture\n")
(self.site / "inside-link.txt").symlink_to(self.site / "download.txt")
self.outside = self.root / "outside-secret.txt"
self.outside.write_bytes(b"MUST-NOT-BE-SERVED-OUTSIDE-ROOT\n")
(self.site / "escape-link.txt").symlink_to(self.outside)
write_json(self.results / "payloads.json", {
name: {"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest()}
for name, body in files.items()})
def spawn(self, argv, name, env=None, affinity=None):
log = (self.results / f"{name}.log").open("w")
self.handles.append(log)
if affinity:
argv = ["taskset", "-c", affinity, *argv]
child = subprocess.Popen([str(x) for x in argv], stdout=log,
stderr=subprocess.STDOUT, env=env, start_new_session=True)
self.children.append(child)
return child
@staticmethod
def stop(process):
if process.poll() is None:
os.killpg(process.pid, signal.SIGTERM)
try:
process.wait(timeout=12)
except subprocess.TimeoutExpired:
os.killpg(process.pid, signal.SIGKILL)
process.wait(timeout=3)
def close(self):
for process in reversed(self.children):
self.stop(process)
for handle in self.handles:
handle.close()
self.temp.cleanup()
write_json(self.results / "fixture-cleanup.json", {
"temporary_state_removed": not self.root.exists(),
"state_was_outside_results": not self.root.is_relative_to(self.results)})
if self.root.exists():
raise RuntimeError("temporary runtime state was not removed; do not package the fixture directory")
def __enter__(self):
return self
def __exit__(self, *_):
self.close()
def base_config(self, reverse_port, static_port):
endpoints = []
for label, port in [("reverse", reverse_port), ("static", static_port)]:
endpoints.append({"id": label, "name": label, "scope": "internal",
"enabled": True, "binding": {
"address": f"127.0.0.1:{port}", "tls": False,
"http2": False, "http3": None}})
return {
"cache": {"enabled": False},
"geoip_updates": {"enabled": False},
"listeners": {"endpoints": endpoints},
"web_services": [{"id": "fixture-static", "name": "Fixture static",
"document_root": str(self.site), "index": ["index.html", "small.html"],
"directory_listing": False}],
"vhosts": [{"id": "reverse", "frontend": {
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
"listener_ids": ["reverse"]},
"backends": [{"id": "origin", "url": f"http://127.0.0.1:{self.origin_port}",
"allow_private": True, "preserve_host": False}],
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []},
{"id": "static", "frontend": {
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
"listener_ids": ["static"]},
"source": {"type": "web_server", "web_service_id": "fixture-static"},
"routes": []}]
}
def rust(self, binary: Path, label: str, max_idle=None):
return RustInstance(self, binary.resolve(), label, max_idle=max_idle)
def nginx(self, executable: Path, frontend=True):
"""Use a separate nginx master for the shared origin and comparator."""
executable = executable.resolve()
result = subprocess.run([str(executable), "-V"], capture_output=True, text=True, timeout=10)
(self.results / "nginx-version.txt").write_text(result.stdout + result.stderr)
origin_conf = self.nginx_config("nginx-origin", f"""
server {{ listen 127.0.0.1:{self.origin_port}; server_name _;
root "{self.site}"; location / {{ add_header Cache-Control "public, max-age=60"; }}
}}
""", workers=self.origin_workers)
origin = self.spawn([executable, "-p", self.root, "-c", origin_conf], "nginx-origin",
affinity=self.origin_cpus)
wait_ready(self.origin_port, origin)
result = {"origin": origin, "origin_port": self.origin_port}
if frontend:
reverse_port, static_port = self.port(), self.port()
conf = self.nginx_config("nginx-frontend", f"""
upstream fixture_origin {{ server 127.0.0.1:{self.origin_port}; keepalive 64; }}
server {{ listen 127.0.0.1:{reverse_port}; server_name _;
location / {{ proxy_pass http://fixture_origin; proxy_http_version 1.1;
proxy_set_header Connection ""; proxy_buffering off; }}
}}
server {{ listen 127.0.0.1:{static_port}; server_name _; root "{self.site}"; }}
""")
process = self.spawn([executable, "-p", self.root, "-c", conf], "nginx-frontend",
affinity=self.server_cpus)
wait_ready(reverse_port, process)
wait_ready(static_port, process)
result.update({"process": process, "reverse_port": reverse_port,
"static_port": static_port})
return result
def nginx_config(self, name, servers, workers=None):
path = self.root / f"{name}.conf"
text = f"""daemon off;
master_process on;
user {pwd.getpwuid(os.geteuid()).pw_name};
worker_processes {workers or self.workers};
pid {self.root}/{name}.pid;
error_log stderr warn;
events {{ worker_connections 8192; }}
http {{
access_log off;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
keepalive_requests 1000000;
default_type application/octet-stream;
types {{ text/html html; text/css css; application/json json; text/plain txt; }}
client_body_temp_path {self.root}/{name}-body;
proxy_temp_path {self.root}/{name}-proxy;
{servers}
}}
"""
path.write_text(text)
(self.results / path.name).write_text(text)
return path
class RustInstance:
def __init__(self, fixture, binary, label, max_idle=None):
self.fixture, self.binary, self.label = fixture, binary, label
if not os.access(binary, os.X_OK):
target = fixture.root / f"{label}-rust-proxy"
shutil.copyfile(binary, target)
target.chmod(0o700)
self.binary = target
self.admin_port, self.reverse_port, self.static_port = (fixture.port() for _ in range(3))
self.data = fixture.root / f"{label}-data"
self.base = fixture.base_config(self.reverse_port, self.static_port)
if max_idle is not None:
self.base["vhosts"][0]["load_balancing"]["keepalive"] = {"max_idle_per_host": max_idle}
self.password = "Fixture!" + secrets.token_hex(20)
self.username = "performance"
self.env = os.environ.copy()
self.env["TOKIO_WORKER_THREADS"] = str(fixture.workers)
self.env["RUST_PROXY_PROXY_LISTEN"] = f"127.0.0.1:{self.reverse_port}"
self.env["RUST_PROXY_UPDATE_CHECK_INTERVAL_SECS"] = "0"
self.env.pop("RUST_PROXY_TLS_LISTEN", None)
self.applies = []
self.cli("admin", "init", "--username", self.username, "--password-stdin",
input=self.password + "\n")
self.start("bootstrap")
self.apply(self.base, "base")
fixture.stop(self.process)
self.start("runtime")
wait_ready(self.static_port, self.process)
version = self.cli("--version").stdout.strip()
write_json(fixture.results / f"{label}-binary.json", {
"label": label, "path": str(binary), "sha256": sha256(binary),
"version": version, "worker_threads": fixture.workers,
"pid": self.process.pid, "protocol": "HTTP/1.1 plaintext loopback",
"cpu_affinity": fixture.server_cpus,
"max_idle_per_host_override": max_idle})
def cli(self, *args, input=None, check=True):
result = subprocess.run([str(self.binary), "--data-dir", str(self.data),
"--admin-listen", f"127.0.0.1:{self.admin_port}", *args],
input=input, text=True, capture_output=True,
env=self.env, timeout=40)
if check and result.returncode:
raise RuntimeError(f"{self.label} CLI {' '.join(args[:2])}: "
f"{result.stdout[-2000:]} {result.stderr[-3000:]}")
return result
def start(self, phase):
self.process = self.fixture.spawn([
self.binary, "--data-dir", self.data, "--admin-listen", f"127.0.0.1:{self.admin_port}",
"--update-check-interval-secs", "0", "serve", "--proxy-listen",
f"127.0.0.1:{self.reverse_port}", "--max-connections", "4096", "--max-inflight", "4096"],
f"{self.label}-{phase}", env=self.env, affinity=self.fixture.server_cpus)
wait_ready(self.admin_port, self.process)
wait_ready(self.reverse_port, self.process)
def apply(self, config, name, expected_success=True):
config_path = self.fixture.root / f"{self.label}-{name}.json"
write_json(config_path, config)
result = self.cli("config", "apply", str(config_path), check=False)
# Configurations with ephemeral test passwords/hashes are intentionally not copied.
safe = copy.deepcopy(config)
for vhost in safe.get("vhosts", []):
if "basic_auth" in vhost:
vhost["basic_auth"]["users"] = "REDACTED fixture users"
write_json(self.fixture.results / "configs" / f"{self.label}-{name}.json", safe)
event = {"name": name, "sha256": sha256(config_path), "returncode": result.returncode,
"expected_success": expected_success, "stdout": result.stdout[-2000:],
"stderr": result.stderr[-3000:]}
self.applies.append(event)
write_json(self.fixture.results / f"{self.label}-config-applies.json", self.applies)
if (result.returncode == 0) != expected_success:
raise RuntimeError(f"config {name}: {result.stdout} {result.stderr}")
return result
def reset(self):
return self.apply(self.base, "restored-base")
def login(self):
status, headers, body = http(self.admin_port, "/api/auth/login", "POST",
{"Origin": f"http://127.0.0.1:{self.admin_port}", "Content-Type": "application/json",
"X-RustProxy-Request": "1"},
json.dumps({"username": self.username, "password": self.password}))
if status != 200:
raise RuntimeError(f"fixture login failed: HTTP {status}: {body[:200]!r}")
self.cookie = headers["set-cookie"].split(";", 1)[0]
self.csrf = json.loads(body)["csrf_token"]
return status
def wait_quiescent(self, timeout=180, quiet_seconds=5, poll_seconds=.5):
"""Wait for the real, one-time ACME bootstrap before any timed work.
This does not disable ACME or change the process environment. The
private fixture has no issuers/jobs; a terminal unavailable state is
valid for HTTP tests, but does not establish ACME issuance support.
"""
from resources import descendants, process_stats
if timeout <= 0 or quiet_seconds <= 0 or poll_seconds <= 0 or timeout <= quiet_seconds:
raise ValueError("quiescence timeout must exceed the positive quiet period")
started = time.monotonic()
deadline = started + timeout
output = self.fixture.results / f"{self.label}-quiescence.json"
report = {"completed": False, "label": self.label, "pid": self.process.pid,
"timeout_seconds": timeout, "required_quiet_seconds": quiet_seconds,
"poll_seconds": poll_seconds, "observations": [],
"acme_issuance_tested": False}
def read_admin(path):
remaining = deadline - time.monotonic()
if remaining <= 0:
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
if self.process.poll() is not None:
raise RuntimeError(f"{self.label} exited during quiescence")
status, _, body = self.admin(path, timeout=min(1, remaining))
if status != 200:
raise RuntimeError(f"{self.label} quiescence {path}: HTTP {status}")
return json.loads(body)
try:
root = process_stats(self.process.pid)
if root is None:
raise RuntimeError(f"{self.label} has no live root process")
identity = (root["pid"], root["start_ticks"])
report["start_ticks"] = root["start_ticks"]
envelope = read_admin("/api/config")
if not isinstance(envelope, dict) or not isinstance(envelope.get("config"), dict):
raise RuntimeError("quiescence requires the actual configuration envelope")
certificates = envelope["config"].get("tls", {}).get("certificates", [])
if not isinstance(certificates, list) or any(not isinstance(cert, dict) for cert in certificates):
raise RuntimeError("unexpected certificate configuration during quiescence")
report["configured_issuers"] = sum(cert.get("issuer") is not None for cert in certificates)
if report["configured_issuers"]:
raise RuntimeError("performance fixture must have no configured ACME issuers")
quiet_started, previous_terminal = None, None
while True:
payload = read_admin("/api/acme")
if not isinstance(payload, dict) or not {"runtime", "job", "renewals"} <= payload.keys():
raise RuntimeError("quiescence requires explicit ACME runtime/job/renewal state")
if payload["job"] is not None or payload["renewals"] != {}:
raise RuntimeError("performance fixture has an unexpected ACME job or renewal")
runtime = payload["runtime"]
if not isinstance(runtime, dict) or runtime.get("status") not in {
"checking", "initializing", "ready", "unavailable"}:
raise RuntimeError("unexpected ACME runtime state during quiescence")
pids = descendants(self.process.pid)
root = process_stats(self.process.pid)
if root is None or (root["pid"], root["start_ticks"]) != identity:
raise RuntimeError(f"{self.label} root process identity changed during quiescence")
now = time.monotonic()
if now >= deadline:
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
terminal = tuple(runtime.get(key) for key in ("status", "mode", "message"))
quiet = runtime["status"] in {"ready", "unavailable"} and pids == [self.process.pid]
if not quiet:
quiet_started, previous_terminal = None, None
elif quiet_started is None or terminal != previous_terminal:
quiet_started, previous_terminal = now, terminal
elapsed_quiet = now - quiet_started if quiet_started is not None else 0
report["observations"].append({
"elapsed_seconds": now - started, "process_count": len(pids),
"acme_status": runtime["status"], "acme_mode": runtime.get("mode"),
"acme_message": runtime.get("message"), "quiet_seconds": elapsed_quiet})
if elapsed_quiet >= quiet_seconds:
report.update({"completed": True, "elapsed_seconds": now - started,
"terminal_status": runtime["status"], "process_count": len(pids)})
write_json(output, report)
return report
write_json(output, report)
time.sleep(min(poll_seconds, max(0, deadline - time.monotonic())))
except (Exception, KeyboardInterrupt) as error:
report.update({"elapsed_seconds": time.monotonic() - started,
"error_type": type(error).__name__, "error": str(error)})
write_json(output, report)
raise
def admin(self, path, method="GET", body=None, csrf=True, cookie=True, origin=None,
timeout=10):
headers = {"Origin": origin or f"http://127.0.0.1:{self.admin_port}", "X-RustProxy-Request": "1"}
if cookie:
headers["Cookie"] = self.cookie
if csrf:
headers["X-CSRF-Token"] = self.csrf
if body is not None:
headers["Content-Type"] = "application/json"
body = json.dumps(body)
return http(self.admin_port, path, method, headers, body, timeout=timeout)
scripts/performance/concurrency_regression.py
-138
View File
@@ -1,138 +0,0 @@
#!/usr/bin/env python3
"""Finite live HTTP concurrency correctness stress; no throughput/latency claim."""
from __future__ import annotations
import argparse
from concurrent.futures import ThreadPoolExecutor
import hashlib
import json
from pathlib import Path
import threading
import time
import traceback
from common import Fixture, http, seal_results, sha256, write_json
from resources import process_stats
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
def active_backends(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
return {"error": f"HTTP {status}"}
values = []
def walk(value):
if isinstance(value, dict):
if "id" in value and "active" in value and "status" in value and "requests" in value:
values.append({key: value[key] for key in ("id", "active", "status", "requests")})
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return values
def batch(rust, port, name, expected, concurrency):
count = max(32, concurrency * 2)
expected_sha = hashlib.sha256(expected).hexdigest()
ready = threading.Barrier(concurrency, timeout=15)
lock = threading.Lock()
running = peak = 0
def fetch(index):
nonlocal running, peak
with lock:
running += 1
peak = max(peak, running)
try:
if index < concurrency:
ready.wait()
status, headers, body = http(port, "/" + name, timeout=15)
digest = hashlib.sha256(body).hexdigest()
return {"index": index, "status": status, "bytes": len(body), "sha256": digest,
"passed": status == 200 and len(body) == len(expected) and digest == expected_sha}
except Exception as error:
return {"index": index, "passed": False, "error": str(error)}
finally:
with lock:
running -= 1
before = process_stats(rust.process.pid)
started = time.monotonic()
with ThreadPoolExecutor(max_workers=concurrency) as executor:
responses = list(executor.map(fetch, range(count)))
elapsed = time.monotonic() - started
# Completion means every client consumed or failed its full response. Give
# asynchronous server bookkeeping a bounded interval to release its leases.
for _ in range(40):
active = active_backends(rust)
if isinstance(active, list) and active and all(row["active"] == 0 for row in active):
break
time.sleep(.05)
after = process_stats(rust.process.pid)
return {"configured_concurrent_clients": concurrency, "peak_running_client_tasks": peak,
"client_tasks_include_barrier": True, "requests": count, "elapsed_seconds": elapsed,
"expected_bytes": len(expected), "expected_sha256": expected_sha,
"response_failures": sum(not response["passed"] for response in responses),
"passed": all(response["passed"] for response in responses)
and isinstance(active, list) and bool(active) and all(row["active"] == 0 for row in active),
"responses": responses, "active_backends_after_drain": active,
"server_before": before, "server_after_drain": after}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be new or empty")
reports = {}
for specification in args.binary:
label, binary = specification.split("=", 1)
cases, error = [], None
try:
with Fixture(args.out / label, workers=4, origin_workers=2) as fixture:
fixture.nginx(args.nginx, frontend=False)
rust = fixture.rust(Path(binary), label)
rust.login()
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
for name in PAYLOADS:
expected = (fixture.site / name).read_bytes()
for concurrency in [1, 16, 128]:
result = batch(rust, port, name, expected, concurrency)
case = {"case": f"concurrency.{source}.{name}.c{concurrency}", **result}
cases.append(case)
write_json(args.out / label / "cases" / f"{source}-{name}-c{concurrency}.json", case)
print(f"{label} {'PASS' if case['passed'] else 'FAIL'} {case['case']}", flush=True)
if rust.process.poll() is not None:
raise RuntimeError("RustProxy exited during concurrency regression")
except Exception as exc:
error = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / label / "failure.json", error)
report = {"kind": "live HTTP/1.1 finite concurrency functional stress", "simulation": False,
"binary_sha256": sha256(Path(binary)), "throughput_measurement": False,
"passed_cases": sum(case["passed"] for case in cases),
"failed_cases": sum(not case["passed"] for case in cases),
"requests": sum(case["requests"] for case in cases),
"response_failures": sum(case["response_failures"] for case in cases),
"completed": error is None and len(cases) == 18,
"cases": [{key: case[key] for key in ("case", "passed", "requests", "response_failures")} for case in cases],
"limitations": ["configured client concurrency is not proof of simultaneous active server exchanges",
"Python client is not a throughput or latency benchmark",
"FD/RSS are before/after snapshots, not peaks or a long-term leak test"]}
reports[label] = report
write_json(args.out / label / "report.json", report)
write_json(args.out / "comparison.json", reports)
seal_results(args.out)
return 0 if all(report["completed"] and not report["failed_cases"] for report in reports.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
scripts/performance/lb_regression.py
-223
View File
@@ -1,223 +0,0 @@
#!/usr/bin/env python3
"""Small live two-origin failover and active-exchange regression, not a benchmark."""
from __future__ import annotations
import argparse
from collections import Counter
import copy
import hashlib
from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
from pathlib import Path
import threading
import time
import traceback
from common import Fixture, http, seal_results, write_json
class Origin:
def __init__(self, port, name):
self.name, self.port = name, port
self.fail_get = False
self.counts, self.post_hashes = Counter(), []
self.release, self.lock = threading.Event(), threading.Lock()
origin = self
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def log_message(self, *_):
pass
def do_GET(self):
self.respond()
def do_POST(self):
self.respond()
def respond(self):
body = self.rfile.read(int(self.headers.get("Content-Length", "0")))
with origin.lock:
origin.counts[(self.command, self.path)] += 1
if self.command == "POST":
origin.post_hashes.append(hashlib.sha256(body).hexdigest())
hold = self.path == "/hold"
status = 503 if self.command == "POST" or origin.fail_get else 200
payload = b"firstlast" if hold else origin.name.encode()
try:
self.send_response(status)
self.send_header("Content-Length", str(len(payload)))
self.send_header("Content-Type", "application/octet-stream")
self.send_header("Cache-Control", "no-store")
# Fresh connections make refused-listener tests independent
# of already accepted upstream keepalive connections.
self.send_header("Connection", "close")
self.end_headers()
if hold:
self.wfile.write(payload[:5])
self.wfile.flush()
origin.release.wait(15)
self.wfile.write(payload[5:])
else:
self.wfile.write(payload)
self.wfile.flush()
except (BrokenPipeError, ConnectionResetError):
pass
self.close_connection = True
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
self.closed = False
def close(self):
self.release.set()
if not self.closed:
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=3)
self.closed = True
def backend_views(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"status unavailable: {status}")
found = {}
def walk(value):
if isinstance(value, dict):
if value.get("id") in ("a", "b") and "active" in value and "status" in value:
found[value["id"]] = {key: value[key] for key in value
if key in ("id", "status", "active", "requests", "errors", "bytes_out")
or key.startswith("retry_") or key.startswith("connect_errors")}
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return found
def run(check, fixture, binary, label):
a, b = Origin(fixture.origin_port, "a"), Origin(fixture.port(), "b")
try:
rust = fixture.rust(binary, label)
rust.login()
config = copy.deepcopy(rust.base)
vhost = config["vhosts"][0]
vhost["backends"] = [{"id": origin.name, "url": f"http://127.0.0.1:{origin.port}",
"allow_private": True, "preserve_host": False} for origin in [a, b]]
vhost["load_balancing"] = {"algorithm": "round_robin", "health_check": {"mode": "off"},
"slow_start_seconds": 0, "passive_health": {"enabled": False},
"retry": {"max_retries": 1, "retry_statuses": [503]}}
vhost["routes"] = [{"id": "hold-a", "path_prefix": "/hold", "backend_id": "a"}]
rust.apply(config, "lb-two-origins")
replies = [http(rust.reverse_port, "/rr") for _ in range(8)]
counts = Counter(body.decode() for status, _, body in replies if status == 200)
check("lb.round_robin_both_origins", counts == {"a": 4, "b": 4}, counts=dict(counts))
a.fail_get = True
replies = [http(rust.reverse_port, "/retry-status") for _ in range(6)]
check("lb.get_503_failover", all(status == 200 and body == b"b" for status, _, body in replies)
and a.counts[("GET", "/retry-status")] > 0,
failed_origin_attempts=a.counts[("GET", "/retry-status")])
views = backend_views(rust)
check("lb.retry_observations_recorded", sum(view.get("retry_attempts_total", 0) for view in views.values()) > 0,
backends=views)
a.fail_get = False
payload = b"one POST must be attempted once"
replies = [http(rust.reverse_port, "/post-no-replay", "POST", body=payload) for _ in range(4)]
attempts = sum(origin.counts[("POST", "/post-no-replay")] for origin in [a, b])
check("lb.post_503_never_replayed", all(status == 503 for status, _, _ in replies) and attempts == 4,
client_requests=4, origin_attempts=attempts)
check("lb.post_body_preserved", len(a.post_hashes + b.post_hashes) == 4 and
all(digest == hashlib.sha256(payload).hexdigest() for digest in a.post_hashes + b.post_hashes))
for mode in ["drain", "disable"]:
vhost["backends"][0].update({"enabled": True, "drain": False})
rust.apply(config, f"lb-{mode}-before")
a.release.clear()
connection = HTTPConnection("127.0.0.1", rust.reverse_port, timeout=10)
try:
connection.request("GET", "/hold")
response = connection.getresponse()
first = response.read(5)
before = backend_views(rust)
check(f"lb.{mode}.lease_spans_response_body", response.status == 200 and first == b"first"
and before.get("a", {}).get("active") == 1, backends=before)
vhost["backends"][0].update({"drain": mode == "drain", "enabled": mode != "disable"})
rust.apply(config, f"lb-{mode}-active")
replies = [http(rust.reverse_port, "/new") for _ in range(4)]
check(f"lb.{mode}.new_requests_use_healthy_member",
all(status == 200 and body == b"b" for status, _, body in replies))
during = backend_views(rust)
check(f"lb.{mode}.active_lease_survives_hot_edit", during.get("a", {}).get("active") == 1,
backends=during)
a.release.set()
last = response.read()
check(f"lb.{mode}.admitted_body_finishes", first + last == b"firstlast")
for _ in range(20):
after = backend_views(rust)
if after.get("a", {}).get("active") == 0:
break
time.sleep(.05)
check(f"lb.{mode}.lease_released_after_body", after.get("a", {}).get("active") == 0,
backends=after)
finally:
a.release.set()
connection.close()
vhost["backends"][0].update({"enabled": True, "drain": False})
rust.apply(config, "lb-connect-refused")
a.close()
replies = [http(rust.reverse_port, "/retry-connect") for _ in range(6)]
views = backend_views(rust)
check("lb.connect_refused_failover", all(status == 200 and body == b"b" for status, _, body in replies)
and views.get("a", {}).get("connect_errors_total", 0) > 0, backends=views)
rust.reset()
check("lb.static_vhost_survives_reload", http(rust.static_port)[0] == 200)
finally:
a.close()
b.close()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be new or empty")
reports = {}
for specification in args.binary:
label, binary = specification.split("=", 1)
cases = []
def check(case, passed, **details):
cases.append({"case": case, "passed": passed, **details})
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(args.out / label / "checks.json", cases)
try:
with Fixture(args.out / label) as fixture:
run(check, fixture, Path(binary), label)
except Exception as exc:
check("lb.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"passed": sum(case["passed"] for case in cases),
"failed": sum(not case["passed"] for case in cases), "cases": cases,
"simulation": False, "kind": "live RustProxy two-origin HTTP/1.1 regression",
"not_covered": ["sticky affinity", "active health probes", "long-running connection soak",
"TLS origin failures", "DNS rotation"]}
reports[label] = report
write_json(args.out / label / "report.json", report)
write_json(args.out / "comparison.json", reports)
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
raise SystemExit(main())
scripts/performance/peer.py
-223
View File
@@ -1,223 +0,0 @@
#!/usr/bin/env python3
"""A protocol peer, not a RustProxy emulator; used only as the test upstream.
Performance comparisons use nginx as the upstream, never this Python fixture.
"""
from __future__ import annotations
import base64
import collections
import hashlib
import json
import socket
import struct
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import urlsplit, parse_qs
class Peer:
def __init__(self, port, site):
self.site = site
self.counts = collections.Counter()
self.received = {}
self.lock = threading.Lock()
self.started_upload = threading.Event()
self.release_stream = threading.Event()
peer = self
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def log_message(self, *_):
pass
def emit(self, status, body=b"", headers=None):
self.send_response(status)
self.send_header("Content-Length", str(len(body)))
if "Content-Type" not in (headers or {}):
self.send_header("Content-Type", "application/octet-stream")
for key, value in (headers or {}).items():
self.send_header(key, value)
self.end_headers()
if self.command != "HEAD" and body:
self.wfile.write(body)
def upload(self):
chunks = []
if self.headers.get("Transfer-Encoding", "").lower() == "chunked":
while True:
length = int(self.rfile.readline().split(b";", 1)[0], 16)
if length == 0:
while self.rfile.readline().strip():
pass
break
payload = self.rfile.read(length)
self.rfile.read(2)
chunks.append(payload)
peer.started_upload.set()
else:
remaining = int(self.headers.get("Content-Length", 0))
while remaining:
payload = self.rfile.read(min(remaining, 1024))
if not payload:
break
chunks.append(payload)
remaining -= len(payload)
peer.started_upload.set()
return b"".join(chunks)
def do_POST(self):
self.handle_method()
def do_PUT(self):
self.handle_method()
def do_HEAD(self):
self.handle_method()
def do_GET(self):
self.handle_method()
def handle_method(self):
try:
self.route()
except (BrokenPipeError, ConnectionResetError, socket.timeout):
self.close_connection = True
def route(self):
parsed = urlsplit(self.path)
path, query = parsed.path, parse_qs(parsed.query)
with peer.lock:
peer.counts[(self.command, path)] += 1
count = peer.counts[(self.command, path)]
body = self.upload() if self.command in ("POST", "PUT") or self.headers.get("Content-Length") or self.headers.get("Transfer-Encoding") else b""
if path == "/no-replay":
self.close_connection = True
self.connection.shutdown(socket.SHUT_RDWR)
self.connection.close()
return
if path in ("/echo", "/stream-upload"):
data = {"method": self.command, "path": self.path,
"headers": {key.lower(): value for key, value in self.headers.items()},
"body_bytes": len(body), "body_sha256": hashlib.sha256(body).hexdigest()}
with peer.lock:
peer.received[path] = data
self.emit(200, json.dumps(data).encode(), {"Content-Type": "application/json",
"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
return
if path == "/stream-download":
self.send_response(200)
self.send_header("Transfer-Encoding", "chunked")
self.send_header("Content-Type", "application/octet-stream")
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(b"5\r\nfirst\r\n")
self.wfile.flush()
peer.release_stream.wait(5)
self.wfile.write(b"4\r\nlast\r\n0\r\n\r\n")
self.wfile.flush()
return
if path == "/ws":
key = self.headers.get("Sec-WebSocket-Key", "")
accept = base64.b64encode(hashlib.sha1(
(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").encode()).digest()).decode()
self.send_response(101)
self.send_header("Upgrade", "websocket")
self.send_header("Connection", "Upgrade")
self.send_header("Sec-WebSocket-Accept", accept)
self.end_headers()
self.wfile.flush()
frame = self.rfile.read(2)
if len(frame) == 2:
length = frame[1] & 127
if length == 126:
length = struct.unpack("!H", self.rfile.read(2))[0]
mask = self.rfile.read(4) if frame[1] & 128 else b"\0\0\0\0"
data = self.rfile.read(length)
data = bytes(byte ^ mask[i % 4] for i, byte in enumerate(data))
self.wfile.write(bytes([0x81, len(data)]) + data)
self.wfile.flush()
self.close_connection = True
return
if path.startswith("/status/"):
self.emit(int(path.rsplit("/", 1)[1]), b"origin status payload\n",
{"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
return
if path == "/redirect":
self.emit(302, b"redirect body\n", {"Location": "/small.html", "Cache-Control": "no-store"})
return
if path in ("/private", "/cookie", "/no-store", "/vary", "/fresh", "/revalidate", "/concurrent"):
cache_control = "public, max-age=60"
headers = {"Content-Type": "text/plain"}
if body:
headers["X-Received-Body-Bytes"] = str(len(body))
if path == "/private":
cache_control = "private, max-age=60"
if path == "/no-store":
cache_control = "no-store"
if path == "/cookie":
headers["Set-Cookie"] = "application-session=fixture; Path=/"
if path == "/vary":
headers["Vary"] = "Accept-Language"
if path == "/concurrent":
time.sleep(.08)
if path == "/revalidate":
cache_control = "public, max-age=1, stale-if-error=10"
headers["ETag"] = '"fixture-v1"'
if self.headers.get("If-None-Match") == headers["ETag"]:
self.emit(304, b"", {**headers, "Cache-Control": cache_control})
return
data = f"{path}:count={count}:lang={self.headers.get('Accept-Language', '')}".encode()
if path == "/revalidate":
data = b"stable revalidated object"
self.emit(200, data, {**headers, "Cache-Control": cache_control})
return
if path == "/no-transform":
self.emit(200, b"unchanged compression fixture " * 1024,
{"Content-Type": "text/plain", "Cache-Control": "no-transform, public, max-age=60"})
return
file = peer.site / path.lstrip("/")
if not file.is_file() or ".." in file.parts:
self.emit(404, b"fixture not found")
return
payload = file.read_bytes()
tag = '"' + hashlib.sha256(payload).hexdigest() + '"'
headers = {"Cache-Control": "public, max-age=60", "ETag": tag,
"Accept-Ranges": "bytes", "X-Origin-Fixture": "yes"}
headers["Content-Type"] = {".html": "text/html", ".txt": "text/plain",
".css": "text/css", ".json": "application/json"}.get(file.suffix, "application/octet-stream")
if self.headers.get("If-None-Match") == tag:
self.emit(304, b"", headers)
return
range_value = self.headers.get("Range")
if range_value and self.headers.get("If-Range", tag) == tag:
try:
first, last = range_value.removeprefix("bytes=").split("-", 1)
start = int(first) if first else max(0, len(payload) - int(last))
end = min(int(last), len(payload) - 1) if first and last else len(payload) - 1
if start >= len(payload) or end < start:
self.emit(416, b"", {**headers, "Content-Range": f"bytes */{len(payload)}"})
else:
self.emit(206, payload[start:end + 1], {**headers,
"Content-Range": f"bytes {start}-{end}/{len(payload)}"})
return
except ValueError:
pass
self.emit(200, payload, headers)
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
self.server.daemon_threads = True
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
def count(self, method, path):
with self.lock:
return self.counts[(method, path)]
def close(self):
self.release_stream.set()
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=2)
scripts/performance/profile_syscalls.py
-284
View File
@@ -1,284 +0,0 @@
#!/usr/bin/env python3
"""Profile selected server syscalls separately from throughput measurements.
Runs each RustProxy binary sequentially against a disposable loopback fixture.
The measured operation is one persistent HTTP/1.1 connection requesting the
64-KiB static file with cache disabled and Accept-Encoding: identity.
Every response is validated; syscall counts include background server work.
strace elapsed/CPU times must not be interpreted as uninstrumented performance.
"""
from __future__ import annotations
import argparse
import hashlib
from http.client import HTTPConnection
import json
import os
from pathlib import Path
import platform
import re
import shutil
import signal
import subprocess
import sys
import time
import traceback
from common import Fixture, seal_results, sha256, write_json
from resources import descendants
SYSCALLS = ("read", "pread64", "write", "writev", "openat", "close", "fcntl", "lseek", "futex")
REQUEST_PATH = "/large.bin"
def thread_tracers(pid: int) -> dict[int, int]:
result = {}
for task in Path(f"/proc/{pid}/task").iterdir():
try:
status = (task / "status").read_text()
except FileNotFoundError:
continue # A thread may exit between the directory read and status.
match = re.search(r"^TracerPid:\s+(\d+)$", status, re.MULTILINE)
if match:
result[int(task.name)] = int(match.group(1))
return result
def wait_attached(tracer, server, timeout=10):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if tracer.poll() is not None:
raise RuntimeError("strace exited before attachment; see strace diagnostic log (ptrace may be unavailable)")
if server.poll() is not None:
raise RuntimeError("server exited during strace attachment")
traced = thread_tracers(server.pid)
if traced and all(traced.values()):
return sorted(traced)
time.sleep(.01)
raise RuntimeError("strace did not attach to every server thread within 10 seconds")
def detach(tracer, server):
# Do not signal the process group: only the independently started tracer.
if tracer.poll() is None:
tracer.send_signal(signal.SIGINT)
try:
tracer.wait(timeout=10)
except subprocess.TimeoutExpired:
tracer.terminate()
try:
tracer.wait(timeout=3)
except subprocess.TimeoutExpired:
tracer.kill()
tracer.wait(timeout=3)
raise RuntimeError("strace failed to finish on SIGINT; profile is incomplete")
if server.poll() is not None:
raise RuntimeError("server exited while the tracer was detached")
traced = thread_tracers(server.pid)
if not traced or any(traced.values()):
raise RuntimeError("server still has traced threads after strace exited")
return {"signal": "SIGINT to strace PID only", "strace_returncode": tracer.returncode,
"server_alive": True, "remaining_traced_threads": 0}
def parse_summary(path: Path, requests: int):
rows = {name: {"calls": 0, "errors": 0, "calls_per_request": 0.0}
for name in SYSCALLS}
total = None
for line in path.read_text().splitlines():
fields = line.split()
if len(fields) not in (5, 6) or fields[-1] not in (*SYSCALLS, "total"):
continue
percent, seconds, usecs, calls = map(float, fields[:4])
errors = int(fields[4]) if len(fields) == 6 else 0
row = {"calls": int(calls), "errors": errors,
"calls_per_request": int(calls) / requests,
"strace_system_seconds": seconds, "strace_percent_time": percent,
"strace_us_per_call": usecs}
if fields[-1] == "total":
total = row
else:
rows[fields[-1]] = row
if total is None:
raise RuntimeError(f"no complete strace summary found in {path}")
if sum(row["calls"] for row in rows.values()) != total["calls"]:
raise RuntimeError("parsed syscall counts do not match the raw strace total")
return {"selected_syscalls": rows, "selected_total": total}
def request_checked(connection, expected_socket, expected_sha, phase, sequence, evidence):
if connection.sock is not expected_socket:
raise RuntimeError("keepalive connection was replaced before the next request")
connection.request("GET", REQUEST_PATH,
headers={"Accept-Encoding": "identity", "Connection": "keep-alive"})
response = connection.getresponse()
payload = response.read()
digest = hashlib.sha256(payload).hexdigest()
headers = {key.lower(): value for key, value in response.getheaders()}
valid = (response.status == 200 and len(payload) == 65536 and digest == expected_sha
and headers.get("content-length") == "65536"
and headers.get("content-encoding", "identity") == "identity"
and connection.sock is expected_socket and not response.will_close)
result = {"phase": phase, "sequence": sequence, "status": response.status,
"bytes": len(payload), "sha256": digest,
"content_length": headers.get("content-length"),
"content_encoding": headers.get("content-encoding"),
"connection_reused": connection.sock is expected_socket,
"will_close": response.will_close, "passed": valid}
evidence.write(json.dumps(result, sort_keys=True) + "\n")
evidence.flush()
if not valid:
raise RuntimeError(f"response verification failed in {phase} request {sequence}: {result}")
def profile_variant(fixture, label, binary, args):
rust = fixture.rust(binary, label)
rust.login()
rust.wait_quiescent()
raw = args.out / "raw"
raw.mkdir(exist_ok=True)
summary_path = raw / f"{label}.strace.txt"
diagnostic_path = raw / f"{label}.strace.log"
responses_path = raw / f"{label}.responses.jsonl"
expected_sha = sha256(fixture.site / "large.bin")
quiescence_path = fixture.results / f"{label}-quiescence.json"
config_path = fixture.results / "configs" / f"{label}-base.json"
config_sha = sha256(config_path)
if config_sha != rust.applies[-1]["sha256"]:
raise RuntimeError("saved fixture configuration differs from the applied configuration")
command = [str(args.strace), "-f", "-c", "-e", "trace=" + ",".join(SYSCALLS),
"-p", str(rust.process.pid), "-o", str(summary_path)]
connection = HTTPConnection("127.0.0.1", rust.static_port, timeout=args.timeout)
tracer = None
result = {"variant": label, "binary": str(binary), "binary_sha256": sha256(binary),
"config": str(config_path.relative_to(args.out)), "config_sha256": config_sha,
"quiescence": str(quiescence_path.relative_to(args.out)),
"quiescence_sha256": sha256(quiescence_path),
"pid": rust.process.pid, "request_path": REQUEST_PATH, "payload_bytes": 65536,
"payload_sha256": expected_sha, "cache_enabled": False,
"request_encoding": "identity", "concurrency": 1,
"warmup_requests": args.warmup, "profile_requests": args.requests,
"strace_command": command, "raw_summary": str(summary_path.relative_to(args.out)),
"raw_diagnostics": str(diagnostic_path.relative_to(args.out)),
"responses": str(responses_path.relative_to(args.out)), "passed": False}
try:
with responses_path.open("w") as responses, diagnostic_path.open("w") as diagnostics:
connection.connect()
expected_socket = connection.sock
result["client_local_address"] = expected_socket.getsockname()
for index in range(args.warmup):
request_checked(connection, expected_socket, expected_sha, "warmup", index + 1, responses)
result["processes_before_attach"] = descendants(rust.process.pid)
if result["processes_before_attach"] != [rust.process.pid]:
raise RuntimeError("server spawned a child after quiescence; refusing contaminated syscall trace")
environment = os.environ.copy()
environment["LC_ALL"] = "C"
tracer = subprocess.Popen(command, stdout=diagnostics, stderr=diagnostics,
env=environment, start_new_session=True)
result["initial_traced_threads"] = wait_attached(tracer, rust.process)
started = time.monotonic()
try:
for index in range(args.requests):
request_checked(connection, expected_socket, expected_sha, "profile", index + 1, responses)
if tracer.poll() is not None:
raise RuntimeError("strace exited before all profile requests completed")
finally:
result["instrumented_request_loop_seconds"] = time.monotonic() - started
result["detach"] = detach(tracer, rust.process)
tracer = None
# An actual HTTP response after detaching proves the server stayed usable.
request_checked(connection, expected_socket, expected_sha, "after_detach", 1, responses)
result["processes_after_detach"] = descendants(rust.process.pid)
if result["processes_after_detach"] != [rust.process.pid]:
raise RuntimeError("server has a child after profiling; syscall trace needs review")
result.update(parse_summary(summary_path, args.requests))
result["verified_responses"] = args.requests + args.warmup + 1
result["passed"] = True
return result
finally:
if tracer is not None:
detach(tracer, rust.process)
connection.close()
fixture.stop(rust.process)
write_json(args.out / f"{label}.json", result)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--nginx", type=Path, required=True, help="nginx executable for the shared fixture origin")
parser.add_argument("--strace", type=Path, default=shutil.which("strace"))
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--requests", type=int, default=500)
parser.add_argument("--warmup", type=int, default=20)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=4)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--timeout", type=float, default=20)
args = parser.parse_args()
if args.requests < 1 or args.warmup < 0 or args.workers < 1 or args.origin_workers < 1 or args.timeout <= 0:
parser.error("requests/workers/timeout must be positive, warmup must be nonnegative")
if args.strace is None:
parser.error("strace is required; pass --strace /absolute/path")
if args.out.exists() and (not args.out.is_dir() or any(args.out.iterdir())):
parser.error("--out must be a new or empty directory; existing evidence is never overwritten")
variants = {}
for value in args.binary:
label, separator, filename = value.partition("=")
if not separator or not re.fullmatch(r"[A-Za-z][A-Za-z0-9_-]{0,48}", label) or label in variants:
parser.error("each --binary must use a unique filesystem-safe LABEL=/ABS/PATH")
binary = Path(filename).resolve()
if not binary.is_file():
parser.error(f"binary does not exist: {binary}")
variants[label] = binary
args.out, args.nginx, args.strace = args.out.resolve(), args.nginx.resolve(), args.strace.resolve()
for executable in (args.nginx, args.strace):
if not executable.is_file() or not os.access(executable, os.X_OK):
parser.error(f"executable is missing or is not executable: {executable}")
args.out.mkdir(parents=True, exist_ok=True)
version = subprocess.run([str(args.strace), "-V"], capture_output=True, text=True, timeout=10)
environment = {"kind": "selected syscall counts with ptrace instrumentation", "simulation": False,
"platform": platform.platform(), "python": sys.version, "strace_version": version.stdout + version.stderr,
"strace_sha256": sha256(args.strace), "nginx_sha256": sha256(args.nginx),
"variant_order": list(variants), "selected_syscalls": SYSCALLS,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
"limitations": ["strace changes scheduling and latency; do not use instrumented times as RPS evidence",
"Counts cover selected syscalls in every traced server thread, including background work",
"Normalized counts divide the whole trace by verified profile requests; they do not isolate per-request causality",
"This is a warm-file-cache, c1, HTTP/1.1 plaintext static-file scenario; no general capacity inference",
"Syscall counts do not directly count Tokio task dispatches or memory allocations"]}
for path in ("/proc/sys/kernel/yama/ptrace_scope", "/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max"):
try:
environment[path] = Path(path).read_text().strip()
except OSError:
environment[path] = None
write_json(args.out / "environment.json", environment)
rows, error = [], None
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
fixture.nginx(args.nginx, frontend=False)
for label, binary in variants.items():
result = profile_variant(fixture, label, binary, args)
rows.append(result)
write_json(args.out / "profiles.json", rows)
print(json.dumps({"variant": label, "requests": args.requests, "passed": result["passed"],
"selected_syscall_calls": result["selected_total"]["calls"]}), flush=True)
except Exception as exc:
error = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", error)
print(json.dumps(error), flush=True)
finally:
write_json(args.out / "profiles.json", rows)
write_json(args.out / "completion.json", {"completed": error is None, "profiles": len(rows),
"simulation": False, "throughput_measurement": False,
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
seal_results(args.out)
return 1 if error else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/regression.py
-107
View File
@@ -1,107 +0,0 @@
#!/usr/bin/env python3
"""Run black-box acceptance tests against one or more real RustProxy binaries.
Example:
python3 scripts/performance/regression.py --binary before=/abs/before \
--binary after=/abs/after --out /abs/results/regression
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import platform
import sys
import time
import traceback
from common import Fixture, seal_results, write_json
from peer import Peer
import checks_cache
import checks_policy
import checks_proxy
import checks_static
GROUPS = {"static": checks_static.run, "proxy": checks_proxy.run,
"policy": checks_policy.run, "cache": checks_cache.run,
"transform": checks_cache.transforms}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", required=True, type=Path)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--groups", default=",".join(GROUPS))
args = parser.parse_args()
groups = args.groups.split(",")
if any(group not in GROUPS for group in groups):
parser.error(f"groups must be selected from {','.join(GROUPS)}")
args.out = args.out.resolve()
args.out.mkdir(parents=True, exist_ok=True)
metadata = {"kind": "live RustProxy black-box HTTP acceptance", "simulation": False,
"python": sys.version, "platform": platform.platform(),
"cpu_affinity": sorted(os.sched_getaffinity(0)) if hasattr(os, "sched_getaffinity") else None,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"groups": groups,
"not_covered": ["TLS/ALPN", "HTTP/2", "HTTP/3", "PHP-FPM",
"Tinyauth verifier", "real DNS rebinding", "24-hour soak",
"concurrent filesystem rename-race proof"]}
write_json(args.out / "environment.json", metadata)
reports = {}
for binary_arg in args.binary:
label, separator, name = binary_arg.partition("=")
if not separator or not label.replace("-", "").replace("_", "").isalnum():
parser.error("--binary must be LABEL=/ABS/PATH with a simple unique label")
if label in reports:
parser.error("binary labels must be unique")
cases = []
started = time.monotonic()
output = args.out / label
def check(case, passed, **details):
row = {"case": case, "passed": passed, **details}
cases.append(row)
print(f"{label} {'SKIP' if passed is None else 'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(output / "checks.json", cases)
try:
with Fixture(output, workers=args.workers) as fixture:
peer = Peer(fixture.origin_port, fixture.site)
try:
rust = fixture.rust(Path(name), label)
check("runtime.started", True)
for group in groups:
try:
rust.reset()
GROUPS[group](check, rust, peer)
except Exception as exc:
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
finally:
rust.reset()
check("runtime.still_alive", rust.process.poll() is None)
finally:
peer.close()
except Exception as exc:
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"label": label, "kind": metadata["kind"], "simulation": False,
"seconds": time.monotonic() - started, "passed": sum(c["passed"] is True for c in cases),
"failed": sum(c["passed"] is False for c in cases),
"skipped": sum(c["passed"] is None for c in cases), "cases": cases}
reports[label] = report
write_json(output / "report.json", report)
print(f"{label}: {report['passed']} passed, {report['failed']} failed, {report['skipped']} skipped", flush=True)
comparison = {"variants": {name: {key: report[key] for key in ["passed", "failed", "skipped", "seconds"]}
for name, report in reports.items()},
"cases": [{"case": case, **{name: next((entry["passed"] for entry in report["cases"]
if entry["case"] == case), "not_run") for name, report in reports.items()}}
for case in sorted({entry["case"] for report in reports.values() for entry in report["cases"]})]}
write_json(args.out / "comparison.json", comparison)
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/resources.py
-228
View File
@@ -1,228 +0,0 @@
"""Linux /proc sampling outside the server process; no synthetic resource data."""
from __future__ import annotations
import os
from pathlib import Path
import threading
import time
def cgroup_cpu_stat(path=Path("/sys/fs/cgroup/cpu.stat")):
values = {key: int(value) for key, value in (line.split() for line in path.read_text().splitlines())}
if not {"nr_throttled", "throttled_usec"} <= values.keys():
raise RuntimeError(f"{path} lacks cgroup v2 throttling counters")
return {"path": str(path), "observed_monotonic": time.monotonic(), "values": values}
def cgroup_cpu_delta(before, after):
delta = {key: after["values"][key] - value for key, value in before["values"].items()
if key in after["values"]}
if any(value < 0 for value in delta.values()):
raise RuntimeError("cgroup CPU counters reset during the measured run")
return delta
def parse_cpu_set(value):
if not value:
raise ValueError("an explicit, nonempty CPU set is required")
result = set()
for part in value.split(","):
bounds = part.strip().split("-")
if len(bounds) not in (1, 2) or any(not item.isascii() or not item.isdigit() for item in bounds):
raise ValueError(f"invalid CPU set {value!r}; use comma-separated numbers or ranges")
start, end = int(bounds[0]), int(bounds[-1])
if start > end or end - start > 65536:
raise ValueError(f"invalid CPU range {part!r}")
result.update(range(start, end + 1))
return result
def validate_cpu_sets(values, allowed):
groups = {label: parse_cpu_set(value) for label, value in values.items()}
for label, cpus in groups.items():
if not cpus <= allowed:
raise ValueError(f"{label} CPUs {sorted(cpus - allowed)} are outside the allowed affinity")
for other, other_cpus in groups.items():
if label < other and cpus & other_cpus:
raise ValueError(f"{label} and {other} CPU sets overlap: {sorted(cpus & other_cpus)}")
return groups
def affinity_snapshot(root, expected, expected_processes=None):
"""Verify every currently visible thread in the root process and children."""
tasks = []
for pid in descendants(root):
try:
threads = list(Path(f"/proc/{pid}/task").iterdir())
except FileNotFoundError:
continue
for thread in threads:
try:
actual = set(os.sched_getaffinity(int(thread.name)))
except ProcessLookupError:
continue
if actual != expected:
raise RuntimeError(f"affinity mismatch for PID {pid} TID {thread.name}: "
f"expected {sorted(expected)}, got {sorted(actual)}")
tasks.append({"pid": pid, "tid": int(thread.name), "cpus": sorted(actual)})
if not tasks or not any(task["pid"] == root for task in tasks):
raise RuntimeError(f"cannot inspect affinity of live process {root}")
processes = len({task["pid"] for task in tasks})
if expected_processes is not None and processes != expected_processes:
raise RuntimeError(f"process count mismatch for PID {root}: "
f"expected {expected_processes}, inspected {processes}")
return {"root_pid": root, "processes": processes,
"threads": len(tasks), "tasks": tasks}
def wait_for_affinity(process, expected, timeout=2):
"""Allow taskset to execute before inspecting the generator's inherited mask."""
deadline = time.monotonic() + timeout
while True:
if process.poll() is not None:
raise RuntimeError(f"generator exited before affinity verification: {process.returncode}")
try:
return affinity_snapshot(process.pid, expected)
except RuntimeError:
if time.monotonic() >= deadline:
raise
time.sleep(.01)
def descendants(pid, proc_root=Path("/proc")):
"""Find the process tree through stat PPids, including nginx workers.
/proc/PID/task/PID/children is optional on some Linux kernels and may also
omit children created by another thread. Missing that file is not evidence
that a live process is a leaf. A single PPid-map scan avoids either case.
"""
children, seen = {}, set()
for entry in proc_root.iterdir():
if not entry.name.isdecimal():
continue
current = int(entry.name)
try:
fields = (entry / "stat").read_text().rsplit(")", 1)[1].split()
parent = int(fields[1])
except (FileNotFoundError, ProcessLookupError):
# A non-root process may finish while the snapshot is assembled.
continue
except (OSError, ValueError, IndexError) as error:
raise RuntimeError(f"cannot discover process tree from PID {current}: {error}") from error
if fields[0] in {"Z", "X", "x"}:
# A dead/zombie worker cannot satisfy the configured live count.
continue
seen.add(current)
children.setdefault(parent, []).append(current)
if pid not in seen:
raise RuntimeError(f"no readable live root process for PID {pid}")
result, pending = [], [pid]
while pending:
current = pending.pop()
if current in result:
continue
result.append(current)
pending.extend(sorted(children.get(current, []), reverse=True))
return result
def process_stats(pid):
try:
data = Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()
if data[0] in {"Z", "X", "x"}:
return None
ticks = os.sysconf("SC_CLK_TCK")
pages = os.sysconf("SC_PAGE_SIZE")
return {"pid": pid, "start_ticks": int(data[19]),
"cpu_seconds": (int(data[11]) + int(data[12])) / ticks,
"rss_bytes": int(data[21]) * pages,
"fd": len(list(Path(f"/proc/{pid}/fd").iterdir())),
"threads": int(data[17])}
except (FileNotFoundError, ProcessLookupError):
return None
except (OSError, ValueError, IndexError) as error:
raise RuntimeError(f"cannot read resource counters for PID {pid}: {error}") from error
class Monitor:
def __init__(self, roots, interval=.5, expected_processes=None):
self.roots = roots
self.expected_processes = expected_processes or {}
self.process_identities = {}
self.interval = interval
if interval <= 0:
raise ValueError("resource interval must be positive")
self.samples = []
self.errors = []
self.stop_event = threading.Event()
self.started = time.monotonic()
self.thread = threading.Thread(target=self.run, daemon=True)
def sample(self):
groups = {}
for label, root in self.roots.items():
processes = [stats for pid in descendants(root) if (stats := process_stats(pid)) is not None]
if not any(process["pid"] == root for process in processes):
raise RuntimeError(f"no readable live root process for {label} (PID {root})")
expected = self.expected_processes.get(label)
if expected is not None and len(processes) != expected:
raise RuntimeError(f"resource process count mismatch for {label} (PID {root}): "
f"expected {expected}, read {len(processes)}")
identities = {(process["pid"], process["start_ticks"]) for process in processes}
if expected is not None:
original = self.process_identities.setdefault(label, identities)
if identities != original:
raise RuntimeError(f"process identities changed while measuring {label}: "
f"expected {sorted(original)}, read {sorted(identities)}")
groups[label] = {"processes": processes,
"rss_bytes": sum(p["rss_bytes"] for p in processes),
"fd": sum(p["fd"] for p in processes),
"threads": sum(p["threads"] for p in processes)}
self.samples.append({"seconds": time.monotonic() - self.started, "groups": groups})
def run(self):
while not self.stop_event.wait(self.interval):
try:
self.sample()
except Exception as error:
self.errors.append(str(error))
self.stop_event.set()
def start(self):
self.sample()
self.thread.start()
def stop(self):
self.stop_event.set()
self.thread.join(timeout=3)
if self.thread.is_alive():
self.errors.append("resource sampler did not stop")
try:
self.sample()
except Exception as error:
self.errors.append(str(error))
def summary(self, requests):
if self.errors:
raise RuntimeError("resource sampling failed: " + "; ".join(self.errors))
result = {}
for label in self.roots:
observations = [sample["groups"][label] for sample in self.samples]
totals = {}
for group in observations:
for process in group["processes"]:
key = (process["pid"], process["start_ticks"])
low, high = totals.get(key, (process["cpu_seconds"], process["cpu_seconds"]))
totals[key] = (min(low, process["cpu_seconds"]), max(high, process["cpu_seconds"]))
if not totals:
raise RuntimeError(f"no readable process samples for {label}; refusing zero resource metrics")
seconds = sum(high - low for low, high in totals.values())
result[label] = {"cpu_seconds": seconds,
"cpu_us_per_request": seconds * 1_000_000 / requests if requests else None,
"rss_peak_bytes": max(group["rss_bytes"] for group in observations),
"rss_start_bytes": observations[0]["rss_bytes"],
"rss_end_bytes": observations[-1]["rss_bytes"],
"fd_peak": max(group["fd"] for group in observations),
"fd_start": observations[0]["fd"], "fd_end": observations[-1]["fd"],
"processes_seen": len(totals)}
return result
scripts/performance/simulate_cache_034.py
-112
View File
@@ -1,112 +0,0 @@
#!/usr/bin/env python3
"""Deterministic analytical/synthetic cache-retention scenario, not a RustProxy load test.
Model: 20 vhosts with 100 HOT + 100 COLD uniformly requested objects each.
A new configuration is saved 1,000 times. Eighty percent of saves edit only
keepalive/LB operational controls, and 20% edit content policy on one vhost.
Every save is followed by 500 uniform GETs; every object is public/cacheable and
has infinite TTL. A miss immediately inserts the object in its original tier.
Capacity is assumed adequate and there is no origin latency or concurrent IO.
The baseline flushes both tiers globally on every save, as in 0.33.8.
The candidate changes only the affected vhost after content edits.
"""
import argparse
import json
import random
def simulate(vhosts=20, objects_per_tier=100, saves=1000, requests_per_save=500, seed=3400):
if min(vhosts, objects_per_tier, saves, requests_per_save) < 1:
raise ValueError("all counts must be positive")
rng = random.Random(seed)
objects = vhosts * objects_per_tier * 2
old = set(range(objects))
new = set(range(objects))
old_hits = new_hits = old_misses = new_misses = 0
old_evicted = new_evicted = 0
edits_operational = edits_content = 0
for step in range(saves):
changed_vhost = rng.randrange(vhosts)
content_edit = step % 5 == 0
if content_edit:
edits_content += 1
else:
edits_operational += 1
old_evicted += len(old)
old.clear()
if content_edit:
low = changed_vhost * objects_per_tier * 2
high = low + objects_per_tier * 2
removed = set(range(low, high)) & new
new_evicted += len(removed)
new.difference_update(removed)
for _ in range(requests_per_save):
object_id = rng.randrange(objects)
if object_id in old:
old_hits += 1
else:
old_misses += 1
old.add(object_id)
if object_id in new:
new_hits += 1
else:
new_misses += 1
new.add(object_id)
total = saves * requests_per_save
assert old_hits + old_misses == total
assert new_hits + new_misses == total
assert new_evicted <= old_evicted
assert new_misses <= old_misses
return {
"scenario": {
"vhosts": vhosts,
"hot_objects_per_vhost": objects_per_tier,
"cold_objects_per_vhost": objects_per_tier,
"saves": saves,
"operational_saves": edits_operational,
"single_vhost_content_saves": edits_content,
"gets_per_save": requests_per_save,
"total_gets": total,
"seed": seed,
"assumptions": "All objects public/cacheable, infinite TTL, instantly refilled, uniform GETs; no bandwidth/CPU/origin timings",
},
"legacy_global_purge": {
"invalidated_objects": old_evicted,
"misses": old_misses,
"hits": old_hits,
"hit_ratio_percent": round(100 * old_hits / total, 3),
},
"scoped_cache": {
"invalidated_objects": new_evicted,
"misses": new_misses,
"hits": new_hits,
"hit_ratio_percent": round(100 * new_hits / total, 3),
},
"improvement": {
"fewer_invalidations_percent": round(100 * (1 - new_evicted / old_evicted), 3) if old_evicted else 0,
"fewer_cache_misses_percent": round(100 * (1 - new_misses / old_misses), 3) if old_misses else 0,
"hit_ratio_gain_percentage_points": round(100 * (new_hits - old_hits) / total, 3),
"observed_rps_gain_percent": None,
},
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--vhosts", type=int, default=20)
parser.add_argument("--objects-per-tier", type=int, default=100)
parser.add_argument("--saves", type=int, default=1000)
parser.add_argument("--gets-per-save", type=int, default=500)
parser.add_argument("--seed", type=int, default=3400)
parser.add_argument("--output", help="optional JSON results file")
args = parser.parse_args()
result = simulate(args.vhosts, args.objects_per_tier, args.saves, args.gets_per_save, args.seed)
encoded = json.dumps(result, indent=2, sort_keys=True)
if args.output:
from pathlib import Path
Path(args.output).write_text(encoded + "\n")
print(encoded)
if __name__ == "__main__":
main()
scripts/performance/simulate_nginx_gap_034.py
-98
View File
@@ -1,98 +0,0 @@
#!/usr/bin/env python3
"""Amdahl sensitivity study only: 0.33.8 measured baseline, 0.34.0 not measured.
No synthetic RPS is described as benchmarked RustProxy throughput. Amdahl uses
RPS_new = RPS_old / (1 - eligible_cost_fraction * fraction_eliminated).
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
# Verified medians reproduced from the supplied 0.33.8 evidence. Cache OFF.
BASELINE = [
("reverse_proxy", 1040, 29334.12, 170610.16, False),
("reverse_proxy", 65536, 20735.07, 35393.29, False),
("reverse_proxy", 1048576, 4461.19, 2730.23, False),
("www", 1040, 24906.20, 351605.13, False),
("www", 65536, 20856.67, 203709.91, True),
("www", 1048576, 2833.71, 12611.49, True),
]
# WWW scenarios: share of per-request service time due to file copy/read;
# assume sendfile removes HALF of that part, not the full service time.
WWW_SHARES = {"low": 0.20, "middle": 0.40, "high": 0.60}
COPY_COST_REMOVED = 0.50
# Proxy independent optimization: hypothetical reduction in *total* request
# service time, modeling removal of redundant response-frame mutex, nothing else.
PROXY_TIME_SAVED = {"low": 0.01, "middle": 0.03, "high": 0.05}
def run() -> dict:
items = []
for kind, payload_bytes, rust_rps, nginx_rps, eligible in BASELINE:
baseline_ratio = rust_rps / nginx_rps
row = {
"kind": kind,
"bytes": payload_bytes,
"baseline_version": "0.33.8",
"baseline_rust_rps": rust_rps,
"baseline_nginx_rps": nginx_rps,
"baseline_of_nginx_pct": round(100 * baseline_ratio, 2),
"factor_to_match_nginx": round(nginx_rps / rust_rps, 2),
"sendfile_eligible_only_if_plain_linux_http1_1_no_cache_no_compression": eligible,
"scenarios": {},
}
for level in ("low", "middle", "high"):
if kind == "www" and eligible:
saved_fraction = WWW_SHARES[level] * COPY_COST_REMOVED
elif kind == "reverse_proxy":
saved_fraction = PROXY_TIME_SAVED[level]
else:
saved_fraction = 0.0
simulated_rps = rust_rps / (1.0 - saved_fraction)
row["scenarios"][level] = {
"modeled_total_time_saved_pct": round(100 * saved_fraction, 2),
"simulated_rps": round(simulated_rps),
"simulated_rps_gain_pct": round(100 * (simulated_rps / rust_rps - 1), 2),
"simulated_of_nginx_pct": round(100 * simulated_rps / nginx_rps, 2),
"simulated_remaining_gap_to_nginx_pct_points": round(
100 * (1 - simulated_rps / nginx_rps), 2
),
}
items.append(row)
return {
"kind": "HYPOTHETICAL_AMDAHL_SENSITIVITY_NOT_A_BENCHMARK",
"comparison": "Existing 0.33.8 measurements (5 runs, cache off) vs measured nginx",
"not_measured": "0.34.0 server performance and sendfile syscall/runtime correctness",
"conditions": [
"Identical CPU and network assumptions to supplied 0.33.8 baseline",
"WWW zero-copy is limited to Linux, cleartext HTTP/1.1, GET >=64KiB, no cache, range, transform, compression or TLS",
"WWW low/middle/high: file-copy contributes 20%/40%/60% of total request time; modeled sendfile eliminates 50% of that component",
"Reverse proxy low/middle/high: removing response-frame mutex saves a hypothetical 1%/3%/5% of total service time",
"These fractions are chosen sensitivity parameters; not measured or inferred from profiling",
"No change is assumed for WWW 1040B or out-of-scope protocol paths",
"No network saturation, TLS record processing, queueing, origin latency, filesystem cache misses, CPU contention or concurrency effects modeled",
"No additional speedup due to cache-retention changes is assumed",
],
"formula": "RPS_sim = RPS_0.33.8 / (1 - fraction_of_total_service_time_eliminated)",
"entries": items,
}
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, help="write JSON to this file")
args = parser.parse_args()
payload = json.dumps(run(), indent=2, ensure_ascii=False) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(payload)
print(f"wrote {args.output}")
else:
print(payload)
if __name__ == "__main__":
main()
scripts/performance/soak.py
-325
View File
@@ -1,325 +0,0 @@
#!/usr/bin/env python3
"""Fixed-rate mixed traffic, controlled client closes and cache hot-toggle soak.
The default is a 30-minute live run. Short invocations must be explicitly marked
--exploratory and cannot satisfy the release soak gate. No run is auto-scheduled.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, wait, FIRST_COMPLETED
import copy
import hashlib
from http.client import HTTPConnection
import json
import os
from pathlib import Path
import platform
import statistics
import sys
import time
import traceback
from benchmark import counter_values
from common import Fixture, http, seal_results, sha256, write_json
from resources import Monitor
from telemetry import traffic_analytics_snapshot, traffic_analytics_delta
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
def fetch(index, rust, expected, abort_every):
name = PAYLOADS[index % len(PAYLOADS)]
source = "proxy" if (index // len(PAYLOADS)) % 2 else "static"
port = rust.reverse_port if source == "proxy" else rust.static_port
started = time.monotonic()
case = {"index": index, "source": source, "file": name, "method": "GET", "intentional_abort": False}
try:
if abort_every and index % abort_every == 0:
case.update({"file": "huge.bin", "intentional_abort": True})
connection = HTTPConnection("127.0.0.1", port, timeout=10)
try:
connection.request("GET", "/huge.bin", headers={"Accept-Encoding": "identity"})
response = connection.getresponse()
body = response.read(128)
case.update({"status": response.status, "bytes": len(body),
"cache": response.getheader("X-Cache"),
"passed": response.status == 200 and body == expected["huge.bin"][:128]})
finally:
connection.close()
else:
headers = {"Accept-Encoding": "identity"}
method = "HEAD" if index % 29 == 0 else "GET"
if method == "GET" and index % 37 == 0:
headers["Range"] = "bytes=0-127"
expected_status, expected_body = 206, expected[name][:128]
else:
expected_status, expected_body = 200, b"" if method == "HEAD" else expected[name]
status, response_headers, body = http(port, "/" + name, method, headers)
case.update({"method": method, "status": status, "bytes": len(body),
"cache": response_headers.get("x-cache"),
"passed": status == expected_status and body == expected_body})
if not case["passed"]:
case.update({"expected_status": expected_status, "expected_bytes": len(expected_body),
"body_sha256": hashlib.sha256(body).hexdigest()})
except Exception as exc:
case.update({"passed": False, "error": str(exc)})
case["seconds"] = time.monotonic() - started
return case
def probe_cache_mode(rust, expected, enabled, phase, out, observations):
"""Record two exact GETs per source, separate from the paced workload."""
phase_observations = []
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
replies = []
for attempt in [1, 2]:
try:
status, headers, body = http(port, "/small.html?soak_cache_probe=1",
headers={"Accept-Encoding": "identity"})
replies.append({"attempt": attempt, "status": status, "cache": headers.get("x-cache"),
"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
"body_passed": status == 200 and body == expected["small.html"]})
except Exception as error:
replies.append({"attempt": attempt, "body_passed": False, "error": str(error)})
mode_passed = replies[1].get("cache") == "HIT" if enabled else all(
reply.get("cache") is None for reply in replies)
phase_observations.append({"phase": phase, "source": source, "cache_enabled": enabled,
"responses": replies,
"passed": all(reply["body_passed"] for reply in replies) and mode_passed})
observations.extend(phase_observations)
write_json(out / "cache-probes.json", observations)
if not all(observation["passed"] for observation in phase_observations):
raise RuntimeError(f"cache mode/body probe failed after {phase}; see cache-probes.json")
def backend_drain_snapshot(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
return {"passed": False, "error": f"admin status HTTP {status}"}
backends = []
def walk(value):
if isinstance(value, dict):
if "id" in value and "active" in value and "status" in value and "requests" in value:
backends.append({key: value[key] for key in ("id", "active", "status", "requests")})
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return {"passed": bool(backends) and all(backend["active"] == 0 for backend in backends),
"backends": backends}
def memory_trend(samples, label="server"):
observations = [(sample["seconds"], sample["groups"][label]["rss_bytes"], sample["groups"][label]["fd"])
for sample in samples if sample["seconds"] >= 60]
if len(observations) < 3:
return {"available": False, "reason": "less than three samples after the first minute"}
tail_seconds = observations[-1][0]
early = [row[1] for row in observations if row[0] < observations[0][0] + 60]
late = [row[1] for row in observations if row[0] >= tail_seconds - 60]
x_mean = statistics.mean(row[0] for row in observations)
y_mean = statistics.mean(row[1] for row in observations)
denominator = sum((row[0] - x_mean) ** 2 for row in observations)
slope = sum((row[0] - x_mean) * (row[1] - y_mean) for row in observations) / denominator if denominator else 0
return {"available": True, "rss_first_minute_after_warmup_median": statistics.median(early),
"rss_last_minute_median": statistics.median(late),
"rss_growth_bytes": statistics.median(late) - statistics.median(early),
"rss_linear_bytes_per_hour": slope * 3600, "fd_first": observations[0][2],
"fd_last": observations[-1][2], "fd_peak": max(row[2] for row in observations),
"interpretation": "Descriptive only: cache toggles and allocator retention can change RSS without a leak"}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", type=Path, required=True)
parser.add_argument("--label", default="after")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--duration", type=int, default=1800)
parser.add_argument("--rate", type=float, default=100, help="offered requests per second, not a saturation benchmark")
parser.add_argument("--clients", type=int, default=8)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=2)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--toggle-seconds", type=int, default=120)
parser.add_argument("--sample-seconds", type=int, default=5)
parser.add_argument("--abort-every", type=int, default=101)
parser.add_argument("--cooldown-seconds", type=int, default=15)
parser.add_argument("--exploratory", action="store_true")
args = parser.parse_args()
if args.duration < 1800 and not args.exploratory:
parser.error("the soak gate requires >=1800 seconds; label short diagnostics with --exploratory")
if args.rate <= 0 or not 1 <= args.clients <= 128 or args.toggle_seconds < 1 or args.sample_seconds < 1:
parser.error("rate, sample/toggle intervals and clients must be positive; clients <=128")
args.out = args.out.resolve()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be a new or empty evidence directory")
args.out.mkdir(parents=True, exist_ok=True)
settings = {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()}
write_json(args.out / "environment.json", {
"kind": "actual RustProxy fixed-rate mixed-traffic resource soak", "simulation": False,
"settings": settings, "platform": platform.platform(), "binary_sha256": sha256(args.binary),
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"frontend_connection_policy": "A new HTTP/1.1 TCP connection for each workload request and probe",
"controlled_close_interpretation": "Client closes after reading 128 bytes; the server may already have sent the full body. No server-cancellation count is inferred.",
"traffic_analytics_observation_policy": "Global counters are read before traffic, periodically with status diagnostics, and after final drain. Deltas include cache-mode probes and cooldown, and are separate from HTTP errors.",
"not_covered": ["24-hour soak", "TLS/H2/H3 soak", "multi-backend failures", "PHP-FPM",
"real application write load", "origin DNS/TLS rotation", "production traffic",
"long-lived frontend keepalive connections"]})
totals = {"submitted": 0, "completed": 0, "passed": 0, "failed": 0, "controlled_client_closes": 0,
"response_bytes": 0, "scheduler_late_events": 0, "peak_pending": 0}
failure_examples, progress, toggles, counter_samples = [], [], [], []
traffic_samples, traffic_delta = [], None
cache_probes, cache_status_counts = [], {"proxy": Counter(), "static": Counter()}
drain_snapshot, configuration_restored = None, False
samples, resources = [], {}
fatal, runtime_seconds = None, 0
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
nginx = fixture.nginx(args.nginx, frontend=False)
rust = fixture.rust(args.binary, args.label)
rust.login()
rust.wait_quiescent()
expected = {name: (fixture.site / name).read_bytes() for name in PAYLOADS}
config = copy.deepcopy(rust.base)
config["cache"].update({"expose_status": True})
config["web_services"][0]["cache_control"] = "public, max-age=60"
rust.apply(config, "soak-initial")
probe_cache_mode(rust, expected, False, "initial-off", args.out, cache_probes)
traffic_before = traffic_analytics_snapshot(rust)
traffic_samples.append({"phase": "before", "seconds": 0, "snapshot": traffic_before})
write_json(args.out / "traffic-analytics.json", traffic_samples)
monitor = Monitor({"server": rust.process.pid, "origin": nginx["origin"].pid}, args.sample_seconds)
monitor.start()
start = time.monotonic()
deadline = start + args.duration
next_toggle, next_snapshot, next_submit = start + args.toggle_seconds, start, start
pending = set()
def consume(done):
for future in done:
result = future.result()
totals["completed"] += 1
totals["passed" if result["passed"] else "failed"] += 1
totals["controlled_client_closes"] += int(result["intentional_abort"])
totals["response_bytes"] += result.get("bytes", 0)
cache_status = (result.get("cache") or "absent") if "status" in result else "no_response"
cache_status_counts[result["source"]][cache_status] += 1
if not result["passed"] and len(failure_examples) < 100:
failure_examples.append(result)
try:
with ThreadPoolExecutor(max_workers=args.clients) as pool:
while time.monotonic() < deadline:
now = time.monotonic()
done = {future for future in pending if future.done()}
consume(done)
pending -= done
if now >= next_toggle:
config["cache"]["enabled"] = not config["cache"]["enabled"]
rust.apply(config, f"soak-toggle-{len(toggles) + 1}")
toggles.append({"seconds": time.monotonic() - start, "cache_enabled": config["cache"]["enabled"]})
write_json(args.out / "cache-toggles.json", toggles)
probe_cache_mode(rust, expected, config["cache"]["enabled"],
f"toggle-{len(toggles)}", args.out, cache_probes)
next_toggle += args.toggle_seconds
if now >= next_snapshot:
status, headers, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"admin status became unavailable: HTTP {status}")
counter_samples.append({"seconds": time.monotonic() - start,
"cache_enabled": config["cache"]["enabled"],
"counters": counter_values(json.loads(body))})
traffic_now = traffic_analytics_snapshot(rust)
traffic_delta = traffic_analytics_delta(traffic_before, traffic_now)
traffic_samples.append({"phase": "during", "seconds": time.monotonic() - start,
"snapshot": traffic_now, "delta_from_start": traffic_delta})
row = {"seconds": time.monotonic() - start, **totals,
"cache_enabled": config["cache"]["enabled"], "pending": len(pending),
"observed_cache_status_counts": {source: dict(counts)
for source, counts in cache_status_counts.items()}}
progress.append(row)
write_json(args.out / "progress.json", progress)
write_json(args.out / "status-counters.json", counter_samples)
write_json(args.out / "traffic-analytics.json", traffic_samples)
write_json(args.out / "resources.json", list(monitor.samples))
print(json.dumps(row), flush=True)
next_snapshot = time.monotonic() + args.sample_seconds
now = time.monotonic()
if len(pending) >= args.clients * 2:
done, pending = wait(pending, timeout=.1, return_when=FIRST_COMPLETED)
consume(done)
continue
if now < next_submit:
time.sleep(min(.02, next_submit - now))
continue
if now > next_submit + .25:
totals["scheduler_late_events"] += 1
next_submit = now
totals["submitted"] += 1
pending.add(pool.submit(fetch, totals["submitted"], rust, expected, args.abort_every))
totals["peak_pending"] = max(totals["peak_pending"], len(pending))
next_submit += 1 / args.rate
done, pending = wait(pending, timeout=30)
consume(done)
if pending:
raise RuntimeError("requests failed to settle during bounded soak drain")
runtime_seconds = time.monotonic() - start
rust.reset()
configuration_restored = True
cooldown_until = time.monotonic() + args.cooldown_seconds
while time.monotonic() < cooldown_until:
time.sleep(max(0, min(.5, cooldown_until - time.monotonic())))
probe_cache_mode(rust, expected, False, "restored-off", args.out, cache_probes)
for _ in range(40):
drain_snapshot = backend_drain_snapshot(rust)
if drain_snapshot["passed"]:
break
time.sleep(.05)
write_json(args.out / "backend-drain.json", drain_snapshot)
if not drain_snapshot["passed"]:
raise RuntimeError("backend active exchanges did not drain to zero")
traffic_after = traffic_analytics_snapshot(rust)
traffic_delta = traffic_analytics_delta(traffic_before, traffic_after)
traffic_samples.append({"phase": "after_drain", "seconds": time.monotonic() - start,
"snapshot": traffic_after, "delta_from_start": traffic_delta})
finally:
monitor.stop()
samples = monitor.samples
resources = monitor.summary(totals["completed"])
write_json(args.out / "resources.json", samples)
write_json(args.out / "status-counters.json", counter_samples)
write_json(args.out / "traffic-analytics.json", traffic_samples)
write_json(args.out / "failure-examples.json", failure_examples)
except Exception as exc:
fatal = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", fatal)
report = {"completed": fatal is None, "simulation": False, "exploratory": args.exploratory,
"duration_seconds": runtime_seconds, "soak_duration_gate": runtime_seconds >= 1800,
"totals": totals, "resources": resources, "memory_trend": memory_trend(samples) if samples else None,
"cache_toggles": len(toggles), "errors": failure_examples,
"traffic_analytics": {"observations": len(traffic_samples), "delta": traffic_delta,
"includes_cache_mode_probes_and_cooldown": True},
"cache_mode_probes": {"source_checks": len(cache_probes),
"requests": sum(len(probe["responses"]) for probe in cache_probes),
"passed": bool(cache_probes) and all(probe["passed"] for probe in cache_probes)},
"observed_cache_status_counts": cache_status_counts,
"configuration_restored": configuration_restored, "backend_drain": drain_snapshot,
"actual_completed_rps": totals["completed"] / runtime_seconds if runtime_seconds else None,
"passed": fatal is None and totals["failed"] == 0 and runtime_seconds >= args.duration}
write_json(args.out / "report.json", report)
seal_results(args.out)
print(json.dumps(report), flush=True)
return 0 if report["passed"] else 1
if __name__ == "__main__":
sys.exit(main())
scripts/performance/telemetry.py
-47
View File
@@ -1,47 +0,0 @@
"""Selected counters from existing authenticated APIs, without traffic records."""
from __future__ import annotations
import json
import time
def traffic_analytics_snapshot(rust, timeout=2):
"""Read global analytics queue loss; never retain paths, IPs or user agents.
queue_dropped is process-global even when the API's vhost query is used.
recorded_requests is asynchronous analytics work, not an HTTP success count.
Call outside a timed load interval so this control-plane read is not charged
to the request path being measured.
"""
status, _, body = rust.admin("/api/traffic", timeout=timeout)
if status != 200:
raise RuntimeError(f"traffic analytics counter capture failed: HTTP {status}")
payload = json.loads(body)
if not isinstance(payload, dict) or payload.get("scope") != "*" or payload.get("since") != "process_start":
raise RuntimeError("traffic analytics requires the global process-start scope")
analytics = payload.get("analytics")
if not isinstance(analytics, dict) or analytics.get("scope") != "*":
raise RuntimeError("traffic analytics response lacks the global queue view")
data = analytics.get("data")
if not isinstance(data, dict):
raise RuntimeError("traffic analytics response lacks the recorded counters")
values = {"queue_capacity": analytics.get("queue_capacity"),
"queue_dropped": analytics.get("queue_dropped"),
"recorded_requests": data.get("requests")}
if any(type(value) is not int or value < 0 for value in values.values()) or not values["queue_capacity"]:
raise RuntimeError("traffic analytics queue counters are absent or malformed")
return {"endpoint": "/api/traffic", "scope": "*", "since": "process_start",
"pid": rust.process.pid, "observed_monotonic": time.monotonic(), **values}
def traffic_analytics_delta(before, after):
"""Keep telemetry loss separate from failed HTTP requests and log rotation."""
for key in ["pid", "endpoint", "scope", "since", "queue_capacity"]:
if before[key] != after[key]:
raise RuntimeError(f"traffic analytics identity changed during capture: {key}")
result = {"queue_dropped_delta": after["queue_dropped"] - before["queue_dropped"],
"recorded_requests_delta": after["recorded_requests"] - before["recorded_requests"],
"interval_seconds": after["observed_monotonic"] - before["observed_monotonic"]}
if any(value < 0 for value in result.values()):
raise RuntimeError("traffic analytics counters or clock reset during capture")
return result
scripts/performance/transport_clients.py
-191
View File
@@ -1,191 +0,0 @@
"""Strict H2/TLS and H3/QUIC clients for local RustProxy acceptance tests.
The generated fixture certificate is explicitly trusted. Neither transport
silently falls back to HTTP/1.1 or disables certificate verification.
"""
from __future__ import annotations
import asyncio
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from pathlib import Path
import ssl
import httpx
from aioquic.asyncio import QuicConnectionProtocol, connect
from aioquic.h3.connection import H3Connection, MessageError
from aioquic.h3.events import DataReceived, HeadersReceived
from aioquic.quic.configuration import QuicConfiguration
from aioquic.quic.events import ConnectionTerminated, ProtocolNegotiated, StreamReset
@dataclass
class Response:
status: int
headers: dict[str, str]
body: bytes
protocol: str
alpn: str | None
stream_id: int | None = None
class H2Client:
def __init__(self, client: httpx.AsyncClient, port: int):
self.client, self.port = client, port
async def request(self, method, path, headers=None, body=None):
async def chunks():
for offset in range(0, len(body), 16384):
yield body[offset:offset + 16384]
await asyncio.sleep(0)
response = await self.client.request(
method, f"https://127.0.0.1:{self.port}{path}",
headers=headers or {}, content=chunks() if body is not None else None,
)
network = response.extensions.get("network_stream")
tls = network.get_extra_info("ssl_object") if network else None
alpn = tls.selected_alpn_protocol() if tls else None
if response.http_version != "HTTP/2" or alpn != "h2":
raise AssertionError(f"H2 required, got {response.http_version}, ALPN={alpn!r}")
return Response(response.status_code, dict(response.headers), response.content,
"HTTP/2", alpn, response.extensions.get("stream_id"))
@dataclass
class Pending:
future: asyncio.Future
status: int = 0
headers: dict[str, str] = field(default_factory=dict)
body: bytearray = field(default_factory=bytearray)
class HeadAwareH3Connection(H3Connection):
"""Supply request-method context missing from aioquic 1.7.0's length check.
RFC 9110 section 8.6 permits HEAD's Content-Length to describe a GET body.
Still reject any actual HEAD DATA; keep aioquic's exact length check for all
other requests. This changes only the test client, never the target server.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.head_streams = set()
def _check_content_length(self, stream):
if stream.stream_id in self.head_streams:
if stream.content_length:
raise MessageError("HEAD response contained DATA")
return
super()._check_content_length(stream)
def validate_head_adapter():
"""Keep adapter checks separate from the real-server acceptance totals."""
from types import SimpleNamespace
connection = HeadAwareH3Connection.__new__(HeadAwareH3Connection)
connection.head_streams = {0}
for stream_id, declared, received, valid in [(0, 1040, 0, True), (0, 1040, 1, False),
(4, 1040, 1040, True), (4, 1040, 0, False)]:
stream = SimpleNamespace(stream_id=stream_id, expected_content_length=declared,
content_length=received)
try:
connection._check_content_length(stream)
accepted = True
except MessageError:
accepted = False
assert accepted == valid, "H3 fixture HEAD adapter changed ordinary framing validation"
return {"cases": 4, "passed": 4, "kind": "test-client self-test, not RustProxy coverage"}
class H3Client(QuicConnectionProtocol):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.http = HeadAwareH3Connection(self._quic)
self.pending: dict[int, Pending] = {}
self.alpn = None
self.authority = None
def quic_event_received(self, event):
if isinstance(event, ProtocolNegotiated):
self.alpn = event.alpn_protocol
if isinstance(event, ConnectionTerminated):
for item in self.pending.values():
if not item.future.done():
item.future.set_exception(ConnectionError(
f"QUIC closed code={event.error_code}: {event.reason_phrase}"))
if isinstance(event, StreamReset):
item = self.pending.get(event.stream_id)
if item is not None and not item.future.done():
item.future.set_exception(ConnectionError(
f"H3 stream reset code={event.error_code}"))
for http_event in self.http.handle_event(event):
if not isinstance(http_event, (HeadersReceived, DataReceived)):
continue
item = self.pending.get(http_event.stream_id)
if item is None or item.future.done():
continue
if isinstance(http_event, HeadersReceived):
for name, value in http_event.headers:
if name == b":status":
item.status = int(value)
elif not name.startswith(b":"):
item.headers[name.decode("ascii").lower()] = value.decode("latin-1")
else:
item.body.extend(http_event.data)
if len(item.body) > 4 * 1024 * 1024:
item.future.set_exception(ValueError("fixture response exceeded 4 MiB bound"))
continue
if http_event.stream_ended:
item.future.set_result(Response(item.status, item.headers, bytes(item.body),
"HTTP/3", self.alpn, http_event.stream_id))
async def request(self, method, path, headers=None, body=None):
if self.alpn != "h3":
raise AssertionError(f"H3 required, got ALPN={self.alpn!r}")
stream_id = self._quic.get_next_available_stream_id()
item = Pending(asyncio.get_running_loop().create_future())
self.pending[stream_id] = item
if method == "HEAD":
self.http.head_streams.add(stream_id)
fields = [(b":method", method.encode("ascii")), (b":scheme", b"https"),
(b":authority", self.authority.encode("ascii")),
(b":path", path.encode("ascii"))]
fields += [(name.lower().encode("ascii"), value.encode("latin-1"))
for name, value in (headers or {}).items()]
try:
self.http.send_headers(stream_id, fields, end_stream=body is None)
self.transmit()
if body is not None:
if not body:
self.http.send_data(stream_id, b"", end_stream=True)
self.transmit()
for offset in range(0, len(body), 16384):
end = min(offset + 16384, len(body))
self.http.send_data(stream_id, body[offset:end], end_stream=end == len(body))
self.transmit()
await asyncio.sleep(0)
return await asyncio.wait_for(item.future, timeout=10)
finally:
self.pending.pop(stream_id, None)
self.http.head_streams.discard(stream_id)
@asynccontextmanager
async def client(protocol: str, port: int, certificate: Path):
if protocol == "h2":
context = ssl.create_default_context(cafile=str(certificate))
async with httpx.AsyncClient(http2=True, verify=context, trust_env=False,
timeout=10, limits=httpx.Limits(max_connections=1)) as session:
yield H2Client(session, port)
elif protocol == "h3":
config = QuicConfiguration(is_client=True, alpn_protocols=["h3"],
server_name="localhost", idle_timeout=10)
config.verify_mode = ssl.CERT_REQUIRED
config.load_verify_locations(cafile=str(certificate))
async with connect("127.0.0.1", port, configuration=config,
create_protocol=H3Client) as session:
session.authority = f"127.0.0.1:{port}"
yield session
else:
raise ValueError(f"unsupported fixture protocol {protocol!r}")
scripts/performance/transport_regression.py
-281
View File
@@ -1,281 +0,0 @@
#!/usr/bin/env python3
"""Real, bounded RustProxy H2/TLS, H3/QUIC and H1 WebSocket acceptance.
Install Python dependencies aioquic, httpx[http2] and websockets. Example:
python3 scripts/performance/transport_regression.py --binary before=/abs/binary \
--binary after=/abs/binary --out /abs/results --python-deps /abs/optional/pydeps
Only fresh temporary databases and loopback listeners are used. Certificates
are generated by RustProxy and verified by clients; no protocol fallback is used.
This is an interoperability/functional suite, not a throughput benchmark.
"""
from __future__ import annotations
import argparse
import asyncio
import copy
import hashlib
import importlib.metadata
import json
from pathlib import Path
import platform
import socket
import sys
import time
import traceback
from common import Fixture, seal_results, sha256, write_json
from peer import Peer
def response_details(response):
return {"status": response.status, "protocol": response.protocol,
"alpn": response.alpn, "stream_id": response.stream_id,
"bytes": len(response.body), "sha256": hashlib.sha256(response.body).hexdigest(),
"cache": response.headers.get("x-cache"),
"cache_reason": response.headers.get("x-cache-reason")}
def tls_port(fixture):
while True:
port = fixture.port()
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as udp:
try:
udp.bind(("127.0.0.1", port))
return port
except OSError:
continue
def transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin):
config = copy.deepcopy(rust.base)
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
identifier = f"{source}-tls"
config["listeners"]["endpoints"].append({
"id": identifier, "name": identifier, "scope": "internal", "enabled": True,
"binding": {"address": f"127.0.0.1:{port}", "tls": True, "http2": True,
"http3": {"max_concurrent_connections": 16,
"max_concurrent_bidistream": 16,
"max_concurrent_unistream": 16, "max_idle_timeout": 10}}})
vhost = next(v for v in config["vhosts"] if v["id"] == source)
vhost["frontend"]["listener_ids"].append(identifier)
vhost["http2_enabled"] = True
vhost["http3_enabled"] = True
config["listeners"]["endpoints"].append({
"id": "websocket", "name": "WebSocket", "scope": "internal", "enabled": True,
"binding": {"address": f"127.0.0.1:{websocket_port}", "tls": False,
"http2": False, "http3": None}})
config["vhosts"].append({
"id": "websocket", "frontend": {"hosts": ["127.0.0.1", "localhost"],
"listener_ids": ["websocket"]},
"websockets": True,
"backends": [{"id": "echo", "url": f"http://127.0.0.1:{websocket_origin}",
"allow_private": True}],
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []})
config["cache"].update({"enabled": False, "expose_status": True, "expose_hits": True,
"expose_cache_status": True, "expose_only_hits": False})
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
config["web_services"][0]["cache_control"] = "public, max-age=60"
return config
async def http_cases(check, session, protocol, source, cache, fixture):
prefix = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
expected = (fixture.site / "small.html").read_bytes()
path = f"/transport-{protocol}.html"
first = await session.request("GET", path)
check(prefix + ".get", first.status == 200 and first.body == expected,
**response_details(first))
second = await session.request("GET", path)
check(prefix + ".cache_delivery", second.status == 200 and second.body == expected
and ((first.headers.get("x-cache") == "MISS" and second.headers.get("x-cache") == "HIT")
if cache else second.headers.get("x-cache") not in ("MISS", "HIT")),
first_cache=first.headers.get("x-cache"), **response_details(second))
head = await session.request("HEAD", path)
check(prefix + ".head", head.status == 200 and head.body == b""
and head.headers.get("content-length") == str(len(expected)), **response_details(head))
etag = second.headers.get("etag")
check(prefix + ".validator", bool(etag), etag=etag)
ranged = await session.request("GET", path, {"range": "bytes=11-40", "if-range": etag or ""})
check(prefix + ".range", ranged.status == 206 and ranged.body == expected[11:41]
and ranged.headers.get("content-range") == f"bytes 11-40/{len(expected)}",
**response_details(ranged))
mismatch = await session.request("GET", path, {"range": "bytes=11-40", "if-range": '"different"'})
check(prefix + ".if_range_mismatch", mismatch.status == 200 and mismatch.body == expected,
**response_details(mismatch))
conditional = await session.request("GET", path, {"if-none-match": etag or ""})
check(prefix + ".conditional_304", conditional.status == 304 and conditional.body == b"",
**response_details(conditional))
invalid = await session.request("GET", path, {"range": "bytes=999999-"})
check(prefix + ".range_416", invalid.status == 416 and invalid.body == b""
and invalid.headers.get("content-range") == f"bytes */{len(expected)}",
**response_details(invalid))
for name in ["large.bin", "huge.bin"]:
expected_file = (fixture.site / name).read_bytes()
response = await session.request("GET", "/" + name)
check(prefix + ".get." + name, response.status == 200 and response.body == expected_file,
**response_details(response))
parallel = await asyncio.gather(*(session.request("GET", "/large.bin") for _ in range(4)))
expected_large = (fixture.site / "large.bin").read_bytes()
check(prefix + ".multiplex_four_streams", all(r.status == 200 and r.body == expected_large for r in parallel)
and len({r.stream_id for r in parallel}) == 4,
stream_ids=[r.stream_id for r in parallel], response_bytes=[len(r.body) for r in parallel])
if source == "reverse":
for payload in [b"", (bytes(range(256)) * 512) + b"streamed-end"]:
response = await session.request("POST", "/echo?transport=preserved",
{"content-type": "application/octet-stream"}, payload)
data = json.loads(response.body) if response.status == 200 else {}
check(prefix + f".post_stream_{len(payload)}", response.status == 200
and data.get("method") == "POST" and data.get("body_bytes") == len(payload)
and data.get("body_sha256") == hashlib.sha256(payload).hexdigest()
and data.get("path") == "/echo?transport=preserved",
upload_bytes=len(payload), **response_details(response))
else:
response = await session.request("POST", path, body=b"static-method-gate")
check(prefix + ".post_static_method_gate", response.status == 405, **response_details(response))
async def websocket_cases(check, port, cache):
import websockets
prefix = f"h1.websocket.cache_{'on' if cache else 'off'}"
async with websockets.connect(f"ws://127.0.0.1:{port}/echo", open_timeout=5,
close_timeout=3, ping_interval=None, compression=None) as session:
text = "RustProxy echo: Zażółć gęślą jaźń"
await session.send(text)
returned = await asyncio.wait_for(session.recv(), timeout=5)
check(prefix + ".text_echo", returned == text)
binary = bytes(range(256)) * 256
await session.send(binary)
returned = await asyncio.wait_for(session.recv(), timeout=5)
check(prefix + ".binary_echo", returned == binary, bytes=len(returned))
pong = await session.ping(b"rustproxy-transport")
await asyncio.wait_for(pong, timeout=5)
check(prefix + ".ping_pong", True)
await session.close(code=1000, reason="fixture complete")
check(prefix + ".normal_close", session.close_code == 1000, close_code=session.close_code)
async def run_variant(binary, label, output, workers):
import websockets
from transport_clients import client
cases = []
started = time.monotonic()
def check(case, passed, **details):
cases.append({"case": case, "passed": bool(passed), **details})
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(output / "checks.json", cases)
try:
with Fixture(output, workers=workers) as fixture:
plain = (fixture.site / "small.html").read_bytes()
for name in ["transport-h2.html", "transport-h3.html", "transport-shared.html"]:
(fixture.site / name).write_bytes(plain)
peer = Peer(fixture.origin_port, fixture.site)
websocket_origin, websocket_port = fixture.port(), fixture.port()
async def echo(session, _path=None):
async for message in session:
await session.send(message)
try:
async with websockets.serve(echo, "127.0.0.1", websocket_origin,
ping_interval=None, compression=None, close_timeout=3):
rust = fixture.rust(binary, label)
reverse_tls, static_tls = tls_port(fixture), tls_port(fixture)
config = transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin)
rust.apply(config, "transport-listeners")
fixture.stop(rust.process)
rust.start("transport")
certificate = rust.data / "certs" / "default.pem"
write_json(output / "transport.json", {
"certificate_sha256": sha256(certificate), "certificate_verification": True,
"h2_alpn_required": "h2", "h3_alpn_required": "h3",
"reverse_tls_tcp_udp": reverse_tls, "static_tls_tcp_udp": static_tls,
"websocket_h1_tcp": websocket_port, "fallback_allowed": False})
for cache in [False, True]:
if cache:
config["cache"]["enabled"] = True
rust.apply(config, "transport-cache-on")
for protocol in ["h2", "h3"]:
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
group = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
try:
async with client(protocol, port, certificate) as session:
await http_cases(check, session, protocol, source, cache, fixture)
except Exception as exc:
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
try:
await websocket_cases(check, websocket_port, cache)
except Exception as exc:
check(f"h1.websocket.cache_{'on' if cache else 'off'}.runner", False,
error=str(exc), traceback=traceback.format_exc())
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
try:
async with client("h2", port, certificate) as h2:
warm = await h2.request("GET", "/transport-shared.html")
async with client("h3", port, certificate) as h3:
hit = await h3.request("GET", "/transport-shared.html")
check(f"cache_cross_protocol.{source}.h2_to_h3",
warm.status == 200 and hit.status == 200 and hit.body == plain
and hit.headers.get("x-cache") == "HIT",
first_cache=warm.headers.get("x-cache"), **response_details(hit))
except Exception as exc:
check(f"cache_cross_protocol.{source}.runner", False,
error=str(exc), traceback=traceback.format_exc())
check("runtime.still_alive", rust.process.poll() is None)
finally:
peer.close()
except Exception as exc:
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"label": label, "simulation": False, "seconds": time.monotonic() - started,
"passed": sum(case["passed"] for case in cases),
"failed": sum(not case["passed"] for case in cases), "cases": cases}
write_json(output / "report.json", report)
print(f"{label}: {report['passed']} passed, {report['failed']} failed", flush=True)
return report
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--python-deps", type=Path)
args = parser.parse_args()
if args.python_deps:
sys.path.insert(0, str(args.python_deps.resolve()))
variants = {}
for value in args.binary:
label, separator, path = value.partition("=")
if not separator or not label.replace("-", "").replace("_", "").isalnum() or label in variants:
parser.error("--binary must be unique SIMPLE_LABEL=/ABS/PATH")
variants[label] = Path(path).resolve()
args.out = args.out.resolve()
args.out.mkdir(parents=True, exist_ok=True)
from transport_clients import validate_head_adapter
write_json(args.out / "environment.json", {
"kind": "real RustProxy transport acceptance", "simulation": False,
"python": sys.version, "platform": platform.platform(),
"dependencies": {name: importlib.metadata.version(name) for name in ["aioquic", "httpx", "h2", "websockets"]},
"aioquic_head_adapter": validate_head_adapter(),
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"not_covered": ["production certificates", "TLS upstreams", "H3 upstreams", "H2/H3 WebSocket extended CONNECT",
"network loss or migration", "24-hour soak", "real-browser navigation"]})
reports = {}
for label, binary in variants.items():
reports[label] = asyncio.run(run_variant(binary, label, args.out / label, args.workers))
write_json(args.out / "comparison.json", {
"variants": {label: {key: report[key] for key in ["passed", "failed", "seconds"]}
for label, report in reports.items()},
"cases": [{"case": case, **{label: next((row["passed"] for row in report["cases"] if row["case"] == case), "not_run")
for label, report in reports.items()}}
for case in sorted({row["case"] for report in reports.values() for row in report["cases"]})]})
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/wrk_report.lua
-14
View File
@@ -1,14 +0,0 @@
-- No response hook: benchmarks must not be limited by Lua body processing.
-- Known payload/status/headers are verified separately before measurement.
done = function(summary, latency, requests)
io.write("RUSTPROXY_BENCH_JSON {")
io.write(string.format('"requests":%d,"duration_us":%d,"bytes":%d,',
summary.requests, summary.duration, summary.bytes))
io.write(string.format('"p50_us":%.3f,"p95_us":%.3f,"p99_us":%.3f,',
latency:percentile(50), latency:percentile(95), latency:percentile(99)))
io.write(string.format('"latency_mean_us":%.3f,"latency_max_us":%.3f,', latency.mean, latency.max))
io.write(string.format('"errors_connect":%d,"errors_read":%d,"errors_write":%d,"errors_timeout":%d,"errors_status":%d',
summary.errors.connect, summary.errors.read, summary.errors.write,
summary.errors.timeout, summary.errors.status))
io.write("}\n")
end
scripts/preflight.py
+1 -1
View File
@@ -74,7 +74,7 @@ def check(root: Path, repair_known: bool = False) -> list[str]:
flat = module.parent.with_suffix(".rs")
if flat.is_file():
raise ValueError(f"Conflicting Rust modules: {flat.relative_to(root)} and {module.relative_to(root)}")
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/TODO.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
if not (root / path).is_file():
raise ValueError(f"Incomplete RustProxy 0.31.9 source tree: missing {path}")
# The Reader uses compile-time include_str! for its bundled Markdown. A
scripts/tests/cache_age_policy_034_model.py
-62
View File
@@ -1,62 +0,0 @@
#!/usr/bin/env python3
"""Deterministic oracle for HTTP Age / local freshness-cap algebra.
No binary execution, no claim of measured throughput. Run with Python 3 only.
"""
import json
import random
from pathlib import Path
def old_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
age = max(header_age, apparent_age) + response_delay
return max(0, min(max_age, local_limit) - age)
def new_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
# RFC 9111 corrected_initial_age = max(apparent_age, age_value + response_delay)
age = max(apparent_age, header_age + response_delay)
lifetime = min(max_age, age + local_limit)
return max(0, lifetime - age)
def run():
cases = [
('user_cdn_example', 2_592_000, 1_553_483, 0, 0, 86_400),
('user_cdn_example_7d', 2_592_000, 1_553_483, 0, 0, 604_800),
('origin_nearly_stale', 2_592_000, 2_591_990, 0, 0, 86_400),
('origin_stale', 2_592_000, 2_592_010, 0, 0, 86_400),
('normal_small', 3_600, 0, 0, 0, 86_400),
('date_is_older', 3_600, 10, 600, 2, 86_400),
('rfc_delay_case', 3_600, 3, 10, 2, 86_400),
]
report = {'reference': 'RFC 9111 4.2.3 (Age) and 4.2.1 (freshness lifetime)',
'kind': 'algebra simulation; not RustProxy RPS benchmark',
'scenarios': [], 'randomized_checks': 0, 'false_miss_cases_old': 0}
for name, max_age, age, apparent, delay, local in cases:
old = old_freshness(max_age, age, apparent, delay, local)
new = new_freshness(max_age, age, apparent, delay, local)
report['scenarios'].append({'scenario': name, 'origin_max_age_seconds': max_age,
'received_age_seconds': age, 'apparent_age_seconds': apparent,
'response_delay_seconds': delay, 'local_max_ttl_seconds': local,
'old_freshness_seconds': old, 'new_freshness_seconds': new})
rng = random.Random(0x3400)
for _ in range(100_000):
origin = rng.randrange(0, 1 << 24)
header_age = rng.randrange(0, 1 << 24)
apparent = rng.randrange(0, 1 << 24)
delay = rng.randrange(0, 300)
local = rng.randrange(0, 604_801)
corrected_age = max(apparent, header_age + delay)
new = new_freshness(origin, header_age, apparent, delay, local)
expected = min(max(0, origin - corrected_age), local)
assert new == expected, (origin, header_age, apparent, delay, local, new, expected)
assert new <= local and new <= max(0, origin - corrected_age)
old = old_freshness(origin, header_age, apparent, delay, local)
report['false_miss_cases_old'] += old == 0 < new
report['randomized_checks'] += 1
return report
if __name__ == '__main__':
report = run()
print(json.dumps(report, ensure_ascii=False, indent=2))
scripts/tests/cache_age_transform_034_live.py
-212
View File
@@ -1,212 +0,0 @@
#!/usr/bin/env python3
"""Loopback integration regression: transformed Age, HOT/COLD, diagnostics, isolation.
Run after compiling:
python3 scripts/tests/cache_age_transform_034_live.py --binary target/release/rust-proxy
To reproduce the original regression with the user-supplied 0.34.0 binary:
python3 scripts/tests/cache_age_transform_034_live.py --binary /path/to/old/rust-proxy --expect-old-age-bug
No internet and no user data; test config/data/passwords are in a disposable temporary dir.
"""
import argparse
import collections
import copy
import json
import sys
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'performance'))
from common import Fixture, http, sha256, write_json
BODY = b'/* test cache */\nbody { color: red; padding: 0px; }\n'
# Fixed sample from Brotli encoder, embedded to avoid Python dependency on brotli.
BROTLI_BODY = b"\x1b3\x00\xe8\x1d\x07\xee\xbe\r\xa2\xbc3'\x8cP\xe9\xad\xa0\xf2\xb2\xa7\xe4\xfa\x12\x84\xac\x0c\x1cr\xc0\xfe\xc0>\x0fR*a\x9c\x9e2\xe8\x08\xcb\x1cP\x881\xac\xdd\x88s5(\x9d\x01"
COUNTS = collections.Counter()
class Origin(BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_GET(self):
path = self.path.split('?', 1)[0]
if path not in {'/aged.css', '/normal.css', '/private.css', '/no-store.css', '/other.css', '/br.css', '/aged-br.css'}:
self.send_error(404)
return
COUNTS[path] += 1
self.send_response(200)
if path in ('/aged.css', '/aged-br.css'):
policy = 'public, max-age=2592000'
elif path == '/private.css':
policy = 'private, max-age=2592000'
elif path == '/no-store.css':
policy = 'no-store'
else:
policy = 'public, max-age=3600'
self.send_header('Cache-Control', policy)
self.send_header('Content-Type', 'text/css')
payload = BROTLI_BODY if path in ('/br.css', '/aged-br.css') else BODY
self.send_header('Content-Length', str(len(payload)))
if path in ('/br.css', '/aged-br.css'):
self.send_header('Content-Encoding', 'br')
self.send_header('ETag', '"origin-css"')
self.send_header('Last-Modified', 'Thu, 07 Jan 2016 20:04:59 GMT')
self.send_header('Vary', 'Accept-Encoding')
if path in ('/aged.css', '/aged-br.css'):
self.send_header('Age', '1553483')
self.end_headers()
self.wfile.write(payload)
def request(port, path, encoding='identity'):
status, headers, body = http(port, path, headers={'Accept-Encoding': encoding})
return {'status': status, 'x_cache': headers.get('x-cache'),
'x_cache_hits': headers.get('x-cache-hits'), 'x_cache_resident_age': headers.get('x-cache-resident-age'),
'cache_status': headers.get('cache-status'), 'age': headers.get('age'),
'body': body.decode('utf-8', 'replace'), 'content_length': headers.get('content-length'),
'etag': headers.get('etag'), 'encoding': headers.get('content-encoding'), 'origin_count': COUNTS[path.split('?', 1)[0]]}
def dash(rust):
status, _, body = rust.admin('/api/cache?limit=200')
assert status == 200, ('cache dashboard', status, body[:200])
data = json.loads(body)['dashboard']
return {'objects': [{k: row.get(k) for k in ('vhost','path','tier','transformed',
'hits','age_seconds','resident_age_seconds','remaining_ttl_seconds')} for row in data['objects']],
'domains': data['domains']}
def find(d, path):
matches = [item for item in d['objects'] if item['path'] == path and item['vhost'] == 'reverse']
assert len(matches) == 1, (path, matches)
return matches[0]
def run(binary: Path, output: Path, expect_old_age_bug: bool):
outcome = {'binary_sha256': sha256(binary), 'expect_old_age_bug': expect_old_age_bug,
'tests': [], 'passed': False}
output.mkdir(parents=True, exist_ok=True)
with Fixture(output, workers=2) as fixture:
origin = ThreadingHTTPServer(('127.0.0.1', fixture.origin_port), Origin)
thread = threading.Thread(target=origin.serve_forever, daemon=True)
thread.start()
try:
rust = fixture.rust(binary, 'cache-age-034')
rust.login()
cfg = copy.deepcopy(rust.base)
cfg['cache'] = {'enabled': True, 'max_ttl_seconds': 86_400,
'expose_status': True, 'expose_hits': True,
'expose_cache_status': True, 'expose_resident_age': True}
cfg['transform'] = {'enabled': True, 'css': True}
for vhost in cfg['vhosts']:
vhost['transform'] = {'enabled': True, 'css': True}
rust.apply(cfg, 'fresh-cache-transform')
for path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css', '/private.css', '/no-store.css'):
before = COUNTS[path]
encoding = 'br' if path in ('/br.css', '/aged-br.css') else 'identity'
first, second = request(rust.reverse_port, path, encoding), request(rust.reverse_port, path, encoding)
row = {'path': path, 'first': first, 'second': second,
'origin_delta': COUNTS[path] - before}
outcome['tests'].append(row)
if path in ('/aged.css', '/aged-br.css') and expect_old_age_bug:
assert first['x_cache'] == second['x_cache'] == 'MISS', row
assert row['origin_delta'] == 2, row
elif path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css'):
assert first['x_cache'] == 'MISS' and second['x_cache'] == 'HIT', row
assert row['origin_delta'] == 1, row
assert second['cache_status'] and 'transformed' in second['cache_status'], row
assert first['etag'] == second['etag'] and first['body'] == second['body'], row
else:
assert first['x_cache'] == second['x_cache'] == 'MISS', row
print(path, first['x_cache'], second['x_cache'], 'origin+', row['origin_delta'], flush=True)
snapshot = dash(rust)
outcome['tests'].append({'dashboard': snapshot})
assert find(snapshot, '/normal.css')['transformed'], snapshot
aged = find(snapshot, '/aged.css')
if expect_old_age_bug:
assert aged['remaining_ttl_seconds'] == 0, aged
else:
assert aged['transformed'] and aged['remaining_ttl_seconds'] >= 86_000, aged
assert aged['age_seconds'] >= 1_553_483, aged
assert aged['resident_age_seconds'] < 10, aged
aged_br = find(snapshot, '/aged-br.css')
assert aged_br['transformed'] and aged_br['remaining_ttl_seconds'] >= 86_000, aged_br
# Query-string bypass is an explicit administrator rule. Even a
# transformed CSS with public max-age stays BYPASS while it is ON,
# then the *same* URL must become MISS -> HIT after disabling it.
# This is not a transform regression: it is a request policy.
cfg['cache']['bypass_query'] = True
rust.apply(cfg, 'query-bypass-enabled')
query_path = '/normal.css?v=transformed-public'
before = COUNTS['/normal.css']
q_bypass = [request(rust.reverse_port, query_path) for _ in range(2)]
assert [row['x_cache'] for row in q_bypass] == ['BYPASS', 'BYPASS'], q_bypass
assert COUNTS['/normal.css'] - before == 2, q_bypass
cfg['cache']['bypass_query'] = False
rust.apply(cfg, 'query-bypass-disabled')
q_cached = [request(rust.reverse_port, query_path) for _ in range(2)]
assert [row['x_cache'] for row in q_cached] == ['MISS', 'HIT'], q_cached
assert q_cached[1]['cache_status'] and 'transformed' in q_cached[1]['cache_status']
assert COUNTS['/normal.css'] - before == 3, q_cached
outcome['tests'].append({'query_bypass_policy': {'bypass': q_bypass, 'cached': q_cached}})
# Operational LB tuning must not invalidate a cached representation.
before = COUNTS['/normal.css']
cfg['vhosts'][0]['load_balancing']['keepalive'] = {'max_idle_per_host': 16}
rust.apply(cfg, 'only-backend-keepalive-changed')
normal = request(rust.reverse_port, '/normal.css')
assert normal['x_cache'] == 'HIT' and COUNTS['/normal.css'] == before
outcome['tests'].append({'keepalive_update_without_purge': normal})
# Diagnostics can be intentionally hidden on MISS and remain visible on HIT.
cfg['cache']['expose_only_hits'] = True
rust.apply(cfg, 'only-show-hit-headers')
before = COUNTS['/other.css']
first, second = request(rust.reverse_port, '/other.css'), request(rust.reverse_port, '/other.css')
assert first['x_cache'] is None and second['x_cache'] == 'HIT'
assert COUNTS['/other.css'] - before == 1
outcome['tests'].append({'only_hits': [first, second]})
if not expect_old_age_bug:
# Cold path on a 1-entry HOT tier: demotion and on-disk reuse.
cfg['cache'].update({'max_entries': 1, 'cold_enabled': True, 'cold_on_lru': True,
'cold_path': 'cache/age-regression-cold'})
cfg['cache'].pop('expose_only_hits', None)
rust.apply(cfg, 'cold-enabled-one-hot-entry')
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'MISS'
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'HIT'
assert request(rust.reverse_port, '/other.css')['x_cache'] == 'MISS'
deadline = time.monotonic() + 4
while True:
snapshot = dash(rust)
matches = [x for x in snapshot['objects'] if x['path'] == '/aged.css' and x['tier'] == 'cold']
if matches:
assert matches[0]['remaining_ttl_seconds'] >= 86_000, matches
break
if time.monotonic() > deadline:
raise AssertionError('aged transformed CSS never demoted to COLD: ' + str(snapshot))
time.sleep(.05)
cold_hit = request(rust.reverse_port, '/aged.css')
assert cold_hit['x_cache'] == 'HIT' and 'transformed' in cold_hit['cache_status'], cold_hit
outcome['tests'].append({'cold': {'row': matches[0], 'hit': cold_hit}})
outcome['passed'] = True
finally:
origin.shutdown()
origin.server_close()
write_json(output / 'cache-age-transform-result.json', outcome)
return outcome
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--binary', type=Path, required=True)
parser.add_argument('--output', type=Path, default=Path('/tmp/rustproxy-cache-age-034-live'))
parser.add_argument('--expect-old-age-bug', action='store_true')
args = parser.parse_args()
try:
report = run(args.binary.resolve(), args.output.resolve(), args.expect_old_age_bug)
except Exception as error:
print('FAIL', type(error).__name__, str(error)[:1000], file=sys.stderr)
raise
print('PASS', len(report['tests']), 'scenarios; see', args.output / 'cache-age-transform-result.json')
scripts/tests/cache_hot_034_static_test.py
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env python3
"""Source-only hot-cache safety guards; NOT a Rust compile, concurrency test or RPS result."""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
read = lambda path: (ROOT / path).read_text()
hot = read("src/cache/lookup/hot.rs")
entry = read("src/cache/model/cache.rs")
model = read("src/cache/model/maintenance.rs")
inv = read("src/cache/invalidation/invalidate.rs")
dash = read("src/cache/dashboard.rs")
assert 'headers: Arc<HeaderMap>' in entry
assert 'Arc::clone(&entry.headers)' in hot
assert 'drop(store);' in hot
assert hot.index('drop(store);\n\n let mut response') < hot.index('headers.as_ref().clone()')
assert 'self.domain_stats.saved_body_hit(&ticket.scope, &ticket.domain, body_bytes)' in hot
assert 'store.lru.remove(&old_sequence)' in hot and 'store.lru.insert(sequence, storage_key)' in hot
assert '.sweep()' not in hot
assert 'fn sweep(' not in entry
stats = read('src/cache/stats.rs')
assert 'fn saved_body_hit(' in stats
assert 'const DOMAIN_STATS_LIMIT: usize = 4096' in stats
assert 'ArcSwap<PublishedDomains>' in stats
assert 'fn record_reason(&self, reason:' in stats
assert 'self.store.lock()' not in stats
assert 'if self.ban_rule_count.load(Ordering::Acquire) == 0' in inv
assert 'bans.iter().any(|rule| rule.matches_hot' in inv
assert 'fn matching_hot_key(' in model and 'entry.vary.is_empty()' in model
assert 'self.domain_stats.clear()' in dash
assert 'self.domain_stats.record(scope, domain, outcome, reason);' in dash
assert 'include!("tests/hot_hit_034_tests.rs")' in read('src/cache/tests.rs')
for path in (ROOT / 'src').rglob('*.rs'):
if '/tests/' not in path.as_posix() and path.name != 'tests.rs':
assert len(path.read_text().splitlines()) <= 500, path
print('PASS HOT path static invariants (no Rust compilation or throughput claim)')
scripts/tests/cache_scope_034_static_test.py
-40
View File
@@ -1,40 +0,0 @@
#!/usr/bin/env python3
"""0.34.0 source-wiring checks only; do not mistake these for Rust tests."""
from pathlib import Path
import tomllib
root = Path(__file__).resolve().parents[2]
read = lambda path: (root / path).read_text()
assert tomllib.loads(read("Cargo.toml"))["package"]["version"] == "0.34.0"
assert 'version = "0.34.0"' in read("Cargo.lock")
app = read("src/runtime/updates.rs")
assert "CacheInvalidation::Scopes(scopes)" in app
assert "this.cache.purge_scope(&scope)" in app
assert "this.cache.set_control_revision(envelope.revision)" in app
assert app.index("this.cache.set_control_revision(envelope.revision)") < app.index("match invalidation")
source = read("src/runtime/types.rs")
assert "view.load_balancing = Default::default();" in source
assert "member.max_concurrent_requests = 0;" in source
assert "fn vhost_cache_revision" in source
assert "&web_services" in source
assert "service.sendfile = crate::web_server::SendfileMode::Off;" in source
assert "&config.php_fpm_profiles" in source
for filename in ("src/proxy/forward/route_cache.rs", "src/proxy/forward/web_source.rs"):
assert "ticket_scoped_for_config(" in read(filename)
assert "snapshot.envelope.revision" in read(filename)
assert "expected_control_revision" in read("src/cache/tickets.rs")
assert "file.scope_revision != expected_scope_revision" in read("src/cache/cold/startup.rs")
assert "scope_revision: draft.scope_revision" in read("src/cache/cold/worker.rs")
body = read("src/proxy/body.rs")
assert "response_backend: Option<Arc<crate::balance::Lease>>" in body
assert "lease.bytes(count, false);" in body
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
assert "lease.bytes(count, true);" in body
assert "cold_scope_fingerprint_preserves_unmodified_vhost" in read("src/cache/tests/cold_tier_tests.rs")
assert "stale_snapshot_cannot_issue_cache_ticket" in read("src/cache/tests/scope_tests.rs")
for file in (root / 'src').rglob('*.rs'):
if '/tests/' in file.as_posix() or file.name == 'tests.rs':
continue
if len(file.read_text().splitlines()) > 500:
raise AssertionError(f"production Rust source exceeds 500 lines: {file}")
print('PASS source wiring for scoped invalidation, cold fingerprints, stale-snapshot barrier, response accounting and <=500-line modules (STATIC ONLY)')
scripts/tests/dashboard_chart_headers_0335_browser_test.py
-97
View File
@@ -1,97 +0,0 @@
#!/usr/bin/env python3
"""Overview traffic/connection heading cleanup against Chromium and the bundled JS.
Runs a simulated authenticated DOM rather than a live RustProxy service.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
ROUTER = (ROOT / 'src/admin/router.rs').read_text(encoding='utf-8')
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text(encoding='utf-8'))
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text(encoding='utf-8')
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
bundle = app_bundle()
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=bundle)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
currentPanel = 'overview';
samples.splice(0); samples.push(1.5, 85.5, 12.3);
redrawOverviewHistoryCharts();
}''')
for density in ('normal', 'compact', 'ultra'):
for span in (3, 6):
data = page.evaluate('''({density,span}) => {
document.documentElement.dataset.dashboardDensity = density;
const visible = ['traffic_chart', 'connection_chart'];
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !visible.includes(id));
applyDashboardLayout([...visible,...hidden], hidden,
{traffic_chart:span,connection_chart:span});
const traffic = document.querySelector('[data-dashboard-widget="traffic_chart"]');
const connection = document.querySelector('[data-dashboard-widget="connection_chart"]');
const heading = traffic.querySelector('.card-heading');
const peak = traffic.querySelector('#chart-peak');
const peakRect = peak.getBoundingClientRect();
const cardRect = traffic.getBoundingClientRect();
const headingRect = heading.getBoundingClientRect();
return {
title: heading.querySelector('h2')?.textContent.trim(),
peak: peak.textContent,
peakInHeading: peak.parentElement === heading,
peakNotInFooter: !traffic.querySelector('.chart-footer #chart-peak'),
noTrafficEyebrow: !traffic.querySelector('.small-label'),
noTrafficLiveBadge: !traffic.querySelector('.pill.live'),
noConnectionLiveBadge: !connection.querySelector('.pill.live'),
connectionTitleIntact: !!connection.querySelector('.card-heading h3.overview-chart-heading[data-i18n="connectionActivity"]'),
peakInsideCard: peakRect.left >= cardRect.left - 1 && peakRect.right <= cardRect.right + 1,
peakInsideHeading: peakRect.top >= headingRect.top - 1 && peakRect.bottom <= headingRect.bottom + 1,
browserOverflow: document.documentElement.scrollWidth - innerWidth,
};
}''', {'density':density,'span':span})
assert data['title'] == 'Requests per second', (width, density, span, data)
assert data['peak'] == 'max 85.5 req/s', (width, density, span, data)
for key in ('peakInHeading', 'peakNotInFooter', 'noTrafficEyebrow',
'noTrafficLiveBadge', 'noConnectionLiveBadge',
'connectionTitleIntact', 'peakInsideCard', 'peakInsideHeading'):
assert data[key], (width, density, span, key, data)
assert data['browserOverflow'] <= 2, (width, density, span, data)
assert not errors, (width, errors)
print(f'PASS traffic/connection chart headers at {width}px (all densities, spans 3 and 6)')
page.close()
browser.close()
print('PASS Overview chart heading cleanup')
scripts/tests/dashboard_chart_typography_0337_browser_test.py
-168
View File
@@ -1,168 +0,0 @@
#!/usr/bin/env python3
"""All chart titles and series labels in simulated Chromium (no live Rust server).
Covers Overview, diagnostics, Web Server, Traffic, Cache/Transform, Vhosts,
Streams, and DOM-created chart/legend variants. Preserve localized DOM text.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
CHECK = r'''(settings) => {
const { density, compactBody, theme } = settings;
document.documentElement.dataset.dashboardDensity = density;
document.documentElement.dataset.theme = theme;
document.body.dataset.density = compactBody ? 'compact' : 'comfortable';
const titleSelectors = [
'#panel-overview .overview-wide-chart .card-heading h2',
'#panel-overview .activity-card .card-heading h2',
'#panel-overview :is(.system-card, .connection-card) .overview-chart-heading',
'#panel-overview :is(.stat-chart, .stat-donut) > span:first-child',
'.mini-live-chart > .mini-live-chart-head > span:first-child',
'.traffic-chart-card > h3.subheading',
'.function-live-chart-heading > strong',
'.vhost-chart-head > span:first-child',
];
const monoSelectors = [
'.overview-wide-summary', '.overview-wide-legend',
'.overview-donut-legend', '.www-method-legend', '.stat-chart-legend',
'.mini-chart-legend', '.connection-legend',
'.traffic-protocol-legend', '.traffic-chart-legend',
'.function-live-chart-legend', '.stream-chart-legend',
'.chart-footer', '.vhost-chart-head > :not(:first-child)',
'.mini-live-chart-head > b', '.mini-live-chart-head > .mini-live-chart-actions',
'.network-interface-rate',
'.function-live-chart-summary > b', '.function-live-chart-heading small',
'.function-live-chart-samples',
'#panel-overview .stat-chart > small',
'#panel-overview .overview-wide-chart > .fine',
'#panel-overview .www-top-row', '.traffic-bar-row',
'#panel-overview .system-card .card-heading h2',
'#panel-overview .system-card .card-heading .meta',
'#panel-overview .system-card .memory-line',
'#panel-overview .connection-card .card-heading h2',
];
const legendSelectors = [
'.overview-wide-legend', '.overview-donut-legend', '.www-ring-legend',
'.www-method-legend', '.stat-chart-legend', '.mini-chart-legend',
'.connection-legend', '.traffic-protocol-legend', '.traffic-chart-legend',
'.function-live-chart-legend', '.stream-chart-legend', '.stream-traffic-legend'
];
const legendNodes = [...document.querySelectorAll(legendSelectors.join(','))];
const requestLegend = document.querySelector('[data-dashboard-widget="traffic_chart"] .traffic-protocol-legend');
const referenceLegendSize = getComputedStyle(requestLegend).fontSize;
const titleNodes = Array.from(document.querySelectorAll(titleSelectors.join(',')));
const monoNodes = Array.from(document.querySelectorAll(monoSelectors.join(',')));
const base = document.createElement('div');
base.style.fontFamily = 'var(--mono)';
document.body.append(base);
const mono = getComputedStyle(base).fontFamily;
base.style.fontFamily = 'var(--sans)';
const sans = getComputedStyle(base).fontFamily;
base.remove();
const titleErrors = titleNodes.flatMap(node => {
const css = getComputedStyle(node);
const inOverview = !!node.closest('#panel-overview');
const isSmallCaption = inOverview && !!node.closest('.mini-live-chart-head');
const wantedFont = isSmallCaption ? mono : sans;
const wantedSize = isSmallCaption ? '10px' : '12px';
const wantedCase = inOverview ? 'none' : 'uppercase';
const problems = [];
if (css.fontFamily !== wantedFont) problems.push(`font=${css.fontFamily}`);
if (css.fontSize !== wantedSize) problems.push(`size=${css.fontSize}`);
if (css.textTransform !== wantedCase) problems.push(`case=${css.textTransform}`);
return problems.map(problem => `${node.outerHTML.slice(0, 100)}: ${problem}`);
});
const monoErrors = monoNodes.flatMap(node => {
const all = [node, ...node.querySelectorAll('*')].filter(el => el.tagName.toLowerCase() !== 'svg');
return all.filter(el => getComputedStyle(el).fontFamily !== mono)
.map(el => `${el.outerHTML.slice(0, 100)}: ${getComputedStyle(el).fontFamily}`);
});
const legendErrors = legendNodes.flatMap(root => {
const children = [root, ...root.querySelectorAll('span, b, strong, small, li, em')];
return children.flatMap(node => {
const style = getComputedStyle(node);
const issues = [];
if (style.fontSize !== referenceLegendSize) issues.push(`size=${style.fontSize}`);
if (style.fontFamily !== mono) issues.push(`font=${style.fontFamily}`);
return issues.map(issue => `${node.outerHTML.slice(0, 95)} ${issue}`);
});
});
// Keep localized Overview titles in sentence case; other sections retain
// their independent uppercase chart-heading convention.
const compression = document.querySelector('[data-dashboard-widget="wide_compression_activity"]');
const compressionHeading = compression.querySelector('.card-heading h2');
const compressionLegend = compression.querySelector('.overview-wide-legend');
const originalTranslation = compressionHeading.textContent.trim() === 'Aktywność kompresji';
const legendFonts = [...compressionLegend.querySelectorAll('span, b')]
.map(el => getComputedStyle(el).fontFamily);
const unrelatedHeading = document.querySelector('#panel-traffic .card-heading h2');
const numericReading = document.querySelector('#system-cpu');
return {
counts: {titles: titleNodes.length, mono: monoNodes.length, legends: legendNodes.length},
titleErrors, monoErrors, legendErrors, referenceLegendSize, mono, sans,
originalTranslation, compressionTitleSize: getComputedStyle(compressionHeading).fontSize,
compressionTitleCase: getComputedStyle(compressionHeading).textTransform,
compressionLegendIsMono: legendFonts.every(font => font === mono),
unrelatedTitleSize: getComputedStyle(unrelatedHeading).fontSize,
numericReadingIsMono: getComputedStyle(numericReading).fontFamily === mono,
};
}'''
with sync_playwright() as api:
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 960})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
// Representative dynamically rendered Vhost/Web Service/Stream charts.
for (const panel of ['panel-vhosts','panel-web-server','panel-streams']) {
document.getElementById(panel).insertAdjacentHTML('beforeend', `
<div class="vhost-chart"><div class="vhost-chart-head">
<span class="small-label">Connection history</span>
<span class="fine">42 req/s</span></div>
<svg viewBox="0 0 300 54"><path d=""></path></svg>
<div class="stream-chart-legend"><span>RX 2 MB/s</span><span>TX 1 MB/s</span></div>
</div>`);
}
const comp = document.querySelector('[data-dashboard-widget="wide_compression_activity"] .card-heading h2');
comp.textContent = 'Aktywność kompresji';
// The traffic renderer inserts these series dynamically.
document.querySelectorAll('.traffic-chart-legend').forEach(root => {
root.innerHTML = `<li><span class="traffic-chart-label">GET requests</span><b>12</b></li>`;
});
const sample = document.createElement('div');
sample.innerHTML = `<div class="www-top-row"><span>php-worker</span><b>11/s</b></div>
<div class="traffic-bar-row"><span class="traffic-bar-label">GET</span><b class="traffic-bar-value">22</b></div>
<div class="network-interface-rate">100 Mb/s</div>`;
document.getElementById('panel-overview').append(sample);
}''')
for density in ('normal','compact','ultra'):
for compact_body in (False, True):
for theme in ('dark', 'light'):
result = page.evaluate(CHECK, {'density': density, 'compactBody': compact_body, 'theme': theme})
description = (width, density, compact_body, theme)
assert result['counts']['titles'] >= 43, (description, result['counts'])
assert result['counts']['mono'] >= 90, (description, result['counts'])
assert not result['titleErrors'], (description, result['titleErrors'][:10])
assert not result['monoErrors'], (description, result['monoErrors'][:10])
assert result['counts']['legends'] >= 39, (description, result['counts'])
assert result['referenceLegendSize'] == '9px', description
assert not result['legendErrors'], (description, result['legendErrors'][:10])
assert result['originalTranslation'], description
assert result['compressionTitleSize'] == '12px', description
assert result['compressionTitleCase'] == 'none', description
assert result['compressionLegendIsMono'], description
assert result['numericReadingIsMono'], description
assert result['unrelatedTitleSize'] != '12px', (description, result['unrelatedTitleSize'])
print(f'PASS all chart families at {width}px (3 dashboard densities, 2 interface densities, 2 themes)')
page.close()
browser.close()
scripts/tests/dashboard_chart_width_0333_browser_test.py
-110
View File
@@ -1,110 +0,0 @@
#!/usr/bin/env python3
"""Real-browser regression for Overview chart content when card width changes.
Runs the production HTML/CSS/assembled admin JS against a simulated DOM and
fake network-interface samples. This does not exercise Rust HTTP/SSE.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
scripts = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
scripts.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
scripts.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
scripts.append(json.loads(line[:-1]))
return '\n'.join(scripts)
def inspect(page):
return page.evaluate('''() => {
const selectors = {
'websocket_protocol':'[data-dashboard-widget="websocket_protocol"] svg',
'wide_transform_activity':'[data-dashboard-widget="wide_transform_activity"] svg',
'www_requests':'[data-dashboard-widget="www_requests"] svg',
'traffic_chart':'[data-dashboard-widget="traffic_chart"] svg',
'system_chart':'[data-dashboard-widget="system_chart"] svg',
'cache_diagnostics':'[data-dashboard-widget="cache_diagnostics"] .mini-live-chart svg',
'network_diagnostics':'[data-dashboard-widget="network_diagnostics"] .mini-live-chart svg',
'network_interface':'[data-dashboard-widget="network_diagnostics"] .network-interface-row svg',
};
return Object.fromEntries(Object.entries(selectors).map(([name, selector]) => {
const svg = document.querySelector(selector);
if (!svg) throw new Error('Missing SVG: ' + name);
const box = svg.getBoundingClientRect();
const matrix = svg.getScreenCTM();
const viewBox = svg.viewBox.baseVal;
const contentWidth = Math.hypot(matrix.a, matrix.b) * viewBox.width;
const series = svg.querySelector('.overview-wide-series');
return [name, {width:box.width, contentWidth,
preservation:svg.getAttribute('preserveAspectRatio'),
lineStroke: series ? getComputedStyle(series).vectorEffect : null}];
}));
}''')
with sync_playwright() as browser_api:
browser = browser_api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for screen_width in (320, 375, 800, 1440, 1920):
page = browser.new_page(viewport={'width': screen_width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
currentPanel = 'overview';
renderNetworkInterfaceCharts([{name:'eth0',rx_bytes_per_second:12500,tx_bytes_per_second:5200}], true);
}''')
ids = ['websocket_protocol', 'wide_transform_activity', 'www_requests', 'traffic_chart',
'system_chart', 'cache_diagnostics', 'network_diagnostics']
results = []
for span in (3, 12):
page.evaluate('''({ids,span}) => {
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id));
const sizes = Object.fromEntries(ids.map(id => [id, span]));
applyDashboardLayout([...ids, ...hidden], hidden, sizes);
}''', {'ids': ids, 'span': span})
measured = inspect(page)
results.append(measured)
for chart, value in measured.items():
assert value['width'] > 50, (screen_width, span, chart, value)
assert abs(value['width'] - value['contentWidth']) <= 2, (
'chart not filling its own SVG viewport', screen_width, span, chart, value)
assert value['preservation'] == 'none', (screen_width, chart, value)
if value['lineStroke'] is not None:
assert value['lineStroke'] == 'non-scaling-stroke', (chart, value)
if screen_width >= 1440:
for name in results[0]:
assert results[1][name]['width'] > results[0][name]['width'] * 1.4, (
'chart does not grow when widening card', screen_width, name, results)
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, screen_width
assert not errors, (screen_width, errors)
print('PASS SVG chart uses full width at spans 3 and 12:', screen_width)
page.close()
browser.close()
print('PASS resizable Overview chart content regression')
scripts/tests/dashboard_cpu_autoscale_0335_browser_test.py
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env python3
"""Legacy CPU plot regression: low and high real values on a fixed 0-100% axis.
Autoscaling was intentionally replaced by 0.33.5 fixed scaling. This test
retains the original history/reset and label checks using the current scale.
Chromium is simulated, not a live RustProxy server.
"""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
def inspect(page):
return page.evaluate('''() => {
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
const backendSvg = get('backend_traffic').querySelector('svg');
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
const bounds = cpu.getBoundingClientRect();
const matrix = cpuSvg.getScreenCTM();
return {
cpuHeight: cpuSvg.getBoundingClientRect().height,
protocolHeight: protocolSvg.getBoundingClientRect().height,
backendHeight: backendSvg.getBoundingClientRect().height,
svgWidth: cpuSvg.getBoundingClientRect().width,
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
overflow: document.documentElement.scrollWidth - innerWidth,
};
}''')
def render_cpu(page, revision, system, process):
return page.evaluate('''({revision,system,process}) => {
window.rustProxyWideOverview.render({revision,live_system:{
cpu_percent:system, process_cpu_percent:process
}});
const read = name => {
const prefix = `overview-wide-wide_cpu-${name}`;
return {
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
label: document.getElementById(`${prefix}-value`).textContent,
};
};
return {system:read('system'),process:read('process')};
}''', {'revision': revision, 'system': system, 'process': process})
def ys(path):
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': viewport, 'height': 950})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const grid = document.getElementById('dashboard-overview-grid');
grid.classList.add('is-aligned');
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic'].includes(card.dataset.dashboardWidget);
}
window.currentPanel = 'overview';
window.$ = id => document.getElementById(id);
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
window.fmtBytes = value => `${value} B`;
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
}''')
page.add_script_tag(content=SCALE_JS)
page.add_script_tag(content=WIDE_JS)
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
result = inspect(page)
assert result['protocolHeight'] >= 124, (viewport, density, result)
assert abs(result['cpuHeight'] - result['protocolHeight']) <= 1, (
'CPU must use standard chart height', viewport, density, result)
assert result['backendHeight'] > 0, (viewport, density, result)
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
# Measured 0.5-0.8% stay numerically visible without axis auto-expansion.
first = render_cpu(page, 1, 0.5, 0.2)
second = render_cpu(page, 1, 0.8, 0.6)
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
assert ys(second['system']['path']) == [136, 136], (viewport, second)
assert ys(second['process']['path']) == [137, 136], (viewport, second)
# Multi-core values display accurately but clip at the fixed 100% ceiling.
third = render_cpu(page, 1, 3.4, 2.2)
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
high = render_cpu(page, 1, 250, 123)
assert high['system']['label'] == '250.0%'
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
# Revision changes reset the history. Zero stays at the plot baseline.
idle = render_cpu(page, 2, 0, 0)
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
missing = render_cpu(page, 3, None, None)
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
assert not errors, (viewport, errors)
print(f'PASS legacy CPU labels/history on fixed Y-scale at {viewport}px (all densities)')
page.close()
browser.close()
scripts/tests/dashboard_cpu_fixed_scale_0335_browser_test.py
-149
View File
@@ -1,149 +0,0 @@
#!/usr/bin/env python3
"""0.33.5 fixed 0-100% CPU plot and height regression with production HTML/CSS/JS.
Uses Chromium with simulated authenticated DOM and status samples, not a live
RustProxy server, SSE stream or real CPU telemetry.
"""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
def inspect(page):
return page.evaluate('''() => {
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
const connection = get('connection_chart');
const backendSvg = get('backend_traffic').querySelector('svg');
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
const bounds = cpu.getBoundingClientRect();
const matrix = cpuSvg.getScreenCTM();
return {
cpuHeight: cpuSvg.getBoundingClientRect().height,
cpuCardHeight: bounds.height,
connectionCardHeight: connection.getBoundingClientRect().height,
protocolHeight: protocolSvg.getBoundingClientRect().height,
backendHeight: backendSvg.getBoundingClientRect().height,
svgWidth: cpuSvg.getBoundingClientRect().width,
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
gridPath: cpuSvg.querySelector('.chart-grid').getAttribute('d'),
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
overflow: document.documentElement.scrollWidth - innerWidth,
};
}''')
def render_cpu(page, revision, system, process):
return page.evaluate('''({revision,system,process}) => {
window.rustProxyWideOverview.render({revision,live_system:{
cpu_percent:system, process_cpu_percent:process
}});
const read = name => {
const prefix = `overview-wide-wide_cpu-${name}`;
return {
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
label: document.getElementById(`${prefix}-value`).textContent,
};
};
return {system:read('system'),process:read('process')};
}''', {'revision': revision, 'system': system, 'process': process})
def ys(path):
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': viewport, 'height': 950})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const grid = document.getElementById('dashboard-overview-grid');
grid.classList.add('is-aligned');
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic', 'connection_chart'].includes(card.dataset.dashboardWidget);
}
window.currentPanel = 'overview';
window.$ = id => document.getElementById(id);
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
window.fmtBytes = value => `${value} B`;
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
}''')
page.add_script_tag(content=SCALE_JS)
page.add_script_tag(content=WIDE_JS)
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
result = inspect(page)
assert result['protocolHeight'] >= 124, (viewport, density, result)
assert abs(result['cpuHeight'] - 125) <= 1, (
'CPU height must remain fixed at 125px', viewport, density, result)
assert result['backendHeight'] > 0, (viewport, density, result)
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
assert result['gridPath'] == 'M0 47H600 M0 77H600 M0 107H600', result
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
# Low measured CPU values stay near zero instead of filling the plot.
first = render_cpu(page, 1, 0.5, 0.2)
second = render_cpu(page, 1, 0.8, 0.6)
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
assert ys(second['system']['path']) == [136, 136], (viewport, second)
assert ys(second['process']['path']) == [137, 136], (viewport, second)
# The Y axis is always 0-100% regardless of recorded history.
third = render_cpu(page, 1, 3.4, 2.2)
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
high = render_cpu(page, 1, 250, 123)
assert high['system']['label'] == '250.0%' and high['process']['label'] == '123.0%'
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
mid = render_cpu(page, 1, 50, 25)
assert ys(mid['system']['path'])[-1] == 77, (viewport, mid)
assert ys(mid['process']['path'])[-1] == 107, (viewport, mid)
# Revision changes reset history; missing readings do not plot a zero.
idle = render_cpu(page, 2, 0, 0)
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
full = render_cpu(page, 2, 100, 100)
assert ys(full['system']['path'])[-1] == 17 and ys(full['process']['path'])[-1] == 17
missing = render_cpu(page, 3, None, None)
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
# Samples may change paths, but not the plot/card or neighboring card sizes.
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
unchanged = inspect(page)
assert abs(unchanged['cpuHeight'] - 125) <= 1, (viewport, density, unchanged)
for value in (0.1, 99.9, 250, 0.0):
before = inspect(page)
render_cpu(page, 4, value, value)
after = inspect(page)
for key in ('cpuHeight', 'cpuCardHeight', 'connectionCardHeight'):
assert abs(before[key] - after[key]) < 1, (viewport, density, value, key, before, after)
assert not errors, (viewport, errors)
print(f'PASS CPU fixed 0-100% Y-scale and 125px plot height at {viewport}px (all densities)')
page.close()
browser.close()
scripts/tests/dashboard_layout_0331_browser_test.py
-199
View File
@@ -1,199 +0,0 @@
#!/usr/bin/env python3
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
The authenticated app bundle runs against simulated status/preferences; this is
not a Rust server integration test.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
BUNDLE = app_bundle()
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
def inspect(page, ids):
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
}))''', ids)
def assert_visible(page, visible, hidden):
result = inspect(page, visible + hidden)
for widget in visible:
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
for widget in hidden:
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in [320, 375, 800, 1440]:
context = browser.new_context(viewport={'width': width, 'height': 950})
page = context.new_page()
errors = []
page.on('pageerror', lambda e: errors.append(str(e)))
load(page)
result = page.evaluate('''() => {
const prefs = ADMIN_APPEARANCE_DEFAULTS;
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = JSON.parse(JSON.stringify(prefs));
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
appConfirm = async () => true;
return {
order: ADMIN_DASHBOARD_WIDGETS,
hidden: ADMIN_DEFAULT_HIDDEN_WIDGETS,
density: document.documentElement.dataset.dashboardDensity,
samples:window.rustProxyOverviewSamples.getLimit(),
serviceCount:document.querySelectorAll('#dashboard-overview-grid [data-dashboard-widget]').length,
};
}''')
assert result['order'] == PRESET['dashboard_widgets'], (width, result['order'])
assert result['hidden'] == PRESET['dashboard_hidden_widgets'], (width, result['hidden'])
assert result['samples'] == 600 and result['density'] == 'compact' and result['serviceCount'] == 78, result
assert_visible(page, ['backend_traffic', 'extra_response_latency', 'wide_transform_activity'],
['backend_health', 'stream_sessions', 'www_status', 'bandwidth'])
# Reproduce the previously flaky operation: opt in hidden backend +
# Stream cards and opt out a visible backend card, WITHOUT saving yet.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
outcome = page.evaluate('''() => {
const add = id => {
const label = dashboardWidgetLabel(id);
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
.find(node => node.firstElementChild.textContent === label);
if (!item) throw new Error('no hidden card to restore: ' + id);
item.querySelector('button').click();
};
add('backend_health');
add('stream_sessions');
const card = document.querySelector('[data-dashboard-widget="backend_traffic"]');
card.querySelector('.overview-inline-hide').click();
return window.rustProxyInlineLayout.currentDraft();
}''')
assert 'backend_traffic' in outcome['hidden'] and 'backend_health' not in outcome['hidden']
assert 'stream_sessions' not in outcome['hidden']
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
for _ in range(5):
page.evaluate('''() => {
const status = {revision:4,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
}''')
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
# Unrelated settings update must not repaint the saved layout over the draft.
page.evaluate('''() => applyAdminPreferences({...window.__savedDashboard, ui_scale:105})''')
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
assert page.evaluate('window.rustProxyInlineLayout.isEditing()')
page.locator('#overview-inline-cancel').click()
assert not page.evaluate('window.rustProxyInlineLayout.isEditing()')
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions'])
# Save under live traffic, ensure the in-flight preference application
# uses the current draft before the editor exits.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
page.evaluate('''() => {
const label = dashboardWidgetLabel('backend_health');
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
.find(node => node.firstElementChild.textContent === label);
item.querySelector('button').click();
document.querySelector('[data-dashboard-widget="backend_traffic"] .overview-inline-hide').click();
}''')
page.locator('#overview-inline-save').click()
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
assert_visible(page, ['backend_health'], ['backend_traffic'])
saved = page.evaluate('''() => window.__savedDashboard.dashboard_hidden_widgets''')
assert 'backend_traffic' in saved and 'backend_health' not in saved
page.evaluate('''() => {
const status={backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
}''')
assert_visible(page, ['backend_health'], ['backend_traffic'])
# Restore factory layout through the new named-presets modal.
page.locator('#overview-layout-open').evaluate('(node) => node.hidden = false')
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions', 'www_status'])
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
assert page.evaluate('window.__savedDashboard.dashboard_hidden_widgets') == PRESET['dashboard_hidden_widgets']
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2
assert not errors, (width, errors)
context.close()
print('PASS inline layout defaults, add/hide, live ticks, other preferences, cancel/save', width)
# An already-persisted sample choice must override the factory 600.
context = browser.new_context()
page = context.new_page()
# This sandbox blocks all network origins; stub browser storage before
# loading the bundle rather than navigating to a real host.
page.evaluate("""() => Object.defineProperty(window, 'localStorage', {
configurable: true,
value: {getItem: key => key === 'rustproxy.overview.liveChartSamples' ? '120' : null,
setItem: () => undefined},
})""")
load(page)
assert page.evaluate('window.rustProxyOverviewSamples.getLimit()') == 120
context.close()
browser.close()
print('PASS RustProxy 0.33.1 simulated browser layout regression')
scripts/tests/dashboard_metric_visuals_ui_test.mjs
-134
View File
@@ -1,134 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Deterministic Overview widget regression; simulated DOM/status, no running API.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const html = renderAdminIndex(root);
const registry = JSON.parse(fs.readFileSync(path.join(root, 'src/admin/web/dashboard-widgets.json'), 'utf8'));
const lang = JSON.parse(fs.readFileSync(path.join(root, 'langs/en_US.json'), 'utf8')).translations;
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
const widgets = new Map(), ids = new Map();
class Element {
constructor() { this.hidden = false; this.attributes = new Map(); this.style = {}; this.textContent = '-'; }
setAttribute(key, value) { this.attributes.set(key, String(value)); }
removeAttribute(key) { this.attributes.delete(key); }
getAttribute(key) { return this.attributes.get(key) ?? null; }
}
for (const [, id] of html.matchAll(/\bid="([^"]+)"/g)) {
assert.ok(!ids.has(id), `duplicate DOM id: ${id}`);
ids.set(id, new Element());
}
for (const [, id] of html.matchAll(/data-dashboard-widget="([^"]+)"/g)) widgets.set(id, new Element());
assert.equal(registry.length, widgets.size, 'registry and DOM widget count must match');
for (const entry of registry) assert.ok(widgets.has(entry.id), `registry references missing DOM card ${entry.id}`);
let now = 1000;
const document = {
hidden: false,
addEventListener() {},
querySelector(query) { return widgets.get(query.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) ?? null; },
};
const window = { addEventListener() {}, rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
const context = vm.createContext({
window, document, performance: { now: () => now },
currentPanel: 'overview', $: id => ids.get(id) ?? null,
fmt: (v, digits = 0) => Number(v).toFixed(digits),
fmtBytes: v => `${Number(v)} B`,
t: key => lang[key] || key,
console,
});
for (const script of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-extra-metrics.js', 'app/overview-wide-metrics.js']) {
vm.runInContext(code(script), context, { filename: script });
}
const extra = window.rustProxyExtraOverview, wide = window.rustProxyWideOverview;
const status = {
revision: 9,
system: { open_fds: 60, max_fds: 300 },
cache: { hits: 100, misses: 50, resident_bytes: 150, hot_limit_bytes: 300,
cold_queue_bytes: 25, cold_queue_limit_bytes: 100, cold_bytes: 400, cold_limit_bytes: 1000 },
metrics: { requests: 300, http1_requests: 200, http2_requests: 80, http3_requests: 20,
compression_gzip: 10, compression_br: 10, compression_zstd: 10, compression_deflate: 1 },
transform_metrics: { applied: 10, cache_hits: 10, skipped: 10, errors: 1 },
};
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_memory-meter-fill').style.width, '50%');
assert.equal(ids.get('overview-extra-extra_cold_queue_memory-meter-fill').style.width, '25%');
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '40%');
widgets.get('extra_cache_memory').hidden = false;
widgets.get('extra_cache_hit_rate').hidden = false;
widgets.get('extra_h2_share').hidden = false;
widgets.get('extra_h1_share').hidden = false;
widgets.get('extra_h3_share').hidden = false;
widgets.get('extra_process_fds').hidden = false;
extra.render(status);
assert.equal(ids.get('overview-extra-extra_process_fds-meter-fill').style.width, '20%');
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-'); // first counter sample has no rate
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '-');
now = 4000;
status.cache.hits += 30;
status.cache.misses += 10;
status.metrics.requests += 40;
status.metrics.http1_requests += 10;
status.metrics.http2_requests += 20;
status.metrics.http3_requests += 10;
status.metrics.compression_gzip += 3;
status.metrics.compression_br += 6;
status.metrics.compression_zstd += 9;
status.metrics.compression_deflate += 12;
status.transform_metrics.applied += 15;
status.transform_metrics.cache_hits += 3;
status.transform_metrics.skipped += 6;
status.transform_metrics.errors += 1;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '75.0%');
assert.equal(ids.get('extra-cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
assert.equal(ids.get('extra-cache-ratio-miss-arc').getAttribute('stroke-dashoffset'), '-75');
assert.equal(ids.get('extra-cache-ratio-hit-count').textContent, '30');
assert.equal(ids.get('extra-cache-ratio-miss-count').textContent, '10');
assert.equal(ids.get('overview-extra-extra_h2_share-meter-fill').style.width, '50%');
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '8.3/s');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '10.0/s');
for (const name of ['applied', 'cache_hits', 'skipped', 'errors'])
assert.ok(ids.get(`overview-wide-wide_transform_activity-${name}-line`).getAttribute('d')?.length);
for (const name of ['gzip', 'br', 'zstd', 'deflate'])
assert.ok(ids.get(`overview-wide-wide_compression_activity-${name}-line`).getAttribute('d')?.length);
// No hidden-tab spike or fake zero when a long interval is skipped.
now = 100000;
status.cache.hits += 300;
status.cache.misses += 200;
status.metrics.compression_gzip += 600;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
now = 103000;
status.cache.hits += 5;
status.cache.misses += 5;
status.metrics.compression_gzip += 3;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '50.0%');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '1.0/s');
// Bounded metric with no configured budget remains unknown, not a made-up percentage.
status.cache.cold_limit_bytes = 0;
extra.render(status);
assert.equal(ids.get('overview-extra-extra_cold_storage').textContent, '400 B');
assert.equal(ids.get('overview-extra-extra_cold_storage-meter').getAttribute('aria-valuenow'), null);
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '0%');
extra.donut('cache-ratio', 75, 25);
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
assert.equal(ids.get('cache-ratio-chart').getAttribute('aria-label'), 'HIT 75, MISS 25');
extra.donut('cache-ratio', 0, 0);
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '0 100');
assert.ok(code('app/runtime.js').includes('rustProxyExtraOverview?.donut?.("cache-ratio"'),
'main HIT/MISS tile must share tested donut rendering');
console.log('PASS: bounded meters, COLD tiers, HIT/MISS rings, transform and compression rates, pause/no-limit');
scripts/tests/dashboard_modal_0333_browser_test.py
-176
View File
@@ -1,176 +0,0 @@
#!/usr/bin/env python3
"""Regression: named Overview presets, responsive modal and browser actions.
Runs the real panel DOM/CSS/bundled JavaScript with mocked preferences/API.
This is browser layout testing, not Rust server acceptance.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
PACKS = {
name: {'id': name, **json.loads((ROOT / 'langs' / (name + '.json')).read_text())}
for name in ('en_US', 'pl_pl')
}
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
BUNDLE = app_bundle()
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyDashboardLayout')
page.evaluate('''packs => {
languagePacks = new Map(Object.entries(packs));
defaultLanguage = 'en_US';
language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
}''', PACKS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
document.getElementById('overview-layout-open').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
}''')
# Regression for the replacement named-presets modal, with a simulated SQLite API.
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 800, 1440):
context = browser.new_context(viewport={'width': width, 'height': 880})
page = context.new_page()
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
load(page)
page.evaluate('''() => {
const saved = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.__savedDashboard = saved;
window.__presets = new Map();
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
api = async (path, options={}) => {
const method = options.method || 'GET';
if (path === '/api/dashboard-presets' && method === 'GET')
return [...window.__presets.entries()].map(([name, data]) => ({ name, ...structuredClone(data) }));
if (path.startsWith('/api/dashboard-presets/')) {
const name = decodeURIComponent(path.split('/').slice(3).join('/'));
if (method === 'PUT') { window.__presets.set(name, JSON.parse(options.body)); return {}; }
if (method === 'DELETE') { window.__presets.delete(name); return {}; }
if (method === 'PATCH') {
const changed = JSON.parse(options.body).name;
if (window.__presets.has(changed)) throw new Error('already exists');
window.__presets.set(changed, window.__presets.get(name));
window.__presets.delete(name);
return {};
}
}
throw new Error('unexpected API: ' + method + ' ' + path);
};
appConfirm = async () => true;
applyAdminPreferences(window.__savedDashboard);
document.getElementById('overview-layout-open').hidden = false;
}''')
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
assert page.locator('#overview-presets-list .overview-presets-row').count() == 0
widths = page.evaluate('''() => {
const d = document.getElementById('overview-presets-dialog');
const r = d.getBoundingClientRect();
return {left:r.left,right:r.right,overflow:d.scrollWidth - d.clientWidth};
}''')
assert widths['left'] >= -1 and widths['right'] <= width+1 and widths['overflow'] <= 2, (width, widths)
# Persist the factory arrangement as a user preset.
page.locator('#overview-presets-name').fill('Daily')
page.locator('#overview-presets-save').click()
page.wait_for_function("window.__presets.has('Daily')")
assert page.locator('#overview-presets-list .overview-presets-row').count() == 1
assert page.evaluate("window.__presets.get('Daily').overview_samples") == 600
# Create another view, with customized widths, density and history.
page.evaluate('''async () => {
await window.rustProxyPreferences.update({
dashboard_hidden_widgets:['requests'], dashboard_widget_sizes:{requests:3},
dashboard_density:'ultra',
});
window.rustProxyOverviewSamples.setLimit(120, true);
}''')
page.locator('#overview-presets-name').fill('Detailed')
page.locator('#overview-presets-save').click()
page.wait_for_function("window.__presets.has('Detailed')")
assert page.evaluate('''() => {
const p=window.__presets.get('Detailed');
return [p.dashboard_widget_sizes.requests,p.dashboard_density,p.overview_samples];
}''') == [3,'ultra',120]
# Named load must restore full layout and history.
page.locator('.overview-presets-row').filter(has_text='Daily').locator('[data-preset-action="dashboardPresetsLoad"]').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate('''() => [window.__savedDashboard.dashboard_density,
window.__savedDashboard.dashboard_widget_sizes,
window.rustProxyOverviewSamples.getLimit()]''') == ['compact',None,600]
# Rename and delete remain administrator-scoped operations.
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-name').fill('Nightly')
page.locator('.overview-presets-row').filter(has_text='Detailed').locator('[data-preset-action="dashboardPresetsRename"]').click()
page.wait_for_function("window.__presets.has('Nightly')")
assert not page.evaluate("window.__presets.has('Detailed')")
page.locator('.overview-presets-row').filter(has_text='Nightly').locator('[data-preset-action="dashboardPresetsDelete"]').click()
page.wait_for_function("!window.__presets.has('Nightly')")
assert page.evaluate("window.__presets.has('Daily')")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate("window.__savedDashboard.dashboard_widgets") == json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())['dashboard_widgets']
assert not errors, (width, errors)
print('PASS named Dashboard presets save/load/rename/delete/factory and responsive modal:', width)
context.close()
browser.close()
print('PASS Overview preset modal (simulated SQLite API; no running Rust server)')
scripts/tests/dashboard_overview_labels_0337_browser_test.py
-102
View File
@@ -1,102 +0,0 @@
#!/usr/bin/env python3
"""All Overview widget labels and small values in simulated Chromium.
This is a DOM/CSS regression test, not a live RustProxy integration test.
"""
from pathlib import Path
import json
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
PACKS = {name: json.loads((ROOT / f'langs/{name}.json').read_text(encoding='utf-8'))['translations']
for name in ('en_US', 'pl_pl')}
CHECK = r'''({density, lang, translations}) => {
document.documentElement.dataset.dashboardDensity = density;
document.documentElement.lang = lang === 'pl_pl' ? 'pl-PL' : 'en-US';
const grid = document.getElementById('dashboard-overview-grid');
grid.querySelectorAll('[data-i18n]').forEach(node => {
if (Object.hasOwn(translations, node.dataset.i18n)) node.textContent = translations[node.dataset.i18n];
});
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
const font = value => getComputedStyle(value).fontFamily;
const probe = document.createElement('div');
grid.append(probe);
probe.style.fontFamily = 'var(--sans)'; const sans = font(probe);
probe.style.fontFamily = 'var(--mono)'; const mono = font(probe);
probe.remove();
const titleErrors = [];
const titles = [];
const labelErrors = [];
const numericErrors = [];
for (const card of cards) {
let title;
if (card.classList.contains('stat')) title = card.querySelector(':scope > span:first-child');
else if (card.classList.contains('system-card') || card.classList.contains('connection-card'))
title = card.querySelector('.overview-chart-heading');
else if (card.querySelector('.mini-heading > h3')) title = card.querySelector('.mini-heading > h3');
else title = card.querySelector('.card-heading h2');
if (!title) { titleErrors.push(`${card.dataset.dashboardWidget}: missing title`); continue; }
const style = getComputedStyle(title);
titles.push(title);
if (style.fontFamily !== sans || style.fontSize !== '12px' || style.textTransform !== 'none')
titleErrors.push(`${card.dataset.dashboardWidget}: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
if (title.textContent.trim() === title.textContent.trim().toLocaleUpperCase('pl') &&
/[a-ząćęłńóśżź]{4}/i.test(title.textContent) &&
!/^(?:HIT|MISS|HTTP|CPU|RAM|WWW|RX|TX)/.test(title.textContent.trim()))
titleErrors.push(`${card.dataset.dashboardWidget}: all-uppercase title ${title.textContent.trim()}`);
for (const node of card.querySelectorAll(':scope > small, .mini-metrics dt, .mini-metrics dd, .mini-live-chart-head, .memory-line, .chart-footer, .meta, .small-label')) {
if (font(node) !== mono) labelErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
}
const readings = card.classList.contains('stat')
? [...card.querySelectorAll(':scope > strong'), ...card.querySelectorAll(':scope > small')]
: [...card.querySelectorAll('.mini-metrics dd')];
for (const node of readings) if (font(node) !== mono)
numericErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
}
const nested = [...grid.querySelectorAll('.mini-live-chart > .mini-live-chart-head > span:first-child')];
for (const node of nested) {
const style = getComputedStyle(node);
if (style.fontFamily !== mono || style.fontSize !== '10px' || style.textTransform !== 'none')
labelErrors.push(`mini-live-chart: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
}
const systemNumeric = getComputedStyle(document.querySelector('#system-cpu'));
return {
cardCount: cards.length, titleCount: titles.length, secondaryCount: nested.length,
titleErrors, labelErrors, numericErrors,
systemNumericMono: systemNumeric.fontFamily === mono,
cpuLabel: document.querySelector('[data-dashboard-widget="system_chart"] .overview-chart-heading').textContent,
connectionsLabel: document.querySelector('[data-dashboard-widget="connection_chart"] .overview-chart-heading').textContent,
memoryLabel: document.querySelector('[data-dashboard-widget="memory"] > span:first-child').textContent,
backendLabel: document.querySelector('[data-dashboard-widget="backend_health"] > span:first-child').textContent,
diagnosticsTitle: document.querySelector('[data-dashboard-widget="process_diagnostics"] .mini-heading h3').textContent,
};
}'''
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 900})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
for density in ('normal', 'compact', 'ultra'):
for lang in ('en_US', 'pl_pl'):
result = page.evaluate(CHECK, {'density': density, 'lang': lang, 'translations': PACKS[lang]})
descriptor = (width, density, lang)
assert result['cardCount'] == result['titleCount'] >= 75, (descriptor, result)
assert result['secondaryCount'] >= 6, descriptor
assert not result['titleErrors'], (descriptor, result['titleErrors'][:10])
assert not result['labelErrors'], (descriptor, result['labelErrors'][:10])
assert not result['numericErrors'], (descriptor, result['numericErrors'][:10])
assert result['systemNumericMono'], descriptor
expected = ('System load', 'Connection activity') if lang == 'en_US' else ('Obciążenie systemu', 'Aktywność połączeń')
assert (result['cpuLabel'], result['connectionsLabel']) == expected, (descriptor, result)
assert result['memoryLabel'] and result['backendLabel'] and result['diagnosticsTitle'], descriptor
print(f'PASS all Overview titles and small readings at {width}px (3 densities, 2 languages)')
page.close()
browser.close()
scripts/tests/dashboard_presets_sql_test.py
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env python3
"""Offline SQLite regression for additive admin preset migration and 16-slot upsert."""
from pathlib import Path
import sqlite3
ROOT = Path(__file__).resolve().parents[2]
all_scripts = sorted((ROOT / 'migrations').glob('*.sql'))
assert [p.name for p in all_scripts][-1] == '0010_dashboard_presets.sql'
con = sqlite3.connect(':memory:')
con.execute('PRAGMA foreign_keys = ON')
for _ in range(2):
for script in all_scripts:
con.executescript(script.read_text())
con.execute("INSERT INTO administrator(id,username,password_hash,generation,created_at) VALUES(1,'owner','hash',1,0)")
con.execute("INSERT INTO admin_preferences(administrator_id,document) VALUES(1,'{}')")
upsert = '''
INSERT INTO dashboard_presets(administrator_id, name, document, updated_at)
SELECT a.id, ?, ?, ? FROM administrator a WHERE a.id = 1 AND a.username = ?
AND ((SELECT count(*) FROM dashboard_presets WHERE administrator_id = a.id) < 16
OR EXISTS(SELECT 1 FROM dashboard_presets WHERE administrator_id = a.id AND name = ?))
ON CONFLICT(administrator_id, name) DO UPDATE SET document = excluded.document, updated_at = excluded.updated_at
'''
for i in range(16):
name = f'Preset {i}'
assert con.execute(upsert, (name,'{}',i,'owner',name)).rowcount == 1
assert con.execute(upsert, ('Overflow','{}',16,'owner','Overflow')).rowcount == 0
assert con.execute(upsert, ('Preset 0','{"changed":true}',17,'owner','Preset 0')).rowcount == 1
assert con.execute(upsert, ('Other account','{}',18,'intruder','Other account')).rowcount == 0
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 16
assert con.execute('SELECT document FROM dashboard_presets WHERE name="Preset 0"').fetchone()[0] == '{"changed":true}'
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Preset 1','Preset 0')).rowcount == 0
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Renamed','Preset 0')).rowcount == 1
con.execute('DELETE FROM administrator WHERE id=1')
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 0
assert con.execute('SELECT count(*) FROM admin_preferences').fetchone()[0] == 0
print('PASS SQLite migrations repeated, named-preset CRUD, cap, account ownership and cascade')
scripts/tests/dashboard_resizing_0333_browser_test.py
-200
View File
@@ -1,200 +0,0 @@
#!/usr/bin/env python3
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
The authenticated app bundle runs against simulated status/preferences; this is
not a Rust server integration test.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
BUNDLE = app_bundle()
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
def inspect(page, ids):
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
}))''', ids)
def assert_visible(page, visible, hidden):
result = inspect(page, visible + hidden)
for widget in visible:
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
for widget in hidden:
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
# Test actual production CSS/JS in a browser with simulated preferences/status.
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in [320, 375, 800, 1440, 1920]:
context = browser.new_context(viewport={'width': width, 'height': 1000})
page = context.new_page()
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
load(page)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
document.getElementById('overview-layout-open').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
appConfirm = async () => true;
}''')
# Unchanged previous accounts preserve historic grouped rows.
assert page.evaluate('''() => ({
legacy: document.getElementById('dashboard-overview-grid').dataset.overviewLayoutMode,
widths:window.__savedDashboard.dashboard_widget_sizes,
})''') == {'legacy':'legacy','widths':None}
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
# Widen a number+sparkline card from 2 to 3 and then 4 columns.
selector = '[data-dashboard-widget="http_protocol"]'
old_width = page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width')
page.locator(selector + ' .overview-inline-widen').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '3'
page.locator(selector + ' .overview-inline-widen').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
assert page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width') >= old_width * 1.7
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().sizes.http_protocol') == 4
# Minimize a large multi-series chart to the new 3-column option.
for _ in range(2):
page.locator('[data-dashboard-widget="wide_transform_activity"] .overview-inline-narrow').click()
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
# Real UI controls allow crossing old groups: send mini-chart to the front
# with the upward arrow, then across the original stat/chart boundary.
# A direct drag onto the first card also uses the actual event handlers.
page.evaluate('''() => {
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
const transfer = new DataTransfer();
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true, cancelable:true, dataTransfer:transfer}));
to.dispatchEvent(new DragEvent('drop', {bubbles:true, cancelable:true, dataTransfer:transfer, clientX:0}));
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
}''')
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().order[0]') == 'wide_transform_activity'
# Background status and unrelated preference changes cannot reset live sizes.
page.evaluate('''() => {
const status={revision:5,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
applyAdminPreferences({...window.__savedDashboard, chart_style:'soft'});
}''')
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
# Cancel should remove both size and layout changes.
page.locator('#overview-inline-cancel').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '2'
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '6'
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
# The actual unlocked dashboard remains the only layout editor.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
tile = '[data-dashboard-widget="wide_transform_activity"]'
page.locator(tile + ' .overview-inline-narrow').click()
assert page.locator(tile).get_attribute('data-dashboard-span') == '4'
page.locator(tile + ' .overview-inline-narrow').click()
assert page.locator(tile).get_attribute('data-dashboard-span') == '3'
page.evaluate('''() => {
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
const transfer = new DataTransfer();
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true,cancelable:true,dataTransfer:transfer}));
to.dispatchEvent(new DragEvent('drop', {bubbles:true,cancelable:true,dataTransfer:transfer,clientX:0}));
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
}''')
page.locator('#overview-inline-save').click()
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes.wide_transform_activity') == 3
assert page.evaluate('window.__savedDashboard.dashboard_widgets[0]') == 'wide_transform_activity'
# Reload account settings: new card width persists.
page.evaluate('applyAdminPreferences(window.__savedDashboard)')
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
assert page.evaluate('window.__savedDashboard.dashboard_density') == 'compact'
# Four charts in one row, set through the same layout function used by UI.
data = page.evaluate('''() => {
const ids=['wide_transform_activity','wide_compression_activity','http_protocol','websocket_protocol'];
const order=[...ids,...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
const hidden=[...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
const sizes=Object.fromEntries(ids.map(id => [id,3]));
applyDashboardLayout(order,hidden,sizes);
const grid=document.getElementById('dashboard-overview-grid');
return { gridWidth:grid.getBoundingClientRect().width,
template:getComputedStyle(grid).gridTemplateColumns.split(' ').length,
boxes:ids.map(id=>{const b=document.querySelector(`[data-dashboard-widget="${id}"]`).getBoundingClientRect();return {id,x:Math.round(b.x),y:Math.round(b.y),w:Math.round(b.width),scroll:b.width};})
};
}''')
if data['template'] == 12:
assert len(set(b['y'] for b in data['boxes'])) == 1, (width, data)
assert all(b['w'] > 150 for b in data['boxes']), (width,data)
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2, (width, data)
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
assert page.evaluate('document.getElementById("dashboard-overview-grid").dataset.overviewLayoutMode') == 'legacy'
assert not errors, (width, errors)
print('PASS free-order, resize, live refresh, cancel/save, four chart grid', width, data['template'])
context.close()
browser.close()
print('PASS 0.33.3 resizable Overview browser regression')
scripts/tests/dashboard_resume_rate_charts_ui_test.mjs
-115
View File
@@ -1,115 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Simulated DOM/status regression: no holes or artificial zeros after tab resume.
// This intentionally does not exercise the running Rust server or real browser.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
const html = renderAdminIndex(root);
class Element {
constructor() { this.attributes = new Map(); this.style = {}; this.textContent = ''; this.children = []; this.hidden = false; }
getAttribute(key) { return this.attributes.get(key) ?? null; }
setAttribute(key, value) { this.attributes.set(key, String(value)); }
removeAttribute(key) { this.attributes.delete(key); }
append(...children) { this.children.push(...children); }
replaceChildren(...children) { this.children = children; }
}
const ids = new Map([...html.matchAll(/\bid="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
const widgets = new Map([...html.matchAll(/data-dashboard-widget="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
const listeners = new Map();
const document = {
hidden: false,
addEventListener: (type, handler) => listeners.set(type, handler),
querySelector: selector => widgets.get(selector.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) || null,
createElement: () => new Element(),
};
const samples = {
getLimit: () => 60,
trim: values => { if (values.length > 60) values.splice(0, values.length - 60); },
onChange: () => {},
};
const window = {
addEventListener() {}, rustProxyOverviewSamples: samples,
rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) },
};
let now = 1000;
const context = vm.createContext({
window, document, performance: { now: () => now }, currentPanel: 'overview', latestStatus: null,
$: id => ids.get(id) || null, fmt: (value, n = 0) => Number(value).toFixed(n),
fmtBytes: value => `${value} B`, t: value => value,
});
for (const file of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-wide-metrics.js', 'app/web-overview.js'])
vm.runInContext(code(file), context, { filename: file });
const web = {
requests: 200, status_2xx: 190, status_3xx: 2, status_4xx: 5, status_5xx: 3,
static_requests: 100, php_requests: 80, other_requests: 20,
methods: [160, 20, 10, 5, 5], selected_status: [0, 0, 0, 0, 0, 0, 0, 0],
response_headers_us: 300000, response_bytes: 1200000, active_headers: 2,
};
const php = {
metrics: { requests: 80, response_headers_us: 100000, response_bytes: 320000, failures: Array(9).fill(0) },
capacity_limit: 6, capacity_used: 1,
};
const status = {
revision: 4,
metrics: { requests: 200, http1_requests: 120, http2_requests: 60, http3_requests: 20,
compression_gzip: 10, compression_br: 20, compression_zstd: 5, compression_deflate: 8 },
cache: { hits: 50, misses: 30 },
transform_metrics: { applied: 10, cache_hits: 5, skipped: 1, errors: 0 },
web_server_metrics: { demo: { web, php } },
};
function advance(delta) {
for (const key of Object.keys(status.metrics)) status.metrics[key] += delta;
for (const key of ['hits', 'misses']) status.cache[key] += delta;
for (const key of Object.keys(status.transform_metrics)) status.transform_metrics[key] += delta;
for (const key of ['requests', 'status_2xx', 'static_requests', 'php_requests', 'response_headers_us', 'response_bytes']) web[key] += delta;
for (const key of ['requests', 'response_headers_us', 'response_bytes']) php.metrics[key] += delta;
}
function render() { window.rustProxyWideOverview.render(status); window.rustProxyWebOverview.render(status); }
const line = id => ids.get(id).getAttribute('d') || '';
const compression = 'overview-wide-wide_compression_activity-gzip-line';
const website = 'www_requests-all-line';
const phpLine = 'www_php_requests-line';
const starts = path => (path.match(/M/g) || []).length;
render(); // no first-interval rate: establish baseline only
now = 4000; advance(12); render();
const beforeWide = line(compression), beforeWeb = line(website);
const beforePhp = ids.get(phpLine).getAttribute('points');
assert.equal(starts(beforeWide), 1);
assert.equal(starts(beforeWeb), 1);
assert.ok(beforePhp);
// Background tab -> a new generation, regardless of counter increments.
document.hidden = true; listeners.get('visibilitychange')();
document.hidden = false;
now = 100000; advance(600); render();
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
assert.equal(ids.get('www_requests-value').textContent, '-');
assert.equal(line(compression), beforeWide, 'wide rate chart must not receive a null point');
assert.equal(line(website), beforeWeb, 'WWW rate chart must not receive a null point');
assert.equal(ids.get(phpLine).getAttribute('points'), beforePhp, 'small PHP chart must preserve valid history');
// A fresh interval then connects to previous valid samples without a gap.
now = 103000; advance(9); render();
assert.equal(starts(line(compression)), 1, 'wide chart must have one connected path');
assert.equal(starts(line(website)), 1, 'WWW chart must have one connected path');
assert.ok(line(compression).includes('L'));
assert.ok(line(website).includes('L'));
// Real measured zeros must continue to be plotted, not treated as pauses.
now = 106000; render();
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '0.0/s');
assert.equal(ids.get('www_requests-value').textContent, '0.0/s');
assert.equal(starts(line(compression)), 1);
assert.equal(starts(line(website)), 1);
assert.match(code('web-server.js'), /if \(record && rate !== null\) \{ previous\.push\(rate\)/,
'Web Server chart must also skip unavailable rates instead of plotting a zero dip');
console.log('PASS: resumed wide/WWW/PHP charts stay connected; first invalid rate is unavailable; measured zero persists');
scripts/tests/dashboard_stable_scales_0336_browser_test.py
-129
View File
@@ -1,129 +0,0 @@
#!/usr/bin/env python3
"""0.33.6 plot size/scale checks in simulated Chromium, not a running RustProxy."""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALES = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
RATES = (ROOT / 'src/admin/web/app/rate-sampling.js').read_text(encoding='utf-8')
WIDE = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
WIDE_IDS = ('wide_protocol_rates', 'wide_cpu', 'wide_memory', 'wide_cache_activity',
'wide_connections', 'wide_backend_health', 'wide_latency',
'wide_transform_activity', 'wide_compression_activity',
'www_requests', 'www_latency', 'www_failures', 'www_traffic', 'www_status_trend')
with sync_playwright() as api:
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading','app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('dashboard-overview-grid').classList.add('is-aligned');
for (const card of document.querySelectorAll('#dashboard-overview-grid>[data-dashboard-widget]'))
card.hidden = !['wide_protocol_rates','wide_cpu','wide_memory','wide_cache_activity',
'wide_connections','wide_backend_health','wide_latency',
'wide_transform_activity','wide_compression_activity',
'www_requests','www_latency','www_failures','www_traffic','www_status_trend'].includes(card.dataset.dashboardWidget);
window.currentPanel='overview';
window.$ = id => document.getElementById(id);
window.fmt = (n, digits=0) => Number(n).toFixed(digits);
window.fmtBytes = n => `${n} B`;
window.rustProxyOverviewSamples = {getLimit:()=>60,onChange:()=>{}};
window.rustProxyRefresh = {current:()=>({overview_seconds:3})};
window.__clock = 1000;
Object.defineProperty(performance,'now',{configurable:true,value:()=>window.__clock});
}''')
for script in (RATES, SCALES, WIDE):
page.add_script_tag(content=script)
for density in ('normal','compact','ultra'):
measured = page.evaluate('''density => {
document.documentElement.dataset.dashboardDensity=density;
const results={};
for (const card of document.querySelectorAll('#dashboard-overview-grid > .overview-wide-chart')) {
const svg=card.querySelector('svg.overview-wide-svg');
if (!svg) continue;
const r=svg.getBoundingClientRect();
results[card.dataset.dashboardWidget]={height:r.height, width:r.width,
grid:svg.querySelector('.chart-grid').getAttribute('d'),
contentWidth:Math.hypot(svg.getScreenCTM().a,svg.getScreenCTM().b)*600};
}
document.getElementById('panel-transform').hidden=false;
document.getElementById('panel-cache').hidden=false;
for(const id of ['transform-live-chart','cache-live-chart'])document.getElementById(id).open=true;
results.transformDetails=document.getElementById('transform-live-activity-chart').getBoundingClientRect().height;
results.cacheDetails=document.getElementById('cache-live-activity-chart').getBoundingClientRect().height;
return results;
}''', density)
assert abs(measured['transformDetails'] - 180) < 1, (width,density,measured)
assert abs(measured['cacheDetails'] - 180) < 1, (width,density,measured)
for name in WIDE_IDS:
data=measured[name]
assert abs(data['height']-125)<1, (width,density,name,data)
assert data['width']>50 and abs(data['width']-data['contentWidth'])<2, (width,density,name,data)
assert data['grid']=='M0 47H600 M0 77H600 M0 107H600', (width,density,name,data)
result = page.evaluate('''() => {
const c=window.rustProxyChartScale;
const first=c.peak('probe:rate','rate',[[.2,.7]]);
const count=c.peak('probe:count','count',[[0,1]]);
const latency=c.peak('probe:ms','ms',[[.1,1.5]]);
const memory=c.peak('probe:bytes','bytes',[[1000]]);
const high=c.peak('probe:rate','rate',[[101]]);
const held=c.peak('probe:rate','rate',[[.3]]);
c.reset('probe:');
const reset=c.peak('probe:rate','rate',[[.3]]);
const percent=c.peak('probe:cpu','percent',[[.1,500]]);
const blank=c.peak('probe:blank','rate',[[null,NaN,Infinity]]);
const before={revision:10,live_system:{cpu_percent:0.7,process_cpu_percent:0.2},
metrics:{compression_gzip:50,compression_br:100,compression_zstd:10,compression_deflate:5},
transform_metrics:{applied:2,cache_hits:10,skipped:1,errors:0}};
window.rustProxyWideOverview.render(before);
window.__clock=4000;
before.metrics.compression_gzip+=3;before.metrics.compression_br+=6;
before.metrics.compression_zstd+=0;before.metrics.compression_deflate+=0;
before.transform_metrics.applied+=6;before.transform_metrics.cache_hits+=3;
window.rustProxyWideOverview.render(before);
const get = (id,key) => document.getElementById(`overview-wide-${id}-${key}-line`).getAttribute('d');
const compressionLow=get('wide_compression_activity','gzip');
const transformLow=get('wide_transform_activity','applied');
const cpu=get('wide_cpu','system');
const lowLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
window.__clock=7000;
before.metrics.compression_gzip+=603; // now 201/s => ceiling expands to 500/s
before.transform_metrics.applied+=603;
window.rustProxyWideOverview.render(before);
const highLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
const compressionHigh=get('wide_compression_activity','gzip');
window.__clock=10000;
before.metrics.compression_gzip+=3;
before.transform_metrics.applied+=3;
window.rustProxyWideOverview.render(before);
const afterSpike=get('wide_compression_activity','gzip');
return {first,count,latency,memory,high,held,reset,percent,blank,
lowLabel,highLabel,compressionLow,transformLow,cpu,compressionHigh,afterSpike};
}''')
assert result['first']==100 and result['count']==10 and result['latency']==100, (width,result)
assert result['memory']>=64*1024*1024, (width,result)
assert (result['high'],result['held'],result['reset'],result['percent'],result['blank']) == (200,200,100,100,100), (width,result)
assert result['lowLabel']=='1.0/s' and result['highLabel']=='201.0/s', (width,result)
def yvalues(line): return [int(n) for n in re.findall(r'[ML]\d+ (\d+)',line)]
assert yvalues(result['compressionLow'])[-1]>=135, (width,result)
assert yvalues(result['transformLow'])[-1]>=130, (width,result)
assert yvalues(result['cpu'])==[136,136], (width,result)
assert yvalues(result['compressionHigh'])[-1]<100, (width,result)
assert yvalues(result['afterSpike'])[-1]>=135, (width,result)
assert not errors, (width,errors)
print('PASS all 14 overview-wide plot sizes/grids + full Cache/Transform charts + stable scales:', width)
page.close()
browser.close()
scripts/tests/dashboard_title_color_0337_browser_test.py
-92
View File
@@ -1,92 +0,0 @@
#!/usr/bin/env python3
"""Simulated Chromium regression for unified subdued metric/chart titles.
Tests computed CSS in the assembled panel DOM; not a live Rust server.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
CHECK = r'''({theme, density}) => {
document.documentElement.dataset.theme = theme;
document.documentElement.dataset.dashboardDensity = density;
const grid = document.getElementById('dashboard-overview-grid');
const probe = document.createElement('span');
probe.style.color = 'var(--muted)';
document.body.append(probe);
const expected = getComputedStyle(probe).color;
probe.style.color = 'var(--text)';
const primary = getComputedStyle(probe).color;
probe.remove();
const color = node => getComputedStyle(node).color;
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
const titles = cards.map(card => {
let node;
if (card.classList.contains('stat')) node = card.querySelector(':scope > span:first-child');
else if (card.matches('.system-card,.connection-card')) node = card.querySelector('.overview-chart-heading');
else node = card.querySelector('.mini-heading > h3') || card.querySelector('.card-heading h2');
return [card.dataset.dashboardWidget, node];
});
const otherSelectors = [
'#panel-cache .stats-grid .stat > span:first-child',
'#panel-transform .stats-grid .stat > span:first-child',
'#panel-traffic .stat > span:first-child',
'#panel-cache > .card .card-heading h2',
'#panel-transform > .card .card-heading h2',
'.traffic-chart-card > h3.subheading',
'.function-live-chart-heading > strong',
'.mini-live-chart-head > span:first-child',
'.vhost-chart-head > span:first-child',
'.vhost-live-metric > span',
];
// Include dynamically rendered Vhost/Streams chart and Vhost metric titles.
const container = document.getElementById('panel-vhosts');
const fixture = document.createElement('div');
fixture.innerHTML = `<div class="vhost-chart"><div class="vhost-chart-head">
<span class="small-label">Connections history</span><span class="fine">42/s</span>
</div></div><div class="vhost-live-metric"><span>Requests</span><strong>42</strong></div>`;
container.append(fixture);
const otherTitles = otherSelectors.flatMap(selector => [...document.querySelectorAll(selector)]
.map(node => [selector, node]));
const errors = [...titles, ...otherTitles].flatMap(([name, node]) => {
if (!node) return [`${name} missing title`];
if (color(node) !== expected) return [`${name}: ${color(node)} != ${expected} (${node.textContent.trim().slice(0, 35)})`];
return [];
});
const numeric = [
document.querySelector('#system-cpu'),
document.querySelector('#panel-overview .system-card .card-heading h2'),
document.querySelector('#panel-overview .connection-card .card-heading h2'),
document.querySelector('[data-dashboard-widget="requests"] > strong'),
fixture.querySelector('.vhost-live-metric strong'),
].map(node => ({node: node && node.outerHTML.slice(0, 80), color: node && color(node)}));
fixture.remove();
return {expected, primary, count: titles.length, otherCount: otherTitles.length,
errors, numeric};
}'''
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 768, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
for theme in ('dark', 'light', 'ultra_dark'):
for density in ('normal', 'compact', 'ultra'):
result = page.evaluate(CHECK, {'theme': theme, 'density': density})
descriptor = width, theme, density
assert result['count'] == 78, (descriptor, result['count'])
assert result['otherCount'] >= 35, (descriptor, result['otherCount'])
assert result['primary'] != result['expected'], (descriptor, result)
assert not result['errors'], (descriptor, result['errors'][:25])
assert all(item['color'] == result['primary'] for item in result['numeric']), (descriptor, result['numeric'])
print(f'PASS muted titles, primary numeric readings at {width}px (3 themes, 3 densities)')
page.close()
browser.close()
print('PASS subdued metric/chart title CSS (simulated Chromium, no Rust runtime)')
scripts/tests/overview_samples_0330_ui_test.mjs
-116
View File
@@ -1,116 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Simulated browser/status contract: this does not exercise the Rust server.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const read = file => fs.readFileSync(path.join(root, file), 'utf8');
const html = renderAdminIndex(root);
const controllerSource = read('src/admin/web/app/overview-samples.js');
const store = new Map();
assert.match(html, /id="overview-layout-unlock"/);
// History controls share the connected-core eyebrow row (not a separate
// dashboard row), and navigation shows them only on Overview.
const metaStart = html.indexOf('class="page-intro-meta-row"');
const metaEnd = html.indexOf('class="page-intro-title-row"');
const statusPos = html.indexOf('id="connection-label"');
const selectPos = html.indexOf('id="overview-live-samples"');
assert.ok(metaStart < statusPos && statusPos < selectPos && selectPos < metaEnd);
assert.ok(selectPos < html.indexOf('id="panel-overview"'));
assert.match(html, /id="overview-samples-toolbar" hidden role="group"/);
assert.match(read('src/admin/web/app/core.js'), /\$\("overview-samples-toolbar"\)\.hidden = name !== "overview"/);
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="overview-layout-unlock"'));
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="dashboard-overview-grid"'));
for (const sample of [60, 120, 300, 600]) assert.match(html, new RegExp(`<option value="${sample}">`));
class Element {
constructor() { this.listeners = {}; this.value = ''; this.dataset = {}; this.disabled = false; this.attributes = new Map(); this.textContent = ''; this.hidden = false; }
addEventListener(name, callback) { this.listeners[name] = callback; }
fire(name) { this.listeners[name](); }
setAttribute(name, value) { this.attributes.set(name, String(value)); }
getAttribute(name) { return this.attributes.get(name); }
}
const nodes = new Map();
for (const id of ['overview-live-samples', 'overview-samples-save', 'overview-samples-state']) nodes.set(id, new Element());
const widgets = new Map([['wide_compression_activity', new Element()]]);
const document = {
readyState: 'complete', getElementById: id => nodes.get(id) || null,
querySelector: query => widgets.get(query.match(/data-dashboard-widget="([^"]+)"/)?.[1]) || null,
};
const localStorage = {
getItem: key => store.get(key) ?? null,
setItem: (key, value) => { store.set(key, value); },
};
let now = 1000;
const window = { rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
const context = vm.createContext({ document, localStorage, window, queueMicrotask: callback => callback(),
t: key => key, performance: { now: () => now },
currentPanel: 'overview', latestStatus: null, $: id => { if (!nodes.has(id)) nodes.set(id, new Element()); return nodes.get(id); },
fmt: n => String(n), fmtBytes: n => `${n} B`,
});
vm.runInContext(controllerSource, context);
const samples = window.rustProxyOverviewSamples;
const select = document.getElementById('overview-live-samples');
const save = document.getElementById('overview-samples-save');
const state = document.getElementById('overview-samples-state');
assert.equal(samples.getLimit(), 60);
assert.equal(save.disabled, true);
select.value = '600'; select.fire('change');
assert.equal(samples.getLimit(), 600);
assert.equal(state.dataset.i18n, 'overviewSamplesTemporary');
assert.equal(store.size, 0, 'temporary samples must not touch localStorage');
assert.equal(save.disabled, false);
const range = Array.from({ length: 700 }, (_, i) => i);
let calls = 0;
samples.onChange(() => { samples.trim(range); calls++; });
select.value = '120'; select.fire('change');
assert.equal(calls, 1);
assert.equal(range.length, 120);
assert.equal(range[0], 580);
assert.equal(samples.setLimit(999), false);
assert.equal(samples.getLimit(), 120);
save.fire('click');
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '120');
assert.equal(state.dataset.i18n, 'overviewSamplesSaved');
assert.equal(save.disabled, true);
// Fresh load reads only the saved choice, not a temporary override.
select.value = '300'; select.fire('change');
assert.equal(samples.getLimit(), 300);
const freshDocument = { readyState: 'loading', addEventListener() {} };
const fresh = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
vm.runInContext(controllerSource, fresh);
assert.equal(fresh.window.rustProxyOverviewSamples.getLimit(), 120);
// Permanently saved choice survives a fresh load.
save.fire('click');
const again = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
vm.runInContext(controllerSource, again);
assert.equal(again.window.rustProxyOverviewSamples.getLimit(), 300);
// Wide Overview charts must respect the same limit and redraw immediately when
// the limit shrinks, without recording an extra point or hitting the network.
vm.runInContext(read('src/admin/web/app/rate-sampling.js'), context);
vm.runInContext(read('src/admin/web/app/chart-scales.js'), context);
vm.runInContext(read('src/admin/web/app/overview-wide-metrics.js'), context);
select.value = '600'; select.fire('change');
const status = {
revision: 1, metrics: { compression_gzip: 0, compression_br: 0, compression_zstd: 0, compression_deflate: 0 },
};
context.latestStatus = status;
for (let i = 0; i <= 650; i++) {
now += 3000;
for (const key of ['compression_gzip', 'compression_br', 'compression_zstd', 'compression_deflate']) status.metrics[key] += i % 3 + 1;
window.rustProxyWideOverview.render(status);
}
const line = () => nodes.get('overview-wide-wide_compression_activity-gzip-line').getAttribute('d');
assert.equal((line().match(/ L/g) || []).length, 599, 'maximum 600 samples must be retained');
select.value = '120'; select.fire('change');
assert.equal((line().match(/ L/g) || []).length, 119, 'changing selection must trim and redraw without a new sample');
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '300', 'temporary choice must preserve the saved limit');
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('fetch('));
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('setInterval('));
console.log('PASS: Overview samples 60/120/300/600, temporary vs persistent, immediate trim/redraw, no extra requests');
scripts/tests/panel_html.mjs
-22
View File
@@ -1,22 +0,0 @@
import fs from 'node:fs';
import path from 'node:path';
// Same four compile-time substitutions as src/admin/public.rs.
export function renderAdminIndex(root) {
const web = path.join(root, 'src/admin/web');
let html = fs.readFileSync(path.join(web, 'index.html'), 'utf8');
const metrics = fs.readFileSync(path.join(web, 'partials/overview-metrics.html'), 'utf8')
.split('<!-- RP_METRICS_SPLIT -->');
const charts = fs.readFileSync(path.join(web, 'partials/overview-charts.html'), 'utf8')
.split('<!-- RP_CHARTS_SPLIT -->');
if (metrics.length !== 2 || charts.length !== 2) throw new Error('Invalid Overview partials');
for (const [marker, text] of [
['METRICS_MAIN', metrics[0]], ['CHARTS_MAIN', charts[0]],
['METRICS_WEB', metrics[1]], ['CHARTS_WEB', charts[1]],
]) {
const token = `<!-- RP_${marker} -->`;
if (!html.includes(token)) throw new Error(`Missing ${token}`);
html = html.replace(token, text);
}
return html;
}
scripts/tests/panel_html.py
-18
View File
@@ -1,18 +0,0 @@
"""Return the actual assembled embedded Overview HTML to simulated-browser tests."""
from pathlib import Path
def render_admin_index(root: Path) -> str:
web = root / 'src/admin/web'
html = (web / 'index.html').read_text(encoding='utf-8')
metrics = (web / 'partials/overview-metrics.html').read_text(encoding='utf-8').split('<!-- RP_METRICS_SPLIT -->')
charts = (web / 'partials/overview-charts.html').read_text(encoding='utf-8').split('<!-- RP_CHARTS_SPLIT -->')
assert len(metrics) == len(charts) == 2
for key, fragment in (
('METRICS_MAIN', metrics[0]), ('CHARTS_MAIN', charts[0]),
('METRICS_WEB', metrics[1]), ('CHARTS_WEB', charts[1]),
):
marker = '<!-- RP_' + key + ' -->'
assert html.count(marker) == 1
html = html.replace(marker, fragment)
return html
scripts/tests/progressbar_0332_browser_test.py
-107
View File
@@ -1,107 +0,0 @@
#!/usr/bin/env python3
"""Progressbar Appearance regression using the production CSS in Chromium.
Browser-only visual/CSS verification. Does not execute RustProxy or authenticate.
"""
from __future__ import annotations
from io import BytesIO
from pathlib import Path
from panel_html import render_admin_index
import re
from PIL import Image
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
APP_HTML = render_admin_index(ROOT)
assert re.search(r'<div class="overview-meter system-memory-meter" id="memory-meter"', APP_HTML)
assert 'id="memory-meter-fill"' in APP_HTML
CORNERS = {'default': '3px', 'square': '0px', 'subtle': '2px', 'rounded': '4px', 'extra': '999px'}
STYLES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
HTML = '''<!doctype html><html><body>
<section id="panel-cache"><div class="stats-grid cache-stats">
<article class="stat"><progress id="cp-hot-entries-meter" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-memory" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-cold-entries-meter" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-cold-disk" value="55" max="100"></progress></article>
</div></section>
<main id="panel-overview">
<div class="stat stat-meter"><div class="overview-meter" id="overview-progress"><span class="overview-meter-fill" id="overview-fill" style="width:55%"></span></div></div>
<article class="card system-card"><div class="overview-meter system-memory-meter" id="memory-meter" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="55"><span class="overview-meter-fill" id="memory-meter-fill" style="width:55%"></span></div></article>
<div class="www-segmented" id="www-segmented"><span style="width:55%;background:var(--chart-accent)"></span></div>
<div class="www-top-meter" id="www-top"><span id="www-top-fill" style="width:55%"></span></div>
</main>
<div class="traffic-bar-track" id="traffic-track"><span class="traffic-bar-fill" id="traffic-fill" style="width:55%"></span></div>
<div id="chart-swatch" style="background:var(--chart-accent);width:4px;height:4px"></div>
</body></html>'''
def color_at_progress(page, progress_id: str) -> tuple[int, int, int]:
rect = page.locator(f'#{progress_id}').bounding_box()
assert rect and rect['width'] > 100
screenshot = Image.open(BytesIO(page.screenshot())).convert('RGB')
x = round(rect['x'] + rect['width'] * .22)
y = round(rect['y'] + rect['height'] / 2)
return screenshot.getpixel((x, y))
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML)
page.add_style_tag(content=CSS)
for corner, radius in CORNERS.items():
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
actual = page.evaluate('''() => {
const ids = ['cp-hot-entries-meter','cp-memory','cp-cold-entries-meter','cp-cold-disk',
'overview-progress','overview-fill','memory-meter','memory-meter-fill','www-segmented','www-top','www-top-fill',
'traffic-track','traffic-fill'];
return Object.fromEntries(ids.map(id => [id, getComputedStyle(document.getElementById(id)).borderRadius]));
}''')
assert all(value == radius for value in actual.values()), (corner, radius, actual)
actual_pseudo = page.evaluate('''() => {
const p = document.getElementById('cp-memory');
return ['::-webkit-progress-bar', '::-webkit-progress-value'].map(pseudo => getComputedStyle(p, pseudo).borderRadius);
}''')
assert actual_pseudo == [radius, radius], (corner, actual_pseudo)
for theme in ('dark', 'light'):
page.evaluate('theme => document.documentElement.dataset.theme = theme', theme)
for chart_style in STYLES:
page.evaluate('style => document.documentElement.dataset.chartStyle = style', chart_style)
observed = page.evaluate('''() => {
const color = id => getComputedStyle(document.getElementById(id)).backgroundColor;
const progress = document.getElementById('cp-memory');
return {
chart:color('chart-swatch'), native:getComputedStyle(progress).accentColor,
overview:color('overview-fill'), systemRam:color('memory-meter-fill'), www:color('www-top-fill'), traffic:color('traffic-fill'),
track:getComputedStyle(progress).backgroundColor,
overviewTrack:color('overview-progress'), systemTrack:color('memory-meter'), wwwTrack:color('www-top'),
};
}''')
assert len({observed[key] for key in ('chart', 'native', 'overview', 'systemRam', 'www', 'traffic')}) == 1, (theme, chart_style, observed)
assert len({observed[key] for key in ('track', 'overviewTrack', 'systemTrack', 'wwwTrack')}) == 1, (theme, chart_style, observed)
# Native progress on Chromium is painted by the WebKit pseudo-element;
# a pixel check confirms the color, not just the accent-color property.
if chart_style in ('default', 'neon', 'monochrome'):
color = color_at_progress(page, 'cp-memory')
chart_rgb = tuple(page.evaluate("""() => {
const canvas = document.createElement('canvas');
canvas.width = canvas.height = 1;
const ctx = canvas.getContext('2d');
ctx.fillStyle = getComputedStyle(document.getElementById('chart-swatch')).backgroundColor;
ctx.fillRect(0, 0, 1, 1);
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
}"""))
assert color == chart_rgb, (theme, chart_style, color, chart_rgb)
assert not errors, errors
page.close()
browser.close()
print('PASS: native Cache, System RAM, Overview, WWW and Traffic bars respond to 5 corners and 8 palettes (dark/light, 375/1440px).')
scripts/tests/proxy_1kb_opt3_static_test.py
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env python3
"""Source-only regression guards for the OPT3 patch (NOT a Rust compile)."""
from pathlib import Path
root = Path(__file__).resolve().parents[2]
def read(name):
return (root / name).read_text(encoding='utf-8')
assert 'version = "0.34.0"' in read('Cargo.toml')
acl = read('src/acl.rs')
assert "pub fn routing_path_view(raw: &str)" in acl
assert "return routing_path(raw).map(std::borrow::Cow::Owned)" in acl
assert 'raw.split' in acl and 'raw.len() <= 8192' in acl
forward = read('src/proxy/forward_entry.rs')
assert 'let routing_uri = request.uri().clone();' in forward
assert 'crate::acl::routing_path_view(routing_uri.path())' in forward
assert 'if context.client_ip != peer.0.ip()' in forward
route = read('src/proxy/cache_helpers.rs')
assert 'crate::acl::routing_path_view(request.uri().path())' in route
for name in ('src/proxy/forward/cache_lookup.rs', 'src/proxy/forward/success_finish.rs', 'src/proxy/forward/web_source.rs'):
assert 'path.clone()' not in read(name), name
assert 'if !trusted_peer' in read('src/forwarding.rs')
assert 'let mut ips = chain(incoming).unwrap_or_default();' in read('src/forwarding.rs')
headers = read('src/proxy/headers.rs')
assert 'let no_nominations' in headers
assert 'let nominated: Vec<HeaderName>' in headers
assert 'headers.remove(name)' in headers
assert 'String::from_utf8(encoded)' in read('src/proxy/request_id.rs')
assert 'if reason == "cache_disabled"' in read('src/cache/stats.rs')
assert not read('src/cache/stats.rs').startswith('//!') # previous E0753 build issue
for name in ('src/acl/tests.rs', 'src/proxy/headers/tests.rs', 'src/forwarding/tests.rs'):
assert (root / name).exists()
for source in (root / 'src').rglob('*.rs'):
if '/tests/' in source.as_posix() or source.name == 'tests.rs':
continue
assert len(source.read_text(encoding='utf-8').splitlines()) <= 500, source
print('PASS OPT3 source guards (STATIC ONLY; not cargo check, live HTTP or throughput)')
scripts/tests/proxy_1kb_opt4_static_test.py
-52
View File
@@ -1,52 +0,0 @@
#!/usr/bin/env python3
"""OPT4 source invariants: not compilation or performance proof."""
from pathlib import Path
r = Path(__file__).resolve().parents[2]
def source(name):
return (r / name).read_text(encoding="utf-8")
assert 'version = "0.34.0"' in source('Cargo.toml')
route = source('src/proxy/forward/route_cache.rs')
assert "Cow<'_, str>" in route
assert 'routing_uri.path_and_query()' in route
assert 'Ok(None) => public_suffix.clone()' in route
assert 'std::borrow::Cow::Owned(encoded)' in route
for name in ('src/proxy/forward/cache_lookup.rs','src/proxy/forward/success_finish.rs'):
assert 'public_suffix: public_suffix.clone()' not in source(name)
assert 'public_suffix: public_suffix.to_string()' in source(name)
helpers = source('src/proxy/upstream_helpers.rs')
assert 'fn wrap_observed_request' in helpers
assert 'if body.is_end_stream() { body }' in helpers
assert 'let request = request.map(|body| wrap_observed_request' in helpers
assert 'fn log_prefix' in helpers
assert 'input.is_ascii()' in helpers and 'input.chars().take(max_chars).collect()' in helpers
assert 'path: log_prefix(request.uri().path(), 256)' in helpers
assert 'pub request_origin_uri: http::Uri' in source('src/config/model.rs')
target = source('src/proxy/types.rs')
assert 'fn upstream_request_target(\n uri: Uri' in target
assert 'return Some(uri);' in target
assert 'let mut parts = uri.into_parts();' in target
assert 'upstream_request_target(uri, request.headers_mut()' in target
assert 'upstream_request_target(\n uri, request.headers_mut()' in source('src/proxy/forward/upstream.rs')
forward = source('src/forwarding.rs')
assert 'peer_node.as_deref().unwrap_or(&peer_text)' in forward
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forward
headers = source('src/proxy/headers.rs')
assert 'let no_nominations' in headers
assert 'eq_ignore_ascii_case(b"close")' in headers
assert 'let nominated: Vec<HeaderName>' in headers
for name, test in [
('src/proxy/body/tests.rs','optimized_empty_request_keeps_exchange_until_response'),
('src/proxy/body/tests.rs','optimized_nonempty_request_still_meters_data_and_holds_capacity'),
('src/proxy/tests/authority_tests.rs','optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes'),
('src/proxy/tests/authority_tests.rs','moved_uri_keeps_h1_origin_and_query'),
('src/proxy/headers/tests.rs','absent_and_close_connection_preserve_non_hop_headers'),
]:
assert test in source(name), (name, test)
for p in (r / 'src').rglob('*.rs'):
if '/tests/' in p.as_posix() or p.name == 'tests.rs':
continue
assert len(p.read_text(encoding='utf-8').splitlines()) <= 500, p
assert not source('src/cache/stats.rs').startswith('//!')
print('PASS OPT4 source guard (STATIC ONLY; no cargo / no OPT4 benchmark)')
scripts/tests/proxy_1kb_opt5_static_test.py
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env python3
"""OPT5 invariants, static-only. Does not compile or benchmark Rust."""
from pathlib import Path
root = Path(__file__).resolve().parents[2]
def read(path):
return (root / path).read_text(encoding='utf-8')
assert 'version = "0.34.0"' in read('Cargo.toml')
identifier = read('src/proxy/request_id.rs')
assert 'rand::random()' in identifier
assert 'let mut token: [u8; 32]' in identifier
assert 'sequence.to_be_bytes()' in identifier
assert '*byte ^= sequence_byte;' in identifier
assert 'request_id_hex(&token)' in identifier
assert 'let mut encoded = vec![0u8; digest.len() * 2]' in identifier
assert 'Sha256::new()' not in identifier
forwarding = read('src/forwarding.rs')
assert 'self.context_with_original_url(incoming, peer, local, tls, authority, uri, true)' in forwarding
assert forwarding.count('if include_original_url {') == 2
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forwarding
assert 'crate::config::request_host(authority_text)?;' in forwarding
assert 'frontend.is_some_and(|frontend| frontend.tinyauth.is_some())' in read('src/proxy/forward_entry.rs')
assert '&context.original_url' in read('src/tinyauth.rs')
lease = read('src/balance/lease_runtime.rs')
assert 'let failure_detail = is_5xx.then(|| format!("status={status}"));' in lease
assert 'let detail = failure_detail.as_deref().unwrap_or(reason);' in lease
assert 'observe_passive_with_detail(' in lease
assert 'lease.response(' in read('src/proxy/forward/upstream.rs') or 'lease.response(' in read('src/proxy/forward/success_prepare.rs')
analytics = read('src/traffic.rs')
for literal in ['mpsc::channel::<Sample>(QUEUE_CAPACITY)', 'recv_many(&mut batch, 64)',
'for sample in batch.drain(..)', 'get_mut(sample.vhost.as_str())',
'state.vhosts.len() < MAX_VHOSTS', 'try_reserve()',
'self.dropped.fetch_add(1, Ordering::Relaxed)',
'candidate.eq_ignore_ascii_case(needle.as_bytes())',
'2 => self.statuses.add("2xx", 1)', 'map(Counters::view)']:
assert literal in analytics, literal
assert 'let lower = value.to_ascii_lowercase();' not in analytics
helpers = read('src/proxy/upstream_helpers.rs')
assert 'preferred_encoding(&original_headers, global_compression).is_some()' in helpers
assert 'compression.resolve(' not in helpers
assert 'compression::encode(' in helpers
assert 'wrap_observed_request(body, &exchange' in helpers
for rel, case in [
('src/traffic/tests.rs', 'batched_worker_keeps_exact_counts_and_bounded_vhosts'),
('src/traffic/tests.rs', 'status_top_list_preserves_common_and_extension_family_names'),
('src/forwarding/tests.rs', 'original_url_is_omitted_only_when_unneeded_and_forwarding_is_identical'),
('src/compression/tests.rs', 'unaccepted_codecs_cannot_be_reenabled_by_scoped_overrides'),
('src/balance/tests.rs', 'successful_lease_response_keeps_passive_observability_and_failure_detail'),
('src/proxy/request_id/tests.rs', 'independent_sequential_ids_keep_full_length_and_unique_values'),
]:
assert case in read(rel), (rel, case)
for path in (root / 'src').rglob('*.rs'):
if path.name == 'tests.rs' or '/tests/' in path.as_posix():
continue
assert len(path.read_text(encoding='utf-8').splitlines()) <= 500, path
assert not read('src/cache/stats.rs').startswith('//!')
print('PASS OPT5 static source checks; Rust build and runtime acceptance still required')
scripts/tests/proxy_hotpath_034_static_test.py
-47
View File
@@ -1,47 +0,0 @@
#!/usr/bin/env python3
"""Source-only guardrails for the uncompiled 0.34.0 hot-path candidate."""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def src(path):
return (ROOT / path).read_text()
snapshot = src("src/config/snapshot.rs")
assert "fn host_patterns(host: &str) -> impl Iterator" in snapshot
assert "pub fn select_on_arc(" in snapshot
assert "self.select_on_arc(host, path, local, tls).map(Arc::as_ref)" in snapshot
assert "Vec<Arc<CompiledRoute>>" in src("src/config/model.rs")
assert "route: Option<Arc<crate::config::CompiledRoute>>" in src("src/proxy/types.rs")
assert "request.extensions_mut().insert(SelectedRouteContext" in src("src/proxy/upstream_helpers.rs")
assert "selection.route.as_deref()" in src("src/proxy/forward_entry.rs")
assert "let fallback_request = selected" in src("src/proxy/forward_entry.rs")
assert "filter(|route| route.error_fallback.is_some())" in src("src/proxy/forward_entry.rs")
assert "let Some(next_request) = replay_headers" in src("src/proxy/forward/upstream.rs") or "replay_headers.as_ref().and_then" in src("src/proxy/forward/upstream.rs")
assert "let replay_headers = ((pool_retry_possible || h2_replay_possible)" in src("src/proxy/forward/upstream.rs")
pool = src("src/balance/pool_core.rs")
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
assert "let mut scheduling = lock(&self.scheduling);" in pool
assert "fn candidates_excluding" in pool
assert "health: member.health_settings.clone()" in pool
assert "health: selected.health_settings.clone()" in pool
assert "health_settings: Arc<HealthCheck>" in src("src/balance/health.rs")
assert "healthy_cached: AtomicBool" in src("src/balance/model.rs")
assert "self.healthy_cached.store(h.healthy, Ordering::Release)" in src("src/balance/health.rs")
exchange = src("src/proxy/body.rs")
for field in ("source", "cache_effective_enabled", "transform_effective_enabled"):
assert f"pub {field}: AtomicU8" in exchange
assert f"pub {field}: Mutex" not in exchange
assert "exchange.set_source(" in src("src/proxy/forward/route_cache.rs")
assert "exchange.set_cache_enabled(" in src("src/proxy/forward/web_source.rs")
assert "self.live.receiver_count() != 0" in src("src/logging/format.rs")
assert "fn route_handles_reuse_snapshot_ownership_without_cloning_compiled_routes" in src("src/config/tests.rs")
assert "fn single_backend_fast_path_observes_capacity_and_health_without_affinity_cookie" in src("src/balance/tests.rs")
assert "fn atomic_exchange_tags_keep_original_observability_fields" in src("src/proxy/body/tests.rs")
for path in (ROOT / "src").rglob("*.rs"):
if "/tests/" in path.as_posix() or path.name == "tests.rs" or path.is_relative_to(ROOT / "tests"):
continue
assert len(path.read_text().splitlines()) <= 500, path
assert 'version = "0.34.0"' in src("Cargo.toml")
assert "max_idle_per_host: 64" in src("src/balance/model.rs")
assert "pool_recycle_requests: 25_000" in src("src/balance/model.rs")
print("PASS 0.34.0 proxy hot-path wiring, observability, bounded tuned defaults and source-size limits (STATIC ONLY)")
scripts/tests/rps_hotpath_034_static_test.py
-129
View File
@@ -1,129 +0,0 @@
#!/usr/bin/env python3
"""Source invariants for 0.34.0 RPS optimizations (NOT a Rust build or RPS test).
These guards reject accidental removal of important branch/safety conditions.
They must be paired with ``cargo test`` and live black-box HTTP acceptance.
"""
from pathlib import Path
import json
ROOT = Path(__file__).resolve().parents[2]
def source(path):
return (ROOT / path).read_text(encoding="utf-8")
# Random request identifiers still use the OS-seeded thread-local CSPRNG,
# preserve a 256-bit unpredictable 64-hex token and mix the per-process sequence.
# OPT5 removes the extra SHA-256 operation, not the entropy or correlation ID.
identifier = source("src/proxy/request_id.rs")
assert "rand::random()" in identifier
assert "sequence.to_be_bytes()" in identifier
assert "*byte ^= sequence_byte;" in identifier
assert "request_id_hex(&token)" in identifier
assert "SysRng.try_fill_bytes" not in identifier
# Compiled/validated target authority survives request URI construction.
config = source("src/config/backend.rs")
model = source("src/config/model.rs")
request = source("src/proxy/types.rs")
assert "let request_origin_uri: http::Uri = request_origin.parse()?;" in config
assert "request_origin_uri," in config
assert "pub request_origin_uri: http::Uri," in model
assert "let path: http::uri::PathAndQuery = suffix.parse().ok()?;" in request
assert "let mut parts = backend.request_origin_uri.clone().into_parts();" in request
assert "parts.path_and_query = Some(path);" in request
assert "Uri::from_parts(parts).ok()" in request
# No new unbounded caches or disabled LB safety gates.
pool = source("src/balance/pool_core.rs")
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
assert "try_acquire_active(member.definition.max_concurrent_requests)" in pool
assert "m.state.healthy()" in pool
assert "scheduling.get_mut(member.definition.id.as_str())" in pool
assert "scheduling.entry(member.definition.id.clone())" in pool
health = source("src/balance/lease_runtime.rs")
assert "async fn probe(member: Arc<Member>, settings: Arc<HealthCheck>)" in health
assert "checks.spawn(probe(Arc::clone(member), health.clone()))" in health
assert "tokio::time::timeout" in health
# Configurable keep-alive policy, exact recycle-request threshold and age backup.
upstream = source("src/upstream/client.rs")
assert "self.current.load_full()" in upstream
assert "request_due || age_due" in upstream
assert "let now = monotonic_millis();" in upstream
assert "let age_ms = now.saturating_sub(self.generation_started_ms.load(Ordering::Relaxed));" in upstream
assert "fn maintain_rotation_at(" in upstream
assert "self.factory.keepalive.pool_recycle_requests" in upstream
assert "self.factory.keepalive.pool_recycle_seconds" in upstream
assert "pool_max_idle_per_host(if keepalive.enabled" in upstream
assert "retry_canceled_requests(false)" in upstream
assert "observe_protocol" in upstream
# Request/response streamed payloads retain accounting and timeout/cancellation.
body = source("src/proxy/body.rs")
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
assert "this.request_backend = exchange.backend" in body
assert "lease.bytes(count, true)" in body
assert "lease.bytes(count, false)" in body
assert "exchange.request_bytes.fetch_add(count" in body
assert "exchange.response_bytes.fetch_add(count" in body
assert "get_or_insert_with(|| Box::pin(sleep_until(this.deadline)))" in body
assert "this.finish();" in body
# Persist history and preserve live streaming/sink drop accounting.
logs = source("src/logging/format.rs")
assert "let has_viewers = self.live.receiver_count() != 0;" in logs
assert "let has_sink = self.settings.load().enabled();" in logs
assert "external.take().expect(\"entry\")" in logs
assert "self.live.send(entry.clone())" in logs
assert "self.sink.try_send(entry)" in logs
assert "self.sink_dropped.fetch_add(1" in logs
# Never weaken routing, identity, HTTP version and cache eligibility while tuning.
route = source("src/proxy/forward/route_cache.rs")
assert "admin_session_passthrough" in route
assert "cache_plan.request_policy" in route
assert "exchange.set_cache_enabled" in route
assert "exchange.set_transform_enabled" in route
assert "upstream_uri(&backend, &origin_suffix)" in route
assert "headers::strip_hop_by_hop" in route
assert "pool.select_wait(context.client_ip" in route or ".select_wait(context.client_ip" in route
assert "pool.select_member_wait(id)" in route
assert "max_idle_per_host: 64" in source("src/balance/model.rs")
assert "pool_recycle_requests: 25_000" in source("src/balance/model.rs")
assert "(1..=64).contains(&k.max_idle_per_host)" in source("src/balance/model.rs")
assert "keepalive.max_idle_per_host ?? 64" in source("src/admin/web/vhosts/editor.js")
openapi = json.loads(source("src/admin/openapi.json"))
api_keepalive = openapi["components"]["schemas"]
assert any(isinstance(schema, dict) and "keepalive" in schema.get("properties", {})
and schema["properties"]["keepalive"].get("properties", {})
.get("max_idle_per_host", {}).get("default") == 64
for schema in api_keepalive.values())
assert "| `max_idle_per_host` | 64 | 1..64 |" in source("docs/VHOSTS.md")
assert '"keepalive": { "max_idle_per_host": 4 }' in source("src/balance/tests.rs")
assert "self.route.get_mut()" in source("src/proxy/body.rs")
assert "let client_ip = self" in source("src/proxy/body.rs")
assert "if !headers.contains_key(http::header::COOKIE)" in source("src/proxy/headers.rs")
assert "if !headers.contains_key(http::header::SET_COOKIE)" in source("src/proxy/headers.rs")
assert 'version = "0.34.0"' in source("Cargo.toml")
# Full exchange duration histogram still exports cumulative Prometheus series,
# but only one exclusive bucket is updated per request; per-vhost and Docker
# exporters must reconstruct bucket counts from exactly the same interface.
histogram = source("src/metrics/vhost.rs")
exporter = source("src/prometheus/docker_misc.rs")
assert "let bucket = DURATION_BUCKETS_US.partition_point" in histogram
assert "self.buckets[bucket].fetch_add(1, Ordering::Relaxed);" in histogram
assert "pub fn duration_histogram(&self)" in histogram
assert exporter.count("let (buckets, count) = m.duration_histogram();") == 2
assert "rust_proxy_vhost_exchange_duration_seconds_bucket" in exporter
assert "rust_proxy_docker_vhost_exchange_duration_seconds_bucket" in exporter
assert "exclusive_bins_reconstruct_exact_prometheus_cumulative_boundaries" in source("src/metrics/vhost/tests.rs")
for file in (ROOT / "src").rglob("*.rs"):
if "/tests/" in file.as_posix() or file.name == "tests.rs":
continue
assert len(file.read_text(encoding="utf-8").splitlines()) <= 500, file
print("PASS RPS hot-path source invariants (STATIC ONLY; new binary not compiled)")
scripts/tests/sendfile_034_panel_browser_test.py
-82
View File
@@ -1,82 +0,0 @@
#!/usr/bin/env python3
"""Offline Chromium test: sendfile toggle serializes in real admin bundle.
It uses simulated config and a mocked saveSection; never claims native
sendfile execution, Rust compilation or live server acceptance.
"""
from pathlib import Path
import json
import re
from playwright.sync_api import sync_playwright
from panel_html import render_admin_index
ROOT = Path(__file__).resolve().parents[2]
ROUTER = (ROOT / "src/admin/router.rs").read_text()
start = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert start
array = ROUTER[start.end():]
array = array[:re.search(r'\]\s*\.join\("\\n"\)', array).start()]
chunks = []
for line in array.splitlines():
line = line.strip()
match = re.search(r'include_str!\("([^"]+)"\)', line)
if match:
chunks.append((ROOT / "src/admin" / match.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
chunks.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (f + '.js')).read_text()
for f in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
chunks.append(json.loads(line[:-1]))
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 950})
page.set_default_timeout(4500)
errors = []
page.on('pageerror', lambda err: errors.append(str(err)))
page.set_content(render_admin_index(ROOT))
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
page.add_script_tag(content='\n'.join(chunks))
page.evaluate('''() => {
envelope = {revision: 10, config: {web_services: [{
id: 'site-A', name: 'Static Site', document_root: '/srv/static',
index: ['index.html'], php: {enabled: false}
}], vhosts: [{id: 'vhost-A', keepalive: {max_connections: 8}}], php_fpm_profiles: []}};
currentPanel = 'web-server';
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-web-server').hidden = false;
window.__saved = [];
saveSection = async (section, value) => {
window.__saved.push({section, value: structuredClone(value)});
if (section === 'web-services') envelope.config.web_services = structuredClone(value);
return true;
};
window.rustProxyWebServer.render();
}''')
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
assert page.locator('#web-service-sendfile').input_value() == 'off'
page.evaluate("document.getElementById('web-service-sendfile').value = 'auto'")
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
page.wait_for_function('window.__saved.length === 1')
result = page.evaluate('''() => ({
saved: window.__saved[0],
cacheStillUnrelated: envelope.config.vhosts[0].keepalive.max_connections === 8,
closed: !document.getElementById('web-service-dialog').open
})''')
assert result['saved']['section'] == 'web-services', result
assert result['saved']['value'][0]['sendfile'] == 'auto', result
assert result['saved']['value'][0]['document_root'] == '/srv/static', result
assert result['cacheStillUnrelated'] and result['closed'], result
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
assert page.locator('#web-service-sendfile').input_value() == 'auto'
page.evaluate("document.getElementById('web-service-sendfile').value = 'off'")
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
page.wait_for_function('window.__saved.length === 2')
assert page.evaluate('envelope.config.web_services[0].sendfile') == 'off'
assert not errors, errors
print(f'PASS simulated panel sendfile off -> auto -> off config save at {width}px')
browser.close()
scripts/tests/sendfile_034_static_test.py
-44
View File
@@ -1,44 +0,0 @@
#!/usr/bin/env python3
"""RustProxy 0.34.0 sendfile source-wiring assertions ONLY, not native tests."""
from pathlib import Path
import json
import re
root = Path(__file__).resolve().parents[2]
get = lambda path: (root / path).read_text()
model = get('src/web_server/config/model.rs')
assert 'pub enum SendfileMode' in model and 'sendfile: SendfileMode::Off' in model
assert 'pub sendfile: SendfileMode' in model
assert 'sendfile: self.sendfile' in get('src/web_server/config/compile.rs')
assert 'pub sendfile: SendfileMode' in get('src/web_server/config/validation.rs')
assert 'service.sendfile = crate::web_server::SendfileMode::Off;' in get('src/runtime/types.rs')
source = get('src/proxy/forward/web_source.rs')
for guard in ('!enabled && identity_request', 'request_method == Method::GET',
'response.status() == StatusCode::OK', 'content-encoding',
'transfer-encoding', 'content-range', 'candidate.length',
'http::Version::HTTP_11'):
assert guard in source, guard
assert 'content_encoding.is_none()' in get('src/web_server/response/file.rs')
assert 'single_range.is_none()' in get('src/web_server/response/file.rs')
assert 'opened.file.try_clone()' in get('src/web_server/response/file.rs')
bridge = get('src/server/sendfile.rs')
for item in ('libc::sendfile(', 'MAX_PENDING: usize = 8', 'MIN_SIZE: u64 = 64 * 1024',
'registry.slots.is_empty()', 'native_sendfile_transfers_exact_file',
'sendfile placeholder did not match', 'UnexpectedEof'):
assert item in bridge, item
assert 'pub(crate) mod sendfile' in get('src/server.rs')
assert 'SendfileIo::new(stream)' in get('src/server/tls_runtime.rs')
assert 'sendfile_slots' in get('src/server/http1_connection.rs')
js = get('src/admin/web/web-server.js')
html = get('src/admin/web/index.html')
assert '"web-service-sendfile"' in js and 'sendfile: "off"' in js
assert 'sendfile: $("web-service-sendfile").value' in js
assert html.count('id="web-service-sendfile"') == 1
assert 'value="auto"' in html
for locale in ('en_US', 'pl_pl'):
keys = json.loads(get(f'langs/{locale}.json'))['translations']
assert all(k in keys for k in ('webServiceSendfile', 'webServiceSendfileHelp', 'webServiceSendfileAuto'))
api = json.loads(get('src/admin/openapi.json'))
assert 'sendfile' in json.dumps(api)
print('PASS 0.34.0 sendfile source guards, default OFF, Linux syscall, UI, translations, API and scoped cache (STATIC ONLY)')
scripts/tests/sidebar_nav_0333_browser_test.py
-116
View File
@@ -1,116 +0,0 @@
#!/usr/bin/env python3
"""Verify sidebar colors use only the UI theme; chart palettes affect charts only.
Uses actual production HTML/CSS in headless Chromium, not a live RustProxy
server or authenticated preference/session roundtrip.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
THEMES = ('dark', 'ultra_dark', 'light', 'cream')
PALETTES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
CORNERS = {'default': 0, 'square': 0, 'subtle': 4, 'rounded': 8, 'extra': 12}
STYLE_INFO = r'''() => {
const color = value => {
const c = document.createElement('canvas'); c.width = c.height = 1;
const ctx = c.getContext('2d'); ctx.fillStyle = value; ctx.fillRect(0, 0, 1, 1);
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
};
const root = getComputedStyle(document.documentElement);
const active = document.querySelector('.sidebar .main-navigation .nav.active');
const idle = document.querySelector('.sidebar .main-navigation .nav:not(.active)');
const sa = getComputedStyle(active), si = getComputedStyle(idle);
return {
chartAccent:color(root.getPropertyValue('--chart-accent').trim()),
themeSelected:color(root.getPropertyValue('--nav-active-bg').trim()),
themeHovered:color(root.getPropertyValue('--nav-hover-bg').trim()),
foreground:color(sa.color),
background:color(sa.backgroundColor),
borderLeft:color(sa.borderLeftColor),
borderRight:color(sa.borderRightColor),
radius:parseFloat(sa.borderTopLeftRadius),
idleBackground:color(si.backgroundColor),
mobileBackground:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).backgroundColor),
mobileColor:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).color),
navWidth:document.querySelector('.main-navigation').scrollWidth,
navClientWidth:document.querySelector('.main-navigation').clientWidth,
activeWidth:active.scrollWidth,
activeClientWidth:active.clientWidth,
};
}'''
def contrast(fg, bg):
def luminance(rgb):
channels = [x / 255 for x in rgb]
channels = [x / 12.92 if x <= 0.04045 else ((x + 0.055) / 1.055) ** 2.4 for x in channels]
return .2126 * channels[0] + .7152 * channels[1] + .0722 * channels[2]
lo, hi = sorted((luminance(fg), luminance(bg)))
return (hi + .05) / (lo + .05)
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (375, 1200):
page = browser.new_page(viewport={'width': viewport, 'height': 820})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', '').replace('/assets/app___RUSTPROXY_ASSET_ID__.js', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading');
document.getElementById('app-startup').remove();
document.querySelector('.sidebar').classList.add('mobile-nav-open');
document.querySelector('.mobile-menu-toggle')?.setAttribute('aria-expanded', 'true');
}''')
theme_colors = []
for theme in THEMES:
baseline = None
chart_accents = set()
for palette in PALETTES:
page.evaluate('''([theme, palette]) => {
document.documentElement.dataset.theme = theme;
document.documentElement.dataset.chartStyle = palette;
}''', [theme, palette])
for corner, expected_radius in CORNERS.items():
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
style = page.evaluate(STYLE_INFO)
assert style['radius'] == expected_radius, (viewport, theme, palette, corner, style['radius'])
assert style['borderLeft'] == style['foreground'], (viewport, theme, palette, corner, style)
assert style['background'] == style['themeSelected'], (viewport, theme, palette, corner, style)
if viewport == 375:
assert style['mobileBackground'] == style['background'], (viewport, theme, palette, corner, style)
assert style['mobileColor'] == style['foreground'], (viewport, theme, palette, corner, style)
assert contrast(style['foreground'], style['background']) >= 4.5, (viewport, theme, palette, corner, style)
assert style['navWidth'] <= style['navClientWidth'] + 2, (viewport, theme, palette, corner, style)
assert style['activeWidth'] <= style['activeClientWidth'] + 2, (viewport, theme, palette, corner, style)
chart_accents.add(tuple(style['chartAccent']))
# The entire selected state is unaffected by chart-style changes.
nav_colors = (tuple(style['foreground']), tuple(style['background']), tuple(style['borderLeft']))
if baseline is None:
baseline = nav_colors
else:
assert baseline == nav_colors, (viewport, theme, palette, baseline, nav_colors)
page.locator('.nav[data-panel="vhosts"]').hover()
hovered = page.evaluate(STYLE_INFO)
assert hovered['idleBackground'] == hovered['themeHovered'], (viewport, theme, palette, hovered)
assert hovered['idleBackground'] != style['idleBackground'], (viewport, theme, palette)
page.mouse.move(0, 0)
if viewport == 1200 and theme == 'ultra_dark' and palette == 'cool':
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_desktop.png'))
if viewport == 375 and theme == 'dark' and palette == 'soft':
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_mobile.png'))
assert len(chart_accents) > 1, (viewport, theme, chart_accents)
theme_colors.append(baseline)
assert len(set(theme_colors)) == len(THEMES), (viewport, theme_colors)
page.evaluate("document.documentElement.dir = 'rtl'")
rtl = page.evaluate(STYLE_INFO)
assert rtl['borderRight'] == rtl['foreground'], (viewport, rtl)
assert not errors, errors
print('PASS themed navigation independence: viewport', viewport, 'themes', len(THEMES), 'palettes', len(PALETTES), 'corners', len(CORNERS))
page.close()
browser.close()
print('PASS: sidebar changes with UI theme and corner setting, never with chart style')
scripts/tests/system_ram_meter_0332_browser_test.py
-72
View File
@@ -1,72 +0,0 @@
#!/usr/bin/env python3
"""System Load RAM meter updates using the actual combined panel JavaScript.
This is a simulated authenticated DOM/status test, not a Rust runtime integration test.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
router = (ROOT / 'src/admin/router.rs').read_text()
def bundle_for(section):
match = re.search(r'fn ' + section + r'\([^)]*\)[^{]*\{', router)
assert match, section
text = router[match.end():]
text = text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
result = []
for line in text.splitlines():
line = line.strip()
if 'include_str!("' in line:
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
result.append((ROOT / 'src/admin' / path).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + ''.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() + '\n'
for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
html = render_admin_index(ROOT)
css = (ROOT / 'src/admin/web/style.css').read_text()
js = bundle_for('admin_app_javascript')
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '').replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=css)
page.add_script_tag(content=js)
page.wait_for_timeout(100)
report = page.evaluate('''() => {
currentPanel = "overview";
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const meter = document.getElementById('memory-meter');
const fill = document.getElementById('memory-meter-fill');
const snapshot = (used, total) => ({system: {source:'measured',cpu_percent:2.1,
memory_used_bytes:used, memory_total_bytes:total},
live_system:{cpu_percent:2.1}, cache:{}, metrics:{}, listeners:{}});
const values = [];
for (const [used, total] of [[250,1000],[800,1000],[0,1000],[null,null]]) {
renderDiagnostics(snapshot(used,total), false, true);
values.push({fill:fill.style.width, now:meter.getAttribute('aria-valuenow'),
hidden:meter.hidden, text:meter.getAttribute('aria-valuetext')});
}
return { values, role:meter.getAttribute('role'), cssHeight:getComputedStyle(meter).height,
scrollbar:document.documentElement.scrollWidth>window.innerWidth+2 };
}''')
assert report['role'] == 'progressbar' and report['cssHeight'] == '5px', report
assert report['values'][0]['fill'] == '25%' and report['values'][0]['now'] == '25', report
assert report['values'][1]['fill'] == '80%' and report['values'][1]['now'] == '80', report
assert report['values'][2]['fill'] == '0%' and report['values'][2]['now'] == '0', report
assert report['values'][3]['fill'] == '0%' and report['values'][3]['now'] is None and report['values'][3]['hidden'], report
assert not report['scrollbar'] and not errors, (width, errors, report)
page.close()
browser.close()
print('PASS: System RAM meter live percentage and unknown/zero handling on 375/1440px in bundled panel.')
scripts/tests/test_stream_tcp.py
-137
View File
@@ -1,137 +0,0 @@
#!/usr/bin/env python3
"""TCP Stream black-box test: use against a running RustProxy and mock backends.
The 'selftest' mode validates this harness only; it does NOT run RustProxy.
All traffic uses loopback by default. Do not point it at a production service.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import secrets
import socket
import socketserver
import sys
import threading
import time
MAX_LINE = 1024
class TCPServer(socketserver.ThreadingTCPServer):
allow_reuse_address = True
daemon_threads = True
request_queue_size = 128
class TCPBackend(socketserver.StreamRequestHandler):
def handle(self) -> None:
self.request.settimeout(10)
backend_id = self.server.backend_id.encode('ascii')
try:
while True:
line = self.rfile.readline(MAX_LINE + 1)
if not line or len(line) > MAX_LINE or not line.endswith(b'\n'):
return
self.wfile.write(backend_id + b'|' + line)
self.wfile.flush()
except (OSError, TimeoutError):
return
def run_server(host: str, port: int, backend_id: str) -> None:
with TCPServer((host, port), TCPBackend) as server:
server.backend_id = backend_id
print(f'TCP mock {backend_id} listening on {server.server_address}', flush=True)
try:
server.serve_forever(poll_interval=0.1)
except KeyboardInterrupt:
pass
def check_one(index: int, host: str, port: int, messages: int, timeout: float, hold_ms: float, nonce: str) -> str:
backend = None
with socket.create_connection((host, port), timeout=timeout) as connection:
connection.settimeout(timeout)
with connection.makefile('rb') as reader:
for seq in range(messages):
payload = f'rp-tcp:{nonce}:{index}:{seq}\n'.encode('ascii')
connection.sendall(payload)
response = reader.readline(MAX_LINE + 1)
try:
observed, echoed = response.split(b'|', 1)
except ValueError as exc:
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
assert echoed == payload, f'session {index}: response mismatch: {response!r}'
assert observed, f'session {index}: empty backend ID'
if backend is None:
backend = observed
assert observed == backend, f'session {index}: TCP backend changed during connection'
if hold_ms and seq != messages - 1:
time.sleep(hold_ms / 1000)
return backend.decode('ascii')
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
hold_ms: float, min_backends: int) -> Counter:
nonce = secrets.token_hex(8)
started = time.monotonic()
counts: Counter[str] = Counter()
with ThreadPoolExecutor(max_workers=parallel) as pool:
futures = [pool.submit(check_one, i, host, port, messages, timeout, hold_ms, nonce)
for i in range(sessions)]
for future in as_completed(futures):
counts[future.result()] += 1
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
print(f'TCP PASS: {sessions} sessions, {sessions * messages} messages, '
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
return counts
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='mode', required=True)
server = sub.add_parser('server', help='run a local TCP mock backend')
server.add_argument('--host', default='127.0.0.1')
server.add_argument('--port', type=int, required=True)
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy listener')
client.add_argument('--host', default='127.0.0.1', help='RustProxy TCP listener host')
client.add_argument('--port', type=int, required=True, help='RustProxy TCP listener port')
client.add_argument('--sessions', type=int, default=24)
client.add_argument('--messages', type=int, default=3)
client.add_argument('--parallel', type=int, default=8)
client.add_argument('--timeout', type=float, default=4.0)
client.add_argument('--hold-ms', type=float, default=20.0, help='pause between messages to keep TCP sessions alive')
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
args = parser.parse_args()
if args.mode == 'server':
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
parser.error('--backend-id must be nonempty ASCII without | or newline')
run_server(args.host, args.port, args.backend_id)
elif args.mode == 'client':
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.hold_ms < 0:
parser.error('invalid port, count, timeout or hold delay')
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
args.timeout, args.hold_ms, args.min_backends)
else:
with TCPServer(('127.0.0.1', 0), TCPBackend) as server:
server.backend_id = 'local-mock'
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
print('SELFTEST PASS: tested mock TCP peer, not RustProxy')
finally:
server.shutdown()
thread.join(timeout=2)
if __name__ == '__main__':
try:
main()
except (OSError, AssertionError, UnicodeError) as exc:
print(f'TCP FAIL: {exc}', file=sys.stderr)
sys.exit(1)
scripts/tests/test_stream_udp.py
-135
View File
@@ -1,135 +0,0 @@
#!/usr/bin/env python3
"""UDP Stream black-box test: sends multiple packets from each stable client socket.
Start separately labeled mock backends and target an existing RustProxy UDP
listener. 'selftest' only validates this Python harness, NOT RustProxy.
By default, the server responds PONG to PING for an explicitly configured probe.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import secrets
import socket
import socketserver
import sys
import threading
import time
MAX_DATAGRAM = 1024
class UDPServer(socketserver.UDPServer):
allow_reuse_address = True
class UDPBackend(socketserver.BaseRequestHandler):
def handle(self) -> None:
data, connection = self.request
if data == b'PING':
reply = b'PONG'
elif data and len(data) <= MAX_DATAGRAM:
reply = self.server.backend_id.encode('ascii') + b'|' + data
else:
return
connection.sendto(reply, self.client_address)
def run_server(host: str, port: int, backend_id: str) -> None:
with UDPServer((host, port), UDPBackend) as server:
server.backend_id = backend_id
print(f'UDP mock {backend_id} listening on {server.server_address}; health probe PING -> PONG', flush=True)
try:
server.serve_forever(poll_interval=0.1)
except KeyboardInterrupt:
pass
def check_one(index: int, host: str, port: int, messages: int, timeout: float, pause_ms: float, nonce: str) -> str:
info = socket.getaddrinfo(host, port, type=socket.SOCK_DGRAM)[0]
family, sock_type, proto, _, address = info
backend = None
# One socket = one stable client identity for the entire UDP Stream session.
with socket.socket(family, sock_type, proto) as connection:
connection.settimeout(timeout)
for seq in range(messages):
payload = f'rp-udp:{nonce}:{index}:{seq}'.encode('ascii')
connection.sendto(payload, address)
response, _ = connection.recvfrom(MAX_DATAGRAM + 128)
try:
observed, echoed = response.split(b'|', 1)
except ValueError as exc:
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
assert echoed == payload, f'session {index}: response mismatch {response!r}'
assert observed, f'session {index}: empty backend ID'
if backend is None:
backend = observed
assert observed == backend, f'session {index}: UDP backend changed for the same client socket'
if pause_ms and seq != messages - 1:
time.sleep(pause_ms / 1000)
return backend.decode('ascii')
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
pause_ms: float, min_backends: int) -> Counter:
nonce = secrets.token_hex(8)
started = time.monotonic()
counts: Counter[str] = Counter()
with ThreadPoolExecutor(max_workers=parallel) as pool:
futures = [pool.submit(check_one, i, host, port, messages, timeout, pause_ms, nonce)
for i in range(sessions)]
for future in as_completed(futures):
counts[future.result()] += 1
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
print(f'UDP PASS: {sessions} client sockets, {sessions * messages} datagrams, '
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
return counts
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='mode', required=True)
server = sub.add_parser('server', help='run local UDP echo and optional PING/PONG probe responder')
server.add_argument('--host', default='127.0.0.1')
server.add_argument('--port', type=int, required=True)
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy UDP listener')
client.add_argument('--host', default='127.0.0.1', help='RustProxy UDP listener host')
client.add_argument('--port', type=int, required=True, help='RustProxy UDP listener port')
client.add_argument('--sessions', type=int, default=24)
client.add_argument('--messages', type=int, default=3)
client.add_argument('--parallel', type=int, default=8)
client.add_argument('--timeout', type=float, default=4.0)
client.add_argument('--pause-ms', type=float, default=20.0, help='pause between datagrams on each stable client socket')
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
args = parser.parse_args()
if args.mode == 'server':
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
parser.error('--backend-id must be nonempty ASCII without | or newline')
run_server(args.host, args.port, args.backend_id)
elif args.mode == 'client':
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.pause_ms < 0:
parser.error('invalid port, count, timeout or pause delay')
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
args.timeout, args.pause_ms, args.min_backends)
else:
with UDPServer(('127.0.0.1', 0), UDPBackend) as server:
server.backend_id = 'local-mock'
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
print('SELFTEST PASS: tested mock UDP peer, not RustProxy')
finally:
server.shutdown()
thread.join(timeout=2)
if __name__ == '__main__':
try:
main()
except (OSError, AssertionError, UnicodeError) as exc:
print(f'UDP FAIL: {exc}', file=sys.stderr)
sys.exit(1)
scripts/tests/upgrade_performance_test.py
-80
View File
@@ -1,80 +0,0 @@
#!/usr/bin/env python3
"""Run source-upgrade preservation/refusal regressions in disposable directories."""
from pathlib import Path
import shutil
import sqlite3
import sys
import tempfile
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT))
from scripts.deploy.upgrade import upgrade
def fixture(parent):
target = parent / "installed"
target.mkdir()
(target / "Cargo.toml").write_text('[package]\nname = "rust-proxy"\nversion = "0.33.7"\n')
(target / "src").mkdir()
(target / "src" / "obsolete.rs").write_text("// old managed source\n")
(target / "data").mkdir()
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
db.execute("CREATE TABLE preserved(value TEXT)")
db.execute("INSERT INTO preserved VALUES ('configuration and preferences')")
(target / ".env").write_text("LOCAL_FIXTURE=1\n")
(target / "langs").mkdir()
(target / "langs" / "custom.json").write_text('{"custom": true}\n')
return target
def main():
with tempfile.TemporaryDirectory(prefix="rustproxy-upgrade-test-", dir=ROOT.parent) as temporary:
parent = Path(temporary)
target = fixture(parent)
try:
upgrade(ROOT, target, stopped=False)
except ValueError:
pass
else:
raise AssertionError("upgrade must require stopped confirmation")
assert (target / "src" / "obsolete.rs").is_file()
print("PASS upgrade refuses missing --stopped without modifying source")
if sys.platform != "win32":
import fcntl
with (target / "data" / "process.lock").open("w+") as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
try:
upgrade(ROOT, target, stopped=True)
except ValueError:
pass
else:
raise AssertionError("upgrade must refuse a live process lock")
print("PASS upgrade refuses a data directory locked by a live process")
backup = upgrade(ROOT, target, stopped=True)
assert (target / ".env").read_text() == "LOCAL_FIXTURE=1\n"
assert (target / "langs" / "custom.json").read_text() == '{"custom": true}\n'
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
assert db.execute("SELECT value FROM preserved").fetchone() == ("configuration and preferences",)
assert not (target / "src" / "obsolete.rs").exists()
assert (backup / "src" / "obsolete.rs").is_file()
assert (target / "Cargo.toml").read_bytes() == (ROOT / "Cargo.toml").read_bytes()
assert (target / "src" / "proxy" / "cache_plan.rs").is_file()
print("PASS upgrade preserves SQLite, environment and custom language; backs up old source and replaces managed tree")
if sys.platform != "win32":
shutil.rmtree(target / "src")
(target / "src").symlink_to(backup / "src", target_is_directory=True)
try:
upgrade(ROOT, target, stopped=True)
except ValueError:
pass
else:
raise AssertionError("upgrade must refuse a managed symlink")
assert (backup / "src" / "obsolete.rs").is_file()
print("PASS upgrade refuses managed symlinks without changing their targets")
if __name__ == "__main__":
main()
scripts/tests/vhost_backend_popup_0337_browser_test.py
-268
View File
@@ -1,268 +0,0 @@
#!/usr/bin/env python3
"""Simulated Chromium Vhost backend quick-view, without a Rust server.
Verify both backend triggers, the shared click-to-open upstream H2 protocol
popover without native title flicker, and the two Cache card margins.
Exercise live updates, safe text, keyboard/outside dismissal and mobile sizing.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
import json
import re
import time
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
PACKS = {lang: {'id': lang, **json.loads((ROOT / 'langs' / f'{lang}.json').read_text())}
for lang in ('en_US', 'pl_pl')}
def app_bundle():
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert found
code = ROUTER[found.end():]
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
fragments = []
for line in code.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / f'{name}.js').read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
return '\n'.join(fragments)
CONFIG = {
'vhosts': [{
'id': 'api-one', 'enabled': True, 'source': {'type': 'reverse_proxy'},
'frontend': {'hosts': ['api.example.test'], 'listener_ids': []},
'backends': [
{'id': 'origin-http', 'url': 'http://10.12.0.5:8080/api', 'protocol': 'http2', 'enabled': True},
{'id': '<origin-dns>', 'url': 'https://hiddenuser:secret@backend.example.test:8443/path', 'protocol': 'auto', 'enabled': True},
]
}, {'id': 'static-only', 'enabled': True,
'source': {'type': 'web_server', 'web_service_id': 'static-test'},
'frontend': {'hosts': ['static.example.test'], 'listener_ids': []}, 'backends': []}],
'listeners': {'endpoints': []}, 'web_services': [], 'acl_profiles': [],
}
def snapshot(first=200):
return {'backend_pools': [{
'id': 'api-one', 'health_mode': 'on',
'members': [
{'id': 'origin-http', 'status': 'up', 'last_status': first,
'requests': 12, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
{'id': '<origin-dns>', 'status': 'down', 'last_status': 0,
'requests': 0, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
]
}], 'vhost_metrics': {}}
def protocol_snapshot(h2_count):
status = snapshot()
status['backend_pools'][0]['members'][0].update({
'upstream_protocol_observed': 'h2',
'upstream_protocol_last_seen_ms': int(time.time() * 1000),
'upstream_protocol_http1_total': 3,
'upstream_protocol_http2_total': h2_count,
})
return status
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440):
page = browser.new_page(viewport={'width': width, 'height': 820})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
page.evaluate('''([config, status, packs]) => {
languagePacks = new Map(Object.entries(packs));
defaultLanguage = 'en_US'; language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-vhosts').hidden = false;
currentPanel = 'vhosts';
envelope = {revision: 12, config};
latestStatus = status;
window.rustProxyVhosts.render();
}''', [CONFIG, snapshot(), PACKS])
vhost = page.locator('[data-vhost-card="api-one"]')
backend = vhost.locator('.vhost-detail.vhost-backend-trigger')
health = vhost.locator('.vhost-live-metric.vhost-backend-trigger')
assert backend.count() == health.count() == 1, width
assert backend.get_attribute('aria-expanded') == 'false'
assert backend.get_attribute('aria-haspopup') == 'dialog'
# Clickable backend/health fields must match their non-button peers,
# including for user-selected interface corner styles.
shapes = page.evaluate('''() => {
const root = document.querySelector('[data-vhost-card="api-one"]');
const peers = [
[root.querySelector('.vhost-detail.vhost-backend-trigger'), root.querySelector('.vhost-detail:not(.vhost-backend-trigger)')],
[root.querySelector('.vhost-live-metric.vhost-backend-trigger'), root.querySelector('.vhost-live-metric:not(.vhost-backend-trigger)')]
];
const checks = ['square', 'subtle', 'rounded', 'extra'].map(corner => {
document.documentElement.dataset.cornerStyle = corner;
return {corner, tiles: peers.map(([trigger, peer]) => {
const left = getComputedStyle(trigger), right = getComputedStyle(peer);
return {radius: left.borderRadius, peerRadius: right.borderRadius,
padding: left.padding, peerPadding: right.padding,
font: left.fontFamily, peerFont: right.fontFamily};
})};
});
document.documentElement.removeAttribute('data-corner-style');
return checks;
}''')
for shape in shapes:
for tile in shape['tiles']:
assert tile['radius'] == tile['peerRadius'] == '0px', (width, shape)
assert tile['padding'] == tile['peerPadding'], (width, shape)
assert tile['font'] == tile['peerFont'], (width, shape)
backend.click()
popup = page.locator('#vhost-backend-popup')
assert popup.count() == 1, width
assert backend.get_attribute('aria-expanded') == 'true'
assert popup.get_attribute('role') == 'dialog'
# The anchored popup is a surface, not a permanently rounded tooltip.
# It must follow live interface corner preference changes without
# dismissing/recreating the open backend (or shared H2) popup.
corners = page.evaluate('''() => {
const node = document.getElementById('vhost-backend-popup');
const saved = document.documentElement.getAttribute('data-corner-style');
const out = {};
for (const style of ['default', 'square', 'subtle', 'rounded', 'extra']) {
document.documentElement.dataset.cornerStyle = style;
out[style] = {
popup: getComputedStyle(node).borderRadius,
row: getComputedStyle(node.querySelector('.vhost-backend-popup-row')).borderRadius,
close: getComputedStyle(node.querySelector('.vhost-backend-popup-close')).borderRadius,
reference: getComputedStyle(document.querySelector('[data-vhost-card="api-one"]')).borderRadius,
sameNode: node === document.getElementById('vhost-backend-popup'),
};
}
if (saved === null) document.documentElement.removeAttribute('data-corner-style');
else document.documentElement.dataset.cornerStyle = saved;
return out;
}''')
expected_radii = {'default': ('0px', '0px'), 'square': ('0px', '0px'),
'subtle': ('5px', '4px'), 'rounded': ('12px', '8px'),
'extra': ('18px', '12px')}
for style, (surface, control) in expected_radii.items():
actual = corners[style]
assert actual == {'popup': surface, 'row': surface, 'close': control,
'reference': surface, 'sameNode': True}, (width, style, actual)
values = popup.evaluate('''node => ({
cards: node.querySelectorAll('.vhost-backend-popup-row').length,
text: node.textContent, html: node.innerHTML,
box: (() => { const r = node.getBoundingClientRect();
return {left: r.left, right: r.right, top:r.top, bottom:r.bottom}; })(),
})''')
assert values['cards'] == 2, (width, values)
assert '10.12.0.5:8080' in values['text'] and 'HTTP 200' in values['text'], (width, values)
assert 'backend.example.test:8443' in values['text'], width
assert 'secret' not in values['text'], width
assert '<origin-dns>' in values['text'] and '&lt;origin-dns&gt;' in values['html'], width
assert 'Brak odpowiedzi HTTP' in values['text'], width
assert 'HTTPS · AUTO' in values['text'], width
assert values['box']['left'] >= -1 and values['box']['right'] <= width + 1, (width, values['box'])
assert values['box']['top'] >= -1 and values['box']['bottom'] <= 821, (width, values['box'])
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, width
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', snapshot(503))
assert 'HTTP 503' in popup.inner_text(), width
assert popup.locator('.has-error').count() == 1, width
health.click()
assert popup.count() == 1 and health.get_attribute('aria-expanded') == 'true', width
assert backend.get_attribute('aria-expanded') == 'false', width
page.keyboard.press('Escape')
assert popup.count() == 0 and health.get_attribute('aria-expanded') == 'false', width
assert health.evaluate('(node) => document.activeElement === node'), width
backend.click()
page.locator('#vhost-search').click(force=True)
assert popup.count() == 0, width
# A static-only vhost presents a translated empty state, not stale pool values.
page.locator('[data-vhost-card="static-only"] .vhost-detail.vhost-backend-trigger').click()
assert 'Brak skonfigurowanych backendów.' in popup.inner_text(), width
popup.locator('.vhost-backend-popup-close').click()
assert popup.count() == 0, width
backend.click()
assert popup.count() == 1, width
# The existing backend popover is also used by the H2 policy badge.
protocol = vhost.locator('[data-upstream-protocol-backend="origin-http"]')
assert protocol.count() == 1 and protocol.evaluate('n => n.tagName') == 'BUTTON', width
assert protocol.get_attribute('title') is None, width
protocol.click()
assert backend.get_attribute('aria-expanded') == 'false', width
assert protocol.get_attribute('aria-expanded') == 'true', width
assert 'Brak odpowiedzi backendu' in popup.inner_text(), width
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(8))
assert 'origin-http · H2 compatible → H2' in protocol.inner_text(), width
assert 'Ostatnia wersja HTTP' in popup.inner_text(), width
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '8'], width
page.evaluate('''() => {
window.__popupReference = document.getElementById('vhost-backend-popup');
window.__headingReference = window.__popupReference.querySelector('h4');
window.__popupRemoved = 0;
window.__popupObserver = new MutationObserver(records => {
for (const record of records) for (const item of record.removedNodes) {
if (item === window.__popupReference) window.__popupRemoved++;
}
});
window.__popupObserver.observe(document.body, {childList: true});
}''')
for count in range(9, 19):
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(count))
unchanged = page.evaluate('''() => ({
samePopup: document.getElementById('vhost-backend-popup') === window.__popupReference,
sameHeading: window.__popupReference.querySelector('h4') === window.__headingReference,
removed: window.__popupRemoved,
expanded: document.querySelector('[data-upstream-protocol-backend="origin-http"]').getAttribute('aria-expanded'),
})''')
assert unchanged == {'samePopup': True, 'sameHeading': True, 'removed': 0, 'expanded': 'true'}, (width, unchanged)
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '18'], width
page.evaluate('window.__popupObserver.disconnect()')
assert protocol.get_attribute('title') is None, width
# A stale observation should change the displayed information in
# place, not dismiss/recreate the click popover.
stale = protocol_snapshot(18)
stale['backend_pools'][0]['members'][0]['upstream_protocol_last_seen_ms'] -= 600000
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', stale)
assert 'H2 compatible → ?' in protocol.inner_text(), width
assert popup.locator('.vhost-backend-popup-list').is_hidden(), width
assert page.evaluate('document.getElementById("vhost-backend-popup") === window.__popupReference'), width
page.keyboard.press('Escape')
assert popup.count() == 0 and protocol.get_attribute('aria-expanded') == 'false', width
assert protocol.evaluate('node => document.activeElement === node'), width
# Cache Domains and Stored objects are the only directly adjacent
# table cards and need a visible, small gap at any viewport width.
cache_gap = page.evaluate('''() => {
const panel = document.getElementById('panel-cache');
panel.hidden = false;
const [domains, stored] = panel.querySelectorAll(':scope > article.card');
const gap = stored.getBoundingClientRect().top - domains.getBoundingClientRect().bottom;
panel.hidden = true;
return gap;
}''')
assert 9.5 <= cache_gap <= 10.5, (width, cache_gap)
page.evaluate('window.rustProxyVhosts.reset()')
assert popup.count() == 0, width
assert not errors, (width, errors)
print(f'PASS Vhosts shared backend/H2 popover and Cache gap at {width}px')
page.close()
browser.close()
print('PASS Vhost quick-view with simulated runtime status (not live Rust)')
scripts/tests/web_observability_0330_browser_test.py
-140
View File
@@ -1,140 +0,0 @@
from pathlib import Path
from panel_html import render_admin_index
import re, json
from playwright.sync_api import sync_playwright
root=Path(__file__).resolve().parents[2]
router=(root/'src/admin/router.rs').read_text()
def concatenate(section):
pattern=r'fn '+section+r'\([^)]*\)[^{]*\{'
m=re.search(pattern,router)
assert m, section
text=router[m.end():]
# The array contents are bounded by ].join("\\n") before the next function.
text=text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
result=[]
for line in text.splitlines():
line=line.strip()
if 'include_str!("' in line:
path=re.search(r'include_str!\("([^"]+)"\)',line).group(1)
result.append((root/'src/admin'/path).read_text())
elif line=='admin_vhosts_javascript(),':
result.append('(() => {\n'+''.join((root/'src/admin/web/vhosts'/f'{part}.js').read_text()+'\n' for part in ['core','protocol','health','rates','editor','render'])+'})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
bundle=concatenate('admin_app_javascript')
assert 'window.rustProxyOverviewSamples =' in bundle
assert bundle.index('window.rustProxyOverviewSamples =') < bundle.index('window.rustProxyOverviewSamples.onChange(')
css=(root/'src/admin/web/style.css').read_text()
html=render_admin_index(root)
with sync_playwright() as p:
browser=p.chromium.launch(headless=True, executable_path='/usr/bin/chromium',args=['--no-sandbox','--allow-file-access-from-files'])
for width in [320,375,800,1440]:
page=browser.new_page(viewport={'width':width,'height':900})
errors=[]
page.on('pageerror',lambda e: errors.append(str(e)))
# Execute in a fully populated DOM without disallowed filesystem/network navigation.
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js','').replace('/assets/style___RUSTPROXY_ASSET_ID__.css',''))
page.add_style_tag(content=css)
page.add_script_tag(content=bundle)
page.wait_for_timeout(550)
state=page.evaluate('''() => ({ controller:typeof window.rustProxyOverviewSamples,
handle:typeof window.rustProxyOverviewSamples?.onChange,
web:typeof window.rustProxyWebOverview, services:typeof window.rustProxyWebServer,
registry: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
htmlWidth:document.documentElement.scrollWidth,windowWidth:window.innerWidth })''')
print(width,state,'errors',errors)
assert state['controller']=='object' and state['handle']=='function' and state['web']=='object'
assert state['registry']==12
assert not [e for e in errors if 'onChange' in e or 'ReferenceError' in e or 'SyntaxError' in e or 'TypeError' in e],errors
assert state['htmlWidth'] <= state['windowWidth'] + 2
# The selector belongs to the same header row as the core connection;
# hiding it on other panels must not leave an empty dashboard row.
layout = page.evaluate('''() => {
document.getElementById('workspace').hidden = false;
setPanel('overview', false);
const status = document.getElementById('connection-label');
const sample = document.getElementById('overview-samples-toolbar');
const row = document.querySelector('.page-intro-meta-row');
const rect = node => { const r = node.getBoundingClientRect();
return {x:r.x,y:r.y,width:r.width,height:r.height,right:r.right}; };
const overview = { status:rect(status),sample:rect(sample),row:rect(row),
visible: !sample.hidden,inside: row.contains(sample) && row.contains(status),
overflow: document.documentElement.scrollWidth > innerWidth + 2 };
setPanel('web-server', false);
const notOverview = sample.hidden && !status.hidden;
setPanel('overview', false);
return {overview,notOverview,visibleAgain:!sample.hidden};
}''')
assert layout['overview']['inside'] and layout['overview']['visible']
assert layout['notOverview'] and layout['visibleAgain']
assert not layout['overview']['overflow'], (width, layout)
if width >= 800:
assert abs(layout['overview']['sample']['y'] - layout['overview']['status']['y']) <= 20, (width, layout)
assert layout['overview']['sample']['right'] <= layout['overview']['row']['right'] + 2, (width, layout)
# A genuinely sampled WWW metrics payload (not a screenshot-only fixture).
report = page.evaluate('''() => {
Object.defineProperty(window.performance, "now", {configurable:true, value:()=>window.__clock});
window.__clock = 1000;
currentPanel = "overview";
document.getElementById('workspace').hidden = false;
document.getElementById('login').hidden = true;
document.getElementById('panel-overview').hidden = false;
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
function snap(k) {
return {revision:7, web_server_metrics:{
"public-site": {web: {requests:100+k*30, status_2xx:86+k*24, status_3xx:3+k,
status_4xx:8+k*2, status_5xx:3+k*3, response_headers_us:4000000+k*1300000,
response_bytes:100000+k*20000, static_requests:70+k*20, php_requests:25+k*7,
other_requests:5+k*3, methods:[90+k*25,8+k*4,0,0,2+k],
selected_status:[80+k*18,2,2,1,5+k,0,0,0], active_headers:0},
php: {metrics:{requests:25+k*7, response_headers_us:2000000+k*700000,
response_bytes:30000+k*5000,failures:[1+k,1+k,1+k,0,0,0,0,0,0]},
capacity_limit:32, capacity_used:8, capacity_available:24}}}
};
}
const before = snap(0), after = snap(1);
// Saved custom layout from before 0.33.0 must not enable new cards.
const savedLayout = applyDashboardLayout(['requests', 'errors'],
normalizeHiddenDashboardWidgets([], ['requests', 'errors']));
const newCardsHidden = [...document.querySelectorAll('[data-dashboard-widget^="www_"]')]
.every(card=>card.hidden);
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
window.rustProxyWebOverview.render(before, true);
window.__clock = 4000;
window.rustProxyWebOverview.render(after, true);
let ids = ['www_requests-all-line', 'www_requests-php-line',
'www_latency-web-line', 'www_traffic-php-line', 'www_status_trend-5xx-line'];
const linePaths = ids.map(id=>document.getElementById(id)?.getAttribute('d'));
const statusText = document.getElementById('www_status-value').textContent;
const capacityText = document.getElementById('www_capacity-value').textContent;
const rateText = document.getElementById('www_php_requests-value').textContent;
const bars = document.querySelectorAll('#www_top_services-bars .www-top-row').length;
window.rustProxyOverviewSamples.setLimit(120);
const historyLimit = window.rustProxyOverviewSamples.getLimit();
currentPanel = "web-server";
envelope = {config:{web_services:[{id:'public-site', name:'Public site',document_root:'/srv/www/public'}],vhosts:[]}};
window.rustProxyWebServer.render();
const details = document.querySelector('.www-service-panel');
if (!details || details.tagName !== 'SECTION') throw Error('Missing always-visible WWW metrics');
window.__clock = 7000;
window.rustProxyWebServer.renderLive(before, true);
window.__clock = 10000;
window.rustProxyWebServer.renderLive(after, true);
const value = details.querySelector('[data-web-live-field="requests"]')?.textContent;
const phpCapacity = details.querySelector('[data-web-live-field="php"]')?.textContent;
return {linePaths,statusText,capacityText,rateText,bars,historyLimit,value,phpCapacity,newCardsHidden,
width:document.documentElement.scrollWidth};
}''')
print('sampled WWW',width,report)
assert all(report['linePaths']), report
assert report['statusText'] != '-' and report['capacityText'] == '8 / 32', report
assert report['rateText'] not in ('-', '0.0/s'), report
assert report['bars'] == 1 and report['historyLimit'] == 120 and report['newCardsHidden'], report
assert report['value'] and report['phpCapacity'] == '8 / 32', report
assert report['width'] <= width + 2, report
assert not [e for e in errors if any(x in e for x in ('ReferenceError', 'TypeError', 'SyntaxError'))], errors
page.close()
browser.close()
print('PASS: actual composed admin bundle starts without a JS TypeError')
scripts/tests/web_observability_0330_source_test.py
-84
View File
@@ -1,84 +0,0 @@
#!/usr/bin/env python3
"""Static contracts for 0.33.0 WWW observability (not Rust compilation)."""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
import tomllib
ROOT = Path(__file__).resolve().parents[2]
read = lambda file: (ROOT / file).read_text()
manifest = tomllib.loads(read('Cargo.toml'))
lock = tomllib.loads(read('Cargo.lock'))
version = manifest['package']['version']
assert tuple(map(int, version.split('.'))) >= (0, 33, 0)
assert [p['version'] for p in lock['package'] if p['name'] == 'rust-proxy'] == [version]
registry = json.loads(read('src/admin/web/dashboard-widgets.json'))
widgets = [w for w in registry if w['id'].startswith('www_')]
assert len(widgets) == 12
assert all(w['default_hidden'] is True for w in widgets)
assert len({w['id'] for w in registry}) == len(registry)
html = render_admin_index(ROOT)
english = json.loads(read('langs/en_US.json'))['translations']
polish = json.loads(read('langs/pl_pl.json'))['translations']
for widget in widgets:
assert f'data-dashboard-widget="{widget["id"]}" hidden' in html, widget['id']
assert widget['label_key'] in english, widget['label_key']
assert widget['label_key'] in polish, widget['label_key']
if widget['description_key']:
assert widget['description_key'] in english
assert widget['description_key'] in polish
# Safely assemble the admin app before core.js: the earlier release crashed on
# registering onChange callbacks before rustProxyOverviewSamples was defined.
router = read('src/admin/router.rs')
assert router.index('include_str!("web/app/overview-samples.js")') < router.index('include_str!("web/app/core.js")')
assert '"web/app/web-overview.js"' in router
assert '"web_server_metrics"' in read('src/admin/status_live.rs')
assert 'web_server_metrics_payload(&snapshot)' in read('src/admin/status.rs')
# record_web is only consumed by web_server::serve; expose no wider than the
# ResponseKind enum argument. This is a static visibility contract, not rustc.
metrics = read('src/web_server/metrics.rs')
assert 'pub(super) enum ResponseKind' in metrics
assert 'pub(super) fn record_web(' in metrics
assert 'pub(crate) fn record_web(' not in metrics
assert 'service.web_metrics.record_web(' in read('src/web_server/serve.rs')
families = read('src/prometheus/web_php.rs')
required = {
'rust_proxy_web_server_requests_total', 'rust_proxy_web_server_responses_total',
'rust_proxy_web_server_request_kind_total', 'rust_proxy_web_server_methods_total',
'rust_proxy_web_server_status_code_total', 'rust_proxy_web_server_response_header_duration_microseconds_total',
'rust_proxy_web_server_response_body_bytes_total', 'rust_proxy_web_server_requests_active',
'rust_proxy_web_server_compression_total', 'rust_proxy_php_requests_total',
'rust_proxy_php_responses_total', 'rust_proxy_php_response_header_duration_microseconds_total',
'rust_proxy_php_response_body_bytes_total', 'rust_proxy_php_concurrency_available',
'rust_proxy_php_concurrency_limit', 'rust_proxy_php_concurrency_in_use', 'rust_proxy_php_failures_total'
}
assert all(f'"{name}"' in families for name in required), required
assert 'PHP_FAILURE_NAMES' in families
assert re.search(r'\["GET", "POST", "PUT", "DELETE", "other"\]', families)
assert 'let wl = labels(&[("web_service", service.id.as_str())]);' in families
j = json.loads(read('grafana/rustproxy-overview.json'))
assert j['title'].startswith('RustProxy 0.33.')
ids = [p['id'] for p in j['panels']]
assert len(ids) == len(set(ids))
assert any(v.get('name') == 'web_service' and v.get('multi') and v.get('includeAll') for v in j['templating']['list'])
for title in ('Web Server 0.33.0', 'PHP-FPM / FastCGI', 'Per Web Service', 'WWW / PHP correlation'):
assert any(p['title'] == title and p['type'] == 'row' for p in j['panels'])
extra = [p for p in j['panels'] if p['id'] >= 89]
for i, p in enumerate(extra):
a = p['gridPos']
assert 0 <= a['x'] and a['w'] > 0 and a['x'] + a['w'] <= 24
for q in extra[i + 1:]:
b = q['gridPos']
overlap = a['x'] < b['x'] + b['w'] and b['x'] < a['x'] + a['w'] and a['y'] < b['y'] + b['h'] and b['y'] < a['y'] + a['h']
assert not overlap, (p['id'], q['id'])
production = [p for p in (ROOT / 'src').rglob('*.rs') if p.name != 'tests.rs' and 'tests' not in p.parts]
large = [str(p.relative_to(ROOT)) for p in production if len(p.read_text().splitlines()) > 500]
assert not large, large
print('PASS RustProxy 0.33.0 source version, opt-in widgets, localizations, status API,')
print(' exporter families, Grafana geometry/filtering and Rust module size (STATIC ONLY)')
scripts/tests/web_server_0330_ui_test.py
-153
View File
@@ -1,153 +0,0 @@
#!/usr/bin/env python3
"""Simulated Web Server telemetry against the production admin JS/CSS bundle.
Browser-only test: does not assert live RustProxy, PHP-FPM, or network behavior.
Run: python3 scripts/tests/web_server_0330_ui_test.py
"""
from __future__ import annotations
import json
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
router = (ROOT / 'src/admin/router.rs').read_text()
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', router)
assert match, 'admin_app_javascript definition missing'
portion = router[match.end():]
portion = portion[:re.search(r'\]\s*\.join\("\\n"\)', portion).start()]
fragments = []
for line in portion.splitlines():
line = line.strip()
if 'include_str!("' in line:
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
fragments.append((ROOT / 'src/admin' / path).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
BUNDLE = '\n'.join(fragments)
assert BUNDLE.index('window.rustProxyOverviewSamples =') < BUNDLE.index('window.rustProxyOverviewSamples.onChange(')
WEB = {
'requests': 100, 'status_2xx': 90, 'status_3xx': 2, 'status_4xx': 5, 'status_5xx': 3,
'response_headers_us': 2_500_000, 'response_bytes': 10_000, 'active_headers': 2,
'static_requests': 70, 'php_requests': 25, 'other_requests': 5,
'methods': [80, 10, 5, 2, 3], 'selected_status': [75, 5, 2, 0, 5, 1, 1, 1],
'failures': [0] * 9, 'compression': [5, 3, 2, 1],
}
PHP = {
'requests': 25, 'status_2xx': 22, 'status_3xx': 0, 'status_4xx': 1, 'status_5xx': 2,
'response_headers_us': 1_000_000, 'response_bytes': 2_000,
'failures': [1, 2, 3, 4, 5, 6, 7, 8, 9],
}
def observation(factor: int):
web = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
for key, value in WEB.items()}
php = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
for key, value in PHP.items()}
web['active_headers'] = 2
return {'revision': 10, 'web_server_metrics': {
'site-A': {'web': web, 'php': {'metrics': php, 'capacity_limit': 32, 'capacity_used': 18}}
}}
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 800, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda err: errors.append(str(err)))
page.set_content(render_admin_index(ROOT))
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
page.add_script_tag(content=BUNDLE)
page.evaluate("""() => {
document.querySelector('#workspace').hidden = false;
document.querySelector('#panel-overview').hidden = false;
document.querySelectorAll('[data-dashboard-widget^="www_"]').forEach(n => n.hidden = false);
currentPanel = 'overview';
window.__clock = 1000;
Object.defineProperty(performance, 'now', { configurable: true, value: () => window.__clock });
}""")
first, second = observation(1), observation(2)
state = page.evaluate("""([one, two]) => {
const api = window.rustProxyWebOverview;
api.render(one);
window.__clock = 4000;
api.render(two);
return {
rps: document.getElementById('www_requests-all-value').textContent,
php: document.getElementById('www_requests-php-value').textContent,
latency: document.getElementById('www_latency-web-value').textContent,
capacity: document.getElementById('www_capacity-value').textContent,
types: document.getElementById('www_kind-value').textContent,
status: document.getElementById('www_status-value').textContent,
bytes: document.getElementById('www_traffic-value').textContent,
top: document.getElementById('www_top_services-value').textContent,
layoutWidth: document.documentElement.scrollWidth,
screenWidth: window.innerWidth,
children: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
sampleCount: document.getElementById('www_requests-all-line').getAttribute('d'),
limit: window.rustProxyOverviewSamples.getLimit(),
};
}""", [first, second])
assert state['children'] == 12, state
assert state['rps'] != '-' and state['php'] != '-', state
assert 'ms' in state['latency'] and '18' in state['capacity'], state
assert state['types'].endswith('%') and state['status'].endswith('%'), state
assert '/s' in state['top'], state
assert state['layoutWidth'] <= state['screenWidth'] + 2, (width, state)
assert not errors, (width, errors)
# Exercise the always-visible per-service metrics with the same status,
# without any real HTTP/PHP request or extra polling.
details = page.evaluate("""(status) => {
envelope = { revision: 10, config: {
web_services: [{ id: 'site-A', name: 'Example', document_root: '/var/www/site-A',
index: ['index.html'], php: { enabled: true } }],
vhosts: [], php_fpm_profiles: []
}};
currentPanel = 'web-server';
window.rustProxyWebServer.render();
const detail = document.querySelector('#web-service-list .www-service-panel');
if (!detail) return { missing: true };
window.rustProxyWebServer.renderLive(status, false);
return { text: detail.textContent, rows: detail.querySelectorAll('[data-web-live-field]').length,
alwaysVisible: detail.tagName === 'SECTION' && !detail.querySelector('summary') };
}""", second)
assert not details.get('missing') and details['rows'] == 7 and details['alwaysVisible'], (width, details)
assert '18 / 32' in details['text'], (width, details)
# The first reading after an overdue browser interval must establish a
# fresh baseline without adding a fake 0/s dip to the service chart.
continuity = page.evaluate("""([first, second, paused, resumed]) => {
const api = window.rustProxyWebServer;
const chart = document.querySelector('#web-service-list [data-web-live-id="site-A"] polyline');
const rate = document.querySelector('#web-service-list [data-web-live-field="rate"]');
window.__clock = 5000; api.renderLive(first);
window.__clock = 8000; api.renderLive(second);
const before = chart.getAttribute('points');
window.__clock = 100000; api.renderLive(paused);
const during = chart.getAttribute('points'), unavailable = rate.textContent;
window.__clock = 103000; api.renderLive(resumed);
return { before, during, unavailable, after: chart.getAttribute('points'), resumedRate: rate.textContent };
}""", [observation(2), observation(3), observation(10), observation(12)])
assert continuity['before'] == continuity['during'], (width, continuity)
assert continuity['unavailable'] == '-', (width, continuity)
assert continuity['resumedRate'] != '-', (width, continuity)
assert continuity['after'] != continuity['before'], (width, continuity)
assert not errors, (width, errors)
old = page.evaluate("""() => {
currentPanel = 'overview';
window.rustProxyWebOverview.render({ revision: 11, web_server_metrics: {} });
return {
rate: document.getElementById('www_requests-value').textContent,
status: document.getElementById('www_status-value').textContent,
top: document.getElementById('www_top_services-bars').childElementCount,
};
}""")
assert old == {'rate': '-', 'status': '-', 'top': 0}, (width, old)
assert not errors, (width, errors)
print(f'PASS simulated Web Server cards and always-visible service metrics at {width}px: {state["rps"]}, {state["php"]}, layout {state["layoutWidth"]}px')
page.close()
browser.close()
print('PASS production JS bundle + all twelve visible WWW widgets (simulated data only)')
scripts/tests/web_server_0334_browser_test.py
-155
View File
@@ -1,155 +0,0 @@
#!/usr/bin/env python3
"""WWW/Vhost visual parity in production UI files, using simulated status.
Chromium exercises the real assembled admin JS/CSS and responsive layouts.
It does not establish Rust, SSE or PHP-FPM runtime acceptance.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert found
code = ROUTER[found.end():]
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
fragments = []
for line in code.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
return '\n'.join(fragments)
def status(step):
return {'web_server_metrics': {
'site-A': {'web': {
'requests': step * 120, 'response_headers_us': step * 2_400_000,
'status_5xx': step * 4, 'response_bytes': step * 20_480,
'static_requests': step * 80, 'php_requests': step * 30,
'other_requests': step * 10,
}, 'php': {'capacity_used': 3, 'capacity_limit': 16}},
'site-B': {'web': {
'requests': step * 24, 'response_headers_us': step * 100_000,
'status_5xx': 0, 'response_bytes': step * 8_192,
'static_requests': step * 24, 'php_requests': 0,
'other_requests': 0,
}, 'php': None},
}}
with sync_playwright() as playwright:
browser = playwright.chromium.launch(
headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 640, 800, 980, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda e: errors.append(str(e)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
initial = page.evaluate('''(status) => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-web-server').hidden = false;
currentPanel = 'web-server';
envelope = {revision: 10, config: {web_services: [
{id: 'site-A', name: 'Example service', document_root: '/var/www/a', index: ['index.html'], php: {enabled: true}},
{id: 'site-B', name: 'Static service', document_root: '/var/www/b', index: ['index.html']}
], vhosts: [], php_fpm_profiles: []}};
window.__clock = 1000;
Object.defineProperty(performance, 'now', {configurable: true, value: () => window.__clock});
latestStatus = status;
window.rustProxyWebServer.render();
const list = document.querySelector('#web-service-list');
const cards = [...list.querySelectorAll(':scope > .vhost-card')];
const metricPanels = cards.map(card => card.querySelector('.www-service-panel'));
const grid = metricPanels[0].querySelector('.vhost-live-grid');
const chart = metricPanels[0].querySelector('.vhost-chart');
const svg = chart.querySelector('svg');
const svgBox = svg.getBoundingClientRect(), svgMatrix = svg.getScreenCTM();
const rect = node => node.getBoundingClientRect();
const expectedMetricLabels = ['webLiveRequests','webLiveRate','webLiveErrors',
'webLiveLatency','webLivePhpCapacity','webLiveBytes'];
return {
cards: cards.length, sectionTags: metricPanels.map(panel => panel.tagName),
collapsibles: list.querySelectorAll('details, summary').length,
liveFields: metricPanels.map(panel => panel.querySelectorAll('[data-web-live-field]').length),
labels: [...grid.children].map(tile => tile.querySelector('span')?.dataset.i18n),
tileClasses: [...grid.children].map(tile => tile.className),
values: [...grid.querySelectorAll('strong')].map(el => el.textContent),
phpMissing: metricPanels[1].querySelector('[data-web-live-field="php"]').textContent,
columns: getComputedStyle(grid).gridTemplateColumns.split(' ').length,
chartWidth: rect(chart).width, gridWidth: rect(grid).width,
cardWidth: rect(cards[0]).width, listWidth: rect(list).width,
svgWidth: svgBox.width, svgContentWidth: svg.viewBox.baseVal.width * Math.hypot(svgMatrix.a, svgMatrix.b),
preserveAspect: svg.getAttribute('preserveAspectRatio'),
stroke: getComputedStyle(svg.querySelector('polyline')).vectorEffect,
chartLabel: !!chart.querySelector('.vhost-chart-head .small-label'),
rendered: getComputedStyle(metricPanels[0]).display !== 'none' && rect(chart).height > 60,
fitsViewport: document.documentElement.scrollWidth <= innerWidth + 2,
expectedMetricLabels,
};
}''', status(1))
expected_cols = 2 if width <= 640 else 3 if width <= 980 else 6
assert initial['cards'] == 2, (width, initial)
assert initial['sectionTags'] == ['SECTION', 'SECTION'], (width, initial)
assert initial['collapsibles'] == 0, (width, initial)
assert initial['liveFields'] == [7, 7], (width, initial)
assert initial['labels'] == initial['expectedMetricLabels'], (width, initial)
assert initial['tileClasses'] == ['vhost-live-metric'] * 6, (width, initial)
assert initial['values'][0] != '-' and initial['values'][4] == '3 / 16', (width, initial)
assert initial['phpMissing'] != '-', (width, initial)
assert initial['columns'] == expected_cols, (width, initial)
assert initial['chartWidth'] > 0.95 * initial['gridWidth'], (width, initial)
assert initial['listWidth'] - initial['cardWidth'] < 3, (width, initial)
assert abs(initial['svgWidth'] - initial['svgContentWidth']) <= 2, (width, initial)
assert initial['preserveAspect'] == 'none' and initial['stroke'] == 'non-scaling-stroke', (width, initial)
assert initial['chartLabel'] and initial['rendered'] and initial['fitsViewport'], (width, initial)
sampled = page.evaluate('''([first, second, last]) => {
const api = window.rustProxyWebServer;
const panel = document.querySelector('[data-web-live-id="site-A"]');
const rate = panel.querySelector('[data-web-live-field="rate"]');
window.__clock = 4000; api.renderLive(first, true);
window.__clock = 7000; api.renderLive(second, true);
const before = {rate: rate.textContent, points: panel.querySelector('polyline').getAttribute('points')};
// Service-list redraw must show the current status without opening anything.
latestStatus = last;
api.render();
const updated = document.querySelector('[data-web-live-id="site-A"]');
const total = updated.querySelector('[data-web-live-field="requests"]').textContent;
const split = updated.querySelector('[data-web-live-field="split"]').textContent;
return {before, total, split,
pointsAfterRedraw: updated.querySelector('polyline').getAttribute('points'),
metricsRemainVisible: !!updated.querySelector('.vhost-live-metric'),
overflow: document.documentElement.scrollWidth - innerWidth};
}''', [status(2), status(3), status(4)])
assert sampled['before']['rate'] not in ('-', '0/s', '0.0/s'), (width, sampled)
assert sampled['before']['points'].count(' ') >= 1, (width, sampled)
assert sampled['total'] != initial['values'][0], (width, sampled)
assert sampled['split'] != '-' and sampled['metricsRemainVisible'], (width, sampled)
assert sampled['pointsAfterRedraw'] == sampled['before']['points'], (width, sampled)
assert sampled['overflow'] <= 2, (width, sampled)
assert not errors, (width, errors)
print(f'PASS 0.33.4 WWW live metrics visual parity at {width}px; {expected_cols} columns')
page.close()
browser.close()
print('PASS always-visible full-width WWW/Vhost parity with simulated updates (not Rust runtime)')
src/admin/documentation/documents.rs
-11
View File
@@ -236,7 +236,6 @@ pub(super) static READER_DOCUMENTS: &[(&str, &str, &str)] = &[
"/docs/SECURITY_AUDIT.md"
)),
),
reader_doc!("docs/SECURITY_AUDIT_0.31.0.md", "audits"),
(
"docs/SETTINGS.md",
"guides",
@@ -247,16 +246,6 @@ pub(super) static READER_DOCUMENTS: &[(&str, &str, &str)] = &[
"guides",
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/STREAMS.md")),
),
(
"docs/TEST_REPORT.md",
"guides",
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/TEST_REPORT.md")),
),
(
"docs/TODO.md",
"guides",
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/TODO.md")),
),
(
"docs/TRANSFORM.md",
"guides",
src/admin/tests.rs
+1 -1
View File
@@ -2370,7 +2370,7 @@ mod tests {
.any(|document| document["path"] == "docs/PHP_FPM.md"));
assert!(documents
.iter()
.any(|document| document["path"] == "docs/SECURITY_AUDIT_0.31.0.md"));
.any(|document| document["path"] == "docs/SECURITY_AUDIT.md"));
assert!(documents
.iter()
.any(|document| document["path"] == "docs/TEST_REPORT.md"));
src/logging/error_runtime.rs
+48 -40
View File
@@ -48,53 +48,61 @@ fn redact_regex(pattern: &'static str) -> &'static regex::Regex {
}
fn is_secret_field(name: &str) -> bool {
matches!(
name.to_ascii_lowercase().as_str(),
"authorization"
| "proxy_authorization"
| "api_token"
| "access_token"
| "application_secret"
| "consumer_key"
| "key_secret"
| "backup_password"
| "password"
| "credential"
| "credentials"
| "secret"
| "token"
| "private_key"
| "master_key"
)
const FIELDS: &[&str] = &[
"authorization",
"proxy_authorization",
"api_token",
"access_token",
"application_secret",
"consumer_key",
"key_secret",
"backup_password",
"password",
"credential",
"credentials",
"secret",
"token",
"private_key",
"master_key",
];
// ASCII-insensitive matching has the same semantics as to_ascii_lowercase,
// without allocating a new key for every field in every JSON access log.
FIELDS.iter().any(|field| name.eq_ignore_ascii_case(field))
}
fn redact_text_cow(input: &str) -> std::borrow::Cow<'_, str> {
let mut value = std::borrow::Cow::Borrowed(input);
// Keep the original order: a replacement can expose a pattern handled by
// the next rule. Ordinary, non-secret values stay borrowed (no six copies).
for (pattern, replacement) in [
("pem", "[REDACTED PRIVATE KEY]"),
("auth", "$1[REDACTED]"),
("query", "$1[REDACTED]"),
("escaped_json", "$1[REDACTED]$2"),
("json", "$1[REDACTED]$2"),
("kv", "$1[REDACTED]"),
] {
let regex = redact_regex(pattern);
if regex.is_match(value.as_ref()) {
value = std::borrow::Cow::Owned(
regex.replace_all(value.as_ref(), replacement).into_owned(),
);
}
}
value
}
pub fn redact_text(input: &str) -> String {
let mut value = redact_regex("pem")
.replace_all(input, "[REDACTED PRIVATE KEY]")
.into_owned();
value = redact_regex("auth")
.replace_all(&value, "$1[REDACTED]")
.into_owned();
value = redact_regex("query")
.replace_all(&value, "$1[REDACTED]")
.into_owned();
// Debug-formatted fields can contain a JSON string with escaped quote
// delimiters. Redact that form before plain JSON, retaining quote escapes
// and consuming escaped quotes inside a secret instead of leaking its tail.
value = redact_regex("escaped_json")
.replace_all(&value, "$1[REDACTED]$2")
.into_owned();
value = redact_regex("json")
.replace_all(&value, "$1[REDACTED]$2")
.into_owned();
redact_regex("kv")
.replace_all(&value, "$1[REDACTED]")
.into_owned()
redact_text_cow(input).into_owned()
}
fn redact_value(value: &mut Value) {
match value {
Value::String(text) => *text = redact_text(text),
Value::String(text) => {
if let std::borrow::Cow::Owned(redacted) = redact_text_cow(text) {
*text = redacted;
}
}
Value::Array(items) => {
for item in items {
redact_value(item);
src/logging/format.rs
+16 -8
View File
@@ -138,11 +138,21 @@ impl Logs {
pub fn push(&self, entry: LogEntry) {
let has_viewers = self.live.receiver_count() != 0;
let has_sink = self.settings.load().enabled();
// The usual history-only configuration needs no deep copy at all.
// When external consumers are present, clone exactly for history and
// keep the original entry available for streaming/sink delivery.
// Reserve the bounded sink slot BEFORE cloning a log entry. Under
// saturation, the failed enqueue must not copy 30+ owned fields.
let sink_permit = if has_sink {
match self.sink.try_reserve() {
Ok(permit) => Some(permit),
Err(_) => {
self.sink_dropped.fetch_add(1, Ordering::Relaxed);
None
}
}
} else {
None
};
let mut external = Some(entry);
let history_entry = if has_viewers || has_sink {
let history_entry = if has_viewers || sink_permit.is_some() {
external.as_ref().expect("entry").clone()
} else {
external.take().expect("entry")
@@ -165,10 +175,8 @@ impl Logs {
if has_viewers {
let _ = self.live.send(entry.clone());
}
// Snapshot the sink choice once. This preserves sink drop
// accounting and avoids a second ArcSwap load per request.
if has_sink && self.sink.try_send(entry).is_err() {
self.sink_dropped.fetch_add(1, Ordering::Relaxed);
if let Some(permit) = sink_permit {
permit.send(entry);
}
}
}
src/logging/runtime.rs
+67 -1
View File
@@ -41,6 +41,10 @@ fn format_entry(entry: &LogEntry, format: &JsonFormat) -> Value {
Value::Object(object)
}
// Bound both write size and event count; never wait for the batch to fill.
const FILE_BATCH_EVENTS: u64 = 64;
const FILE_BATCH_BYTES: usize = 128 * 1024;
type LokiClient = Client<hyper_rustls::HttpsConnector<HttpConnector>, Body>;
fn loki_client() -> Result<LokiClient> {
let mut roots = RootCertStore::empty();
@@ -114,12 +118,74 @@ async fn sink_loop(
let mut vhost_files: BTreeMap<PathBuf, File> = BTreeMap::new();
let mut vhost_directory: Option<PathBuf> = None;
let mut logstash: Option<LogstashState> = None;
let mut deferred_entry: Option<LogEntry> = None;
let mut file_batch = Vec::with_capacity(8 * 1024);
loop {
let entry = tokio::select! {
let entry = if let Some(entry) = deferred_entry.take() {
entry
} else {
tokio::select! {
_ = shutdown.cancelled() => break,
entry = rx.recv() => match entry { Some(entry) => entry, None => break },
}
};
let config = settings.load_full();
// The common local-file-only sink has no per-event network or routing
// dependency. Coalesce consecutive JSONL records into one bounded write.
// Other sink combinations retain their original per-event semantics.
if config.file.enabled && !config.file.per_vhost
&& !config.logstash.enabled && !config.loki.enabled
{
vhost_files.clear();
vhost_directory = None;
let path = PathBuf::from(&config.file.path);
if file.as_ref().is_none_or(|state| state.path != path) {
file = match open_access_file(path.clone()).await {
Ok(opened) => Some(FileState { path: path.clone(), file: opened }),
Err(error) => {
tracing::warn!(%error, path=%path.display(), "access log file unavailable");
None
}
};
}
file_batch.clear();
let mut count = 0u64;
let mut next = Some(entry);
while let Some(item) = next.take() {
let mut payload = format_entry(&item, &config.format);
redact_value(&mut payload);
let mut line = serde_json::to_vec(&payload).unwrap_or_default();
line.push(b'\n');
file_batch.extend_from_slice(&line);
count += 1;
if count >= FILE_BATCH_EVENTS || file_batch.len() >= FILE_BATCH_BYTES {
break;
}
match rx.try_recv() {
Ok(item) if Arc::ptr_eq(&settings.load_full(), &config) => {
next = Some(item);
}
Ok(item) => {
// A pending entry must be handled with the new snapshot;
// a batch never silently crosses a configuration change.
deferred_entry = Some(item);
break;
}
Err(_) => break,
}
}
let written = match file.as_mut() {
Some(state) => state.file.write_all(&file_batch).await.is_ok(),
None => false,
};
if written {
sent.fetch_add(count, Ordering::Relaxed);
} else {
errors.fetch_add(count, Ordering::Relaxed);
file = None;
}
continue;
}
let mut payload = format_entry(&entry, &config.format);
redact_value(&mut payload);
// File and Logstash consume the same JSONL record. Serialize once and
src/logging/tests.rs
+63
View File
@@ -458,4 +458,67 @@ mod tests {
assert_eq!(logs.stats().queue_dropped, 1);
}
#[test]
fn redaction_leaves_ordinary_strings_borrowed_and_preserves_secret_matching() {
use std::borrow::Cow;
assert!(matches!(redact_text_cow("GET /index.html"), Cow::Borrowed(_)));
assert!(matches!(redact_text_cow("hello € world"), Cow::Borrowed(_)));
assert!(matches!(redact_text_cow("?api_token=topsecret"), Cow::Owned(_)));
let mut value = json!({
"ToKeN": "secret-value",
"ordinary": "hello € world",
"nested": {"AUTHORIZATION": "Bearer private"},
"path": "/index?password=topsecret"
});
redact_value(&mut value);
assert_eq!(value["ToKeN"], "[REDACTED]");
assert_eq!(value["ordinary"], "hello € world");
assert_eq!(value["nested"]["AUTHORIZATION"], "[REDACTED]");
assert_eq!(value["path"], "/index?password=[REDACTED]");
}
#[tokio::test]
async fn local_file_log_batch_keeps_every_jsonl_record_and_counters() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("access.jsonl");
let mut settings = Settings::default();
settings.file.enabled = true;
settings.file.path = path.to_string_lossy().into_owned();
settings.format.fields.insert("request_number".into(), "id".into());
settings.format.fields.insert("request_path".into(), "path".into());
settings.format.static_fields.insert("service".into(), "rustproxy".into());
let stop = CancellationToken::new();
let logs = Logs::start(settings, stop.clone());
const COUNT: usize = 180;
for id in 0..COUNT {
let mut item = entry();
item.id = id as u64;
item.path = format!("/test/{id}?token=secret");
logs.push(item);
}
let lines = tokio::time::timeout(Duration::from_secs(10), async {
loop {
if logs.stats().sent == COUNT as u64 {
if let Ok(text) = tokio::fs::read_to_string(&path).await {
if text.lines().count() == COUNT {
break text;
}
}
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
}).await.expect("all accepted local file records must be written");
stop.cancel();
assert_eq!(logs.stats().queue_dropped, 0);
assert_eq!(logs.stats().errors, 0);
assert_eq!(logs.stats().sent, COUNT as u64);
for (id, line) in lines.lines().enumerate() {
let record: Value = serde_json::from_str(line).unwrap();
assert_eq!(record["request_number"], id as u64);
assert_eq!(record["request_path"], format!("/test/{id}?token=[REDACTED]"));
assert_eq!(record["service"], "rustproxy");
assert_eq!(record.as_object().unwrap().len(), 3);
}
}
}
src/proxy/tests/authority_tests.rs
+55 -15
View File
@@ -36,7 +36,12 @@ mod authority_tests {
fn preserve_host_sets_http2_authority_without_changing_network_origin() {
use crate::config::UpstreamProtocol;
for (protocol, scheme, expected_authority, remove_host) in [
(UpstreamProtocol::Http1, "https", "backend.internal:443", false),
(
UpstreamProtocol::Http1,
"https",
"backend.internal:443",
false,
),
(UpstreamProtocol::Auto, "https", "unitraklub.pl", false),
(UpstreamProtocol::Http2, "https", "unitraklub.pl", false),
(UpstreamProtocol::H2, "https", "unitraklub.pl", true),
@@ -59,7 +64,11 @@ mod authority_tests {
let mut headers = HeaderMap::new();
headers.insert(HOST, HeaderValue::from_static("unitraklub.pl"));
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
assert_eq!(target.authority().unwrap().as_str(), expected_authority, "{protocol:?}");
assert_eq!(
target.authority().unwrap().as_str(),
expected_authority,
"{protocol:?}"
);
assert_eq!(target.path(), "/block_refresh/block/10");
assert_eq!(headers.contains_key(HOST), !remove_host);
assert_eq!(backend.origin, format!("{scheme}://10.87.20.2:443"));
@@ -70,8 +79,10 @@ mod authority_tests {
fn optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes() {
assert_eq!(log_prefix("/small.html", 256), "/small.html");
assert_eq!(log_prefix("abcdefghijklmnopqrstuvwxyz", 5), "abcde");
assert_eq!(log_prefix("za\u{17c}\u{f3}\u{142}\u{107}\u{1f525}", 4),
"za\u{17c}\u{f3}");
assert_eq!(
log_prefix("za\u{17c}\u{f3}\u{142}\u{107}\u{1f525}", 4),
"za\u{17c}\u{f3}"
);
assert_eq!(log_prefix("", 0), "");
}
@@ -83,8 +94,14 @@ mod authority_tests {
let uri = upstream_uri(backend, "/keep?key=hello%2Fworld").unwrap();
let mut headers = HeaderMap::new();
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
assert_eq!(target.path_and_query().unwrap().as_str(), "/keep?key=hello%2Fworld");
assert_eq!(target.authority().unwrap(), backend.request_origin_uri.authority().unwrap());
assert_eq!(
target.path_and_query().unwrap().as_str(),
"/keep?key=hello%2Fworld"
);
assert_eq!(
target.authority().unwrap(),
backend.request_origin_uri.authority().unwrap()
);
}
#[test]
@@ -109,8 +126,16 @@ mod authority_tests {
let backend = &snapshot.pools[0].members[0].backend;
let mut headers = HeaderMap::new();
headers.insert(HOST, HeaderValue::from_static("public.example"));
let req = retry_request(&Method::GET, Version::HTTP_2, &headers, backend,
&HeaderValue::from_static("public.example"), "/retry", true).unwrap();
let req = retry_request(
&Method::GET,
Version::HTTP_2,
&headers,
backend,
&HeaderValue::from_static("public.example"),
"/retry",
true,
)
.unwrap();
assert_eq!(req.uri().authority().unwrap().as_str(), "public.example");
assert!(req.headers().get(HOST).is_none());
}
@@ -361,7 +386,10 @@ mod authority_tests {
#[test]
fn backend_diagnostics_are_opt_in_and_strip_spoofed_values() {
let mut response = Response::new(Body::empty());
response.headers_mut().insert("x-rustproxy-backend", HeaderValue::from_static("canonical-spoof"));
response.headers_mut().insert(
"x-rustproxy-backend",
HeaderValue::from_static("canonical-spoof"),
);
response
.headers_mut()
.insert("x-rust-proxy-backend", HeaderValue::from_static("spoofed"));
@@ -388,7 +416,10 @@ mod authority_tests {
response
.extensions_mut()
.insert(OriginTiming(Duration::from_millis(12)));
response.headers_mut().insert("x-rustproxy-backend", HeaderValue::from_static("canonical-spoof"));
response.headers_mut().insert(
"x-rustproxy-backend",
HeaderValue::from_static("canonical-spoof"),
);
apply_backend_diagnostic_values(&mut response, true, true);
assert!(response.headers().get("x-rustproxy-backend").is_none());
assert_eq!(
@@ -623,16 +654,25 @@ mod graceful_h2_tests {
config.vhosts[0].backends[0].tls_server_name = Some("backend.example".into());
let snapshot = crate::config::compile(config, 1, &validation_policy()).unwrap();
let backend = &snapshot.pools[0].members[0].backend;
assert_eq!(backend.request_origin_uri.to_string(), backend.request_origin);
assert_eq!(
backend.request_origin_uri.to_string(),
backend.request_origin
);
for suffix in [
"/", "/static/main.css", "/path?x=1&y=2", "/p%2Fq?url=https%3A//example.org/a",
"/?q=%C5%82&empty=", "/double//slash", "/item#frag",
"/",
"/static/main.css",
"/path?x=1&y=2",
"/p%2Fq?url=https%3A//example.org/a",
"/?q=%C5%82&empty=",
"/double//slash",
"/item#frag",
] {
let previous = format!("{}{suffix}", backend.request_origin).parse::<Uri>().ok();
let previous = format!("{}{suffix}", backend.request_origin)
.parse::<Uri>()
.ok();
assert_eq!(upstream_uri(backend, suffix), previous, "{suffix}");
}
assert!(upstream_uri(backend, "http://attacker.invalid/").is_none());
assert_eq!(backend.origin, "https://127.0.0.1:9000");
}
}