v0.34.0_8
This commit is contained in:
1 parent
b6a3e12225
commit
480c3ec769
92 files changed
+298
-10035
No files matched your search
@@ -10,7 +10,7 @@ Maintain English documentation and Markdown files. Keep translations in discover
|
|||||||
CHANGELOG.md uses version headings with no release dates. Patch increments may be
|
CHANGELOG.md uses version headings with no release dates. Patch increments may be
|
||||||
bundled; substantial optional integrations raise the minor version. 0.7.0 adds managed renewal, server deployment helpers, Basic Auth and visual locations. The reported compression threshold type mismatch is removed without narrowing. Cloudflare Tunnel and Docker discovery are deferred.
|
bundled; substantial optional integrations raise the minor version. 0.7.0 adds managed renewal, server deployment helpers, Basic Auth and visual locations. The reported compression threshold type mismatch is removed without narrowing. Cloudflare Tunnel and Docker discovery are deferred.
|
||||||
|
|
||||||
Deliver a complete archive with dev.sh, tests and an updated verification report.
|
Deliver a complete archive with dev.sh and the existing tests. Keep verification results outside the repository; do not add iteration reports, standalone benchmark scripts, or micro-optimization notes to CHANGELOG.md.
|
||||||
Do not include databases, credentials, real certificates/keys, target directories,
|
Do not include databases, credentials, real certificates/keys, target directories,
|
||||||
node_modules or fabricated Cargo.lock content. Preserve an actually resolved
|
node_modules or fabricated Cargo.lock content. Preserve an actually resolved
|
||||||
lockfile when a compiler-enabled environment can produce it.
|
lockfile when a compiler-enabled environment can produce it.
|
||||||
@@ -40,8 +40,8 @@ Keep production Rust source files within 500 lines. Rust unit and integration
|
|||||||
**test files are exempt** from this limit: do not create numbered `part_XX.rs`
|
**test files are exempt** from this limit: do not create numbered `part_XX.rs`
|
||||||
fragments to satisfy source-size checks. Consolidate coherent test suites in
|
fragments to satisfy source-size checks. Consolidate coherent test suites in
|
||||||
`tests.rs` and preserve the existing test cases.
|
`tests.rs` and preserve the existing test cases.
|
||||||
Run `./dev.sh check` and `./dev.sh test` when Rust is available. Run the SQL and upgrade
|
Run `./dev.sh check` and `./dev.sh test` when Rust is available. Run
|
||||||
scripts even when it is not. UI tests use a simulated API; the Python protocol-peer
|
existing SQL and upgrade checks when available; do not restore removed auxiliary scripts. UI tests use a simulated API; the Python protocol-peer
|
||||||
self-test is not a test of RustProxy. Record these distinctions accurately.
|
self-test is not a test of RustProxy. Record these distinctions accurately.
|
||||||
Do not mark a stage accepted or production-ready on the basis of static inspection.
|
Do not mark a stage accepted or production-ready on the basis of static inspection.
|
||||||
Review real protocol behavior, restored data and resource bounds before expanding
|
Review real protocol behavior, restored data and resource bounds before expanding
|
||||||
|
|||||||
+6
-6
@@ -21,7 +21,7 @@
|
|||||||
- Source-only until OPT5 compilation and Rust tests. The compiled OPT4 input
|
- Source-only until OPT5 compilation and Rust tests. The compiled OPT4 input
|
||||||
passed 151 live HTTP checks (0 failed, 1 optional Zstandard skipped).
|
passed 151 live HTTP checks (0 failed, 1 optional Zstandard skipped).
|
||||||
Brief shared-host OPT4/OPT3/nginx tests are mixed at 16/64 connections;
|
Brief shared-host OPT4/OPT3/nginx tests are mixed at 16/64 connections;
|
||||||
see `docs/PROXY_1KB_OPT5_DEV_0.34.0.md`. No version increment.
|
No version increment.
|
||||||
|
|
||||||
- DEV 1 KiB follow-up (OPT4): keep ordinary unmodified public request targets
|
- DEV 1 KiB follow-up (OPT4): keep ordinary unmodified public request targets
|
||||||
borrowed through URI rewriting, avoid creating a metering body adapter for
|
borrowed through URI rewriting, avoid creating a metering body adapter for
|
||||||
@@ -34,7 +34,7 @@
|
|||||||
for complex values, TLS/H2/H3 authority handling and request limits.
|
for complex values, TLS/H2/H3 authority handling and request limits.
|
||||||
- OPT3 compiled binary passed 151 live HTTP checks, 0 failed, 1 skipped
|
- OPT3 compiled binary passed 151 live HTTP checks, 0 failed, 1 skipped
|
||||||
(optional Zstandard). OPT4 was subsequently compiled and tested similarly;
|
(optional Zstandard). OPT4 was subsequently compiled and tested similarly;
|
||||||
see `docs/PROXY_1KB_OPT4_DEV_0.34.0.md` and newer OPT5 analysis.
|
Results should be checked against matched configurations.
|
||||||
|
|
||||||
|
|
||||||
- DEV 1 KiB throughput follow-up (OPT3): keep normalized safe routing
|
- DEV 1 KiB throughput follow-up (OPT3): keep normalized safe routing
|
||||||
@@ -50,7 +50,7 @@
|
|||||||
fields and untrusted IPv6 forwarding. User-supplied OPT2-FIX1 binary passed
|
fields and untrusted IPv6 forwarding. User-supplied OPT2-FIX1 binary passed
|
||||||
89 targeted live checks. A compiled OPT3 executable was subsequently tested
|
89 targeted live checks. A compiled OPT3 executable was subsequently tested
|
||||||
(151 passing HTTP cases; one skipped), but short shared-host A/B did not
|
(151 passing HTTP cases; one skipped), but short shared-host A/B did not
|
||||||
establish a stable RPS gain; see `docs/PROXY_1KB_OPT3_DEV_0.34.0.md`.
|
establish a stable RPS gain.
|
||||||
|
|
||||||
- DEV 1 KiB accounting follow-up: move per-domain HIT/MISS/BYPASS, reason,
|
- DEV 1 KiB accounting follow-up: move per-domain HIT/MISS/BYPASS, reason,
|
||||||
stored-entry and saved-body statistics outside the HOT cache's shared mutex.
|
stored-entry and saved-body statistics outside the HOT cache's shared mutex.
|
||||||
@@ -64,7 +64,7 @@
|
|||||||
- Extend Rust regressions for parallel BYPASS/HIT counting, reset semantics and
|
- Extend Rust regressions for parallel BYPASS/HIT counting, reset semantics and
|
||||||
saturated analytics. The supplied **previous** DEV executable passes 151
|
saturated analytics. The supplied **previous** DEV executable passes 151
|
||||||
actual HTTP checks; these newest source changes are uncompiled and their RPS
|
actual HTTP checks; these newest source changes are uncompiled and their RPS
|
||||||
uplift unmeasured. See `docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`.
|
uplift unmeasured.
|
||||||
|
|
||||||
|
|
||||||
- DEV 1 KiB reverse-proxy throughput follow-up: identify HTTP/1.1 upstream
|
- DEV 1 KiB reverse-proxy throughput follow-up: identify HTTP/1.1 upstream
|
||||||
@@ -77,7 +77,7 @@
|
|||||||
sanitization when the corresponding header is absent. Preserve analytics,
|
sanitization when the corresponding header is absent. Preserve analytics,
|
||||||
history, counters, all security checks and cookie handling when present.
|
history, counters, all security checks and cookie handling when present.
|
||||||
- Add source guards, Rust regressions and a measured-configuration report in
|
- Add source guards, Rust regressions and a measured-configuration report in
|
||||||
`docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. This **source** is not compiled;
|
this source update. This **source** is not compiled;
|
||||||
newly introduced code performance and full protocol parity remain unverified.
|
newly introduced code performance and full protocol parity remain unverified.
|
||||||
No version increment and no claim of reaching 100k RPS / 80% nginx.
|
No version increment and no claim of reaching 100k RPS / 80% nginx.
|
||||||
|
|
||||||
@@ -102,7 +102,7 @@
|
|||||||
supervisor rotation, and keep H1/H2 protocol telemetry. No altered upstream keepalive defaults or cache query policies.
|
supervisor rotation, and keep H1/H2 protocol telemetry. No altered upstream keepalive defaults or cache query policies.
|
||||||
- Add source-only invariants, Rust regression cases and reproducible live
|
- Add source-only invariants, Rust regression cases and reproducible live
|
||||||
before/after/nginx benchmarking instructions. This revision has not yet been
|
before/after/nginx benchmarking instructions. This revision has not yet been
|
||||||
compiled or measured for RPS; see docs/PROXY_RPS_0.34.0.md.
|
compiled or measured for RPS.
|
||||||
|
|
||||||
|
|
||||||
- Cache regression fix: interpret local max TTL as a cap on remaining freshness at receipt, not on the upstream Age clock. Preserve RFC 9111 Age and transformed response validators; prohibit private/no-store caching.
|
- Cache regression fix: interpret local max TTL as a cap on remaining freshness at receipt, not on the upstream Age clock. Preserve RFC 9111 Age and transformed response validators; prohibit private/no-store caching.
|
||||||
|
|||||||
@@ -1,53 +1,5 @@
|
|||||||
# RustProxy 0.34.0
|
# RustProxy 0.34.0
|
||||||
|
|
||||||
## DEV: OPT5 multi-hot-path tuning (source candidate; uncompiled)
|
|
||||||
|
|
||||||
OPT5 builds on the **compiled OPT4** (151/151 executed live HTTP checks passed;
|
|
||||||
1 optional Zstandard check skipped). The shared-host 1,040-byte HTTP/1.1
|
|
||||||
comparison shows OPT4 at about 39,054 RPS with 16 connections and 36,425 RPS
|
|
||||||
with 64; mixed results versus OPT3 mean there is no proven broad RPS gain.
|
|
||||||
OPT5 removes unnecessary ID hashing and unauthenticated original-URL
|
|
||||||
construction, avoids per-success health-detail allocation, batches bounded
|
|
||||||
traffic analytics, avoids routine status/User-Agent allocations, and fast-paths
|
|
||||||
unoffered response compression. All policies, observability, caches and
|
|
||||||
protocol support remain enabled; no version bump. **OPT5 has not been built
|
|
||||||
or measured.** See [`docs/PROXY_1KB_OPT5_DEV_0.34.0.md`](docs/PROXY_1KB_OPT5_DEV_0.34.0.md).
|
|
||||||
|
|
||||||
## DEV: OPT4 small-response proxy hot path (subsequently compiled)
|
|
||||||
|
|
||||||
The compiled OPT4 binary passed 151 real HTTP acceptance checks (1 optional
|
|
||||||
Zstandard case skipped). OPT4 removes avoidable allocations and string work
|
|
||||||
for ordinary uncached HTTP/1.1 requests, with regressions for empty request
|
|
||||||
bodies, URI/Host/Connection semantics and Unicode log truncation. Its brief
|
|
||||||
shared-host RPS comparison is mixed, so no stable improvement is claimed. No settings or functionality disabled;
|
|
||||||
version remains 0.34.0. Details:
|
|
||||||
[`docs/PROXY_1KB_OPT4_DEV_0.34.0.md`](docs/PROXY_1KB_OPT4_DEV_0.34.0.md).
|
|
||||||
|
|
||||||
|
|
||||||
## DEV: small-response proxy hot path, OPT3 source candidate
|
|
||||||
|
|
||||||
Seven compatible optimizations reduce path copies and work constructing
|
|
||||||
forwarded headers, log IPs, hop-by-hop headers, request IDs and cache-off
|
|
||||||
reason totals. HTTP/1.1 proxy with a 1,040-byte response remains the first
|
|
||||||
benchmark priority; local Web Server is secondary. No version or policy changes.
|
|
||||||
A subsequently supplied compiled OPT3 executable passed 151 real HTTP
|
|
||||||
assertions (one optional codec skipped). Two short same-host benchmark runs
|
|
||||||
did **not** establish a stable RPS gain over OPT2. See the historical
|
|
||||||
[`OPT3 source report`](docs/PROXY_1KB_OPT3_DEV_0.34.0.md)
|
|
||||||
and the newer [`OPT4 analysis`](docs/PROXY_1KB_OPT4_DEV_0.34.0.md).
|
|
||||||
|
|
||||||
## DEV: cache counter contention and saturated analytics (source candidate)
|
|
||||||
|
|
||||||
Per-domain cache outcome/reason accounting now uses bounded immutable indexes
|
|
||||||
and atomic counters rather than the HOT cache's global store mutex on every
|
|
||||||
request. The analytics producer reserves a queue slot before cloning fields;
|
|
||||||
full-queue/drop behavior and its counter remain unchanged. Per-vhost
|
|
||||||
latency histograms now update one exclusive atomic bin per completion and
|
|
||||||
reconstruct unchanged cumulative Prometheus buckets during scraping. The supplied
|
|
||||||
previous DEV binary passed 151 live HTTP assertions, but this newest code
|
|
||||||
**has not been built**. No version bump, no measured RPS claim. See
|
|
||||||
[`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`](docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md).
|
|
||||||
|
|
||||||
## 0.34.0: HOT HIT critical-section follow-up (source candidate)
|
## 0.34.0: HOT HIT critical-section follow-up (source candidate)
|
||||||
|
|
||||||
Immutable response headers are shared until a HIT; the mutable per-response
|
Immutable response headers are shared until a HIT; the mutable per-response
|
||||||
@@ -57,12 +9,6 @@ shorter paths. The existing stale-for-revalidation policy makes the previous
|
|||||||
full HOT sweep unnecessary. Exact LRU, generation barriers, BAN, COLD,
|
full HOT sweep unnecessary. Exact LRU, generation barriers, BAN, COLD,
|
||||||
transformation/ETag, TTL and cache bypass settings remain unchanged.
|
transformation/ETag, TTL and cache bypass settings remain unchanged.
|
||||||
|
|
||||||
This is **uncompiled source**; do not infer network RPS or production acceptance
|
|
||||||
from source checks. See
|
|
||||||
[`docs/CACHE_HOTPATH_0.34.0.md`](docs/CACHE_HOTPATH_0.34.0.md).
|
|
||||||
|
|
||||||
## 0.34.0: reverse-proxy hot-path optimization revision (source candidate)
|
|
||||||
|
|
||||||
This revision retains the **0.34.0** version and all prior scoped-cache,
|
This revision retains the **0.34.0** version and all prior scoped-cache,
|
||||||
`sendfile`, security, logging and vhost/administration controls. In the
|
`sendfile`, security, logging and vhost/administration controls. In the
|
||||||
uncached reverse-proxy hot path, request observation now passes the immutable
|
uncached reverse-proxy hot path, request observation now passes the immutable
|
||||||
@@ -87,9 +33,7 @@ subscribers only while subscribers exist; history and external sinks remain.
|
|||||||
|
|
||||||
These changes have **not** been compiled or timed in this environment, and
|
These changes have **not** been compiled or timed in this environment, and
|
||||||
there is no claimed 0.34.0 RPS improvement. Build and run the regression and
|
there is no claimed 0.34.0 RPS improvement. Build and run the regression and
|
||||||
nginx benchmark matrix before deployment. See
|
nginx benchmark matrix before deployment.
|
||||||
[`docs/PROXY_HOTPATH_0.34.0.md`](docs/PROXY_HOTPATH_0.34.0.md) for invariants,
|
|
||||||
checks and exact measurement commands.
|
|
||||||
|
|
||||||
## 0.34.0: scoped cache, proxy accounting and opt-in Linux sendfile
|
## 0.34.0: scoped cache, proxy accounting and opt-in Linux sendfile
|
||||||
|
|
||||||
@@ -116,12 +60,6 @@ existing streaming path. The default is `off`; toggling this transport option
|
|||||||
does not purge HOT/COLD cache. **Native compilation and live protocol tests
|
does not purge HOT/COLD cache. **Native compilation and live protocol tests
|
||||||
remain pending; do not deploy this candidate without them.**
|
remain pending; do not deploy this candidate without them.**
|
||||||
|
|
||||||
An offline cache-retention simulation for a 20-vhost workload shows 92.123%
|
|
||||||
fewer invalidated objects and 92.113% fewer cache misses than the old global
|
|
||||||
purge policy **under its stated synthetic assumptions**. These are not RPS
|
|
||||||
figures; real 0.34.0 performance remains unmeasured.
|
|
||||||
|
|
||||||
See `docs/TEST_REPORT_0.34.0.md` for verification scope and remaining gates.
|
|
||||||
|
|
||||||
## 0.33.8: request-path performance and compatibility verification
|
## 0.33.8: request-path performance and compatibility verification
|
||||||
|
|
||||||
@@ -150,12 +88,9 @@ to exercise the existing documented behavior, including scheme-isolated cache
|
|||||||
keys, purge generations, private administration responses and bundled error pages.
|
keys, purge generations, private administration responses and bundled error pages.
|
||||||
|
|
||||||
The LB panel, database schema, default TLS/caching policies and bounded pool
|
The LB panel, database schema, default TLS/caching policies and bounded pool
|
||||||
recycling retain their existing behavior. Live process regression, verified TLS,
|
recycling retain their existing behavior. Historical verification methods and
|
||||||
H2/H3/QUIC and WebSocket runners, repeated nginx comparisons, resource sampling,
|
results are documented in [docs/TEST_REPORT.md](docs/TEST_REPORT.md) and
|
||||||
syscall profiling and a mixed-traffic soak are included in
|
[docs/PERFORMANCE_0.33.8.md](docs/PERFORMANCE_0.33.8.md).
|
||||||
[`scripts/performance/`](scripts/performance/README.md). See the
|
|
||||||
[verification contract](docs/TEST_REPORT.md) and the separately generated
|
|
||||||
[performance results](docs/PERFORMANCE_0.33.8.md) for measured outcomes and limits.
|
|
||||||
|
|
||||||
## 0.33.7: named dashboard presets, panel components and chart typography
|
## 0.33.7: named dashboard presets, panel components and chart typography
|
||||||
|
|
||||||
@@ -780,7 +715,6 @@ Historical release information remains in [CHANGELOG.md](CHANGELOG.md) and [docs
|
|||||||
- [docs/FILE_LOGS.md](docs/FILE_LOGS.md) - stored-log explorer, protocol-error records, bounds and filesystem security.
|
- [docs/FILE_LOGS.md](docs/FILE_LOGS.md) - stored-log explorer, protocol-error records, bounds and filesystem security.
|
||||||
- [docs/PHP_FPM.md](docs/PHP_FPM.md) - implemented FastCGI/PHP-FPM integration, safety model and runtime requirements.
|
- [docs/PHP_FPM.md](docs/PHP_FPM.md) - implemented FastCGI/PHP-FPM integration, safety model and runtime requirements.
|
||||||
- [docs/TEST_REPORT.md](docs/TEST_REPORT.md) - verification results and environment limitations.
|
- [docs/TEST_REPORT.md](docs/TEST_REPORT.md) - verification results and environment limitations.
|
||||||
- [docs/TODO.md](docs/TODO.md) - unresolved follow-up only.
|
|
||||||
|
|
||||||
## Release history
|
## Release history
|
||||||
|
|
||||||
|
|||||||
+1
-21
@@ -12,15 +12,12 @@
|
|||||||
retain 0.34.0, TLS/H2/H3, WebSocket, cache, ACL, logs and bounded queues.
|
retain 0.34.0, TLS/H2/H3, WebSocket, cache, ACL, logs and bounded queues.
|
||||||
- [ ] Compile and test OPT5; run full real-protocol regression and >=7
|
- [ ] Compile and test OPT5; run full real-protocol regression and >=7
|
||||||
30-second CPU-isolated A/B/nginx runs before accepting a throughput gain.
|
30-second CPU-isolated A/B/nginx runs before accepting a throughput gain.
|
||||||
|
|
||||||
- [x] OPT4 source candidate: borrow original no-rewrite public target,
|
- [x] OPT4 source candidate: borrow original no-rewrite public target,
|
||||||
bypass the request body adapter only for bodies reporting end-of-stream,
|
bypass the request body adapter only for bodies reporting end-of-stream,
|
||||||
avoid redundant upstream URI refcount operations, and simplify common
|
avoid redundant upstream URI refcount operations, and simplify common
|
||||||
forwarding/header/log branches without weakening validation.
|
forwarding/header/log branches without weakening validation.
|
||||||
- [x] Add OPT4 regression tests for body accounting and permits, Unicode,
|
- [x] Add OPT4 regression tests for body accounting and permits, Unicode,
|
||||||
URI identity, and safe Connection header filtering.
|
URI identity, and safe Connection header filtering.
|
||||||
- [ ] Complete OPT4/OPT5 Rust unit suites and extended TLS/H2/H3/WS/LB
|
|
||||||
acceptance; the short OPT4 RPS comparison did not prove a stable gain.
|
|
||||||
|
|
||||||
|
|
||||||
- [x] DEV 1 KiB: remove shared HOT store mutex from cache outcome/reason
|
- [x] DEV 1 KiB: remove shared HOT store mutex from cache outcome/reason
|
||||||
@@ -30,8 +27,6 @@
|
|||||||
- [x] Move per-vhost histogram aggregation to scrape time; retain the same
|
- [x] Move per-vhost histogram aggregation to scrape time; retain the same
|
||||||
cumulative Prometheus and Docker-vhost bucket semantics while one exclusive
|
cumulative Prometheus and Docker-vhost bucket semantics while one exclusive
|
||||||
atomic bin is updated per completed exchange.
|
atomic bin is updated per completed exchange.
|
||||||
- [ ] Build/test revised counters source; repeat 1 KiB source A/B and pinned
|
|
||||||
nginx benchmark (100k RPS / 80% target still open).
|
|
||||||
|
|
||||||
- [x] Share compiled route handles and reuse middleware routing in forwarding;
|
- [x] Share compiled route handles and reuse middleware routing in forwarding;
|
||||||
avoid exact-host wildcard allocation and full-route cloning.
|
avoid exact-host wildcard allocation and full-route cloning.
|
||||||
@@ -39,10 +34,6 @@
|
|||||||
between leases; reduce health locks and fallback/retry header snapshots.
|
between leases; reduce health locks and fallback/retry header snapshots.
|
||||||
- [x] Convert three exchange metadata locks to atomic tags; avoid unused live
|
- [x] Convert three exchange metadata locks to atomic tags; avoid unused live
|
||||||
log clones and add Rust regression fixtures for modified semantics.
|
log clones and add Rust regression fixtures for modified semantics.
|
||||||
- [ ] Compile this hot-path revision, run Rust tests and verify live LB health,
|
|
||||||
sticky/session behavior and HTTP/2 graceful shutdown retries.
|
|
||||||
- [ ] Profile CPU flamegraphs and measure real gains vs 0.33.8/nginx; tune
|
|
||||||
idle pool/recycle only as a separately labelled experiment.
|
|
||||||
|
|
||||||
- [x] Disconnect keepalive/LB/diagnostic-only vhost saves from cache revisioning.
|
- [x] Disconnect keepalive/LB/diagnostic-only vhost saves from cache revisioning.
|
||||||
- [x] Invalidate only a vhost scope when its origin/representation policy changes.
|
- [x] Invalidate only a vhost scope when its origin/representation policy changes.
|
||||||
@@ -50,16 +41,11 @@
|
|||||||
interrupted content update or restart.
|
interrupted content update or restart.
|
||||||
- [x] Reduce response-body per-frame backend mutex operations without losing metrics.
|
- [x] Reduce response-body per-frame backend mutex operations without losing metrics.
|
||||||
- [x] Add Rust regression cases and preserve unchanged LB controls/API.
|
- [x] Add Rust regression cases and preserve unchanged LB controls/API.
|
||||||
- [ ] Compile, run the full Rust suites and repeat live HTTP/TLS/H2/H3/WS/LB
|
|
||||||
regressions with a compatible Rust toolchain.
|
regressions with a compatible Rust toolchain.
|
||||||
- [ ] Run paired 0.33.8 vs 0.34.0 vs nginx benchmarks; record real RPS/p95/p99,
|
|
||||||
CPU, RSS, cache retention after saves, and rollback on failed acceptance.
|
CPU, RSS, cache retention after saves, and rollback on failed acceptance.
|
||||||
- [x] Implement opt-in Linux HTTP/1.1 `sendfile(2)` and a per-Web-Service
|
- [x] Implement opt-in Linux HTTP/1.1 `sendfile(2)` and a per-Web-Service
|
||||||
Delivery toggle, with TLS/H2/H3/range/cache/compression fallback and bounded
|
Delivery toggle, with TLS/H2/H3/range/cache/compression fallback and bounded
|
||||||
per-connection slots. Implementation is source-only until native tests run.
|
per-connection slots. Implementation is source-only until native tests run.
|
||||||
- [ ] Compile and validate native `sendfile`/fallback functionality and safety
|
|
||||||
on the target Linux OS; compare live WWW RPS against nginx.
|
|
||||||
- [ ] Complete long-running soak, external dependency integration and staged rollout.
|
|
||||||
|
|
||||||
## 0.33.8 - Performance with retained functionality
|
## 0.33.8 - Performance with retained functionality
|
||||||
|
|
||||||
@@ -76,10 +62,6 @@
|
|||||||
- [x] Add repeatable live HTTP, verified TLS/H2/H3/WebSocket, nginx comparison,
|
- [x] Add repeatable live HTTP, verified TLS/H2/H3/WebSocket, nginx comparison,
|
||||||
resource sampling, syscall profiling and mixed-traffic soak runners. Measured
|
resource sampling, syscall profiling and mixed-traffic soak runners. Measured
|
||||||
outcomes and limits are recorded in `docs/PERFORMANCE_0.33.8.md` and raw evidence.
|
outcomes and limits are recorded in `docs/PERFORMANCE_0.33.8.md` and raw evidence.
|
||||||
- [ ] Complete deployment-specific 24-hour soak and remote-network testing with
|
|
||||||
actual PHP-FPM, Tinyauth and DNS infrastructure before production rollout.
|
|
||||||
- [ ] Evaluate further proxy pool/queue costs and a secure static-file sendfile
|
|
||||||
path from measured profiles; preserve TLS/H2/H3 and all observability semantics.
|
|
||||||
|
|
||||||
## 0.33.7 - named dashboard presets, focused HTML extraction and chart typography
|
## 0.33.7 - named dashboard presets, focused HTML extraction and chart typography
|
||||||
|
|
||||||
@@ -107,8 +89,6 @@
|
|||||||
sentence case, all other panels uppercase); use mono for full legends,
|
sentence case, all other panels uppercase); use mono for full legends,
|
||||||
captions and chart data across all chart families.
|
captions and chart data across all chart families.
|
||||||
- [x] Validate SQL migration and run available simulated-browser/JS tests.
|
- [x] Validate SQL migration and run available simulated-browser/JS tests.
|
||||||
- [ ] Build/test Rust and run the new endpoint/UI against a real server with
|
|
||||||
representative persisted data. The production acceptance gate remains open.
|
|
||||||
|
|
||||||
## 0.33.6 - Stable dashboard chart ranges and plot heights
|
## 0.33.6 - Stable dashboard chart ranges and plot heights
|
||||||
|
|
||||||
@@ -745,4 +725,4 @@ Future Web Server work must retain the 0.25 split between reusable **Web Service
|
|||||||
- Race-resistant filesystem confinement for non-Unix targets before enabling local file serving there.
|
- Race-resistant filesystem confinement for non-Unix targets before enabling local file serving there.
|
||||||
- Real PHP-FPM release-host acceptance over Unix socket and loopback TCP.
|
- Real PHP-FPM release-host acceptance over Unix socket and loopback TCP.
|
||||||
|
|
||||||
See [`docs/TODO.md`](docs/TODO.md) for unresolved work and release acceptance, and [`docs/ROADMAP_HISTORY.md`](docs/ROADMAP_HISTORY.md) for completed historical milestones.
|
See [`docs/ROADMAP_HISTORY.md`](docs/ROADMAP_HISTORY.md) for completed historical milestones.
|
||||||
+20
-91
@@ -1,13 +1,13 @@
|
|||||||
61bf40a0572889dc8b0dda9eae4e93e44b0de9364b3c798106c85700ffb37c23 .dockerignore
|
61bf40a0572889dc8b0dda9eae4e93e44b0de9364b3c798106c85700ffb37c23 .dockerignore
|
||||||
4140a2956264b44f259e6a37fc00ba02097bbe68b09296e04d3d243c1287129b .env.example
|
4140a2956264b44f259e6a37fc00ba02097bbe68b09296e04d3d243c1287129b .env.example
|
||||||
9e6356475bf587091bab72246ec9e325b498f7e48a6220e9a18baa50d0a385a6 .gitignore
|
9e6356475bf587091bab72246ec9e325b498f7e48a6220e9a18baa50d0a385a6 .gitignore
|
||||||
46bab1d2c2d6e0a80d57a6b232b1ba4c220884b95cd45ddccbafce2aa160cddb AGENTS.md
|
5135a996f12151e481942bf3b4c46c42b675c1cf7086bd44733f3be260177d8c AGENTS.md
|
||||||
3e46648fd8c91548ea77dcac4f6cbdbc5741c64b33240c44a50c138597880e99 CHANGELOG.md
|
e7bea33973a805fdd1fd3af967555004b3509ef9b81576819c707c3dd5c47497 CHANGELOG.md
|
||||||
ca201e3cd3f9842eae0006d0b40a0a72bcd3d4da6e4e5ecaf200d1b9a1390db2 Cargo.lock
|
ca201e3cd3f9842eae0006d0b40a0a72bcd3d4da6e4e5ecaf200d1b9a1390db2 Cargo.lock
|
||||||
fec5f1bae9a59ef1fabd27c5e45d31448dd8f811736eebfa8826b3806a7e3349 Cargo.toml
|
fec5f1bae9a59ef1fabd27c5e45d31448dd8f811736eebfa8826b3806a7e3349 Cargo.toml
|
||||||
25c7f3c8162cbf5cc156439774118a81dae9fdc52b611ec7e40c230fc99a772c Dockerfile
|
25c7f3c8162cbf5cc156439774118a81dae9fdc52b611ec7e40c230fc99a772c Dockerfile
|
||||||
d48bcdcda6d9261d7b358c39b8bdf93d39b722b3751af3b5ebe192e16bd8b4e6 README.md
|
f0d89d95166d47b6ac2033cc8f5eaf2246c0c70139437394c98e10a95ab3d0dc README.md
|
||||||
7bba42713fa944908f5f3611841bf6b36377240ae4b77ec692ff48ea662b44f9 ROADMAP.md
|
b1d6174f509c743622cb27c92e047b724ac9d44bbf1c0987178da8a3bbd74b01 ROADMAP.md
|
||||||
da8311e3db945290666324a2f24bf4521b365272d7ac3f862ded1be993eea646 UPDATE_CHECK.toml
|
da8311e3db945290666324a2f24bf4521b365272d7ac3f862ded1be993eea646 UPDATE_CHECK.toml
|
||||||
f645f218dd062bf6cb54a8ee634290562cad364a4cccf09f16e3c8303582af8c build.rs
|
f645f218dd062bf6cb54a8ee634290562cad364a4cccf09f16e3c8303582af8c build.rs
|
||||||
ea8d091843f86f4c66a5c76b20268b2feb397070851b6e05c5bc7d7a8d9157eb deploy/README.md
|
ea8d091843f86f4c66a5c76b20268b2feb397070851b6e05c5bc7d7a8d9157eb deploy/README.md
|
||||||
@@ -27,12 +27,12 @@ e52b31d0abf65ae3034f6cb367cca113633014ce8d1d799ad4dd5624d2c7b8bb docker-agent/n
|
|||||||
5220d669171ec3ad65f50e0bc8de5178dadaa4904def8e1a4f61002aa9e9a55b docker-agent/server.go
|
5220d669171ec3ad65f50e0bc8de5178dadaa4904def8e1a4f61002aa9e9a55b docker-agent/server.go
|
||||||
875faa3969df9409d79f050a84b4183496f557edda3d34aaec19fe556cf8c471 docs/ACME_PROVIDERS.md
|
875faa3969df9409d79f050a84b4183496f557edda3d34aaec19fe556cf8c471 docs/ACME_PROVIDERS.md
|
||||||
ec15163f5f51c9212003b7edd1fc380321311c2496fbaa9b45c838281b9af475 docs/ACME_RFC2136.md
|
ec15163f5f51c9212003b7edd1fc380321311c2496fbaa9b45c838281b9af475 docs/ACME_RFC2136.md
|
||||||
00aef5eea37a8f56d7ea95e63ddcfdf39bac041168af29582b5ab2ab62d4c81e docs/API.md
|
70588369169c06e71925d6f45f98258153626cccbd168a77eb882a4a7ded7702 docs/API.md
|
||||||
f27053f8a93d7883025a04922d015e0b697cad653e714be632b757e167297de1 docs/API_ACCESS.md
|
f27053f8a93d7883025a04922d015e0b697cad653e714be632b757e167297de1 docs/API_ACCESS.md
|
||||||
d30551a52ffd34c5963094d2826ba39795210a4e6a4e4d7333066dae7456fc50 docs/ARCHITECTURE.md
|
d30551a52ffd34c5963094d2826ba39795210a4e6a4e4d7333066dae7456fc50 docs/ARCHITECTURE.md
|
||||||
fb99fac0a490af262a0ecdad593ffe5164004c2166fe7ddd607476bcc56a1623 docs/CACHE.md
|
7f61bc6e5c82b6519b50525eb2516298a5e28dbb28537d31eaf2f56bb7d5ff59 docs/CACHE.md
|
||||||
9113f3abbfe81bafc4855d8680df8aa2a381ecc358cc3e1a0f78b6b756421117 docs/CACHE_DIAGNOSTICS.md
|
9113f3abbfe81bafc4855d8680df8aa2a381ecc358cc3e1a0f78b6b756421117 docs/CACHE_DIAGNOSTICS.md
|
||||||
a8fd061f8bdd920158fa42972f9d26418cbfe9d2cd8757ba1d9d1d12e5c908ee docs/CACHE_HOTPATH_0.34.0.md
|
de75fae9e387b2e8b3d478677963f8d67a124f3f02b746de285ca8b697b3816b docs/CACHE_HOTPATH_0.34.0.md
|
||||||
3822d72e12f82c37a9e70ab40c7e89e89145ad37f29914e4702db4bf024a5501 docs/CHANGELOG_TECHNICAL_ARCHIVE.md
|
3822d72e12f82c37a9e70ab40c7e89e89145ad37f29914e4702db4bf024a5501 docs/CHANGELOG_TECHNICAL_ARCHIVE.md
|
||||||
c8e69e492179ce39958cc63615cf46c4ac08cfdb7f3483aaab66868a3abcd687 docs/CONTAINER.md
|
c8e69e492179ce39958cc63615cf46c4ac08cfdb7f3483aaab66868a3abcd687 docs/CONTAINER.md
|
||||||
833ddb12f23dcac796223521388b19c6ec3ea04c5550cf1c5f4ea4eb7c58f3a0 docs/CUSTOM_LOCATIONS.md
|
833ddb12f23dcac796223521388b19c6ec3ea04c5550cf1c5f4ea4eb7c58f3a0 docs/CUSTOM_LOCATIONS.md
|
||||||
@@ -56,33 +56,22 @@ be53a58afbeddbd7deb4b5f6447f2dc74db0327839bd83407c20915df98dcfa0 docs/NOT_FOUND
|
|||||||
e68d5a5cf0991de9308f78afb0264e02979f7463a5f395eb7a6e9ec2bd3a3d54 docs/PHP_FPM.md
|
e68d5a5cf0991de9308f78afb0264e02979f7463a5f395eb7a6e9ec2bd3a3d54 docs/PHP_FPM.md
|
||||||
bc068a176056f5ea4d3b637071e644ed58a2140494c233ef22aa66e9700e3299 docs/PHP_FPM_ISOLATION.md
|
bc068a176056f5ea4d3b637071e644ed58a2140494c233ef22aa66e9700e3299 docs/PHP_FPM_ISOLATION.md
|
||||||
ad6e3f003b820a4f45e30e39577d120a69d167c70910bda4891971759458c3dd docs/PROMETHEUS.md
|
ad6e3f003b820a4f45e30e39577d120a69d167c70910bda4891971759458c3dd docs/PROMETHEUS.md
|
||||||
401ab1bb2c7f3961742195d81f5daf5c9cf81609a4b175bea1d10d2a4b727082 docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md
|
df6dc1070f94eeee7d4273251b93ab03b34e6cf0b02789d5b8f538a37814b382 docs/README.md
|
||||||
f13ee0005d57d2576ccc970745f17069096df4afa23d712ea0f748da85433f7d docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md
|
|
||||||
c9bbdd1cd8df02b2a19016b7bbeee1cf55917614ac672f1f94e0a05db7d309ef docs/PROXY_1KB_OPT3_DEV_0.34.0.md
|
|
||||||
27002c4d4e19bb7d58521b8ba4cd5faf180d2917b6ac200dd7fc3664c2060145 docs/PROXY_1KB_OPT4_DEV_0.34.0.md
|
|
||||||
8c26aa9ebebf54d912749e3336e548b59d805997366be2324157c3e56b887f5f docs/PROXY_1KB_OPT5_DEV_0.34.0.md
|
|
||||||
deea89aed410c0619bdbdb12437edc7aff9dca2fec7fdc059e918c649fc36b51 docs/PROXY_HOTPATH_0.34.0.md
|
|
||||||
a7d6189b0e8cb854918456e0063101ee77bfb2419a9ba0430eb0e3f8678c0f09 docs/PROXY_RPS_0.34.0.md
|
|
||||||
887d1fdcc3b546b024afcd18b1716b8e3a74a3800012bf4bda7a122d2eabe233 docs/README.md
|
|
||||||
6e46a137b9c88dac8cb9e3fdca877430092392de9913d746278dbff8652460c2 docs/REDIRECTS.md
|
6e46a137b9c88dac8cb9e3fdca877430092392de9913d746278dbff8652460c2 docs/REDIRECTS.md
|
||||||
0dfd0d431c5aedc67abac1ce5de040ffa790ec9578fd7d6f91a987c333d73358 docs/REFERENCES.md
|
0dfd0d431c5aedc67abac1ce5de040ffa790ec9578fd7d6f91a987c333d73358 docs/REFERENCES.md
|
||||||
540fa46716d562bfc846d2f7bac0515380831ddbdcaa01ff4ac86be4ff3d3aa0 docs/RELEASE_0.34.0_CACHE_FIX_PL.md
|
|
||||||
d85c5567fc6a7714b96ad7cfe4ee9ff2085f84c4f31534a6484340d03a73e0f7 docs/RENEWAL.md
|
d85c5567fc6a7714b96ad7cfe4ee9ff2085f84c4f31534a6484340d03a73e0f7 docs/RENEWAL.md
|
||||||
cd4ed3af26eb51c9ff6ef3dda9d52ff5120bcfc78c842bce34a639cf85b6aea9 docs/ROADMAP_HISTORY.md
|
82bbcd1c3a3bb6a44065f96f160f8633b2f2450ab4723412570cc885a6de459b docs/ROADMAP_HISTORY.md
|
||||||
759bf2987f2b602488829d1856fa33d57c2a3b23bc681ef5a931b9633af3a533 docs/SECRET_STORE_BACKUP.md
|
759bf2987f2b602488829d1856fa33d57c2a3b23bc681ef5a931b9633af3a533 docs/SECRET_STORE_BACKUP.md
|
||||||
ec7e32fd19e6ba6ed1732f1f3dd92ac0860609e8c0bbd3c1998840cdfb24bf54 docs/SECURITY.md
|
ec7e32fd19e6ba6ed1732f1f3dd92ac0860609e8c0bbd3c1998840cdfb24bf54 docs/SECURITY.md
|
||||||
620b69e566d27af04f0604f4ca3f88cf95cdfea7fbc6ae9dd92446d2b2b5d9be docs/SECURITY_AUDIT.md
|
620b69e566d27af04f0604f4ca3f88cf95cdfea7fbc6ae9dd92446d2b2b5d9be docs/SECURITY_AUDIT.md
|
||||||
007537ee971463072b2482dfa4fdcf1955f837ced0b711d4eef3b6267d16d4f6 docs/SECURITY_AUDIT_0.31.0.md
|
3434a638ef0198d659ad77e0a70d0540cf4f203e10e147713e6c2530ba84324a docs/SETTINGS.md
|
||||||
4dcf3243a13d9ae1827b5cefdbb9e27bd6310321c50d4d5934527d44437df6ef docs/SETTINGS.md
|
|
||||||
8fc112cb45a2f35ef5b0dd4371aa6f43aca4df85db62c6ae02e56d74a3b702ae docs/STREAMS.md
|
8fc112cb45a2f35ef5b0dd4371aa6f43aca4df85db62c6ae02e56d74a3b702ae docs/STREAMS.md
|
||||||
95148bfa1d2f8ff52b591c9acefc6c8992dadc4ebda43a195101d9b31d67a576 docs/TEST_REPORT.md
|
95148bfa1d2f8ff52b591c9acefc6c8992dadc4ebda43a195101d9b31d67a576 docs/TEST_REPORT.md
|
||||||
47e3cf646b2884db18684385ca7ff18d929b6de6a5206ed5e7ad27a71d350052 docs/TEST_REPORT_0.34.0.md
|
|
||||||
cfb452d8496839e832b5fbacdd4e52525b0ecc3112fbb62340c1e580bd5c1717 docs/TODO.md
|
|
||||||
751c9b13b47d0959a8575d6e82bdecba988b950a374ba0b084fbba08b4b65ebe docs/TRANSFORM.md
|
751c9b13b47d0959a8575d6e82bdecba988b950a374ba0b084fbba08b4b65ebe docs/TRANSFORM.md
|
||||||
e6d270894cafe96f8aa9cf58c9609223d03c9bcf6f872adfcbfb767a7b313f0a docs/TRANSPORT.md
|
e6d270894cafe96f8aa9cf58c9609223d03c9bcf6f872adfcbfb767a7b313f0a docs/TRANSPORT.md
|
||||||
811390a8c4ad6439dda4d532e35edc389db70f185ec728604ab35e22033f7e07 docs/VHOSTS.md
|
915da874d0b24e39b359fb70cc557730f7053a8334ced9ed406b6fa933c0dfaf docs/VHOSTS.md
|
||||||
09b60fb7b40db4d0139fa2a9e69a2aa9f28d4e61b8643521cfc7e936d7df8774 docs/VHOST_ACCESS.md
|
09b60fb7b40db4d0139fa2a9e69a2aa9f28d4e61b8643521cfc7e936d7df8774 docs/VHOST_ACCESS.md
|
||||||
ca5d1d1d7e8aa372dcd3e2bd56bb7a9b8977953482ece560cfef58a19314242e docs/WEB_SERVER.md
|
cdf208dc6438d11c8119b6de81cad7a838eaeeb9cbbb5417e691253760d69ddf docs/WEB_SERVER.md
|
||||||
c336f998027330c58d5afbbbb3a3a682fd44ae3997643882fe342633822f8bb3 docs/screenshots/README.md
|
c336f998027330c58d5afbbbb3a3a682fd44ae3997643882fe342633822f8bb3 docs/screenshots/README.md
|
||||||
5502e8ce44276213803b66c58e3a4d804a1354f5ed5d83f47ee805e8c44ab088 examples/acl-headers.json
|
5502e8ce44276213803b66c58e3a4d804a1354f5ed5d83f47ee805e8c44ab088 examples/acl-headers.json
|
||||||
2431dd88a2d9ad2cf3cd4b8fcdfa602563ab1013c612d8917fddbe7febfb2dc3 examples/cache-dashboard.json
|
2431dd88a2d9ad2cf3cd4b8fcdfa602563ab1013c612d8917fddbe7febfb2dc3 examples/cache-dashboard.json
|
||||||
@@ -136,73 +125,13 @@ a641bd35400c56f3b68b0a92e97b602d06090f03972ca8f01fa744e90dd97403 scripts/deploy
|
|||||||
2cc658faaa610b21b48942c59b955525539ad4503fd60f2a49c7e626a22faaff scripts/migration/migrate_npm.py
|
2cc658faaa610b21b48942c59b955525539ad4503fd60f2a49c7e626a22faaff scripts/migration/migrate_npm.py
|
||||||
a459d95bbb0076f33ade8b273b813308167f9d8d45242b309a2d18320ca0093c scripts/migration/migrate_npm_api.py
|
a459d95bbb0076f33ade8b273b813308167f9d8d45242b309a2d18320ca0093c scripts/migration/migrate_npm_api.py
|
||||||
31ccad3ec3c880119f3b7002f0c6652f8b43634edad8ab3763cb0e4497d0bb1f scripts/migration/migrate_npm_core.py
|
31ccad3ec3c880119f3b7002f0c6652f8b43634edad8ab3763cb0e4497d0bb1f scripts/migration/migrate_npm_core.py
|
||||||
bf934d59052988a528f3b7e7e3957b5a1ff1cf17db0d543e165cf76600e2137a scripts/performance/README.md
|
e380bf369dcfd16e0e82d243d072bf8739e005d66b049c9b849a0d8b2d34162c scripts/preflight.py
|
||||||
939287dadf7309689d29d2171757c2d80e91d898c3f9112454b943d08bd611fb scripts/performance/ab_1kb_dev.py
|
|
||||||
0380406a059ee9ec27473f8a46435f638f73a6dc729382ae1561c08dbf7ccb53 scripts/performance/benchmark.py
|
|
||||||
9ac220163256e08513597c397c016a48345b114d7876b439259cc310b36c83fc scripts/performance/checks_cache.py
|
|
||||||
f09125d081d0eed9a44fdca75de14e6bbf09dcb00fe1f027ba2270f89e3ff192 scripts/performance/checks_policy.py
|
|
||||||
96e01db050987af431a4662fce34752842bb2575237fd3239bcfc10ae55def6b scripts/performance/checks_proxy.py
|
|
||||||
c1d8c184811ae8fb1960d1ada08d8a4a95405c3b83dd18e1a27d711b596140be scripts/performance/checks_static.py
|
|
||||||
1044eb8d225e2fa117d5435697abdf0a014aa2f9063d4b41566e74ef47d50a34 scripts/performance/common.py
|
|
||||||
0f1fa62ee3429509dc21fe1cbfa7159e38d260ada98bfc6376e8078c9840ef48 scripts/performance/concurrency_regression.py
|
|
||||||
044da66a00a04f558a6b6e6157db0a7656dd5ee22adf0b0a47482c7f951380e9 scripts/performance/lb_regression.py
|
|
||||||
1b601d68d15909d786ac63890731f0592bd9057a68993a1d90bc70d6618e6f82 scripts/performance/peer.py
|
|
||||||
830a360609e2b8ebbb1ef6f3d89e3e6151e8fefb36c3673d20df68ff8932eabb scripts/performance/profile_syscalls.py
|
|
||||||
ceb26650fc8b63d0107eb9100ccd572e5c70f7a0ce5cc533a94083e20485095d scripts/performance/regression.py
|
|
||||||
d6480962a23f55d19a54aba27648ca1b28b50f03afe2decbfa272efd142df756 scripts/performance/resources.py
|
|
||||||
2dcd4c7e39dd58fd77c3c713d9bdee5025cace7191ca5f12bd6e7fe28f333358 scripts/performance/simulate_cache_034.py
|
|
||||||
4a5af2d3870d9517b4ad9c008789490cfb07affe98ffd629e51f7674ff54a642 scripts/performance/simulate_nginx_gap_034.py
|
|
||||||
f85dd0c29d5a96ad59a71e035ac4303c7b6a70a8d61280a15cdb966b32a218c7 scripts/performance/soak.py
|
|
||||||
fe6ff5bfd76ac7a2752a8a2e1e45c63e549c4a61c753008671e478df3ebff304 scripts/performance/telemetry.py
|
|
||||||
7c1ba46c623e1ade47b872fcdf4526a48ca85c56515dffeac626188b0357d23c scripts/performance/transport_clients.py
|
|
||||||
bd19431c89d7dcfe6743dffff9c4b16260a976d1936bdc4c11d4fcf6b3a7dda6 scripts/performance/transport_regression.py
|
|
||||||
1d747278da3fca093ed91c96d4a9480d0f711d0773d673906dbb56081635603c scripts/performance/wrk_report.lua
|
|
||||||
b5b3135449f6be4e97088b76ef7cf2428e2de3c7944c9cbbcb1edd2016c0fd4a scripts/preflight.py
|
|
||||||
3d6a0503afc25beab79ddc86864bcbb15248ded7368d33aa64321b8cdff59933 scripts/resolve_dependencies_0312.sh
|
3d6a0503afc25beab79ddc86864bcbb15248ded7368d33aa64321b8cdff59933 scripts/resolve_dependencies_0312.sh
|
||||||
20124173452b4514bad8850c1637569a9a191882478451e549bd16b432605fc1 scripts/runtime/acme_provider.py
|
20124173452b4514bad8850c1637569a9a191882478451e549bd16b432605fc1 scripts/runtime/acme_provider.py
|
||||||
e69813fb1d3a4cdcfb7243964cc323fc21a603087bf2e84188fa42b9d3cee4b5 scripts/runtime/acme_rfc2136.py
|
e69813fb1d3a4cdcfb7243964cc323fc21a603087bf2e84188fa42b9d3cee4b5 scripts/runtime/acme_rfc2136.py
|
||||||
174c8059625e3315a3d9fefe74c17f78ed850ff25ee15bfdabc148107dd344eb scripts/start_stream_backends.sh
|
174c8059625e3315a3d9fefe74c17f78ed850ff25ee15bfdabc148107dd344eb scripts/start_stream_backends.sh
|
||||||
046b4282550f5cb4e53906665c91c7442be70fc68fd0743466fd832582f7c979 scripts/test_stream_tcp.py
|
046b4282550f5cb4e53906665c91c7442be70fc68fd0743466fd832582f7c979 scripts/test_stream_tcp.py
|
||||||
f0b48ec0cc4411755cc0b4128eebce710375afa99344cb76198cf31074f8eb63 scripts/test_stream_udp.py
|
f0b48ec0cc4411755cc0b4128eebce710375afa99344cb76198cf31074f8eb63 scripts/test_stream_udp.py
|
||||||
fafd577ccbb58485c47de35fc125e5c64a3a7628ae40d01d4cd924fce2f414db scripts/tests/cache_age_policy_034_model.py
|
|
||||||
b75a381d1b41bfb2ea7d2ce09728249b5e1465f76d57e2c3f151af72a5ae4d03 scripts/tests/cache_age_transform_034_live.py
|
|
||||||
dbab8d5457546a1ffe8233514ee94225a73974fe5d6f3b1f5fadd06885fa228c scripts/tests/cache_hot_034_static_test.py
|
|
||||||
6cb79535f742d1a41685f486688dc2f1330ea362e3da1a15d4fa32561094282f scripts/tests/cache_scope_034_static_test.py
|
|
||||||
efb7a1fde2c74f30cd4a1abb88ae07c1cac7378bb33685dd09d52bb30849daac scripts/tests/dashboard_chart_headers_0335_browser_test.py
|
|
||||||
eb2c9e5c968938b6fbe4675b547ba1f537519878b9095e1fea133f4d83f96fc8 scripts/tests/dashboard_chart_typography_0337_browser_test.py
|
|
||||||
6a908fd434d2a8a80adfd06a66d4b83c4174e448d453a8c10b1812475d036677 scripts/tests/dashboard_chart_width_0333_browser_test.py
|
|
||||||
252b401c29e2db50e105d7fb044bcae14752748261a67da1a4025d71bfa2608c scripts/tests/dashboard_cpu_autoscale_0335_browser_test.py
|
|
||||||
41a2b3765006c138324ae1312f5c1edfed6ce48051230f85779e8d2aad34d2a3 scripts/tests/dashboard_cpu_fixed_scale_0335_browser_test.py
|
|
||||||
de583adde771e0d6f64ea0d5aed45b1c942a7c78b7193598eeeb77fff00c2803 scripts/tests/dashboard_layout_0331_browser_test.py
|
|
||||||
1c813c78dfe96ba4d2c9470c93dd24e81be06e6485ebdc39cbfa30f8d32aa0fe scripts/tests/dashboard_metric_visuals_ui_test.mjs
|
|
||||||
f14626c5ec006477062b496bdc0025caa3366c7037ee598e15cd55e29041d58f scripts/tests/dashboard_modal_0333_browser_test.py
|
|
||||||
f934f5ce66105a1394b93761a6081a51133837ac038f5faf67a236f676b15c1b scripts/tests/dashboard_overview_labels_0337_browser_test.py
|
|
||||||
df87420f16c418aa17a8ba8808e506095681446d32a7d76ae0244222ed37928a scripts/tests/dashboard_presets_sql_test.py
|
|
||||||
df1457797241aab114a5671ce3f0229815dd29d06ab6f6254c0a1048576af6e1 scripts/tests/dashboard_resizing_0333_browser_test.py
|
|
||||||
149910e6e8d569443a724fd21701e2fd8f6b5e360991d4f710f5d21364402bce scripts/tests/dashboard_resume_rate_charts_ui_test.mjs
|
|
||||||
3868dc7b241f1cf15bc297104e29449ec3e35f75cb66078439bc4b5c4ee11040 scripts/tests/dashboard_stable_scales_0336_browser_test.py
|
|
||||||
fa42ec0475e4398d647b3c094309b4332cd2b5649db223cae0b8e821d6b44a28 scripts/tests/dashboard_title_color_0337_browser_test.py
|
|
||||||
3c1b2bda6410d3348b0f5a05c70494cca9636339bed8217fc4abc0a30981faae scripts/tests/overview_samples_0330_ui_test.mjs
|
|
||||||
72185f59f31bf04a4e88d53fe2fca52509f3663cb5f40b1d3bc319a5f945ef23 scripts/tests/panel_html.mjs
|
|
||||||
b0136544af717b75df2cb107900ea65c3f945fdd0040369dbb93919008fb954c scripts/tests/panel_html.py
|
|
||||||
8d1903e2b17b3d59cd0a5bec3e298e468b3d88ed2dda8b724239cc45d8353c59 scripts/tests/progressbar_0332_browser_test.py
|
|
||||||
7fc86a34a47c6e1f0f9fe2748d0ea7f59e455f8bf4e5ad081e185356e43721b0 scripts/tests/proxy_1kb_opt3_static_test.py
|
|
||||||
ec05272ad8c14c7a0265a84cc7b6ebc0dca0651413106d655681681873b88fc8 scripts/tests/proxy_1kb_opt4_static_test.py
|
|
||||||
37306b175190aa72b143d263060fa6441244eb40b2bcb54c82071cc87d73b54d scripts/tests/proxy_1kb_opt5_static_test.py
|
|
||||||
2a6e75c05f3860acf65ea8132f0083904478b690145115ec017c120d70f26cca scripts/tests/proxy_hotpath_034_static_test.py
|
|
||||||
0c50def966cbc95c22339d1c506763fab4d9e5c176a161205ab88d79af5be920 scripts/tests/rps_hotpath_034_static_test.py
|
|
||||||
5be5cdc60324f4f0f6638640000b888a5babb0a9bbf29511c824a0ba38878c2a scripts/tests/sendfile_034_panel_browser_test.py
|
|
||||||
978789bfe89d43034dfebddba49b9b17daedb1be031732734f95375e56c78306 scripts/tests/sendfile_034_static_test.py
|
|
||||||
1bb99b148d0297235b707f16865ca077329246fc19ecbb20a143bdeb56a23f37 scripts/tests/sidebar_nav_0333_browser_test.py
|
|
||||||
12f2b18b0ed2171c49eb8cde2c2b88fc627054c2f574c04bd9847821b00cd4c1 scripts/tests/system_ram_meter_0332_browser_test.py
|
|
||||||
046b4282550f5cb4e53906665c91c7442be70fc68fd0743466fd832582f7c979 scripts/tests/test_stream_tcp.py
|
|
||||||
f0b48ec0cc4411755cc0b4128eebce710375afa99344cb76198cf31074f8eb63 scripts/tests/test_stream_udp.py
|
|
||||||
6fa1667a844b7bc2adaa78738339b58ac6c94a6d6e7be34573433dcd323ae809 scripts/tests/upgrade_performance_test.py
|
|
||||||
f0a2e3daa0a49aac54723cb5a281360e49a97c076281372c5cca2a85e096c02c scripts/tests/vhost_backend_popup_0337_browser_test.py
|
|
||||||
23cb1f790cd5cee1b76ecd85f6a2a44ef2e16b0840bf06beaa2ae13048063b2f scripts/tests/web_observability_0330_browser_test.py
|
|
||||||
3babdd0fd6e2998755ae88615112f6ea056f5c808245abb6de60fbda555594d1 scripts/tests/web_observability_0330_source_test.py
|
|
||||||
f55efa99245bbdbc98fe3202e52952d224ac3200612561580d52f6e251810049 scripts/tests/web_server_0330_ui_test.py
|
|
||||||
0dd1cef06a1c3e1546e2fb9b085bf369dfbed02107b9ab01b4fb4f3b894099c0 scripts/tests/web_server_0334_browser_test.py
|
|
||||||
22e880e98c7b9d700c7a4c02b9b1d543e5dbd1f4ecdca42813a1083165b32a19 scripts/version.py
|
22e880e98c7b9d700c7a4c02b9b1d543e5dbd1f4ecdca42813a1083165b32a19 scripts/version.py
|
||||||
9bf4e4a61824f82b773b4e8ffa0909b9cf0a42bfc14d695bd64da240cc634a61 src/acl.rs
|
9bf4e4a61824f82b773b4e8ffa0909b9cf0a42bfc14d695bd64da240cc634a61 src/acl.rs
|
||||||
63efab1e95d41134fc30cc57320c32810bbae9f81a640bdb24b1be2059e63d8a src/acl/dynamic.rs
|
63efab1e95d41134fc30cc57320c32810bbae9f81a640bdb24b1be2059e63d8a src/acl/dynamic.rs
|
||||||
@@ -222,7 +151,7 @@ b4a2a068acfa1143b0032b9cd072c19cf3fdda2580f0b932d11d6368cd2ab3ad src/admin/cert
|
|||||||
51f46a2cb3167c4e3a784dfbb59b8fe0e9d5d3d51365b26699498e31d0afb6a9 src/admin/certificates.rs
|
51f46a2cb3167c4e3a784dfbb59b8fe0e9d5d3d51365b26699498e31d0afb6a9 src/admin/certificates.rs
|
||||||
b5f6d49f1add583cfc8cd4aa36ffe4db56dac4de3b151fe8239fe42f507019cc src/admin/dashboard_presets.rs
|
b5f6d49f1add583cfc8cd4aa36ffe4db56dac4de3b151fe8239fe42f507019cc src/admin/dashboard_presets.rs
|
||||||
5e150d4ef0413a2f92db5b6631669933b84d4b4796633d01476c4b39188cd05c src/admin/documentation.rs
|
5e150d4ef0413a2f92db5b6631669933b84d4b4796633d01476c4b39188cd05c src/admin/documentation.rs
|
||||||
958546c43c05bf4bed8121b24db3b3096f9243f3c56a71e1762592b755684013 src/admin/documentation/documents.rs
|
3bc1425c0dd142471a48391474ab2551de7d6f2b788280ed4788ab94a435c712 src/admin/documentation/documents.rs
|
||||||
67b901485d5858d5b5bf78a6260035db32241531d99481754c3533fdf404facb src/admin/documentation/tests.rs
|
67b901485d5858d5b5bf78a6260035db32241531d99481754c3533fdf404facb src/admin/documentation/tests.rs
|
||||||
1ce779b84aa8f47bdb3c34f83593c1f2f06d2235f0319847096cfcd035434168 src/admin/file_log_search.rs
|
1ce779b84aa8f47bdb3c34f83593c1f2f06d2235f0319847096cfcd035434168 src/admin/file_log_search.rs
|
||||||
ab18acf6fb4ac86fb22635b9219a12dac76d24952c8f15538486267a9b4b7855 src/admin/file_logs.rs
|
ab18acf6fb4ac86fb22635b9219a12dac76d24952c8f15538486267a9b4b7855 src/admin/file_logs.rs
|
||||||
@@ -246,7 +175,7 @@ a40adfbaa908da3ea726d7aa3d45394a70a6a55b6fc0063270ec4d2a2f2026a3 src/admin/sess
|
|||||||
4b8a0483ede651d08489a4d6e5bb616f06423094bbbefcee70b8755d65c04398 src/admin/status_live.rs
|
4b8a0483ede651d08489a4d6e5bb616f06423094bbbefcee70b8755d65c04398 src/admin/status_live.rs
|
||||||
c97881b17b7a42511868b0e48e2f25aab6fb5b613d538f5e1f1967508454e486 src/admin/status_system.rs
|
c97881b17b7a42511868b0e48e2f25aab6fb5b613d538f5e1f1967508454e486 src/admin/status_system.rs
|
||||||
cadd7dca29b9aad8c77c7bc23dd249ea37a15adfa4d636a8ef00273a91ce35dc src/admin/tail.rs
|
cadd7dca29b9aad8c77c7bc23dd249ea37a15adfa4d636a8ef00273a91ce35dc src/admin/tail.rs
|
||||||
ccea3b72782aa9d36e41b89f40a755289d285692155c27cc81a0968f8780fe2a src/admin/tests.rs
|
58c9dbabfcfe37ce41d18e38777ead17781da91b3fc97bff19073fa09b262bd2 src/admin/tests.rs
|
||||||
48fcd818c522fe80f2c7b2e218f87de6a7772ffd0a24e93502780b03ec78e335 src/admin/web/acme.js
|
48fcd818c522fe80f2c7b2e218f87de6a7772ffd0a24e93502780b03ec78e335 src/admin/web/acme.js
|
||||||
c0d08c5b8fc13dec9bf53fd842d224b9df3874fc3bdf31413e3ab182a8cec2aa src/admin/web/app/backend-dashboard.js
|
c0d08c5b8fc13dec9bf53fd842d224b9df3874fc3bdf31413e3ab182a8cec2aa src/admin/web/app/backend-dashboard.js
|
||||||
a085ee336e6669229f3ad2f74536f19bd088412ca79bcaff30dff293f1e1baf9 src/admin/web/app/chart-scales.js
|
a085ee336e6669229f3ad2f74536f19bd088412ca79bcaff30dff293f1e1baf9 src/admin/web/app/chart-scales.js
|
||||||
@@ -416,11 +345,11 @@ f2a8c2a98f4ccfb708f9d75a48ec7b351232be9b8a4364b7e63ba14d28eead7c src/http3/sett
|
|||||||
a31db7eac12b578b1eb5a74fae98bbe36312da95b7d7173135dbc41fc454da21 src/http3/tests.rs
|
a31db7eac12b578b1eb5a74fae98bbe36312da95b7d7173135dbc41fc454da21 src/http3/tests.rs
|
||||||
68fb5ab6a889995a66ad12cb6acf88c5d04f0aca572e2ae15f1a9cd171a5f24b src/lib.rs
|
68fb5ab6a889995a66ad12cb6acf88c5d04f0aca572e2ae15f1a9cd171a5f24b src/lib.rs
|
||||||
2551189a3e5701371026c9abd2c5d44d37495f28fb67e220180421bfd315e3dd src/logging.rs
|
2551189a3e5701371026c9abd2c5d44d37495f28fb67e220180421bfd315e3dd src/logging.rs
|
||||||
18c8d74c21a47f234be93551d7264b1f8f8c00d1bbfbe555d91618c58cea0740 src/logging/error_runtime.rs
|
21ddb01b8f9c7fab60840d18103ef8de4716ff0c890cc1525b472b69af7916ec src/logging/error_runtime.rs
|
||||||
bc3873c6f22d1ee7fa76ae586547c9867e6eeaec8266e3ed1f529c87108df2b6 src/logging/format.rs
|
42b88cefef603fafa9eec3ba83a53069f01f571e98cf923210a6fe605e008e90 src/logging/format.rs
|
||||||
f5815d778cc1785723faed4abb15f9880f22b6524b5289fe1e7158905aac0986 src/logging/model.rs
|
f5815d778cc1785723faed4abb15f9880f22b6524b5289fe1e7158905aac0986 src/logging/model.rs
|
||||||
42fef80af50735cd566c6801dad86ba81be98cf4a8ec7d45c2ea26f53f556a22 src/logging/runtime.rs
|
b90ee5c7badacd0141fba99a8109fced785a11abe11675c8687fa35e68f5e7b1 src/logging/runtime.rs
|
||||||
7ee1eb1d3b52b4c14303f8ca14a1924d4d6bbe75b2e43ce3ca47d7958f792c7d src/logging/tests.rs
|
c83a7b84a9d7226c200394c351c4b3d9c417869828c01a35d4b1297670ab50f2 src/logging/tests.rs
|
||||||
3836ed14b6710e183f048ee4d7807a35366c27a27338057fe4a2811129b63302 src/main.rs
|
3836ed14b6710e183f048ee4d7807a35366c27a27338057fe4a2811129b63302 src/main.rs
|
||||||
fc8e99eac83c083d65101f6402aacbcc8392d66508c31566a50de3ebcb12e7da src/main/admin_runtime.rs
|
fc8e99eac83c083d65101f6402aacbcc8392d66508c31566a50de3ebcb12e7da src/main/admin_runtime.rs
|
||||||
42a9197475823f67c1ca762951ede5d7a2f10124bd6ed4d1a3365f4297be894c src/main/bootstrap.rs
|
42a9197475823f67c1ca762951ede5d7a2f10124bd6ed4d1a3365f4297be894c src/main/bootstrap.rs
|
||||||
@@ -466,7 +395,7 @@ cc7fe83f42acb19d79abe83107427cdc6b0ed14083702f4fe66af22f7cdfc4cc src/proxy/head
|
|||||||
55133ceace2e5262d9c54fa41bea099a27b817d1e09c1cd0e2493fd9cee4da38 src/proxy/redirect_helpers.rs
|
55133ceace2e5262d9c54fa41bea099a27b817d1e09c1cd0e2493fd9cee4da38 src/proxy/redirect_helpers.rs
|
||||||
2280841494c3e14ccc5e3f02b026fe3c4328a3dab04ab449b9286bb69e39c028 src/proxy/request_id.rs
|
2280841494c3e14ccc5e3f02b026fe3c4328a3dab04ab449b9286bb69e39c028 src/proxy/request_id.rs
|
||||||
b9a2307c6aa7f16613d935508a61574dfee964020b1b5be86293df2d684615d8 src/proxy/request_id/tests.rs
|
b9a2307c6aa7f16613d935508a61574dfee964020b1b5be86293df2d684615d8 src/proxy/request_id/tests.rs
|
||||||
e8d1210a6b8e8c6a388d016d22e7c18255cf737c5583124b35cd50bd2592f23c src/proxy/tests/authority_tests.rs
|
3db4188e2cc6d69fdff1b817cc15d0e6b18bda18d4e4729d37c05736c7d8cad6 src/proxy/tests/authority_tests.rs
|
||||||
1aa9bbbfb2659facdaeaf185e11221b1c8cd5fe6e23c7c1dacd3aa55e3db4fcb src/proxy/tests/cache_fast_path_tests.rs
|
1aa9bbbfb2659facdaeaf185e11221b1c8cd5fe6e23c7c1dacd3aa55e3db4fcb src/proxy/tests/cache_fast_path_tests.rs
|
||||||
0f2eacadf1447db8aa1c1a78005aa262e0f747096bfe7e0a0e41a2cd31a0d17f src/proxy/tests/h3_navigation_tests.rs
|
0f2eacadf1447db8aa1c1a78005aa262e0f747096bfe7e0a0e41a2cd31a0d17f src/proxy/tests/h3_navigation_tests.rs
|
||||||
d134b0eb617f82f9971b3f3461379ea37c57bd0079bf54748d0c434b2ef1b475 src/proxy/tests/http_invalidation_tests.rs
|
d134b0eb617f82f9971b3f3461379ea37c57bd0079bf54748d0c434b2ef1b475 src/proxy/tests/http_invalidation_tests.rs
|
||||||
|
|||||||
+1
-1
@@ -21,7 +21,7 @@ additional requests receive 429. Ambiguous duplicate control headers and
|
|||||||
request bodies are rejected. See
|
request bodies are rejected. See
|
||||||
[CACHE](CACHE.md#http-ban--purge-from-authorized-client-ips) for request examples,
|
[CACHE](CACHE.md#http-ban--purge-from-authorized-client-ips) for request examples,
|
||||||
scoping, confirmation and reverse-proxy requirements, and
|
scoping, confirmation and reverse-proxy requirements, and
|
||||||
[0.31.0 security review](SECURITY_AUDIT_0.31.0.md) for audit findings. The authenticated
|
[security review](SECURITY_AUDIT.md) for audit findings. The authenticated
|
||||||
`/api/cache/invalidate` endpoint remains unchanged.
|
`/api/cache/invalidate` endpoint remains unchanged.
|
||||||
|
|
||||||
## Browser documentation and anonymous surface
|
## Browser documentation and anonymous surface
|
||||||
|
|||||||
+1
-1
@@ -232,7 +232,7 @@ separate from the administrator-only selective invalidation API. If an
|
|||||||
application changes a GET resource after POST/PUT/PATCH/DELETE, use origin
|
application changes a GET resource after POST/PUT/PATCH/DELETE, use origin
|
||||||
freshness/validators or one of the explicit invalidation paths; RustProxy
|
freshness/validators or one of the explicit invalidation paths; RustProxy
|
||||||
does not guess the affected URLs. For exposure review and test scenarios, see
|
does not guess the affected URLs. For exposure review and test scenarios, see
|
||||||
[0.31.0 security audit](SECURITY_AUDIT_0.31.0.md).
|
[security audit](SECURITY_AUDIT.md).
|
||||||
|
|
||||||
## Eligibility and freshness
|
## Eligibility and freshness
|
||||||
|
|
||||||
|
|||||||
@@ -54,15 +54,12 @@ Native Rust test file: `src/cache/tests/hot_hit_034_tests.rs`.
|
|||||||
./dev.sh check
|
./dev.sh check
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
CARGO_INCREMENTAL=0 ./dev.sh test
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
CARGO_INCREMENTAL=0 cargo build --release --locked
|
||||||
python3 scripts/tests/cache_hot_034_static_test.py
|
|
||||||
cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
|
cargo test --release --locked --lib manual_hot_lookup_microbenchmark -- --ignored --nocapture
|
||||||
```
|
```
|
||||||
|
|
||||||
For real HTTP RPS use the supplied process regression and same-host nginx
|
For real HTTP RPS, compare two compiled RustProxy 0.34.0 binaries
|
||||||
benchmark (`scripts/performance/README.md`) with **identical** configs. The
|
with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
|
||||||
standard nginx comparator benchmark explicitly disables the cache; it cannot
|
A cache-disabled nginx comparison cannot measure this HOT-specific improvement.
|
||||||
measure this HOT-specific improvement. Compare two compiled RustProxy 0.34.0
|
|
||||||
binaries with the same cache-enabled, cache-warmed vhost and fixed no-query URL.
|
|
||||||
Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
|
Check `X-Cache: HIT`, body SHA-256, exact `Cache-Status`, transform marker, Age,
|
||||||
ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
|
ETag, p95/p99, CPU/request, memory and warm-up/eviction counts while measuring.
|
||||||
|
|
||||||
|
|||||||
@@ -1,147 +0,0 @@
|
|||||||
# RustProxy 0.34.0 DEV — 1 KiB lock contention follow-up
|
|
||||||
|
|
||||||
## Goal and baseline
|
|
||||||
|
|
||||||
**Priority 1: reverse proxy, HTTP/1.1 keep-alive, uncompressed 1,040-byte
|
|
||||||
responses, cache OFF.** Aiming for approximately 100,000 RPS and at least 80%
|
|
||||||
of nginx under the same pinned-CPU benchmark. Priority 2: local static WWW,
|
|
||||||
measured separately. No feature removals and **no version bump**.
|
|
||||||
|
|
||||||
The user supplied a **new executable** (SHA-256
|
|
||||||
`1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa`,
|
|
||||||
`rust-proxy 0.34.0`), apparently built from the prior DEV revision. It passed
|
|
||||||
151 actual black-box HTTP tests with 0 failures and 1 skipped optional zstd
|
|
||||||
decoder. This runtime coverage belongs to the **supplied executable only**,
|
|
||||||
not to the freshly edited source below. Neither `cargo` nor `rustc` is
|
|
||||||
available in the editing environment; the new revision has NOT been compiled.
|
|
||||||
|
|
||||||
## Modified source
|
|
||||||
|
|
||||||
### `Cache::record` and `Cache::record_reason`
|
|
||||||
|
|
||||||
Before, every HIT, MISS and BYPASS acquired the same `Store` mutex used by the
|
|
||||||
HOT object store, exact LRU and cache inventory. With cache disabled, a 1 KiB
|
|
||||||
request still contested that mutex to increment `cache_disabled`.
|
|
||||||
|
|
||||||
`src/cache/stats.rs` now holds bounded per-(scope,domain) statistics separately:
|
|
||||||
|
|
||||||
- An `ArcSwap`-published immutable, borrowed-lookup BTreeMap of domain handles.
|
|
||||||
**The normal path does not lock or allocate**, even for BYPASS. New domains
|
|
||||||
alone use a registration mutex and a copy-on-first-visit registry update.
|
|
||||||
- Atomic counts for HIT, MISS, BYPASS, stored objects and saved body bytes.
|
|
||||||
The common static reasons (`cache_disabled`, `cache_hit`, `cache_miss`,
|
|
||||||
`request_cookie`) use direct atomic counters. Other reason names use another
|
|
||||||
immutable published index and atomic counters; they are registered once.
|
|
||||||
- Hard cardinality remains **4096 (scope, domain) pairs**. When full, new
|
|
||||||
unregistered domains are skipped as before. Scope purge preserves stats.
|
|
||||||
Only a revision-bearing global purge resets the per-domain registry.
|
|
||||||
- The dashboard still exposes the same `DomainCounters`, reason strings,
|
|
||||||
saved byte counts, ratio and per-vhost/object views. Counter snapshots are
|
|
||||||
quiescent-exact, but under concurrent requests individual fields can reflect
|
|
||||||
slightly different instants, consistent with other atomic metrics.
|
|
||||||
- HOT object and COLD representation/LRU access, barriers, memory accounting,
|
|
||||||
cache statuses, hit policy, retry and backend security paths are unchanged.
|
|
||||||
|
|
||||||
Trade-offs: a first-visit registration copies the bounded domain index and
|
|
||||||
allocates one counter set; this is not an adaptive, unbounded cache. The hot
|
|
||||||
path's ArcSwap guard and lookup still cost CPU. Measure that cost in profiles;
|
|
||||||
compare under high-cardinality traffic to identify any excessive registration
|
|
||||||
cost before production acceptance.
|
|
||||||
|
|
||||||
### Traffic analytics queue saturation
|
|
||||||
|
|
||||||
Previously `Exchange::drop` cloned the vhost, route, method, path, IP, user
|
|
||||||
agent and country before trying to enqueue the `Sample`. If the bounded
|
|
||||||
2,048-element MPSC queue was full, all those clones were discarded.
|
|
||||||
|
|
||||||
`Analytics::push_lazy` now acquires the same bounded Tokio queue permit with
|
|
||||||
`try_reserve` first and creates the sample only after a successful reservation.
|
|
||||||
Rejections increment `queue_dropped` exactly as before. Neither the analytics
|
|
||||||
worker, counters/top lists, reset behavior nor request log path is removed.
|
|
||||||
This improves allocation costs **on overflow** but does not turn dropped
|
|
||||||
analytics into delivered samples; monitor the existing drop counter.
|
|
||||||
|
|
||||||
### Per-vhost duration histogram atomics
|
|
||||||
|
|
||||||
Previously `VhostMetrics::complete` incremented every cumulative latency bucket
|
|
||||||
whose threshold covered the observation and the `+Inf` bucket. For an ordinary
|
|
||||||
short response this meant **11 shared atomic RMWs** for the histogram alone,
|
|
||||||
on top of existing exchange/status/byte/latency metrics.
|
|
||||||
|
|
||||||
It now increments **exactly one exclusive bucket** chosen from the same 10
|
|
||||||
latency boundaries. Both the regular vhost and Docker-vhost Prometheus exporters
|
|
||||||
compute the original cumulative `le` buckets, `+Inf` and `_count` from the 11
|
|
||||||
exclusive buckets **at scrape time**. The metric names, labels, boundaries,
|
|
||||||
body-lifetime timing and measurement collection remain intact. The cumulative
|
|
||||||
snapshot may be concurrent with request completions, as before. Added boundary
|
|
||||||
unit tests for exact thresholds and long-duration `+Inf`.
|
|
||||||
|
|
||||||
This is source-only: neither the throughput benefit nor compiled Rust behavior
|
|
||||||
has been verified in the editing environment. Review exporter parity and
|
|
||||||
under-load histogram correctness on the newly compiled executable.
|
|
||||||
|
|
||||||
## Verification and untested claims
|
|
||||||
|
|
||||||
- PASS: source preflight, cache/HOT/LB/RPS/histogram/static source guards and <=500-line
|
|
||||||
production Rust modules.
|
|
||||||
- PASS: SQLite migration + preset constraints, upgrade/lock and source
|
|
||||||
replacement tests.
|
|
||||||
- PASS: compiled user-supplied binary: 151 real HTTP checks, 0 failed,
|
|
||||||
1 optional zstd skip. This binary predates the source changes above.
|
|
||||||
- NOT RUN: Rust compile, Rust unit tests, native regressions or RPS after
|
|
||||||
rebuilding the new source. No numerical speedup attributed to this patch.
|
|
||||||
- Current diagnostic A/B on the user-supplied binary may use
|
|
||||||
`scripts/performance/ab_1kb_dev.py` and must remain labelled *old vs already
|
|
||||||
supplied binary*, **not** old vs newly edited source. The `wrk` 30 s x 7
|
|
||||||
pinned-CPU benchmark remains the production acceptance gate.
|
|
||||||
|
|
||||||
## Repeat on a compiler-enabled benchmark host
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh doctor
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
||||||
python3 scripts/performance/regression.py \
|
|
||||||
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
||||||
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-counter-http
|
|
||||||
python3 scripts/performance/lb_regression.py \
|
|
||||||
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
||||||
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-counter-lb
|
|
||||||
python3 scripts/performance/transport_regression.py \
|
|
||||||
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
||||||
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-counter-transport
|
|
||||||
python3 scripts/performance/benchmark.py \
|
|
||||||
--binary previous=/absolute/path/to/supplied-rust-proxy \
|
|
||||||
--binary edited="$(realpath target/release/rust-proxy)" \
|
|
||||||
--nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
|
|
||||||
--out /tmp/rustproxy034-counter-benchmark \
|
|
||||||
--modes proxy --sizes small --concurrency 16,64,128 \
|
|
||||||
--duration 30 --warmup 5 --repeats 7
|
|
||||||
```
|
|
||||||
|
|
||||||
Explicitly set `max_idle_per_host=64` on **both** before/after vhosts, with the
|
|
||||||
same TLS/H1/H2/health/cache/logging/settings. Verify 1,040 bytes and SHA-256,
|
|
||||||
no failures, latency percentiles, per-request CPU, RSS, file descriptors,
|
|
||||||
backend connects and all log/traffic drop counters. Test concurrent global
|
|
||||||
purges and per-vhost purges during traffic and verify the stats reset behavior
|
|
||||||
and cache HIT/MISS/BYPASS/reason distribution. Run long soaks and both cold/HOT
|
|
||||||
cache tests. The 100k RPS / nginx 80% goal is still **OPEN**.
|
|
||||||
|
|
||||||
## Build fix after first compiler feedback (OPT2-FIX1)
|
|
||||||
|
|
||||||
The first user build of OPT2 failed with six Rust `E0753` diagnostics because
|
|
||||||
`src/cache/stats.rs` starts with `//!` *inner module* documentation but is
|
|
||||||
injected via `include!("stats.rs")` after other items in `src/cache/mod.rs`.
|
|
||||||
The six documentation lines are now ordinary `//` comments; there is no
|
|
||||||
behavior change. Three unnecessary `mut` bindings in
|
|
||||||
`src/cache/dashboard.rs` were also removed (warnings only). Version remains
|
|
||||||
0.34.0; no performance or feature semantics have changed in this fix.
|
|
||||||
|
|
||||||
The compiler-enabled host should rerun `./dev.sh check`,
|
|
||||||
`CARGO_INCREMENTAL=0 ./dev.sh test` and an optimized release build, then
|
|
||||||
performance benchmarks. This editing environment still lacks `cargo` and
|
|
||||||
`rustc`, so the fix has been statically checked, **not compiled** here.
|
|
||||||
@@ -1,137 +0,0 @@
|
|||||||
# RustProxy 0.34.0 DEV: 1 KiB reverse-proxy throughput audit
|
|
||||||
|
|
||||||
## Goal and evidence boundaries
|
|
||||||
|
|
||||||
Priority 1 is short (~1 KiB) HTTP reverse-proxy responses: aim for approximately
|
|
||||||
100,000 requests/s, and eventually at least 80% of nginx under the **same**
|
|
||||||
controlled benchmark, without disabling authentication, ACLs, caching semantics,
|
|
||||||
rate limits, traffic accounting, request IDs, logs, body streaming or protocol
|
|
||||||
support. Priority 2 is local WWW serving, to be tuned and measured separately.
|
|
||||||
This is a development revision; Cargo/package version remains **0.34.0**.
|
|
||||||
|
|
||||||
**This source is not compiled in the current environment.** Neither `cargo`
|
|
||||||
nor `rustc` is available. Only the user's pre-existing executable (SHA-256
|
|
||||||
`453757748455d7b9b50a6140e322ee4d4cf4a030eea29318686f2e28c594b644`)
|
|
||||||
was run. Live configuration A/B tests therefore substantiate a **configuration
|
|
||||||
bottleneck**, not the throughput of this edited source.
|
|
||||||
|
|
||||||
## Real baseline experiment (existing executable only)
|
|
||||||
|
|
||||||
Disposable loopback nginx upstream, 1,040-byte body, HTTP/1.1 persistent
|
|
||||||
frontend connections, caching disabled, identity response encoding, four Tokio
|
|
||||||
workers, ApacheBench (`ab -k -n 12000`), 16/64 concurrent frontend clients.
|
|
||||||
Checks outside the timed intervals verified HTTP 200, exact 1,040 bytes and
|
|
||||||
SHA-256 `62a08d75cdde0416c883d72458385b5e56ae8c14466934ad784587ddb93a548b`;
|
|
||||||
0 failed requests in the shown samples. The host has shared CPU resources;
|
|
||||||
results vary and are not a pinned-CPU or production acceptance benchmark.
|
|
||||||
|
|
||||||
| Existing executable configuration | 16 clients, RPS | 64 clients, RPS |
|
|
||||||
|---|---:|---:|
|
|
||||||
| max_idle_per_host = 4 | 18,226 | 17,935 |
|
|
||||||
| max_idle_per_host = 64 | 30,906 | 24,872 |
|
|
||||||
| Relative change (configuration only) | +69.6% | +38.7% |
|
|
||||||
|
|
||||||
A separate repeat experiment at 64 clients (two runs per variant, in order)
|
|
||||||
produced: `idle4` 14,028 / 14,834; `idle16` 22,205 / 24,711;
|
|
||||||
`idle32` 25,580 / 29,088; `idle64` 36,521 / 33,847;
|
|
||||||
`idle64 + recycle 250,000` 31,168 / 29,753;
|
|
||||||
`idle32_again` 34,798 / 29,194; `idle64_again` 34,677 / 29,965;
|
|
||||||
`idle4_again` 23,503 / 23,514 RPS. The order drift warns against
|
|
||||||
reporting a single percentage as a robust speedup. Raising generation
|
|
||||||
recycling from 25,000 to 250,000 did not establish any improvement; its
|
|
||||||
original limit is retained. All shown variants reported zero HTTP failures.
|
|
||||||
|
|
||||||
A preliminary nginx comparison gave ~75k/117k RPS at 16/64 clients, but later
|
|
||||||
nginx samples were ~50k/93k on the shared host. Do not compute a definitive
|
|
||||||
"% nginx" from these non-isolated runs. The 100k/80% goals remain **OPEN**.
|
|
||||||
|
|
||||||
## Source modifications
|
|
||||||
|
|
||||||
| Location | Change | Preserved behavior / caveat |
|
|
||||||
|---|---|---|
|
|
||||||
| `src/balance/model.rs` | New vhost default `max_idle_per_host`: **4 -> 64** | Configured values **1..64** remain validated; existing explicitly saved `4` stays `4`. Disabled keepalive, 30 s idle timeout, 25k-request and 3,600 s recycling rules are unchanged. Increased idle sockets use more file descriptors and memory. |
|
|
||||||
| `src/admin/web/vhosts/editor.js` | Match the new `64` default when the field is absent | Existing values, limits and all editor settings remain unchanged. |
|
|
||||||
| `src/proxy/body.rs` | At `Exchange::drop`, use exclusive `Mutex::get_mut()` rather than lock/unlock; move owned `route`, `client_ip`, `country`, `upstream_error` into final observation where possible | Exactly the same log fields, analytics sample, counters, permit lifetime, errors and websocket/H3 fields. Analytics still gets a clone when a vhost sample is queued; history still receives the original. |
|
|
||||||
| `src/proxy/headers.rs` | Skip reserved cookie sanitization immediately if the corresponding `Cookie`/`Set-Cookie` header is absent | Full sanitization, admin-cookie separation and restoration restrictions still apply when the header is present. |
|
|
||||||
| `src/balance/tests.rs`, `src/proxy/body/tests.rs`, `src/proxy/headers/tests.rs` | Add compatibility, completion and cookie regression assertions | These Rust tests **require** a compiler before they count as run. |
|
|
||||||
| `scripts/tests/*hotpath*` | Update source-only guards for the new **bounded** default | Static assertions do not prove functional or runtime performance equivalence. |
|
|
||||||
|
|
||||||
**Persistence note:** old vhosts may store `keepalive.max_idle_per_host=4`
|
|
||||||
explicitly; the new code default does not rewrite them. To apply the measured
|
|
||||||
configuration improvement before rebuilding, change that option to `64` in the
|
|
||||||
relevant vhost and verify actual idle socket/FD usage. Where hundreds of origins
|
|
||||||
share a host, test `16` or `32` if the FD/memory trade-off is preferable.
|
|
||||||
|
|
||||||
## Bottlenecks to profile next, not silently disable
|
|
||||||
|
|
||||||
1. `src/cache/dashboard.rs::Cache::record`: obtains a **shared cache-store
|
|
||||||
mutex on every outcome**, including `BYPASS` when cache is disabled through
|
|
||||||
`src/proxy/forward/cache_lookup.rs`. This may compete with HOT cache
|
|
||||||
operations and admin dashboard reads; profile `pthread_mutex`/futex and
|
|
||||||
lock hold times. Candidate: separate bounded, sharded domain/reason
|
|
||||||
counters from the HOT object store while preserving exact dashboard totals,
|
|
||||||
reason labels and reset semantics.
|
|
||||||
2. `src/logging/format.rs::Logs::push`: per-request bounded history
|
|
||||||
`try_lock`, potentially causing recorded log drops under high concurrency.
|
|
||||||
Preserve *existing* drops/accounting, live subscribers and external sinks.
|
|
||||||
Candidate: worker-local bounded batches or sharded history with deterministic
|
|
||||||
bounded merge; validate log order, recent history, retention and drops.
|
|
||||||
3. `src/proxy/body.rs::Exchange::drop` + `src/traffic.rs::Analytics::push`:
|
|
||||||
request-sized log strings, sampling queue sends, atomics and lifecycle
|
|
||||||
observations. Profile allocations and CPU per completed request. Candidate:
|
|
||||||
compact shared immutable identity fields or conditional cloning solely
|
|
||||||
where consumers require ownership, without removing logs or analytics.
|
|
||||||
4. `src/proxy/forward/route_cache.rs`: request-target path copies,
|
|
||||||
header policy and several route/cache eligibility steps. Measure with
|
|
||||||
sampling `perf record -g` before extracting additional fast paths; preserve
|
|
||||||
query handling, canonical host, ACL, authentication, retries and WebSockets.
|
|
||||||
5. `src/upstream/*`: profile H1 pool misses, TCP connect rate, Hyper
|
|
||||||
connection reuse and generation swaps. Increasing recycle thresholds or
|
|
||||||
changing HTTP/2/H2C modes without an isolated performance case is not
|
|
||||||
recommended.
|
|
||||||
6. **Separate WWW priority:** benchmark short static objects independently;
|
|
||||||
existing `sendfile: auto` applies to eligible cleartext H1 objects >=64 KiB
|
|
||||||
and cannot accelerate 1 KiB bodies.
|
|
||||||
|
|
||||||
No lock removal, tracing/log disabling, caching-policy weakening, bypass of
|
|
||||||
security checks, or transformation of complete bodies into buffered copies is
|
|
||||||
proposed without a test proving semantic equivalence.
|
|
||||||
|
|
||||||
## Acceptance gate on a machine with Rust toolchain and isolated load generator
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh doctor
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
||||||
python3 scripts/performance/regression.py \
|
|
||||||
--binary after="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-1kb-regression
|
|
||||||
python3 scripts/performance/lb_regression.py \
|
|
||||||
--binary before=/absolute/path/to/old-rust-proxy \
|
|
||||||
--binary after="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-1kb-lb
|
|
||||||
python3 scripts/performance/transport_regression.py \
|
|
||||||
--binary before=/absolute/path/to/old-rust-proxy \
|
|
||||||
--binary after="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-1kb-transport
|
|
||||||
python3 scripts/performance/benchmark.py \
|
|
||||||
--binary before=/absolute/path/to/old-rust-proxy \
|
|
||||||
--binary after="$(realpath target/release/rust-proxy)" \
|
|
||||||
--nginx /usr/sbin/nginx --wrk /absolute/path/to/wrk \
|
|
||||||
--out /tmp/rustproxy034-1kb-benchmark \
|
|
||||||
--modes proxy --sizes small --concurrency 16,64,128 \
|
|
||||||
--duration 30 --warmup 5 --repeats 7
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the **same explicit** `max_idle_per_host=64` and all other vhost settings
|
|
||||||
for the *source A/B* test. Run 4 vs 64 as a separate *configuration A/B* test;
|
|
||||||
never attribute a settings difference to code. Verify full content checksums,
|
|
||||||
HTTP 2xx/4xx/5xx, p50/p95/p99, drop counters, backend connections,
|
|
||||||
FD/RSS trends, CPU/request and packet/TCP retransmit rates. Test H1/H2/H3,
|
|
||||||
TLS, WebSockets, sticky/rate/ACL, cache MISS/HIT/BYPASS, admin sessions, active
|
|
||||||
failover, cancellation, idle timeouts and a sustained memory/connection soak.
|
|
||||||
Profile the freshly **built** executable separately from timed RPS runs.
|
|
||||||
|
|
||||||
Until compilation and those gates pass, this is a **development source-only
|
|
||||||
candidate**, not a claim of reaching 100k RPS or 80% nginx.
|
|
||||||
@@ -1,88 +0,0 @@
|
|||||||
# RustProxy 0.34.0 DEV — 1,040-byte response, OPT3
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
Primary: uncached reverse proxy, HTTP/1.1 keep-alive, 1,040-byte body,
|
|
||||||
no compression, ACL/security/logging/analytics/counters fully operational.
|
|
||||||
Goal: ~100,000 RPS and >=80% of nginx on the **same isolated host**.
|
|
||||||
Secondary: local Web Server/static files, measured independently.
|
|
||||||
**Version remains exactly 0.34.0**; existing configuration and defaults are
|
|
||||||
unchanged; no functions, checks or logs are disabled.
|
|
||||||
|
|
||||||
## New code optimizations (relative to OPT2-FIX1)
|
|
||||||
|
|
||||||
1. `src/acl.rs` / `src/proxy/cache_helpers.rs`: borrow ordinary validated
|
|
||||||
routing paths using `Cow<str>` during immutable route selection. Calls
|
|
||||||
with escaped (`%xx`) or repeated separators still use the original strict
|
|
||||||
allocator/decoder. Invalid control bytes, dot segments, UTF-8 and ambiguous
|
|
||||||
encodings remain rejected; the original wire path is not rewritten.
|
|
||||||
2. `src/proxy/forward_entry.rs` plus cache/Web Service call sites: retain the
|
|
||||||
original `http::Uri` value so normal paths can remain borrowed during the
|
|
||||||
entire mutable forwarding/rewrite operation. Cache-refresh tasks that need
|
|
||||||
ownership still allocate explicitly and hold their own `String`.
|
|
||||||
3. `src/forwarding.rs`: a **direct, untrusted** TCP peer now obtains the same
|
|
||||||
seven forwarding headers without allocating hop-chain `Vec` structures.
|
|
||||||
Trust chains and the configured `trust_forwarded_proto` logic use the
|
|
||||||
unchanged full parser. IPv4/IPv6 RFC 7239 formatting and true listener
|
|
||||||
port/scheme are preserved; untrusted spoofed values are still ignored.
|
|
||||||
4. `src/proxy/forward_entry.rs`: when effective client IP is identical to
|
|
||||||
the transport peer, reuse the IP stored by observation. No redundant
|
|
||||||
`IpAddr::to_string()` and no per-exchange mutex write. Different/trusted or
|
|
||||||
IPv4-mapped addresses are still stored in canonical form.
|
|
||||||
5. `src/proxy/headers.rs`: plain single-valued `Connection: keep-alive`
|
|
||||||
avoids the temporary `Vec<HeaderName>` while all standard hop-by-hop fields
|
|
||||||
are removed; comma-separated, repeated and complex Connection fields keep
|
|
||||||
the original removal parser to prevent nominated-header leaks.
|
|
||||||
6. `src/proxy/request_id.rs`: preserve SHA-256, CSPRNG, per-process sequence,
|
|
||||||
unpredictable 64-char hexadecimal IDs and wire headers, but write the hex
|
|
||||||
digits into one preallocated byte buffer before UTF-8 verification.
|
|
||||||
7. `src/cache/stats.rs`: select the most frequent `cache_disabled` reason
|
|
||||||
counter directly. The same exact atomic totals, reason labels, 4096-domain
|
|
||||||
limit and reset behavior are retained.
|
|
||||||
|
|
||||||
The changes avoid repeated work in common cases; no numerical throughput
|
|
||||||
benefit has been measured for this new source. In particular, no cache, log,
|
|
||||||
metrics, retry or auth feature was switched off for a synthetic benchmark.
|
|
||||||
|
|
||||||
## Evidence and known limits
|
|
||||||
|
|
||||||
- Provided newly compiled *OPT2-FIX1* executable SHA-256:
|
|
||||||
`6037e98d059672b716f2e1411976bf305cac927901a6f6662d8b703829ff8196`.
|
|
||||||
- PASS **89** HTTP proxy/policy/cache black-box assertions, zero failures,
|
|
||||||
testing that supplied executable, **not** the new OPT3 code.
|
|
||||||
- Source checks, migration/upgrade checks and ZIP checks are separate
|
|
||||||
static/offline verification; they are not equivalent to cargo/test/live.
|
|
||||||
- A single interrupted diagnostic A/B sample at c=64, 3 s, same idle=64,
|
|
||||||
gave older binary 33,752 RPS and OPT2-FIX1 binary 33,556 RPS, zero failed
|
|
||||||
requests; do **not** infer a true performance difference from one sample.
|
|
||||||
- NOT RUN here: `cargo check`, `cargo test`, rebuilt OPT3 integration tests,
|
|
||||||
post-build A/B RPS, long soaking, real TLS/H2/H3 compatibility and 80%-of-nginx
|
|
||||||
acceptance. No Rust toolchain is available in this editing container.
|
|
||||||
- Resource/latency trade-offs (including log/analytics drops, RSS/FDs) require
|
|
||||||
the same measured and pinned server/peer/generator CPU sets for before/after.
|
|
||||||
|
|
||||||
## Rebuild and measure
|
|
||||||
|
|
||||||
From the unpacked `RustProxy/` directory:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
bash dev.sh
|
|
||||||
python3 scripts/performance/regression.py \
|
|
||||||
--binary new=/ABSOLUTE/PATH/NEW/rust-proxy --out /tmp/rustproxy-opt3-http
|
|
||||||
python3 scripts/performance/ab_1kb_dev.py \
|
|
||||||
--binary before=/ABSOLUTE/PATH/OPT2FIX1/rust-proxy \
|
|
||||||
--binary after=/ABSOLUTE/PATH/OPT3/rust-proxy \
|
|
||||||
--nginx /usr/sbin/nginx --ab /usr/bin/ab \
|
|
||||||
--out /tmp/rustproxy-opt3-1kb --concurrency 16,64 \
|
|
||||||
--duration 5 --repeats 3 --workers 4 --max-idle 64
|
|
||||||
```
|
|
||||||
|
|
||||||
For an acceptance-grade evaluation, run the `scripts/performance/benchmark.py`
|
|
||||||
paired before/after/nginx test with separate pinned CPU sets, a verified origin
|
|
||||||
and 30-second runs repeated 7 times. Report median and p95/p99 latency,
|
|
||||||
errors, CPU, RSS, file descriptors, dropped log/analytics events, and all HTTP
|
|
||||||
status/body hashes. Retest TLS/H2/H3/WebSocket, auth, cache semantics and
|
|
||||||
connection/backpressure limits. Compare the **rebuilt** OPT3 binary against
|
|
||||||
OPT2-FIX1 at exactly the same listener/backend settings and compiler profile.
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
# RustProxy 0.34.0 DEV / OPT4: HTTP/1.1 small-body follow-up
|
|
||||||
|
|
||||||
## Objective and validity
|
|
||||||
|
|
||||||
Primary goal: 1,040-byte reverse-proxy response, HTTP/1.1 keep-alive,
|
|
||||||
cache disabled; target about 100k RPS and at least 80% of nginx on the **same
|
|
||||||
isolated host**. Secondary goal: local static Web Server, benchmarked separately.
|
|
||||||
No version bump or feature, metric, logging, ACL, authentication, TLS,
|
|
||||||
HTTP/2/3, stream, retry or cache-policy bypass is introduced.
|
|
||||||
|
|
||||||
The **provided and compiled OPT3 binary** (SHA-256
|
|
||||||
`2efe273340f9cf92c5900c85c0bd1b775c87a9d1eaff1a10ec7635d268b080b1`)
|
|
||||||
passed 151 black-box HTTP assertions: 0 failures and 1 optional Zstandard
|
|
||||||
case skipped. This is evidence for the **input** binary, not for the new
|
|
||||||
OPT4 sources. OPT4 still requires a native Rust build, cargo tests and a
|
|
||||||
post-build benchmark. The diagnostic same-host CPU shares resources and is
|
|
||||||
not an acceptance benchmark.
|
|
||||||
|
|
||||||
## Code changes relative to OPT3
|
|
||||||
|
|
||||||
1. `src/proxy/forward/route_cache.rs`: store normal public path/query as
|
|
||||||
`Cow<'_, str>` referencing the independently retained, immutable original
|
|
||||||
URI. Without a path rewrite both public/origin suffixes stay borrowed;
|
|
||||||
strict encoded/rewrite handling remains an owned validated string.
|
|
||||||
Cache refresh tasks explicitly call `.to_string()` only when they actually
|
|
||||||
must retain an owned target across spawned tasks.
|
|
||||||
2. `src/proxy/upstream_helpers.rs`: skip allocating a request-body metering
|
|
||||||
adapter **only** when `Body::is_end_stream()` establishes that it contains
|
|
||||||
no DATA frames or trailers. Every nonempty/streaming request retains the
|
|
||||||
original metering, timeout, backend lease accounting and cancellation.
|
|
||||||
The response adapter still retains the exchange/inflight permit and handles
|
|
||||||
completion/response bytes for empty requests.
|
|
||||||
3. `src/proxy/types.rs`, `src/proxy/forward/upstream.rs`: consume owned
|
|
||||||
`http::Uri` when preparing the upstream target. HTTP/1.1 returns it directly;
|
|
||||||
H2 takes the same URI into parts for strict authority rewriting. Remove an
|
|
||||||
unnecessary intermediate URI clone; replay still builds independent URIs.
|
|
||||||
4. `src/forwarding.rs`: reuse the formatted IPv4 peer address when creating
|
|
||||||
RFC 7239 `Forwarded` as well as X-Real-IP/X-Forwarded-For. Preserve quoted
|
|
||||||
IPv6 bracket and proxy-chain logic for trusted and untrusted peers.
|
|
||||||
5. `src/proxy/upstream_helpers.rs`: fast-path byte-safe truncation of short
|
|
||||||
or ASCII host/path log values, keeping the original max-*characters*
|
|
||||||
semantics for non-ASCII UTF-8.
|
|
||||||
6. `src/proxy/headers.rs`: avoid nominated-field token parsing for absent,
|
|
||||||
lone `keep-alive`, or lone `close` Connection values. A `close` field on
|
|
||||||
the wire still forces the general parser, so a nominated `Close` header
|
|
||||||
is removed exactly as before. Multi-value, comma lists and malformed tokens
|
|
||||||
retain the previous defensive parser.
|
|
||||||
7. `src/proxy/headers.rs`: similarly avoid the generic token parser when
|
|
||||||
checking HTTP upgrade on plain keep-alive, close, or absent Connection.
|
|
||||||
Complex multi-valued requests and explicit Upgrade remain unchanged.
|
|
||||||
|
|
||||||
## Tests and limits
|
|
||||||
|
|
||||||
- New Rust cases validate empty vs nonempty request-body permits and bytes,
|
|
||||||
H1 URI/authority/query, ASCII/Unicode truncation, and Connection variants
|
|
||||||
including a literal nominated `Close` field. Existing regression coverage
|
|
||||||
still includes sticky backends, cache, ACL and trusted forwarded identity.
|
|
||||||
- Source, SQLite migration and upgrade tests are necessary but **not** Rust
|
|
||||||
compilation or performance evidence. The compile check must run before use.
|
|
||||||
- Run `./dev.sh check && CARGO_INCREMENTAL=0 ./dev.sh test`, followed by
|
|
||||||
the native black-box acceptance suite (proxy, policy, cache, transform,
|
|
||||||
compression, static serving), plus TLS/H2/H3 and websocket tests.
|
|
||||||
|
|
||||||
## Measurements
|
|
||||||
|
|
||||||
An initial diagnostic same-host `ab` sample at concurrency 16 gave
|
|
||||||
OPT2 33,042.55 RPS and compiled OPT3 36,962.11 RPS, without HTTP errors;
|
|
||||||
this single sample is not proof of a reliable uplift. A separate complete
|
|
||||||
12-sample run (two 3-second repetitions each at 16/64 clients) produced:
|
|
||||||
|
|
||||||
| Concurrency | OPT2 median RPS | OPT3 median RPS | nginx median RPS | OPT3/nginx |
|
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
|
||||||
| 16 | 21,608 | 29,906 | 92,586 | 32.3% |
|
|
||||||
| 64 | 38,060 | 34,781 | 123,809 | 28.1% |
|
|
||||||
|
|
||||||
All 12 samples had valid 200 responses and correct 1040-byte body checks.
|
|
||||||
OPT3 was slower than OPT2 at concurrency 64; the variability is too large to
|
|
||||||
claim a sustained uplift or regression based on two short same-host samples.
|
|
||||||
Raw evidence includes run order, response checksum and nginx comparator.
|
|
||||||
No result here measures OPT4, which was not built in this environment.
|
|
||||||
|
|
||||||
## Next evaluation commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
bash dev.sh
|
|
||||||
python3 scripts/performance/regression.py --binary opt4=/absolute/path/target/debug/rust-proxy --out /tmp/opt4-http
|
|
||||||
python3 scripts/performance/ab_1kb_dev.py --binary opt3=/absolute/path/opt3/rust-proxy --binary opt4=/absolute/path/target/debug/rust-proxy --nginx /usr/sbin/nginx --ab /usr/bin/ab --out /tmp/opt4-ab --duration 5 --repeats 3 --concurrency 16,64 --workers 4 --max-idle 64
|
|
||||||
```
|
|
||||||
|
|
||||||
For authoritative acceptance: pin client/server/origin to nonoverlapping CPU
|
|
||||||
sets, align build profile and all flags, compare 30s x7 runs, record median,
|
|
||||||
p95/p99, correct 1040 B response checksum and status, CPU, RSS, FDs, cache
|
|
||||||
stats, analytic/sink queue drops, errors, and both HTTP/1.1 and optional
|
|
||||||
HTTP/2 endpoints. Reject changes on new 4xx/5xx, body mismatches, regressions
|
|
||||||
of access controls or material memory/fd increases.
|
|
||||||
@@ -1,93 +0,0 @@
|
|||||||
# RustProxy 0.34.0 DEV — OPT5, 1,040-byte reverse-proxy optimization
|
|
||||||
|
|
||||||
## Acceptance boundary
|
|
||||||
|
|
||||||
The source package preserves **0.34.0** and all existing configurable services,
|
|
||||||
logs, dashboards, cache/ACL/auth policies, streaming, TLS/H2/H3/WebSocket,
|
|
||||||
failover, retry and bounded-resource behavior. OPT5 is a **source-only** candidate:
|
|
||||||
no Rust compiler is available in the editing environment. The new code's
|
|
||||||
compilation, Rust unit tests, binary throughput and full-protocol interoperability
|
|
||||||
are unverified until the developer builds it. The compiled OPT4 executable is
|
|
||||||
the known test input, **not** a benchmark of OPT5.
|
|
||||||
|
|
||||||
## Compiled OPT4 input verification
|
|
||||||
|
|
||||||
Input: `/mnt/data/rust-proxy(4)`, SHA-256
|
|
||||||
`1c4c701f11b57b1293020061c8af21d5a9df0c86f03290018a90edc007667b24`.
|
|
||||||
|
|
||||||
- The real HTTP regression runner completed **151 passing**, **0 failing** cases;
|
|
||||||
**1 skipped** because Python `zstandard` is not installed. Protocol tests for
|
|
||||||
TLS/H2/H3 are a separate acceptance gate and were **not** passed by this run.
|
|
||||||
- Same-host ApacheBench HTTP/1.1 keep-alive, 1,040-byte body verified by exact
|
|
||||||
SHA-256, cache OFF, 4 Tokio workers, 64 idle HTTP1 upstream connections,
|
|
||||||
3-second warm-and-measure samples, two repeats per variant/concurrency:
|
|
||||||
|
|
||||||
| RPS, median of 2 samples | 16 clients | 64 clients |
|
|
||||||
| --- | ---: | ---: |
|
|
||||||
| OPT3 | 37,017 | 40,668 |
|
|
||||||
| OPT4 | 39,054 | 36,425 |
|
|
||||||
| nginx | 99,808 | 110,783 |
|
|
||||||
| OPT4 vs OPT3 | +5.5% | -10.4% |
|
|
||||||
| OPT4 / nginx | 39.1% | 32.9% |
|
|
||||||
|
|
||||||
There were zero HTTP failures and the response body was verified before/after
|
|
||||||
measurements. Shared CPU and only two short samples are diagnostic, NOT a stable
|
|
||||||
speedup or production proof. The 100k RPS and 80%-of-nginx goals remain unmet.
|
|
||||||
|
|
||||||
## OPT5 changes (8 coherent hot-path optimizations)
|
|
||||||
|
|
||||||
1. `src/proxy/request_id.rs`: eliminate per-request SHA-256. Keep the existing
|
|
||||||
OS-seeded/reseeded thread-local CSPRNG and 256-bit ID; XOR a monotonic
|
|
||||||
sequence into the first 8 nonce bytes. Output remains uniformly random,
|
|
||||||
64 lowercase hex characters and the protected `X-Request-ID` correlation
|
|
||||||
remains. Random IDs continue to be unique with overwhelming probability.
|
|
||||||
2. `src/forwarding.rs` and `src/proxy/forward_entry.rs`: only construct
|
|
||||||
`Context.original_url` for a frontend with configured Tinyauth. This is the
|
|
||||||
*only* runtime consumer of that absolute URL. Public `context(...)` still
|
|
||||||
constructs it by default. Trusted/untrusted forwarding and all validation
|
|
||||||
still execute identically; IPv4/IPv6 and `x-original-url` for Tinyauth remain.
|
|
||||||
3. `src/balance/lease_runtime.rs`: borrow `"ok"` for successful backend health
|
|
||||||
observations instead of allocating a `String`. 5xx failures still record the
|
|
||||||
exact original `status=NNN` detail and slow/passive health counters are kept.
|
|
||||||
4. `src/traffic.rs`: receive 64 samples at a time (Tokio `recv_many`) and apply
|
|
||||||
them under a single dashboard-state mutex. Order within the existing bounded
|
|
||||||
2,048-sample queue is preserved; waiting is outside the mutex and producer
|
|
||||||
backpressure and drop accounting are unchanged.
|
|
||||||
5. `src/traffic.rs`: reuse map entries with borrowed vhost names. New keys are
|
|
||||||
cloned only when a domain first appears, preserving the max 64-vhost bound.
|
|
||||||
6. `src/traffic.rs`: classify status 1xx–5xx using static labels; preserve
|
|
||||||
unknown extension status labels such as `7xx`.
|
|
||||||
7. `src/traffic.rs`: replace per-sample lowercase allocation for User-Agent
|
|
||||||
family detection with ASCII-insensitive byte-window comparisons, and avoid
|
|
||||||
copying all TopTable maps when displaying a single vhost.
|
|
||||||
8. `src/proxy/upstream_helpers.rs`: skip resolving inherited/vhost/web-service
|
|
||||||
compression settings when no encoding is accepted even under the global
|
|
||||||
codec ceiling (for example `Accept-Encoding: identity`). Existing compression
|
|
||||||
handling is retained for every accepted codec and request method.
|
|
||||||
|
|
||||||
The caller's front-end policies remain evaluated on every request. No log,
|
|
||||||
metric, access control or authentication feature was disabled or made optional.
|
|
||||||
|
|
||||||
## Verification and outstanding gates
|
|
||||||
|
|
||||||
Static source, SQL migration and managed-upgrade tests are executed and their
|
|
||||||
outcomes supplied in the external evidence archive. New Rust test fixtures cover
|
|
||||||
request IDs, forwarding/trusted IPv6, compression, healthy/5xx backend detail,
|
|
||||||
analytics batching, top classes, country-independent User-Agent matching,
|
|
||||||
reset and the bounded queue.
|
|
||||||
|
|
||||||
To verify the new build on the developer's system:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
./dev.sh build
|
|
||||||
```
|
|
||||||
|
|
||||||
After building, repeat the actual HTTP regression, LB regression and TLS/H2/H3
|
|
||||||
transport tests and collect at least seven 30-second CPU-isolated, warmed,
|
|
||||||
interleaved OPT4/OPT5/nginx measurements with `wrk`, p95/p99 and CPU/RSS.
|
|
||||||
Treat the non-tested binaries, short shared-host tests and simulated Python
|
|
||||||
checks as distinct types of evidence. Measure the secondary Web Server path
|
|
||||||
separately. Do not alter the version or silently relax cache, auth or timeout
|
|
||||||
policies to reach a RPS target.
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
# RustProxy 0.34.0 — reverse-proxy hot-path candidate
|
|
||||||
|
|
||||||
## What was changed
|
|
||||||
|
|
||||||
1. **Routes:** `Snapshot.routes` and `fallbacks` now hold
|
|
||||||
`Arc<CompiledRoute>` handles. The same compiled location is shared across
|
|
||||||
hostname aliases. `observe` resolves vhost/location once from the accepted
|
|
||||||
listener/TLS-state and stores a request-local handle. `forward_request`
|
|
||||||
consumes the same immutable snapshot/route; internal error fallbacks still
|
|
||||||
resolve the explicitly configured named location independently.
|
|
||||||
2. **Host matching:** wildcard candidate strings are generated lazily only
|
|
||||||
after an exact host failed, without changing longest-specific-suffix order.
|
|
||||||
3. **LB:** single-backend non-cookie pools use a direct admitted lease without
|
|
||||||
temporary candidate vectors or scheduler mutex. Multi-backend weighted
|
|
||||||
round-robin retains its ordering and mutex; least-connection/traffic,
|
|
||||||
adaptive, IP hash and cookie-affinity retain their behavior. IP hash avoids
|
|
||||||
repeated canonicalization/temporary `format!` strings for each candidate.
|
|
||||||
4. **Health:** a `healthy_cached` atomic allows checking healthy backends
|
|
||||||
without the health mutex; all state transitions remain inside the existing
|
|
||||||
state machine, including passive quarantine and half-open recovery.
|
|
||||||
Resolved active/passive per-member policies are held in shared immutable
|
|
||||||
`Arc` handles so a new request lease does not deep-clone health paths,
|
|
||||||
headers or optional host strings.
|
|
||||||
5. **Fallback/retry:** request body+HeaderMap replay templates are made only
|
|
||||||
for configured error fallbacks. Retry headers are cloned only for eligible
|
|
||||||
safe replay requests where another member or H2 graceful close makes retry
|
|
||||||
possible; writes, streaming bodies and WebSockets are not replayed.
|
|
||||||
6. **Telemetry:** three per-request mutex-backed scalar tags are now atomic;
|
|
||||||
their original event strings and `Option<bool>` log fields are decoded at
|
|
||||||
exchange completion. Request ID, method, host and path are moved into the
|
|
||||||
terminal log record instead of deep-cloned in Drop; the traffic sample
|
|
||||||
still receives its own required copies. Logs avoid a broadcast clone if
|
|
||||||
there are no subscribers, preserving history, sink and counters.
|
|
||||||
|
|
||||||
## Preserved boundaries
|
|
||||||
|
|
||||||
All vhost, ACL, GeoIP, Basic Auth, Tinyauth, WebSocket, retry, cache,
|
|
||||||
transform, compression, log, metrics, `sendfile`, PHP-FPM, HTTPS/H2/H3 and
|
|
||||||
admin panel configuration paths are intended to remain compatible. Defaults
|
|
||||||
and SQLite schema are unchanged; no new performance toggle was added. The
|
|
||||||
optimization reuses route objects within one immutable snapshot but does not
|
|
||||||
allow stale snapshot cache fills (existing revision ticket barrier remains).
|
|
||||||
Sticky cookies deliberately retain the general selection path, and upstream
|
|
||||||
request-count recycling is unchanged (default 25,000 per generation).
|
|
||||||
|
|
||||||
## Locally executed checks vs required checks
|
|
||||||
|
|
||||||
- **Executed locally:** scoped-cache/sendfile/hot-path *source-only* guards,
|
|
||||||
Python syntax, SQLite migration and upgrade preservation tests, selected JS
|
|
||||||
panel scripts and simulated editor tests. These do **not** prove Rust builds.
|
|
||||||
- **Prepared, NOT executed:** Rust unit tests for shared routing, single-member
|
|
||||||
LB capacity/health, passive ejection/recovery, atomic log fields, retained
|
|
||||||
sendfile, cache and protocol behavior.
|
|
||||||
- **Unmeasured:** 0.34.0 requests/sec, latency, CPU cost, memory and nginx gap.
|
|
||||||
Do not infer a throughput gain from fewer source-code locks/allocations.
|
|
||||||
|
|
||||||
## Build and acceptance
|
|
||||||
|
|
||||||
Run on the intended Linux builder with exactly the same compiler, dependencies,
|
|
||||||
release flags and workload as the 0.33.8 reference:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh doctor
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
||||||
python3 scripts/tests/proxy_hotpath_034_static_test.py
|
|
||||||
python3 scripts/tests/cache_scope_034_static_test.py
|
|
||||||
python3 scripts/tests/sendfile_034_static_test.py
|
|
||||||
python3 scripts/tests/dashboard_presets_sql_test.py
|
|
||||||
python3 scripts/tests/upgrade_performance_test.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Next run `scripts/performance/regression.py`, `lb_regression.py`,
|
|
||||||
`transport_regression.py` and `concurrency_regression.py` against both binary
|
|
||||||
versions. After functional acceptance, run the paired
|
|
||||||
`scripts/performance/benchmark.py` suite against the unmodified baseline,
|
|
||||||
this candidate and nginx; use the instructions in
|
|
||||||
`scripts/performance/README.md` including separate CPU affinities.
|
|
||||||
|
|
||||||
Measure RPS/p95/p99, CPU microseconds per request, RSS, FDs, pool rotations,
|
|
||||||
health transitions, cache HIT ratio and logger drops. For each payload report
|
|
||||||
`(0.34.0 RPS / 0.33.8 RPS - 1)*100` and `0.34.0 RPS/nginx RPS*100` from real
|
|
||||||
medians. A source-only revision is **not production-accepted**.
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
# RustProxy 0.34.0: hot-path throughput revision
|
|
||||||
|
|
||||||
**Status: uncompiled source candidate.** This revision does not change the
|
|
||||||
application version, public API, dashboard controls, cache policy or tuning
|
|
||||||
defaults. It must not be labelled production-ready until the new Rust binary
|
|
||||||
passes compile, unit tests, live regressions and paired performance comparisons.
|
|
||||||
|
|
||||||
## Code changes and intended cost reductions
|
|
||||||
|
|
||||||
| Focus | Change | Kept behavior / risk to verify |
|
|
||||||
|---|---|---|
|
|
||||||
| Per-request identity | Draw a 256-bit nonce from the OS-seeded/reseeded thread-local CSPRNG instead of asking `getrandom` on every request. Preserve SHA-256 of nonce+monotonic sequence and the existing 64-hex ID format. | IDs remain non-predictable. Check ID uniqueness and raw upstream headers. |
|
|
||||||
| Upstream HTTP request target | Compile the validated scheme/authority `http::Uri` once with each immutable backend; parse only the request path+query and rebuild using typed `Uri::from_parts`. | Never accept a client-supplied authority, bypass DNS/address policies or change SNI. Test H1, H2, IPv6, query and percent-encoding. |
|
|
||||||
| Load balancing | Do not clone the member ID string for an existing smooth-WRR scoreboard entry. Continue the previously introduced no-mutex single-member non-cookie path. | Sticky-cookie modes, capacity limits, retry exclusions, health and drain remain unchanged. |
|
|
||||||
| Active health | Use already-compiled `Arc<HealthCheck>` when dispatching probes, avoiding a clone of its path/headers for each scheduled probe. | Same 250-ms supervisor, timeout, enabled/claim gate and health state transitions. |
|
|
||||||
| Per-frame streaming | Capture the leased backend once at first streamed request DATA (response already captured at wrapper construction). Do not take `Exchange.backend` mutex on every upload chunk. | Request/response byte counters, mid-stream failures, idle timeouts and lease lifetime remain active. |
|
|
||||||
| Pool recycling | Preserve exact request-count and elapsed-time recycling checks per request, with the existing independent 250-ms maintenance cycle. The pool factory/connection policy is unchanged. | Exact lifecycle semantics are more important than saving one cheap clock read; tune idle/age limits only after profiling. |
|
|
||||||
| Log history | Move completed `LogEntry` directly into bounded history when there are no live viewers or enabled external sinks, instead of deeply cloning its strings. Continue cloning and delivering to live viewers/sinks when enabled. | Same JSON fields, history size, drop counters, file/Loki/Logstash forwarding. |
|
|
||||||
| Route/HTTP instrumentation | Remove redundant route-ID mutex/write in forwarding and collapse H1/H2 version tallying into one branch. | Route is recorded once before forwarding; H1/H2 counters remain. |
|
|
||||||
|
|
||||||
These historical changes build on the previous 0.34.0 route-Arc, single-backend LB,
|
|
||||||
conditional retry/fallback-header cloning, atomic exchange flags and cache HOT
|
|
||||||
LRU changes. There are no new UI toggles. This **earlier** revision retained `max_idle_per_host`
|
|
||||||
4 and `pool_recycle_requests` 25,000. The **subsequent DEV 1 KiB revision**
|
|
||||||
changes only the default idle limit to 64, based on live configuration A/B
|
|
||||||
with the old binary (not source A/B); see `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. `sendfile: auto` remains opt-in and applies only to eligible Linux
|
|
||||||
plaintext H1 static WWW responses, not reverse proxy or TLS/H2/H3.
|
|
||||||
|
|
||||||
## Cache transform/query policy
|
|
||||||
|
|
||||||
An enabled `bypass_query` is a deliberate policy: `/theme.css?v=...` is a
|
|
||||||
`BYPASS` even when the origin sends `Cache-Control: public`. Turning the bypass
|
|
||||||
off permits a transformed CSS object to follow `MISS -> HIT` and appear in the
|
|
||||||
cache dashboard. An existing transformed HOT row with hit count and TTL is
|
|
||||||
positive evidence of this. This throughput revision does not modify the
|
|
||||||
cache eligibility logic or discard Age/ETag/Cache-Control headers.
|
|
||||||
|
|
||||||
## Verification already possible here
|
|
||||||
|
|
||||||
- **Old/reproducible reference executable only:** 110 passed, 0 failed, 1 skipped
|
|
||||||
(`compression.codec.zstd` skipped because the optional decoder was missing),
|
|
||||||
from live proxy/policy/cache/transform loopback regression. This does **not**
|
|
||||||
exercise the newly modified Rust source.
|
|
||||||
- **Old/reproducible reference executable only:** query bypass enabled gives
|
|
||||||
`BYPASS -> BYPASS`, disabled gives `MISS -> HIT` and a transformed representation;
|
|
||||||
the older Age regression is also reproducible on this old executable.
|
|
||||||
- Source-only invariants: `scripts/tests/rps_hotpath_034_static_test.py` plus
|
|
||||||
existing route/cache/sendfile checks. SQL migration and upgrade scripts pass.
|
|
||||||
- **Not performed:** Rust compilation or newly added Rust unit tests, live
|
|
||||||
behavior of the optimized binary, HTTP/2 live TLS, comparative RPS, CPU
|
|
||||||
profiling and long-duration memory/health soak. The environment does not
|
|
||||||
include `cargo`, `rustc` or `wrk`. No post-change RPS gain is claimed.
|
|
||||||
|
|
||||||
## Acceptance commands on a Rust-capable Linux host
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./dev.sh check
|
|
||||||
CARGO_INCREMENTAL=0 ./dev.sh test
|
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
||||||
python3 scripts/tests/rps_hotpath_034_static_test.py
|
|
||||||
python3 scripts/tests/cache_age_transform_034_live.py \
|
|
||||||
--binary "$(realpath target/release/rust-proxy)" \
|
|
||||||
--output /tmp/rustproxy034-cache-after
|
|
||||||
python3 scripts/performance/regression.py \
|
|
||||||
--binary optimized="$(realpath target/release/rust-proxy)" \
|
|
||||||
--out /tmp/rustproxy034-http-regression
|
|
||||||
```
|
|
||||||
|
|
||||||
Run `scripts/performance/benchmark.py` from the documented reproducible
|
|
||||||
before/after/nginx command in `scripts/performance/README.md`. Compare the
|
|
||||||
unchanged old 0.34.0 binary with the newly compiled source, not a baseline
|
|
||||||
with different feature flags. Record RPS at 1,040 B, 64 KiB, 1 MiB, p95/p99,
|
|
||||||
CPU/request, RSS, connection counts, errors and exact response-body hashes.
|
|
||||||
Use `perf record -g -p <proxy-pid>` during a stable request interval and inspect
|
|
||||||
`perf report --stdio` to identify the next true bottleneck.
|
|
||||||
|
|
||||||
Do not interpret Python origin self-tests, source inspections, or speculative
|
|
||||||
Amdahl models as measured RustProxy throughput.
|
|
||||||
+2
-4
@@ -7,7 +7,6 @@ This is the canonical index for the current project documentation. The authentic
|
|||||||
- [Project README](../README.md) - product overview, installation entry points and current release notes.
|
- [Project README](../README.md) - product overview, installation entry points and current release notes.
|
||||||
- [ROADMAP](../ROADMAP.md) - planned and completed release scope.
|
- [ROADMAP](../ROADMAP.md) - planned and completed release scope.
|
||||||
- [CHANGELOG](../CHANGELOG.md) - user/operator-visible changes by release.
|
- [CHANGELOG](../CHANGELOG.md) - user/operator-visible changes by release.
|
||||||
- [TODO](TODO.md) - open work only; completed work is not retained as TODO.
|
|
||||||
|
|
||||||
## Functional documentation
|
## Functional documentation
|
||||||
|
|
||||||
@@ -42,7 +41,6 @@ This is the canonical index for the current project documentation. The authentic
|
|||||||
- [MOCK_SERVER](MOCK_SERVER.md) - local Python mock origin used by development/integration checks.
|
- [MOCK_SERVER](MOCK_SERVER.md) - local Python mock origin used by development/integration checks.
|
||||||
- [MOCK_AUTH](MOCK_AUTH.md) - local fake Tinyauth peer for development only.
|
- [MOCK_AUTH](MOCK_AUTH.md) - local fake Tinyauth peer for development only.
|
||||||
- [MEMORY_STRESS](MEMORY_STRESS.md) - sustained-memory audit, offline load simulations, real RustProxy staging stress and RSS measurement procedure.
|
- [MEMORY_STRESS](MEMORY_STRESS.md) - sustained-memory audit, offline load simulations, real RustProxy staging stress and RSS measurement procedure.
|
||||||
- [TEST_REPORT_0.34.0](TEST_REPORT_0.34.0.md) - 0.34.0 source-stage test evidence, simulated cache retention and required acceptance gates.
|
|
||||||
|
|
||||||
## Technical documentation
|
## Technical documentation
|
||||||
|
|
||||||
@@ -61,7 +59,7 @@ Completed roadmap history is retained separately as [ROADMAP_HISTORY](ROADMAP_HI
|
|||||||
|
|
||||||
### Retained audits and historical context
|
### Retained audits and historical context
|
||||||
|
|
||||||
- [SECURITY_AUDIT](SECURITY_AUDIT.md) and [SECURITY_AUDIT_0.31.0](SECURITY_AUDIT_0.31.0.md) - current security audit material and the 0.31.0 audit.
|
- [SECURITY_AUDIT](SECURITY_AUDIT.md) - security audit material.
|
||||||
- [CHANGELOG_TECHNICAL_ARCHIVE](CHANGELOG_TECHNICAL_ARCHIVE.md) and [ROADMAP_HISTORY](ROADMAP_HISTORY.md) - retained long-form technical history and completed milestones.
|
- [CHANGELOG_TECHNICAL_ARCHIVE](CHANGELOG_TECHNICAL_ARCHIVE.md) and [ROADMAP_HISTORY](ROADMAP_HISTORY.md) - retained long-form technical history and completed milestones.
|
||||||
- [ACME_RFC2136](ACME_RFC2136.md) - historical DNS provider details.
|
- [ACME_RFC2136](ACME_RFC2136.md) - historical DNS provider details.
|
||||||
- [INTEGRATIONS_040](INTEGRATIONS_040.md) and [MANAGEMENT](MANAGEMENT.md) - retained design and integration notes.
|
- [INTEGRATIONS_040](INTEGRATIONS_040.md) and [MANAGEMENT](MANAGEMENT.md) - retained design and integration notes.
|
||||||
@@ -72,7 +70,7 @@ The old per-version implementation notes and audits removed from this trimmed so
|
|||||||
## Maintenance rules
|
## Maintenance rules
|
||||||
|
|
||||||
- Put user/operator-visible release changes in `CHANGELOG.md`.
|
- Put user/operator-visible release changes in `CHANGELOG.md`.
|
||||||
- Put future release scope in `ROADMAP.md` and unresolved work in `docs/TODO.md`.
|
- Put future release scope and unresolved work in `ROADMAP.md`.
|
||||||
- Keep detailed implementation evidence in technical documents or version audits, not in CHANGELOG.
|
- Keep detailed implementation evidence in technical documents or version audits, not in CHANGELOG.
|
||||||
- When functionality changes, update the functional document, this index, Reader coverage and cross-links in the same change.
|
- When functionality changes, update the functional document, this index, Reader coverage and cross-links in the same change.
|
||||||
- Remove obsolete or duplicated documents when they no longer provide historical or technical value; retain useful audits with an explicit historical label.
|
- Remove obsolete or duplicated documents when they no longer provide historical or technical value; retain useful audits with an explicit historical label.
|
||||||
@@ -1,93 +0,0 @@
|
|||||||
# RustProxy 0.34.0 — poprawki cache, transformacji i kontrola RPS
|
|
||||||
|
|
||||||
Status: **zmodyfikowane źródła, niekompilowane lokalnie** (brak `cargo` i `rustc`). Numer wersji: 0.34.0.
|
|
||||||
|
|
||||||
## Co odtworzono na binarce użytkownika
|
|
||||||
|
|
||||||
Uruchomiono binarkę Linux ELF w izolowanej konfiguracji z lokalnym origin HTTP, z włączonym cache, transformacją CSS, `max_ttl_seconds=86400`, diagnostyką i API administracyjnym. HTTP/1.1 localhost, pełne GET, brak kontaktu z witryną produkcyjną.
|
|
||||||
|
|
||||||
| Odpowiedź | Pierwszy GET | Drugi GET | Liczba pobrań z origin |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `/normal.css` | MISS | HIT | 1 |
|
|
||||||
| `/aged.css` | MISS | MISS | 2 |
|
|
||||||
| `/br.css` | MISS | HIT | 1 |
|
|
||||||
| `/aged-br.css` | MISS | MISS | 2 |
|
|
||||||
| `/private.css` | MISS | MISS | 2 |
|
|
||||||
| `/no-store.css` | MISS | MISS | 2 |
|
|
||||||
|
|
||||||
Zwykły CSS i skompresowany Brotli dostają HIT. Warianty z `Age: 1553483` mają `MISS → MISS` mimo `Cache-Control: public, max-age=2592000`. Przypadki `private` / `no-store` słusznie nie trafiają do cache. Zmiana keep-alive vhosta nie usuwała poprawnie zachowanego obiektu. `expose_only_hits=true` celowo ukrywa nagłówki na MISS.
|
|
||||||
|
|
||||||
## Główna przyczyna i poprawka
|
|
||||||
|
|
||||||
W `src/cache/policy/response.rs` obowiązywał wzór `ttl = min(origin_max_age, local_max_ttl)` oraz `remaining = ttl - corrected_age`. To jest sprzeczne z zadaniem lokalnego limitu: świeżość w origin nadal trwa, ale CDN `Age > local_max_ttl` dawał sztuczne 0 sekund lokalnej świeżości.
|
|
||||||
|
|
||||||
Poprawiony wzór:
|
|
||||||
|
|
||||||
```text
|
|
||||||
corrected_initial_age = max(apparent_age, upstream_age + response_delay)
|
|
||||||
remaining_origin_freshness = max(0, origin_max_age - corrected_initial_age)
|
|
||||||
remaining_local_freshness = min(remaining_origin_freshness, local_max_ttl)
|
|
||||||
entry_fresh_until_age = corrected_initial_age + remaining_local_freshness
|
|
||||||
```
|
|
||||||
|
|
||||||
Przykład z przesłanych nagłówków: wiek upstream 1 553 483 s = prawie 18 dni. Pierwotna świeżość 2 592 000 s (30 dni), więc pozostało około 12 dni w origin. Po naprawie RustProxy będzie mógł przechować świeżą odpowiedź przez lokalny limit (domyślnie 86 400 s = 1 dzień). Nagłówek Age nadal będzie około 1,55 mln sekund i będzie rósł na HIT — to poprawne.
|
|
||||||
|
|
||||||
`Age` nie wynika z `Last-Modified: 2016`. Najczęściej jest przekazany przez upstream/CDN, a stara data `Date` też mogłaby podnieść apparent_age. Rozróżnienie: `Age` = HTTP wiek całkowity; `resident_age_seconds` = pobyt w HOT/COLD.
|
|
||||||
|
|
||||||
## HOT RPS i panel
|
|
||||||
|
|
||||||
1. HOT HIT nie wyjmuje już i nie wkłada od nowa wpisu oraz indeksu `Vary`, tylko aktualizuje licznik HIT i kolejność LRU w miejscu. Zmiana ma zmniejszyć alokacje i krytyczną sekcję; **zysk RPS jeszcze niezmierzony**.
|
|
||||||
2. Panel Cache ma osobne kolumny **Age** i **Czas w RustProxy**. API `dashboard.objects[]` dodaje `resident_age_seconds` także dla COLD. Pozostałe nazwy pól nie zostały zmienione.
|
|
||||||
3. Licznik `x-cache-hits` dotyczy konkretnego aktualnego wpisu. Gdy obiekt zostanie wyparte z HOT (limit wpisów/pamięć), usunięty lub zastąpiony po MISS, licznik restartuje. W próbce użytkownika TTL wynosił około 604 tys. s (7 dni), więc późniejszy MISS po ~4 minutach **nie wynikał z normalnego wygaśnięcia TTL**. Konkretnej przyczyny epizodu nie można ustalić bez trendów `evictions`, `purges`, `hot_demotions`, `cold_hits`, kluczy/Vary oraz zmian konfiguracji.
|
|
||||||
4. Brak `x-cache` na MISS jest oczekiwany, gdy włączono **nagłówki diagnostyczne tylko na HIT**. Brak wpisu w API może mieć inne przyczyny (np. cache-control `private`, `no-store`, brak polityki origin, limit obiektu) i nie należy ich obchodzić na siłę.
|
|
||||||
5. Nie zmieniano uprawnień, `private`/`no-store`, kluczy Vary, TTL po stronie origin ani zasad selektywnej invalidacji.
|
|
||||||
|
|
||||||
## Wynik symulacji TTL
|
|
||||||
|
|
||||||
Wykonano **100 000 deterministycznych scenariuszy**. Dla każdego sprawdzono: po naprawie lokalna świeżość nie przekracza `local_max_ttl` ani świeżości pozostałej w origin.
|
|
||||||
|
|
||||||
| Scenariusz | Stary pozostały TTL | Po poprawce (symulacja) |
|
|
||||||
|---|---:|---:|
|
|
||||||
| Age = 1 553 483, local cap 1 dzień | 0 s | 86 400 s |
|
|
||||||
| Age = 1 553 483, local cap 7 dni | 0 s | 604 800 s |
|
|
||||||
| Origin kończy świeżość za 10 s | 0 s | 10 s |
|
|
||||||
| Origin jest już nieświeży | 0 s | 0 s |
|
|
||||||
|
|
||||||
W symulacji 33 387 / 100 000 losowych przypadków stara polityka błędnie dawała zero zamiast dodatniego TTL. **To nie jest procent regresji w realnym ruchu**, a wynik losowania parametrów testowych.
|
|
||||||
|
|
||||||
## Nginx — rzeczywiste pomiary starej binarki
|
|
||||||
|
|
||||||
Archiwalne wyniki samej binarki `rust-proxy 0.34.0` na tym środowisku (Linux, 4 vCPU, HTTP/1.1 loopback, ApacheBench, 64 połączenia, cache OFF, 3 powtórzenia), z nginx 1.26.3. Brak miarodajnego benchmarku **nowych źródeł** przed ponowną kompilacją.
|
|
||||||
|
|
||||||
| Endpoint | RustProxy RPS (mediana) | nginx RPS (mediana) | RustProxy / nginx |
|
|
||||||
|---|---:|---:|---:|
|
|
||||||
| proxy 1 KiB | 10,544 | 58,709 | 18.0% |
|
|
||||||
| proxy 64 KiB | 7,893 | 17,358 | 45.5% |
|
|
||||||
| proxy 1 MiB | 1,010 | 1,666 | 60.6% |
|
|
||||||
| www 1 KiB | 11,984 | 71,318 | 16.8% |
|
|
||||||
| www 64 KiB | 9,224 | 26,600 | 34.7% |
|
|
||||||
| www 1 MiB | 1,264 | 1,098 | 115.2% |
|
|
||||||
|
|
||||||
Ograniczenia porównania: ApacheBench może limitować klienta; wyniki nie reprezentują TLS/H2/H3 ani ruchu produkcyjnego. **Nie wolno tych RPS przypisywać nowej rewizji.**
|
|
||||||
|
|
||||||
## Zestaw testów dla nowej kompilacji
|
|
||||||
|
|
||||||
```bash
|
|
||||||
CARGO_INCREMENTAL=0 cargo test --locked --lib
|
|
||||||
CARGO_INCREMENTAL=0 cargo build --release --locked
|
|
||||||
python3 scripts/tests/cache_age_policy_034_model.py
|
|
||||||
python3 scripts/tests/cache_age_transform_034_live.py --binary target/release/rust-proxy --output /tmp/cache-test-034
|
|
||||||
python3 scripts/tests/cache_scope_034_static_test.py
|
|
||||||
python3 scripts/tests/proxy_hotpath_034_static_test.py
|
|
||||||
python3 scripts/tests/sendfile_034_static_test.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Do testów jednostkowych dodano Age, transformowane CSS/HOT, democję do COLD i odczyt COLD po restarcie, logikę LRU/HIT oraz kontrolę zakazów `private`/`no-store`. Test lokalny sprawdza CSS/Brotli, origin count, API, keep-alive config i ukryte nagłówki. Test po kompilacji bez przełącznika `--expect-old-age-bug` ma wykazać `MISS → HIT` dla Age/Brotli. Na starej binarce odtwarzanie regresji przechodzi z `--expect-old-age-bug`.
|
|
||||||
|
|
||||||
Kontrole wykonane na źródłach: źródłowe testy cache scope, proxy hotpath, sendfile, składnia JS, parsowanie tłumaczeń. **Nie uruchomiono testów Rust ani benchmarków poprawionej binarki** — środowisko nie ma `cargo/rustc`.
|
|
||||||
|
|
||||||
## Następny pomiar po kompilacji
|
|
||||||
|
|
||||||
Porównać tę rewizję 0.34.0 z poprzednią 0.34.0 oraz nginx dla 1 KiB / 64 KiB / 1 MiB; osobno cache OFF, HOT HIT i transformacja/Brotli. Raportować RPS, p95/p99, procesor, RSS, nowe TCP, `cache.evictions` i `cache.purges`; nie modyfikować ustawień cache podczas benchmarku.
|
|
||||||
|
|
||||||
Uwaga: `curl -I -X GET` miesza zachowanie HEAD i GET; do testów reprezentacji zalecane `curl -sS -D - -o /dev/null URL`.
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# Historical roadmap archive
|
# Historical roadmap archive
|
||||||
|
|
||||||
> Archived during the RustProxy 0.25 documentation cleanup. This file preserves completed historical roadmap entries. The active roadmap is [`../ROADMAP.md`](../ROADMAP.md); unresolved work is tracked in [`TODO.md`](TODO.md).
|
> Archived during the RustProxy 0.25 documentation cleanup. This file preserves completed historical roadmap entries. The active roadmap is [`../ROADMAP.md`](../ROADMAP.md).
|
||||||
|
|
||||||
# RustProxy roadmap
|
# RustProxy roadmap
|
||||||
|
|
||||||
|
|||||||
@@ -1,148 +0,0 @@
|
|||||||
# RustProxy 0.31.0 - API, panel and HTTP cache-maintenance security review
|
|
||||||
|
|
||||||
Scope: source snapshot supplied for the v0.31.0 maintenance follow-up, reviewed
|
|
||||||
2026-10-10. This is a **code/contract review with offline regressions**, not a
|
|
||||||
penetration test or proof of production safety. The supplied archive has no
|
|
||||||
`.git` directory; the review does not establish parity with remote `master`.
|
|
||||||
|
|
||||||
## Interfaces and trust boundaries
|
|
||||||
|
|
||||||
| Surface | Exposure | Authentication and mutation rules | Assessment |
|
|
||||||
|---|---|---|---|
|
|
||||||
| Public administration bootstrap (`/`, login JS/CSS, `/api/auth/status`, setup/login POST) | Login/public administration listener | Host/origin validation; request marker for unsafe bootstrap actions; one-time setup capability and login throttling | Source-reviewed; live browser flow pending |
|
|
||||||
| Private panel HTML, JS, documentation, OpenAPI | Administration listener | Administrator session; `Cache-Control: no-store`; CSP, frame and MIME protections | Source and anonymous-response test reviewed; runtime pending |
|
|
||||||
| `/api/status`, `/api/system-info`, `/api/traffic`, `/api/cache`, logs, config, preferences, audit | Administration listener | Administrator cookie session; sensitive reads not on anonymous allowlist | Source and regression tests added; runtime pending |
|
|
||||||
| `/api/events`, live-log SSE | Administration listener | Session on subscribe and repeated validity checks; bounded viewers; heartbeat/disconnect cleanup | Source-reviewed; reconnect/load pending |
|
|
||||||
| State-changing admin APIs (including cache/config/preferences) | Administration listener | Valid session, Origin/Host, `X-RustProxy-Request: 1`, CSRF token; audit on cache commands | Source and regression tests added; runtime pending |
|
|
||||||
| Certificate API (`/api/certificates/{id}/reload`, deploy) | Administration listener | Narrow-scoped Key ID + Bearer grant; no general read access; HTTPS required remotely; origin validation when supplied | Source-reviewed; token rotation/revocation live tests pending |
|
|
||||||
| Public `BAN`/`PURGE` | Configured public vhost listener | **No admin session**: separate explicit IP/CIDR allowlist (empty by default), matched listener/vhost/ACL and optional rate limit | Hardened below; reverse/HTTP tests pending |
|
|
||||||
| Separate Prometheus metrics exporter | Distinct optional listener | Configured IP CIDRs and bearer-token policy; not the admin session | Unchanged; deployment exposure audit pending |
|
|
||||||
|
|
||||||
## Findings addressed in this follow-up
|
|
||||||
|
|
||||||
1. **E0716 / compilation defect (high functional impact).** `error()` accepts
|
|
||||||
`&'static str`; a reference to `format!(...)` was a temporary. Invalid
|
|
||||||
selector errors now log the detailed reason server-side and return a fixed,
|
|
||||||
non-sensitive message with status 422. This removes the reported lifetime
|
|
||||||
violation; the Rust compiler must still verify the entire crate.
|
|
||||||
2. **Cache protocol-scope confusion (medium).** Path-only `BAN`/`PURGE`
|
|
||||||
selectors could affect representations of the same host/path across both
|
|
||||||
HTTP and HTTPS. Targets are now built with the trusted external scheme,
|
|
||||||
validated to `http`/`https`, and the matched Host/vhost. IPv6 literal Hosts
|
|
||||||
are bracketed for the URL parser. Other tenants, schemes and hosts remain
|
|
||||||
outside the selector.
|
|
||||||
3. **Unbounded blocking maintenance work (medium availability risk).** Cache
|
|
||||||
invalidation may scan hot/cold entries. Both public HTTP and authenticated
|
|
||||||
admin cache purge/selective invalidation share two maintenance slots and
|
|
||||||
execute scans with `spawn_blocking`. Bursts receive 429 instead of piling
|
|
||||||
up unbounded blocking work on Tokio core threads. The permit stays with the
|
|
||||||
worker after a client disconnects. This is a concurrency bound, **not** a
|
|
||||||
per-client rate quota or a hard time limit for disk work.
|
|
||||||
4. **Ambiguous maintenance input (medium).** Duplicate TTL/confirmation or
|
|
||||||
`Content-Length` headers, transfer-encoded bodies, nonempty bodies and
|
|
||||||
upgrade requests are rejected; `Content-Length: 0` remains compatible with
|
|
||||||
CLI clients. TTL is limited to 1..604800 s. Broad `BAN /` requires the
|
|
||||||
explicit confirmation header; `PURGE` cannot use BAN-only controls.
|
|
||||||
5. **Audit ordering (low/medium).** Removal of an administrator-created BAN
|
|
||||||
rule now records the audit entry before mutating state. Unavailable audit
|
|
||||||
storage blocks the mutation rather than allowing an unaudited removal.
|
|
||||||
6. **Operator safety/usability (low).** An EN/PL help popover beside the HTTP
|
|
||||||
allowlist gives exact `curl` syntax, IP-only authentication semantics and
|
|
||||||
reverse-proxy identity requirements. No new global config variable is added.
|
|
||||||
|
|
||||||
## Existing protections checked in source
|
|
||||||
|
|
||||||
- Admin router's exact anonymous method/path allowlist contains only the
|
|
||||||
expected authentication bootstrap APIs; private docs and assets remain
|
|
||||||
session-protected. Request guard rejects duplicate Host/Origin and checks
|
|
||||||
browser `Sec-Fetch-Site` cross-site signals.
|
|
||||||
- Mutation guard uses an Origin/Host match, per-session CSRF comparison and
|
|
||||||
`X-RustProxy-Request` marker; cookie sessions are HttpOnly/SameSite=Strict
|
|
||||||
and use Secure under HTTPS/trusted forwarded protocol.
|
|
||||||
- Secure response policy includes CSP, `X-Frame-Options: DENY`,
|
|
||||||
`X-Content-Type-Options: nosniff`, referrer restrictions and no-store for
|
|
||||||
private material. A static scan checked 27 first-party browser JS files for
|
|
||||||
direct unsafe HTML injection and dynamic `eval`/`Function` sinks; this scan
|
|
||||||
**cannot establish that the entire DOM is XSS-free**.
|
|
||||||
- IP trust comes from the transport peer or a validated forwarding chain only
|
|
||||||
when the peer is configured trusted. An untrusted client-supplied XFF does
|
|
||||||
not independently grant BAN/PURGE permission. The cache allowlist rejects
|
|
||||||
`/0`, invalid CIDRs, duplicates and more than 32 entries. Requests are never
|
|
||||||
forwarded upstream.
|
|
||||||
- Certificate API tokens have explicit operation/ID grants and cannot use the
|
|
||||||
general admin read API; Bearer grant operations are intentionally separate
|
|
||||||
from cookie + CSRF authentication.
|
|
||||||
|
|
||||||
## HTTP maintenance CLI (allowlisted client only)
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# One URL, including the query string: 200 with JSON result.
|
|
||||||
curl -i -X PURGE 'https://site.example/assets/app.js?v=3'
|
|
||||||
|
|
||||||
# Lazily ban previously cached objects under this path: 202 with JSON result.
|
|
||||||
curl -i -X BAN 'https://site.example/assets/'
|
|
||||||
|
|
||||||
# Entire host: explicit opt-in confirmation header.
|
|
||||||
curl -i -X BAN 'https://site.example/' -H 'X-RustProxy-Confirm-Wide: 1'
|
|
||||||
|
|
||||||
# Optional BAN expiration (30 seconds).
|
|
||||||
curl -i -X BAN 'https://site.example/assets/' -H 'X-RustProxy-Ban-Ttl: 30'
|
|
||||||
```
|
|
||||||
|
|
||||||
Use **Settings -> Cache -> HTTP BAN / PURGE IP allowlist** to grant the
|
|
||||||
caller IP/CIDR. This HTTP interface does not accept admin cookies/tokens as an
|
|
||||||
alternative to the allowlist. The domain, Host header, external scheme and
|
|
||||||
listener must select the intended vhost. Behind a reverse proxy, forward the
|
|
||||||
non-standard methods untouched and **replace**, rather than append untrusted,
|
|
||||||
XFF/Forwarded headers at the edge. Never add the entire reverse-proxy subnet
|
|
||||||
as a maintenance client merely because it is trusted for forwarding.
|
|
||||||
|
|
||||||
## Release security test plan (requires compiled RustProxy)
|
|
||||||
|
|
||||||
1. On the deployment host, run `cargo fmt --all -- --check`,
|
|
||||||
`./dev.sh check`, `./dev.sh test`, and `./dev.sh release`. Address any
|
|
||||||
compiler/regression errors before installation. `cargo audit` or an
|
|
||||||
equivalent advisory scan requires an up-to-date advisory DB and was not run
|
|
||||||
in this offline environment.
|
|
||||||
2. Start a disposable instance with a populated cache under both HTTP and
|
|
||||||
HTTPS for the same vhost/URL. Confirm `PURGE` returns 200 and drops the
|
|
||||||
exact query variant only; `BAN` returns 202 and invalidates only **older**
|
|
||||||
entries on the URL prefix. Confirm the *other* scheme and vhost stay hot.
|
|
||||||
3. From a denied IP, expect 403 on `BAN` and `PURGE`. With an empty allowlist,
|
|
||||||
all clients must be denied. Send duplicate TTL/confirmation/Content-Length,
|
|
||||||
TE/body and `BAN /` without confirmation: expect 400. On a trusted-proxy
|
|
||||||
hop with missing real client identity, expect 403. Test both IPv4 and IPv6.
|
|
||||||
4. From an allowed IP, send 10 concurrent broad maintenance requests; confirm
|
|
||||||
that only two workers run simultaneously and excess return 429. Verify
|
|
||||||
server CPU/IO and audit entries while the cold tier is populated.
|
|
||||||
5. Send GET requests without admin cookies to `/api/config`, `/api/status`,
|
|
||||||
`/api/system-info`, `/api/events`, `/api/logs`, `/docs/openapi.json` and
|
|
||||||
`/assets/app.js`; expect 401 and protective headers. With valid cookie but
|
|
||||||
missing/bad Origin, marker or CSRF on PUT/POST/DELETE, expect 403. Test
|
|
||||||
logout and password/session invalidation, live SSE reconnect/expiry.
|
|
||||||
6. Test the Certificate API with no grant, a valid certificate-specific grant,
|
|
||||||
revoked token, HTTP remote peer, and misuse against `/api/config`/logs.
|
|
||||||
Check errors for token disclosure and audit coverage.
|
|
||||||
7. Test actual browser startup, CSP console, output escaping and form save
|
|
||||||
with malicious HTML strings; inspect reverse-proxy Host/Origin forwarding.
|
|
||||||
Prefer HTTPS administration with a strict `allowed_hosts` list.
|
|
||||||
|
|
||||||
## Residual risk / non-claims
|
|
||||||
|
|
||||||
- **Not executed in this environment:** Cargo format/check/tests/release,
|
|
||||||
actual Rust listener requests, HTTP/2/3, fuzzing, penetration tests,
|
|
||||||
session revocation under load, proxy deployment configuration, dependency
|
|
||||||
CVE scanning and long-running cold-cache resource measurements.
|
|
||||||
- Proxy trust configuration is security-critical. A misconfigured edge that
|
|
||||||
accepts forged forwarded client addresses can let external users impersonate
|
|
||||||
an allowlisted IP. Restrict direct listener reachability and sanitize headers.
|
|
||||||
- A public IP allowlist is not user authentication; anyone using an allowed
|
|
||||||
IP/network can issue destructive maintenance commands. Use narrow CIDRs and
|
|
||||||
optionally enforce additional controls on the edge (mTLS, firewall).
|
|
||||||
- The two-worker concurrency bound mitigates bursts, not all denial-of-service
|
|
||||||
scenarios: configured cache size and storage latency still determine scan
|
|
||||||
cost. Per-IP quotas are not provided here.
|
|
||||||
- Proxy protocol path, certificate verification, arbitrary upstream code,
|
|
||||||
all application content and third-party packages are **not certified safe**
|
|
||||||
by this audit. Production acceptance remains open until the runtime test
|
|
||||||
plan is completed.
|
|
||||||
+1
-1
@@ -350,7 +350,7 @@ The adjacent **info (`i`) popover** in Settings → Cache shows ready-to-copy
|
|||||||
`curl -X PURGE` and `curl -X BAN` examples in English and Polish, including
|
`curl -X PURGE` and `curl -X BAN` examples in English and Polish, including
|
||||||
the explicit `BAN /` confirmation. These public methods authorize by the
|
the explicit `BAN /` confirmation. These public methods authorize by the
|
||||||
source IP/CIDR only; they do not accept a browser admin session as a bypass.
|
source IP/CIDR only; they do not accept a browser admin session as a bypass.
|
||||||
See [0.31.0 security audit](SECURITY_AUDIT_0.31.0.md) for reverse-proxy trust and tests.
|
See [security audit](SECURITY_AUDIT.md) for reverse-proxy trust and tests.
|
||||||
|
|
||||||
## Static-response transformation
|
## Static-response transformation
|
||||||
|
|
||||||
|
|||||||
@@ -1,236 +0,0 @@
|
|||||||
# RustProxy 0.34.0 verification report
|
|
||||||
|
|
||||||
## Acceptance status
|
|
||||||
|
|
||||||
**Source candidate, not verified production release.** Rust (`cargo`, `rustc`) is
|
|
||||||
not installed in this execution environment, and the runtime has no access to
|
|
||||||
its dependency registry. Consequently the modified Rust source was **not
|
|
||||||
compiled**, the new Rust tests were **not executed**, and no 0.34.0 RustProxy
|
|
||||||
binary or live throughput figures are supplied. Source inspection and simulated
|
|
||||||
UI/cache tests do not close the runtime acceptance gate.
|
|
||||||
|
|
||||||
## DEV 1 KiB counters follow-up (source remains uncompiled)
|
|
||||||
|
|
||||||
The newly supplied binary `rust-proxy(1)` identifies as 0.34.0 and has
|
|
||||||
SHA-256 `1a84c58b20945bba9fdd1bc1a44d9e9d6ac7659d714343a46a1a8511a78331fa`.
|
|
||||||
It completed **151 passing real HTTP tests, 0 failures, 1 optional zstd
|
|
||||||
skip**. These tests were run on the supplied binary, not on the subsequently
|
|
||||||
modified source. The latest source moves per-domain cache statistics outside
|
|
||||||
HOT store mutexes and lazily allocates analytics samples only after a queue
|
|
||||||
permit is available. Histogram completion now increments one exclusive
|
|
||||||
latency bin instead of up to 11 cumulative bins; both Prometheus exporters
|
|
||||||
convert them to the same cumulative format at scrape time. Static
|
|
||||||
source/SQL/upgrade checks pass; **Rust compile,
|
|
||||||
Rust unit tests and live new-source RPS remain OPEN**. See
|
|
||||||
`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md` and the separate evidence archive.
|
|
||||||
|
|
||||||
## DEV 1 KiB reverse-proxy follow-up (October 2026; source only)
|
|
||||||
|
|
||||||
The supplied **old executable** passed loopback HTTP/1.1 1,040-byte content
|
|
||||||
checks and ApacheBench 12,000-request samples. The configuration trial
|
|
||||||
`max_idle_per_host=4 -> 64` increased short-run RPS, but shared-CPU noise
|
|
||||||
precludes a stable nginx-parity percentage. The edited sources now use 64 by
|
|
||||||
default for absent/new keepalive fields and reduce exclusive-Drop mutex/clone
|
|
||||||
and absent-cookie overhead. See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`.
|
|
||||||
**No newly modified binary was built or run.** Rust unit tests, runtime
|
|
||||||
regressions, long soaks and post-edit code A/B are still required.
|
|
||||||
|
|
||||||
Current DEV source checks performed on the modified tree:
|
|
||||||
|
|
||||||
| Check | Result | Scope |
|
|
||||||
|---|---|---|
|
|
||||||
| `./dev.sh doctor` | PASS | Source preflight; reports `cargo` and `rustc` absent |
|
|
||||||
| `scripts/tests/rps_hotpath_034_static_test.py` | PASS | Guardrails, defaults, retained routing/security and source-size limits only |
|
|
||||||
| `scripts/tests/proxy_hotpath_034_static_test.py` | PASS | Routing and telemetry source guards only |
|
|
||||||
| `scripts/tests/cache_scope_034_static_test.py`, `cache_hot_034_static_test.py`, `sendfile_034_static_test.py` | PASS | Cache and WWW source guards only |
|
|
||||||
| `scripts/tests/dashboard_presets_sql_test.py` | PASS | SQLite migrations and preset constraints |
|
|
||||||
| `scripts/tests/upgrade_performance_test.py` | PASS | Disposable upgrade/preservation/locking test |
|
|
||||||
| `scripts/tests/sendfile_034_panel_browser_test.py` | PASS, 2 widths | Simulated Chromium admin API, not live Rust |
|
|
||||||
| `scripts/tests/vhost_backend_popup_0337_browser_test.py` | PASS, 4 widths | Simulated Chromium admin API, not live Rust |
|
|
||||||
| Node `--check` (vhost editor), Python `compileall`, JSON parser (OpenAPI) | PASS | Syntax only |
|
|
||||||
| `./dev.sh check`, `./dev.sh test` | BLOCKED, exit 127 | Rust toolchain missing; no Rust unit/build acceptance |
|
|
||||||
| Old supplied binary, `ab`/nginx configuration A/B | COMPLETED, DIAGNOSTIC | 1,040-byte H1 payload; shared-CPU and no new code exercised |
|
|
||||||
|
|
||||||
|
|
||||||
## Later 0.34.0 reverse-proxy optimization revision
|
|
||||||
|
|
||||||
The refreshed source archive additionally reuses immutable route handles,
|
|
||||||
limits lazy host wildcard allocation, fast-paths one-backend non-cookie LB,
|
|
||||||
atomically caches healthy state and exchange scalar log fields, shares
|
|
||||||
compiled per-member health policies across leases, and avoids
|
|
||||||
unneeded fallback/retry header clones and empty-subscriber broadcast clones.
|
|
||||||
The existing cache/sendfile work and all panel controls remain. Added Rust
|
|
||||||
regression cases and the source-only hot-path guard are **not a substitute**
|
|
||||||
for compilation and live HTTP, health, capacity, sticky, and H2 retry tests.
|
|
||||||
No RPS improvement, corrected nginx gap, or full functional parity has been
|
|
||||||
measured for this new revision. Details: `docs/PROXY_HOTPATH_0.34.0.md`.
|
|
||||||
|
|
||||||
## RPS-focused follow-up source revision (not yet compiled)
|
|
||||||
|
|
||||||
See `docs/PROXY_RPS_0.34.0.md` for detailed changes in request IDs, validated
|
|
||||||
upstream URI construction, LB scheduling, probes, streaming byte accounting,
|
|
||||||
unchanged pool generation lifetimes and log-history ownership. Source-only invariants pass.
|
|
||||||
A live test of the PREVIOUS executable (not this source revision) completed
|
|
||||||
110 passing cases, 0 failing cases and 1 skipped optional zstd decoder in the
|
|
||||||
proxy/policy/cache/transform harness. The old executable also confirms query-
|
|
||||||
string bypass is deliberate and reproduces the older Age/TTL bug. The new
|
|
||||||
binary has NOT been built or tested and its RPS gain is unknown.
|
|
||||||
|
|
||||||
## HOT HIT follow-up source revision (not yet compiled)
|
|
||||||
|
|
||||||
See `docs/CACHE_HOTPATH_0.34.0.md`. Response header cloning has moved outside
|
|
||||||
the global HOT lock, and redundant bounded stats key allocation, empty BAN
|
|
||||||
checks and full HOT TTL sweeps were removed. A new Rust concurrency/regression
|
|
||||||
suite and an ignored microbenchmark have been added. Existing COLD and Vary
|
|
||||||
behavior, exact LRU and cache semantics remain. This source has not been built
|
|
||||||
or run under Rust in this environment; no RPS gain is claimed.
|
|
||||||
|
|
||||||
## Changes and correctness boundaries
|
|
||||||
|
|
||||||
- Operational per-vhost saves (upstream keepalive/recycling, connection limits,
|
|
||||||
load-balancing/health policy, timeouts, request gate and diagnostics) retain
|
|
||||||
HOT and COLD entries. Backend origin, routing, content, cache and transform
|
|
||||||
changes retain conservative invalidation. Vhost-specific changes purge only
|
|
||||||
that vhost; global cache, transform, forwarding, headers and Web Service or
|
|
||||||
PHP-FPM profile changes may conservatively purge all tiers.
|
|
||||||
- Scope-specific cold-cache records persist an origin/content compatibility
|
|
||||||
fingerprint. COLD startup rejects stale scope records and deletes their cache
|
|
||||||
artifacts. Legacy COLD metadata defaults its fingerprint to zero; production
|
|
||||||
vhosts have a nonzero SHA-derived fingerprint. A one-time cold refill can
|
|
||||||
occur on the 0.33.8 -> 0.34.0 upgrade because the global identity changed.
|
|
||||||
- A separate administrative configuration revision acts as a cache-ticket
|
|
||||||
publication barrier: a request bound to a pre-save routing snapshot cannot
|
|
||||||
create a new cache ticket after the save. The data revision stays stable for
|
|
||||||
operational-only saves, so previously cached objects remain available.
|
|
||||||
- Reverse-proxy response-body byte metering captures the backend lease once at
|
|
||||||
wrapper creation instead of acquiring a backend-selection mutex per DATA
|
|
||||||
frame. Request-body accounting, error paths and exchange ownership remain.
|
|
||||||
- An **opt-in `sendfile: auto`** setting is wired through Web Service serde,
|
|
||||||
compiled config, the Delivery UI and documented API. The Linux HTTP/1.1
|
|
||||||
TCP transport adapter calls `libc::sendfile` for eligible static files;
|
|
||||||
cache-enabled, ranged, compressed, transformed and TLS/H2/H3 paths retain
|
|
||||||
existing streaming. It is **disabled by default** and must pass native
|
|
||||||
compilation, correctness, transport and load tests before use. No persisted
|
|
||||||
SQLite schema migration or LB UI change is required.
|
|
||||||
|
|
||||||
## Local verification
|
|
||||||
|
|
||||||
| Check | Outcome | Coverage / caveat |
|
|
||||||
|---|---|---|
|
|
||||||
| `./dev.sh doctor` | PASS | Project source layout and tool inventory |
|
|
||||||
| SQL migration/preset test | PASS | All ten migrations re-applied, preset access and limits |
|
|
||||||
| Upgrade test | PASS | Stop/lock guards, SQLite/.env/language preservation, symlink rejection |
|
|
||||||
| Python compileall | PASS | Python syntax only |
|
|
||||||
| Vhost LB/Cache Chromium test | PASS | Simulated browser quick-view, 320/375/768/1440px |
|
|
||||||
| Other Chromium scripts | 15 PASS | Simulated API/CSS/JS, not a live Rust server |
|
|
||||||
| Sidebar navigation Chromium test | INCOMPLETE | Execution time budget; not counted as a pass |
|
|
||||||
| Three Node dashboard scripts | PASS | Simulated UI logic |
|
|
||||||
| `./dev.sh check` and `./dev.sh test` | BLOCKED | `cargo`/`rustc` unavailable |
|
|
||||||
| Rust unit/integration tests (including new scoped COLD regression) | NOT RUN | Requires Rust toolchain |
|
|
||||||
| Live HTTP/protocol/LB regressions | NOT RUN | Requires built 0.34.0 binary |
|
|
||||||
| 0.33.8 / 0.34.0 / nginx paired RPS benchmarks | NOT RUN | No verified 0.34.0 binary |
|
|
||||||
| Zero-copy sendfile WWW setting | SOURCE IMPLEMENTED / NOT RUNTIME VERIFIED | Linux plain H1 static GET only, `off` by default |
|
|
||||||
| Sendfile editor Chromium save/reload | PASS | 375/1440px, simulated admin API, off -> auto -> off |
|
|
||||||
| Sendfile Rust adapter + fallback test code | ADDED / NOT RUN | TCP marker replacement, persistent socket, HEAD, range, conditional; compiler unavailable |
|
|
||||||
| Amdahl sendfile/nginx sensitivity study | PASS MODEL | No runtime evidence for simulated improvements |
|
|
||||||
|
|
||||||
The 0.33.8 report in the supplied evidence archive recorded **706 passing
|
|
||||||
Rust tests**, **152 HTTP assertions**, **111 transport assertions** and
|
|
||||||
**17 LB assertions**. Those are *prior-version* results and are **not evidence
|
|
||||||
that 0.34.0 passes**. This report never transfers previous pass counts.
|
|
||||||
|
|
||||||
## Synthetic cache-retention scenario (not an RPS measurement)
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/simulate_cache_034.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Assume 20 vhosts, 100 HOT + 100 COLD objects each, all public/cacheable with
|
|
||||||
infinite TTL and adequate cache capacity; 1,000 configuration saves (800
|
|
||||||
operational-only, 200 content changes to one vhost), followed by 500 uniformly
|
|
||||||
random GETs per save. A miss immediately reloads an object into its original
|
|
||||||
tier. No networking, filesystem, CPU timing, origin delay or concurrency is
|
|
||||||
modeled. Results with seed 3400:
|
|
||||||
|
|
||||||
| Synthetic metric | Old global purge | 0.34.0 scoped retention |
|
|
||||||
|---|---:|---:|
|
|
||||||
| Object invalidations | 473,540 | 37,300 |
|
|
||||||
| GET misses | 470,006 | 37,069 |
|
|
||||||
| GET hits | 29,994 | 462,931 |
|
|
||||||
| HIT ratio | 5.999% | 92.586% |
|
|
||||||
|
|
||||||
The modeled change yields **92.123% fewer invalidations**, **92.113% fewer
|
|
||||||
misses**, and **+86.587 percentage points** of synthetic cache HIT ratio.
|
|
||||||
**Measured 0.34.0 requests/s gain: unknown.** These values must not be
|
|
||||||
presented as a real speedup or a projection against nginx.
|
|
||||||
|
|
||||||
## Previously measured comparison (0.33.8, not this candidate)
|
|
||||||
|
|
||||||
From the attached 0.33.8 evidence, five paired benchmark runs with cache OFF,
|
|
||||||
median RPS:
|
|
||||||
|
|
||||||
| Mode / payload | 0.33.8 RPS | nginx RPS | 0.33.8 / nginx |
|
|
||||||
|---|---:|---:|---:|
|
|
||||||
| Reverse proxy 1,040 B | 29,334.12 | 170,610.16 | 17.19% |
|
|
||||||
| Reverse proxy 64 KiB | 20,735.07 | 35,393.29 | 58.58% |
|
|
||||||
| Reverse proxy 1 MiB | 4,461.19 | 2,730.23 | 163.40% |
|
|
||||||
| WWW 1,040 B | 24,906.20 | 351,605.13 | 7.08% |
|
|
||||||
| WWW 64 KiB | 20,856.67 | 203,709.91 | 10.24% |
|
|
||||||
| WWW 1 MiB | 2,833.71 | 12,611.49 | 22.47% |
|
|
||||||
|
|
||||||
The 0.33.8 1 MiB proxy path was **2.77% slower than rebuilt 0.33.7**.
|
|
||||||
Removing a per-response-frame mutex is a performance candidate but is **not**
|
|
||||||
a measured correction of that regression. No throughput gains are attributed
|
|
||||||
here to the new source changes.
|
|
||||||
|
|
||||||
## Hypothetical 0.34.0/nginx gap sensitivity (NOT MEASURED)
|
|
||||||
|
|
||||||
Run `python3 scripts/performance/simulate_nginx_gap_034.py`. Using the **real
|
|
||||||
0.33.8 measured medians** above, assume zero-copy removes 50% of the
|
|
||||||
read/copy part of per-request cost; that part hypothetically accounts for
|
|
||||||
20/40/60% of *total* cost for eligible static files. Model **1/3/5%** total
|
|
||||||
service-time savings from removed proxy-frame mutexes, independently.
|
|
||||||
Amdahl: `RPS_sim = RPS_0.33.8 / (1 - hypothetical_fraction_saved)`. The
|
|
||||||
result is a sensitivity calculation, **not a prediction or 0.34.0 benchmark**.
|
|
||||||
|
|
||||||
| Workload | Measured 0.33.8/nginx | Modeled low | Modeled middle | Modeled high |
|
|
||||||
|---|---:|---:|---:|---:|
|
|
||||||
| Proxy 1040 B | 17.19% | 17.37% | 17.73% | 18.10% |
|
|
||||||
| Proxy 64 KiB | 58.58% | 59.18% | 60.40% | 61.67% |
|
|
||||||
| Proxy 1 MiB | 163.40% | 165.05% | 168.45% | 172.00% |
|
|
||||||
| WWW 1040 B | 7.08% | 7.08% | 7.08% | 7.08% |
|
|
||||||
| WWW 64 KiB | 10.24% | 11.38% | 12.80% | 14.63% |
|
|
||||||
| WWW 1 MiB | 22.47% | 24.97% | 28.09% | 32.10% |
|
|
||||||
|
|
||||||
These ratios apply only to eligible Linux plain HTTP/1.1 static workloads
|
|
||||||
with cache OFF. TLS, HTTP/2, HTTP/3, ranged, compressed and object-cached
|
|
||||||
responses are **not** accelerated by this bridge. Bandwidth, contention,
|
|
||||||
queueing, realistic filesystem and actual CPU profiles are not modeled.
|
|
||||||
Even the favorable modeled 1 MiB case remains ~3.1x slower than nginx WWW;
|
|
||||||
small WWW remains ~14.1x slower. Proxy 1 MiB already exceeds nginx in the
|
|
||||||
**prior baseline**, not as a demonstrated new 0.34.0 improvement.
|
|
||||||
|
|
||||||
## Required rollout gates
|
|
||||||
|
|
||||||
1. On a trusted Linux build worker with the pinned Rust toolchain, run
|
|
||||||
`./dev.sh check`, `CARGO_INCREMENTAL=0 ./dev.sh test`, and a locked release
|
|
||||||
build. Resolve any compiler/test issues before a deployment.
|
|
||||||
2. Follow `scripts/performance/README.md` to run `regression.py`,
|
|
||||||
`transport_regression.py`, `lb_regression.py`, and
|
|
||||||
`concurrency_regression.py` against verified 0.33.8 and 0.34.0 binaries.
|
|
||||||
3. Exercise administration saves during simultaneous requests: save keepalive
|
|
||||||
values and confirm unchanged HOT/COLD entries and hashes; change one backend
|
|
||||||
origin and confirm only its scope is purged, both before/after restart. Check
|
|
||||||
that old snapshots cannot recreate invalidated objects.
|
|
||||||
4. Run the five-repeat, 15-second paired benchmark against 0.33.8, 0.34.0 and
|
|
||||||
nginx on the *same host and affinity*, including large proxy responses, CPU,
|
|
||||||
RSS, resource limits and response correctness.
|
|
||||||
5. Compile the source-level Linux `sendfile` bridge and validate on a running
|
|
||||||
RustProxy process: plain HTTP/1 sendfile path, headers/body exactness,
|
|
||||||
keepalive/pipelining, slow clients, cancellation, HEAD, range, cache,
|
|
||||||
compression, TLS/H2/H3 fallback, symlink security and observability. Do
|
|
||||||
**not** activate Auto on production before that gate passes.
|
|
||||||
6. Complete a mixed-traffic soak and staged rollback rehearsal. Until then,
|
|
||||||
do not describe 0.34.0 as production-accepted.
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
# RustProxy TODO
|
|
||||||
|
|
||||||
This file contains open work that remains after the current RustProxy 0.25 Web Server implementation.
|
|
||||||
|
|
||||||
## Web Server follow-up
|
|
||||||
|
|
||||||
- Multipart byte-range responses if required; current single-range/If-Range support remains the default scope.
|
|
||||||
- Named locations only if they can reuse the existing vhost/location router rather than introduce a second router.
|
|
||||||
- Optional PHP `PATH_INFO` script/path splitting after a confinement-safe rule is defined.
|
|
||||||
- Additional FastCGI observability (bounded active/error/latency counters) without per-script labels.
|
|
||||||
- A race-resistant non-Unix filesystem confinement implementation before enabling the Web Server on non-Unix targets.
|
|
||||||
|
|
||||||
## Release acceptance
|
|
||||||
|
|
||||||
- Run `./dev.sh fmt`, `./dev.sh check` and `./dev.sh test` on a host with the stable Rust toolchain.
|
|
||||||
- Run live static-file acceptance over HTTP/1.1, HTTP/2 and HTTP/3, including ACL/auth/custom-error/header behavior.
|
|
||||||
- Run a real PHP-FPM acceptance matrix over a Unix socket and loopback TCP, including `index.php`, front-controller `try_files`, POST bodies, timeouts and source-disclosure failure tests.
|
|
||||||
|
|
||||||
Completed release work belongs in `CHANGELOG.md`; planned version work belongs in `ROADMAP.md`; implementation/audit evidence belongs in the matching audit document.
|
|
||||||
+1
-1
@@ -234,7 +234,7 @@ origin do not expose a selected backend or stale origin timing.
|
|||||||
| `pool_recycle_seconds` | 3600 | 0 (off) or 60..604800 |
|
| `pool_recycle_seconds` | 3600 | 0 (off) or 60..604800 |
|
||||||
| `http1_max_buffer_bytes` | 131072 | 8192..1048576 |
|
| `http1_max_buffer_bytes` | 131072 | 8192..1048576 |
|
||||||
|
|
||||||
The new default of 64 applies when this field is absent in a newly created or deserialized vhost; existing saved values (including 4) remain unchanged. Each idle upstream connection consumes a file descriptor and memory, so use a smaller value for resource-constrained deployments with many origins. See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md` for the live configuration A/B evidence.
|
The new default of 64 applies when this field is absent in a newly created or deserialized vhost; existing saved values (including 4) remain unchanged. Each idle upstream connection consumes a file descriptor and memory, so use a smaller value for resource-constrained deployments with many origins.
|
||||||
|
|
||||||
These configure HTTP idle reuse and TCP keepalive; HTTP/2 also uses pings for active
|
These configure HTTP idle reuse and TCP keepalive; HTTP/2 also uses pings for active
|
||||||
connections, with a 10-second ping timeout and no pinging solely to keep idle
|
connections, with a 10-second ping timeout and no pinging solely to keep idle
|
||||||
|
|||||||
+1
-3
@@ -134,9 +134,7 @@ does not change the listener protocol configuration.
|
|||||||
tested against a running 0.34.0 RustProxy binary in the current environment.
|
tested against a running 0.34.0 RustProxy binary in the current environment.
|
||||||
The included Rust unit/integration test and real-network test cases must pass
|
The included Rust unit/integration test and real-network test cases must pass
|
||||||
on a Linux build worker before enabling in production. Sendfile cannot by
|
on a Linux build worker before enabling in production. Sendfile cannot by
|
||||||
itself close the small-response gap to nginx. See
|
itself close the small-response gap to nginx. Live protocol and performance acceptance is still required.
|
||||||
`docs/TEST_REPORT_0.34.0.md` and
|
|
||||||
`scripts/performance/simulate_nginx_gap_034.py`.
|
|
||||||
|
|
||||||
### Intercept application HTTP errors (0.29.7)
|
### Intercept application HTTP errors (0.29.7)
|
||||||
|
|
||||||
|
|||||||
@@ -1,354 +0,0 @@
|
|||||||
# Live performance and regression harness
|
|
||||||
|
|
||||||
## DEV OPT3 / 1 KiB response: uncompiled source
|
|
||||||
|
|
||||||
`docs/PROXY_1KB_OPT3_DEV_0.34.0.md` describes 7 optimized hot-path sites
|
|
||||||
and how to benchmark their **newly rebuilt** executable. Baseline
|
|
||||||
`rust-proxy(2)` passed 89 real proxy/policy/cache checks. This runtime result
|
|
||||||
is not evidence that uncompiled OPT3 changes work, nor a measured OPT3 RPS gain.
|
|
||||||
|
|
||||||
These scripts exercise actual RustProxy executables through loopback sockets.
|
|
||||||
They do not simulate Rust request handling. The Python server in `peer.py` is a
|
|
||||||
controllable test **upstream**, used to verify streaming, non-replayable writes,
|
|
||||||
headers, cache policy and WebSocket frames. Performance measurements use nginx
|
|
||||||
as the shared upstream instead of Python.
|
|
||||||
|
|
||||||
## DEV 1 KiB histogram writes (source candidate)
|
|
||||||
|
|
||||||
`VhostMetrics::complete` now writes one exclusive histogram bin for each
|
|
||||||
completed exchange instead of up to 11 cumulative atomics. Both Prometheus
|
|
||||||
exporters reconstruct `le` and `+Inf` at scrape time. Unit tests cover boundary
|
|
||||||
semantics. This change has not been compiled or benchmarked: see
|
|
||||||
`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`.
|
|
||||||
|
|
||||||
## DEV 1 KiB cache counter follow-up
|
|
||||||
|
|
||||||
See `docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`. Cache domain metrics no longer
|
|
||||||
lock the HOT store on BYPASS; analytics cloning is deferred until the queue
|
|
||||||
has capacity. The new **source** remains uncompiled in the editing environment.
|
|
||||||
For quick 1,040-byte loopback diagnostics with ApacheBench (not a production
|
|
||||||
acceptance benchmark), use `python3 scripts/performance/ab_1kb_dev.py --help`.
|
|
||||||
The authoritative long-run comparison remains `benchmark.py` with pinned
|
|
||||||
CPU sets, an explicitly equal upstream idle pool and multiple `wrk` repeats.
|
|
||||||
|
|
||||||
## DEV 1 KiB follow-up on 0.34.0 (newer source)
|
|
||||||
|
|
||||||
See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. The previous executable's 4-vs-64
|
|
||||||
idle-connection tuning experiment is real loopback evidence, but the *edited*
|
|
||||||
source has not been compiled. Its new default is 64 for new vhosts, whereas
|
|
||||||
explicitly saved `4` remains `4`. Keep the idle limit equal between before/after
|
|
||||||
*code* benchmarks; measure configuration tuning in a separate labelled series.
|
|
||||||
The source release is not eligible for a claim of 100k RPS or 80% nginx yet.
|
|
||||||
|
|
||||||
## Reverse-proxy optimization revision 0.34.0
|
|
||||||
|
|
||||||
After compiling the refreshed candidate, measure the same unmodified 0.33.8
|
|
||||||
binary and nginx reference as before, without turning off logging, request
|
|
||||||
IDs, ACL, cache-policy enforcement or LB health. Profile request-size 1,040 B
|
|
||||||
first. Before RPS runs execute full proxy, LB, transport and concurrency
|
|
||||||
regressions (including single-member capacity, recovery, sticky cookies,
|
|
||||||
replays on safe methods, rejected non-replayable writes and H2 connection
|
|
||||||
shutdown). Treat `--max-idle` tuning and recycles as separately labelled
|
|
||||||
experiments rather than mixing them into the primary comparison. There are no
|
|
||||||
new measured RPS figures in the source candidate.
|
|
||||||
|
|
||||||
## Follow-up RPS candidate on 0.34.0
|
|
||||||
|
|
||||||
`docs/PROXY_RPS_0.34.0.md` tracks the code changes and their regressions.
|
|
||||||
Compile before/after binaries from the two 0.34.0 source revisions and use
|
|
||||||
the paired benchmark below; a live run against the older binary cannot
|
|
||||||
establish a measured RPS improvement for the uncompiled candidate. Keep
|
|
||||||
`max_idle_per_host`, recycling, HTTP1/H2 settings and all application security
|
|
||||||
and logging features equal between runs. Treat any configuration tuning as a
|
|
||||||
separate experiment. Profile the resulting binary with `perf` only after
|
|
||||||
correctness checks pass.
|
|
||||||
|
|
||||||
## Offline cache-retention model for 0.34.0
|
|
||||||
|
|
||||||
`python3 scripts/performance/simulate_cache_034.py` compares cache invalidation
|
|
||||||
policies under a deterministic synthetic sequence of vhost saves and uniform
|
|
||||||
cacheable GETs. The model contains **no RustProxy server**, no timing and no
|
|
||||||
network traffic; its percentages are **not measured RPS gains**. See
|
|
||||||
`docs/TEST_REPORT_0.34.0.md` for assumptions and the live acceptance gap.
|
|
||||||
|
|
||||||
## Requirements
|
|
||||||
|
|
||||||
- Linux with readable `/proc`, Python 3.10+, executable RustProxy binaries.
|
|
||||||
- nginx and wrk for performance comparisons. Both paths are explicit; no
|
|
||||||
system-wide installation is required.
|
|
||||||
- `taskset` from util-linux for the benchmark's explicit CPU affinity and for
|
|
||||||
optional affinity settings in the other scripts.
|
|
||||||
- Optional Python `brotli` and `zstandard` decoders increase compression
|
|
||||||
acceptance coverage. A missing optional decoder is reported as **skipped**.
|
|
||||||
- Permission to bind unprivileged loopback sockets and start local processes.
|
|
||||||
In a sandbox without sockets or `/proc`, run in the approved local execution
|
|
||||||
environment. Do not replace real runtime results with a peer self-test.
|
|
||||||
|
|
||||||
Each run creates a new private temporary data directory and initializes its own
|
|
||||||
administrator with a random password supplied only over stdin. Configurations
|
|
||||||
are applied through RustProxy's existing authenticated CLI/API and listener
|
|
||||||
changes receive a restart. No historical SQLite database is copied or edited.
|
|
||||||
TLS keys, databases and passwords are never written to the evidence directory.
|
|
||||||
Temporary state is outside the evidence tree and is removed on normal exit.
|
|
||||||
`fixture-cleanup.json` records the cleanup check. Interrupted runs can leave an
|
|
||||||
unrelated `/tmp/rustproxy-perf-*` directory; do not include it in a release.
|
|
||||||
|
|
||||||
Before timed benchmark, soak or syscall-profile work, `wait_quiescent()` waits
|
|
||||||
for the existing authenticated ACME runtime API to report `ready` or
|
|
||||||
`unavailable`. A fresh RustProxy data directory can start a Python/Certbot
|
|
||||||
environment bootstrap even when no certificates have an issuer. The gate
|
|
||||||
requires no configured issuers, jobs or renewal observations, the same live Rust
|
|
||||||
PID and start time, and no child processes for five continuous seconds. It polls
|
|
||||||
every 0.5 seconds and fails after 180 seconds instead of measuring installation
|
|
||||||
work. Selected status, process count and timing evidence is saved in
|
|
||||||
`<label>-quiescence.json`. No PATH, pip settings or application features are
|
|
||||||
changed. An `unavailable` result permits the HTTP fixture to proceed; these runs
|
|
||||||
do not test ACME issuance or certificate-provider availability.
|
|
||||||
|
|
||||||
## Regression tests
|
|
||||||
|
|
||||||
Run from the repository root, providing absolute binary and output paths:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/regression.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--out /absolute/path/results/regression
|
|
||||||
```
|
|
||||||
|
|
||||||
`--groups static,proxy,policy,cache,transform` selects groups; all are enabled by
|
|
||||||
default. `--workers 4` sets the same `TOKIO_WORKER_THREADS` for each binary.
|
|
||||||
The report contains individual pass/fail/skip observations and a comparison
|
|
||||||
matrix. An existing failure in the supplied binary remains a failure in its
|
|
||||||
report; it is not silently converted to a successful baseline expectation.
|
|
||||||
|
|
||||||
Coverage includes:
|
|
||||||
|
|
||||||
- Static GET/HEAD and byte checksums at 1,040 B, 64 KiB and 1 MiB; empty files;
|
|
||||||
ETag/Last-Modified conditional requests; prefix/suffix/open/multipart ranges,
|
|
||||||
If-Range and 416; directory redirect/index/listing; decoded path rejection;
|
|
||||||
deny/inside-root symlinks; precompressed gzip; MIME/download headers;
|
|
||||||
`try_files`, redirects and the compiled `.htaccess` header subset.
|
|
||||||
- Proxy POST/PUT, 1 MiB upload checksums, chunked upload delivery **before the
|
|
||||||
last chunk is sent**, early response delivery, client disconnect, original
|
|
||||||
status/redirect preservation, failed-POST no replay, request/response header
|
|
||||||
scopes, hop-by-hop stripping, request-ID shape/uniqueness and WebSocket echo.
|
|
||||||
- Global host/vhost/profile/location ACL composition, upstream suppression on
|
|
||||||
denial, trusted versus spoofed forwarding, untrusted country hints, method
|
|
||||||
and rate policies, hot changes and rejection of invalid configuration.
|
|
||||||
- Private administration routes, login, session/CSRF and Origin enforcement,
|
|
||||||
Basic Auth, ACL-before-auth, private-origin rejection, retained exchange
|
|
||||||
logging fields and byte accounting.
|
|
||||||
- Cache global/vhost/route OFF and ON, MISS/HIT, Vary isolation, cookie,
|
|
||||||
Authorization/private/no-store bypass, PURGE, concurrent fills, validator
|
|
||||||
revalidation, static cached ranges/conditionals; transforms OFF/ON with cache
|
|
||||||
OFF/ON; compression negotiation, exclusions and scope/global codec ceilings.
|
|
||||||
|
|
||||||
The generic suite does not establish TLS/H2/H3, Tinyauth, real PHP-FPM, real DNS
|
|
||||||
rebinding, concurrent filesystem rename-race safety or long-soak acceptance.
|
|
||||||
Use the dedicated transport tests and existing Rust integration suites for
|
|
||||||
their covered cases. The sequential symlink-target test is explicitly not a
|
|
||||||
concurrent rename-race proof.
|
|
||||||
|
|
||||||
## TLS, HTTP/2, HTTP/3 and WebSocket interoperability
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/transport_regression.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--out /absolute/path/results/transport
|
|
||||||
```
|
|
||||||
|
|
||||||
This suite requires `aioquic`, `httpx[http2]` and `websockets`; use
|
|
||||||
`--python-deps /absolute/path/pydeps` for a separate dependency directory.
|
|
||||||
It verifies the temporary server certificate, negotiated protocol, response
|
|
||||||
status and bytes for TLS H2 and QUIC H3, including HEAD, validators, ranges,
|
|
||||||
cache ON/OFF, 1 MiB bodies and concurrent streams. Dedicated WebSocket checks
|
|
||||||
include bidirectional echo and a normal close handshake. Client protocol
|
|
||||||
self-checks are reported separately from RustProxy acceptance cases.
|
|
||||||
This establishes covered interoperability, not TLS/H2/H3 throughput or
|
|
||||||
production-certificate compatibility.
|
|
||||||
|
|
||||||
## Two-origin load-balancer regression
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/lb_regression.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--out /absolute/path/results/lb
|
|
||||||
```
|
|
||||||
|
|
||||||
This separate, small suite sends real requests through RustProxy to two local
|
|
||||||
TCP origins. It verifies equal round-robin allocation, GET failover after a 503
|
|
||||||
and after a stopped origin refuses connections, and the absence of POST replay.
|
|
||||||
A controlled response pauses after its first body fragment while configuration
|
|
||||||
changes drain or disable its member. New requests must use the other member,
|
|
||||||
the admitted response must finish intact, and its active lease must remain
|
|
||||||
counted until completion and then return to zero. The main HTTP case matrix is
|
|
||||||
unchanged. This suite does not cover sticky sessions, active health probes,
|
|
||||||
DNS rotation or TLS upstream failures.
|
|
||||||
|
|
||||||
## Finite concurrency correctness stress
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/concurrency_regression.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--nginx /absolute/path/nginx --out /absolute/path/results/concurrency
|
|
||||||
```
|
|
||||||
|
|
||||||
This separate functional check uses 1, 16 and 128 concurrent client tasks for
|
|
||||||
proxy and static responses at 1,040 B, 64 KiB and 1 MiB. Each finite batch sends
|
|
||||||
`max(32, 2 * clients)` requests and verifies every status, length and body SHA-256.
|
|
||||||
After the clients finish, backend active-exchange counters must settle to zero.
|
|
||||||
RSS and file descriptors are recorded before the batch and after its drain.
|
|
||||||
These are not peak resource measurements or throughput results. Client tasks
|
|
||||||
start behind a barrier; configured concurrency does not establish that the same
|
|
||||||
number of server exchanges were active simultaneously.
|
|
||||||
|
|
||||||
## Before/after/nginx benchmark
|
|
||||||
|
|
||||||
First verify starts, loopback routing and exact payload hashes without invoking
|
|
||||||
the load generator:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/benchmark.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
|
|
||||||
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
|
|
||||||
--out /absolute/path/results/preflight --prepare-only
|
|
||||||
```
|
|
||||||
|
|
||||||
For a host with CPUs 0–8 available, a reproducible primary run is:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/benchmark.py \
|
|
||||||
--binary before=/absolute/path/rust-proxy-before \
|
|
||||||
--binary after=/absolute/path/rust-proxy-after \
|
|
||||||
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
|
|
||||||
--out /absolute/path/results/primary \
|
|
||||||
--workers 4 --origin-workers 2 --client-threads 4 \
|
|
||||||
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
|
|
||||||
--concurrency 64 --sizes small,large,huge --modes proxy,static \
|
|
||||||
--duration 15 --warmup 3 --repeats 5
|
|
||||||
```
|
|
||||||
|
|
||||||
Choose explicit, nonempty and disjoint CPU sets allowed in the run environment;
|
|
||||||
adapt all three sets on a smaller host. Frontends use the same CPU set and worker
|
|
||||||
count, the shared origin uses another set, and the generator uses a third. The
|
|
||||||
harness checks each process/thread's actual affinity and refuses unavailable
|
|
||||||
resource counters. Four wrk threads share the three client CPUs in this example.
|
|
||||||
The structured result records thread count and effective connection count; at
|
|
||||||
64 connections, four threads keep exactly 64 connections, avoiding wrk's integer
|
|
||||||
division of connections between threads. Only one frontend is loaded at a time.
|
|
||||||
Stop builds, other tests and profilers during throughput measurement.
|
|
||||||
|
|
||||||
The primary run takes approximately 27 minutes: six scenarios, three variants,
|
|
||||||
five samples, and 18 seconds per sample including excluded warm-up. To explore
|
|
||||||
concurrency, repeat in a fresh output directory with
|
|
||||||
`--concurrency 1,16,64,128`. More scenarios intentionally take more time.
|
|
||||||
Short sanity checks require the explicit `--exploratory` option and cannot
|
|
||||||
satisfy the >=5 samples / >=15-second measurement gate.
|
|
||||||
|
|
||||||
Cache is explicitly disabled for the primary benchmark. Transforms retain their
|
|
||||||
disabled defaults. Identity encoding is requested. Logging history, analytics,
|
|
||||||
request IDs, origin policy, body accounting, health/lease plumbing and normal
|
|
||||||
application functionality remain active. nginx uses `sendfile on`, HTTP/1.1,
|
|
||||||
keep-alive and streaming proxy delivery. The generated exact configurations are
|
|
||||||
saved, so this intentional difference in nginx/application features is visible.
|
|
||||||
No cache-disabled claim is inferred from a cache-on speedup.
|
|
||||||
|
|
||||||
Optional keep-alive tuning uses `--max-idle label=64`. Keep this as a separately
|
|
||||||
labelled experiment. The harness does not turn off bounded pool recycling.
|
|
||||||
|
|
||||||
## Recorded evidence and interpretation
|
|
||||||
|
|
||||||
- `schedule.json`: complete deterministic crossover order, rotating and
|
|
||||||
reversing variants between repetitions. No best-run selection.
|
|
||||||
- `preflight.json`, `fixture/payloads.json`, binary/version/config metadata:
|
|
||||||
exact status, body SHA-256, payload sizes and inputs.
|
|
||||||
- Per-sample exact status/body checks before and after the measured interval,
|
|
||||||
plus a wire-bytes-per-request sanity check, detect wrong content and gross
|
|
||||||
transfer-size anomalies. These do not hash every response inside wrk.
|
|
||||||
- `raw/*.txt` and `raw/*.json`: unmodified wrk output and structured requests,
|
|
||||||
duration, bytes, RPS, p50/p95/p99, effective concurrency, and
|
|
||||||
connect/read/write/timeout/status errors.
|
|
||||||
- `raw/*-resources.json`: timestamped `/proc` samples for each server process
|
|
||||||
and its children, plus the separate origin process group.
|
|
||||||
- `raw/*-affinity.json`: observed process/thread CPU masks for the front end,
|
|
||||||
origin and generator.
|
|
||||||
- `runs.csv/json`, `summary.csv/json`: all samples, medians, RPS range and
|
|
||||||
coefficient of variation, latency medians, server CPU time per completed
|
|
||||||
request, peak/end RSS and file descriptors.
|
|
||||||
- `status/*.json`: numeric drop and pool-generation/rotation counters read
|
|
||||||
outside each measured interval. Warm-up output is kept separately.
|
|
||||||
- `completion.json` and `SHA256SUMS`: completion state and hashes of evidence.
|
|
||||||
Any generator/network/status error stops the run with `failure.json`; partial
|
|
||||||
data remains available and is not presented as completed acceptance.
|
|
||||||
|
|
||||||
Use `(after_median / before_median - 1) * 100` for throughput improvement and
|
|
||||||
`after_median / nginx_median * 100` for relative nginx throughput. Rebuilding
|
|
||||||
both revisions with one compiler, target and resolved lockfile is required for
|
|
||||||
source-change attribution. Compare the supplied executable separately: build
|
|
||||||
flags and dependency versions can change its performance.
|
|
||||||
|
|
||||||
These are same-host, closed-loop HTTP/1.1 diagnostics. They do not establish
|
|
||||||
production capacity, TLS/H2/H3 performance, an open-loop latency SLA, allocator
|
|
||||||
rates, CPU cycles or 24-hour resource stability. wrk latency follows wrk's own
|
|
||||||
histogram/correction behavior. RSS and descriptor peaks are sampled observations,
|
|
||||||
so shorter spikes may be missed. Summed nginx RSS can double-count shared pages.
|
|
||||||
Origin or generator CPU saturation can limit observed throughput; inspect their
|
|
||||||
resource data and move the generator/origin to separate machines for release
|
|
||||||
measurements. Run profilers separately so profiling overhead does not enter the
|
|
||||||
headline A/B throughput numbers.
|
|
||||||
|
|
||||||
## Mixed-traffic resource soak
|
|
||||||
|
|
||||||
Run this in its own phase, after correctness and throughput checks:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/soak.py \
|
|
||||||
--binary /absolute/path/rust-proxy-after --label after \
|
|
||||||
--nginx /absolute/path/nginx --out /absolute/path/results/soak \
|
|
||||||
--duration 1800 --rate 100 --clients 8 \
|
|
||||||
--workers 4 --origin-workers 2 --server-cpus 0-3 --origin-cpus 4-5 \
|
|
||||||
--toggle-seconds 120 --sample-seconds 5 --abort-every 101
|
|
||||||
```
|
|
||||||
|
|
||||||
The bounded client executor alternates proxy/static and 1,040 B/64 KiB/1 MiB
|
|
||||||
payloads, verifies full bytes and occasional HEAD/range semantics, closes some
|
|
||||||
clients after an initial body prefix, and applies live cache OFF/ON changes.
|
|
||||||
Every workload request opens a new frontend HTTP/1.1 connection; upstream
|
|
||||||
keepalive retains its configured defaults. This exercises connection churn.
|
|
||||||
The primary wrk benchmark separately uses persistent frontend connections.
|
|
||||||
The nominal request rate is paced; scheduler delays and actual completion rate
|
|
||||||
are reported instead of hiding an under-driven client. Pending jobs are bounded
|
|
||||||
to twice the client count. The script restores base configuration, drains and
|
|
||||||
rechecks both listeners before shutdown. After each cache apply, two separate
|
|
||||||
GETs per source verify exact contents and require a HIT on the second GET when
|
|
||||||
enabled; globally disabled cache must expose no X-Cache header. These probe
|
|
||||||
requests are recorded separately from paced traffic. Backend active-exchange
|
|
||||||
counters must settle to zero after the final drain and cooldown.
|
|
||||||
|
|
||||||
Five-second resource/status records, body/status failures, observed cache-status
|
|
||||||
counts, cache transitions/probes, RSS/FD trend and completed traffic totals are
|
|
||||||
retained. Controlled client closes mean that the client closes after reading a
|
|
||||||
prefix. The server may already have sent the complete response, so those counts
|
|
||||||
are not presented as proven server-side cancellations.
|
|
||||||
Memory trend is descriptive: switching caching and allocator retention affect
|
|
||||||
RSS. This is a 30-minute resource gate, not a 24-hour leak proof. A short
|
|
||||||
`--exploratory --duration ...` sanity run stays explicitly outside that gate.
|
|
||||||
|
|
||||||
## 0.34.0 hypothesis-only sendfile/nginx gap simulator
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/performance/simulate_nginx_gap_034.py --output /tmp/nginx-gap.json
|
|
||||||
```
|
|
||||||
|
|
||||||
This is **not** a benchmark of RustProxy 0.34.0. It reproduces measured
|
|
||||||
0.33.8/nginx medians with cache OFF and applies explicitly hypothetical
|
|
||||||
Amdahl service-time reductions (WWW and reverse proxy separately). A real
|
|
||||||
RustProxy binary is required to validate the code path, detect regressions
|
|
||||||
and measure latency/RPS against nginx under identical conditions.
|
|
||||||
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Short, same-host ab diagnostic for 1,040-byte HTTP/1 proxy responses.
|
|
||||||
|
|
||||||
Requires an existing executable and nginx; keeps the same explicitly supplied
|
|
||||||
idle-pool limit across variants. Not a substitute for wrk/CPU-isolated acceptance.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
import argparse
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
|
|
||||||
from common import Fixture, http, write_json
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument('--binary', action='append', required=True, metavar='LABEL=/abs/path')
|
|
||||||
parser.add_argument('--nginx', default='/usr/sbin/nginx', type=Path)
|
|
||||||
parser.add_argument('--ab', default='/usr/bin/ab', type=Path)
|
|
||||||
parser.add_argument('--out', required=True, type=Path)
|
|
||||||
parser.add_argument('--concurrency', default='16,64')
|
|
||||||
parser.add_argument('--duration', type=int, default=4)
|
|
||||||
parser.add_argument('--repeats', type=int, default=2)
|
|
||||||
parser.add_argument('--workers', type=int, default=4)
|
|
||||||
parser.add_argument('--max-idle', type=int, default=64)
|
|
||||||
args = parser.parse_args()
|
|
||||||
binaries = dict(item.split('=', 1) for item in args.binary)
|
|
||||||
if len(binaries) != len(args.binary) or not all(Path(x).is_absolute() for x in binaries.values()):
|
|
||||||
parser.error('binary labels must be distinct and paths must be absolute')
|
|
||||||
if args.duration < 2 or args.repeats < 1:
|
|
||||||
parser.error('duration >=2 seconds and repeats >=1 required')
|
|
||||||
concurrency = [int(v) for v in args.concurrency.split(',')]
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
records = []
|
|
||||||
write_json(args.out / 'metadata.json', {
|
|
||||||
'simulation': False, 'binary_labels': list(binaries), 'nginx': str(args.nginx),
|
|
||||||
'max_idle_for_every_rust_variant': args.max_idle, 'duration_s': args.duration,
|
|
||||||
'repeats': args.repeats, 'concurrency': concurrency, 'workers': args.workers,
|
|
||||||
'limitations': 'same-host shared CPU and closed-loop ab; not build-level before/after of uncompiled source'
|
|
||||||
})
|
|
||||||
with Fixture(args.out, workers=args.workers) as fixture:
|
|
||||||
nginx = fixture.nginx(args.nginx, frontend=True)
|
|
||||||
instances = {name: fixture.rust(Path(binary), name, max_idle=args.max_idle)
|
|
||||||
for name, binary in binaries.items()}
|
|
||||||
for name, instance in instances.items():
|
|
||||||
instance.login()
|
|
||||||
print('Waiting for quiescence:', name, flush=True)
|
|
||||||
instance.wait_quiescent(timeout=150, quiet_seconds=3)
|
|
||||||
sources = {**{name: r.reverse_port for name, r in instances.items()}, 'nginx': nginx['reverse_port']}
|
|
||||||
payload = (fixture.site / 'small.html').read_bytes()
|
|
||||||
for name, port in sources.items():
|
|
||||||
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
|
|
||||||
if code != 200 or body != payload:
|
|
||||||
raise AssertionError(f'{name} precheck failed: {code}, {len(body)} bytes')
|
|
||||||
write_json(args.out / 'payload-check.json', {
|
|
||||||
'status': 200, 'bytes': len(payload), 'sha256': hashlib.sha256(payload).hexdigest(),
|
|
||||||
'all_variants_verified': True
|
|
||||||
})
|
|
||||||
labels = list(sources)
|
|
||||||
for c in concurrency:
|
|
||||||
for repeat in range(args.repeats):
|
|
||||||
order = labels[repeat % len(labels):] + labels[:repeat % len(labels)]
|
|
||||||
if repeat % 2:
|
|
||||||
order.reverse()
|
|
||||||
for name in order:
|
|
||||||
port = sources[name]
|
|
||||||
cmd = [str(args.ab), '-k', '-t', str(args.duration), '-n', '1000000', '-c', str(c),
|
|
||||||
'-H', 'Accept-Encoding: identity', f'http://127.0.0.1:{port}/small.html']
|
|
||||||
prefix = f'c{c}-rep{repeat+1}-{name}'
|
|
||||||
subprocess.run([str(args.ab), '-k', '-n', '256', '-c', str(c),
|
|
||||||
f'http://127.0.0.1:{port}/small.html'],
|
|
||||||
capture_output=True, timeout=25, check=True)
|
|
||||||
p = subprocess.run(cmd, capture_output=True, text=True, timeout=args.duration + 45)
|
|
||||||
raw = p.stdout + '\n' + p.stderr
|
|
||||||
(args.out / f'{prefix}.txt').write_text(raw)
|
|
||||||
if p.returncode:
|
|
||||||
raise RuntimeError(f'{prefix} ab failed: {p.returncode} {p.stderr}')
|
|
||||||
def field(label):
|
|
||||||
m = re.search(r'^' + re.escape(label) + r':\s*(\d+(?:\.\d+)?)', raw, re.M)
|
|
||||||
return float(m.group(1)) if m else None
|
|
||||||
# ApacheBench tracks both network failures and bad body lengths.
|
|
||||||
if not re.search(r'^Failed requests:\s+0\s*$', raw, re.M):
|
|
||||||
raise RuntimeError(f'{prefix} failed requests: {raw[-1600:]}')
|
|
||||||
if not re.search(r'^Non-2xx responses:\s*0\s*$', raw, re.M) and 'Non-2xx responses:' in raw:
|
|
||||||
raise RuntimeError(f'{prefix} received non-2xx')
|
|
||||||
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
|
|
||||||
if code != 200 or body != payload:
|
|
||||||
raise AssertionError(f'{prefix} postcheck failed')
|
|
||||||
rps = field('Requests per second')
|
|
||||||
if rps is None:
|
|
||||||
raise RuntimeError(f'{prefix} missing RPS')
|
|
||||||
sample = {'variant': name, 'concurrency': c, 'repeat': repeat+1,
|
|
||||||
'rps': rps, 'completed': field('Complete requests'), 'failed': 0,
|
|
||||||
'expected_sha256': hashlib.sha256(payload).hexdigest(), 'status': 200}
|
|
||||||
records.append(sample)
|
|
||||||
write_json(args.out / 'samples.json', records)
|
|
||||||
print(sample, flush=True)
|
|
||||||
write_json(args.out / 'complete.json', {'complete': True, 'samples': len(records)})
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,479 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Interleaved before/after/nginx process benchmarks with immutable raw evidence.
|
|
||||||
|
|
||||||
This is a same-host diagnostic experiment, not a simulation or release sign-off.
|
|
||||||
Runs >=5 measured samples per variant/scenario by default, each >=15 seconds.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import csv
|
|
||||||
import hashlib
|
|
||||||
import itertools
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import platform
|
|
||||||
import resource
|
|
||||||
import statistics
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, http, seal_results, sha256, write_json
|
|
||||||
from resources import (Monitor, affinity_snapshot, cgroup_cpu_delta, cgroup_cpu_stat,
|
|
||||||
parse_cpu_set, validate_cpu_sets, wait_for_affinity)
|
|
||||||
from telemetry import traffic_analytics_delta, traffic_analytics_snapshot
|
|
||||||
|
|
||||||
|
|
||||||
SIZES = {"small": "small.html", "large": "large.bin", "huge": "huge.bin"}
|
|
||||||
DROP_COUNTER_FIELDS = {
|
|
||||||
"logs_dropped": "history_dropped_delta",
|
|
||||||
"logging.sinks.queue_dropped": "logging_sinks_queue_dropped_delta",
|
|
||||||
"logging.error_file.queue_dropped": "logging_error_file_queue_dropped_delta",
|
|
||||||
"cache.cold_queue_dropped": "cache_cold_queue_dropped_delta",
|
|
||||||
}
|
|
||||||
DROP_INTERVAL_NOTE = ("Delta brackets measured wrk plus one postflight request and possible in-flight completions; "
|
|
||||||
"percentage uses wrk completed requests as an approximate denominator. "
|
|
||||||
"History rejection does not imply rejection by live broadcast or external sinks.")
|
|
||||||
|
|
||||||
|
|
||||||
def usage_snapshot():
|
|
||||||
usage = resource.getrusage(resource.RUSAGE_CHILDREN)
|
|
||||||
return usage.ru_utime + usage.ru_stime
|
|
||||||
|
|
||||||
|
|
||||||
def effective_threads(requested, concurrency):
|
|
||||||
if requested < 1 or concurrency < 1:
|
|
||||||
raise ValueError("client threads and concurrency must be positive")
|
|
||||||
# wrk floor-divides the connection count between threads. Select a divisor
|
|
||||||
# so its advertised and effective connection counts remain identical.
|
|
||||||
return next(value for value in range(min(requested, concurrency), 0, -1)
|
|
||||||
if concurrency % value == 0)
|
|
||||||
|
|
||||||
|
|
||||||
def wrk_run(args, port, path, duration, concurrency, prefix, roots=None, expected_processes=None):
|
|
||||||
threads = effective_threads(args.client_threads, concurrency)
|
|
||||||
command = [str(args.wrk), "--latency", "-t", str(threads),
|
|
||||||
"-c", str(concurrency), "-d", f"{duration}s", "--timeout", "5s",
|
|
||||||
"-H", "Accept-Encoding: identity", "-s", str(Path(__file__).with_name("wrk_report.lua")),
|
|
||||||
f"http://127.0.0.1:{port}/{path}"]
|
|
||||||
if args.client_cpus:
|
|
||||||
command = ["taskset", "-c", args.client_cpus, *command]
|
|
||||||
affinities = {}
|
|
||||||
expected_processes = expected_processes or {}
|
|
||||||
if roots:
|
|
||||||
for label, pid in roots.items():
|
|
||||||
expected = parse_cpu_set(args.server_cpus if label == "server" else args.origin_cpus)
|
|
||||||
affinities[label] = affinity_snapshot(pid, expected, expected_processes.get(label))
|
|
||||||
monitor = Monitor(roots, args.resource_interval, expected_processes) if roots else None
|
|
||||||
if monitor:
|
|
||||||
monitor.start()
|
|
||||||
cpu_start = usage_snapshot()
|
|
||||||
started = time.monotonic()
|
|
||||||
process = None
|
|
||||||
stdout, stderr = "", ""
|
|
||||||
try:
|
|
||||||
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
|
||||||
affinities["generator"] = wait_for_affinity(process, parse_cpu_set(args.client_cpus))
|
|
||||||
stdout, stderr = process.communicate(timeout=duration + 30)
|
|
||||||
finally:
|
|
||||||
if process is not None and process.poll() is None:
|
|
||||||
process.kill()
|
|
||||||
stdout, stderr = process.communicate(timeout=5)
|
|
||||||
if monitor:
|
|
||||||
monitor.stop()
|
|
||||||
prefix.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
prefix.with_suffix(".txt").write_text(stdout + "\n" + stderr)
|
|
||||||
write_json(prefix.with_name(prefix.name + "-affinity.json"), affinities)
|
|
||||||
if monitor:
|
|
||||||
write_json(prefix.with_name(prefix.name + "-resources.json"), monitor.samples)
|
|
||||||
if monitor.errors:
|
|
||||||
write_json(prefix.with_name(prefix.name + "-resource-errors.json"), monitor.errors)
|
|
||||||
cpu_seconds = usage_snapshot() - cpu_start
|
|
||||||
wall_seconds = time.monotonic() - started
|
|
||||||
raw = stdout + "\n" + stderr
|
|
||||||
if process.returncode:
|
|
||||||
raise RuntimeError(f"wrk exited {process.returncode}: {raw[-2000:]}")
|
|
||||||
line = next((line for line in stdout.splitlines() if line.startswith("RUSTPROXY_BENCH_JSON ")), None)
|
|
||||||
if line is None:
|
|
||||||
raise RuntimeError("wrk produced no structured report")
|
|
||||||
result = json.loads(line.removeprefix("RUSTPROXY_BENCH_JSON "))
|
|
||||||
if result["requests"] <= 0 or result["duration_us"] <= 0:
|
|
||||||
write_json(prefix.with_suffix(".json"), result)
|
|
||||||
raise RuntimeError("wrk completed no requests or reported an invalid duration; raw evidence retained")
|
|
||||||
result.update({"command": command, "wall_seconds": wall_seconds,
|
|
||||||
"client_threads": threads, "requested_connections": concurrency,
|
|
||||||
"client_threads_requested": args.client_threads,
|
|
||||||
"effective_connections": threads * (concurrency // threads),
|
|
||||||
"generator_cpu_seconds": cpu_seconds,
|
|
||||||
"rps": result["requests"] * 1_000_000 / result["duration_us"],
|
|
||||||
"error_total": sum(value for key, value in result.items() if key.startswith("errors_"))})
|
|
||||||
if monitor:
|
|
||||||
result["resources"] = monitor.summary(result["requests"])
|
|
||||||
write_json(prefix.with_suffix(".json"), result)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def payload_check(port, path, expected):
|
|
||||||
"""A full exact-body check outside the wrk measurement interval."""
|
|
||||||
try:
|
|
||||||
status, headers, body = http(port, "/" + path, headers={"Accept-Encoding": "identity"})
|
|
||||||
return {"status": status, "bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
|
|
||||||
"expected_bytes": len(expected), "expected_sha256": hashlib.sha256(expected).hexdigest(),
|
|
||||||
"content_length": headers.get("content-length"), "content_encoding": headers.get("content-encoding"),
|
|
||||||
"approximate_header_bytes": 19 + sum(len(key) + len(value) + 4 for key, value in headers.items()),
|
|
||||||
"passed": status == 200 and body == expected}
|
|
||||||
except Exception as error:
|
|
||||||
return {"passed": False, "error": str(error), "expected_bytes": len(expected),
|
|
||||||
"expected_sha256": hashlib.sha256(expected).hexdigest()}
|
|
||||||
|
|
||||||
|
|
||||||
def transfer_sanity(result, payload_bytes, header_bytes, concurrency):
|
|
||||||
"""Check gross truncation/size anomalies; this is not per-response hashing."""
|
|
||||||
requests, wire_bytes = result["requests"], result["bytes"]
|
|
||||||
minimum = requests * payload_bytes
|
|
||||||
# wrk counts headers and partially received final responses in its wire bytes.
|
|
||||||
header_allowance = max(1024, 2 * header_bytes)
|
|
||||||
maximum = (requests + concurrency) * (payload_bytes + header_allowance)
|
|
||||||
return {"passed": requests > 0 and minimum <= wire_bytes <= maximum,
|
|
||||||
"wire_bytes_per_request": wire_bytes / requests if requests else None,
|
|
||||||
"expected_payload_bytes_per_request": payload_bytes,
|
|
||||||
"minimum_wire_bytes": minimum, "maximum_wire_bytes": maximum,
|
|
||||||
"header_allowance_per_response": header_allowance,
|
|
||||||
"inflight_response_allowance": concurrency}
|
|
||||||
|
|
||||||
|
|
||||||
def counter_values(value, prefix=""):
|
|
||||||
"""Retain numeric drop and rotation counters without session/config data."""
|
|
||||||
found = {}
|
|
||||||
if isinstance(value, dict):
|
|
||||||
for key, child in value.items():
|
|
||||||
path = f"{prefix}.{key}" if prefix else key
|
|
||||||
if isinstance(child, (int, float)) and any(part in key for part in ["dropped", "rotations", "generation", "requests_in_generation"]):
|
|
||||||
found[path] = child
|
|
||||||
found.update(counter_values(child, path))
|
|
||||||
elif isinstance(value, list):
|
|
||||||
for index, child in enumerate(value):
|
|
||||||
found.update(counter_values(child, f"{prefix}[{index}]"))
|
|
||||||
return found
|
|
||||||
|
|
||||||
|
|
||||||
def validate_drop_counters(values):
|
|
||||||
for name in DROP_COUNTER_FIELDS:
|
|
||||||
if type(values.get(name)) is not int or values[name] < 0:
|
|
||||||
raise RuntimeError(f"missing or invalid cumulative status counter: {name}")
|
|
||||||
return values
|
|
||||||
|
|
||||||
|
|
||||||
def status_counter_snapshot(rust):
|
|
||||||
if rust is None:
|
|
||||||
return None
|
|
||||||
status, _, body = rust.admin("/api/status")
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"{rust.label} status capture failed: HTTP {status}")
|
|
||||||
return counter_values(json.loads(body))
|
|
||||||
|
|
||||||
|
|
||||||
def drop_counter_report(before, after, requests):
|
|
||||||
if requests <= 0:
|
|
||||||
raise ValueError("drop-rate denominator must contain completed requests")
|
|
||||||
if (before is None) != (after is None):
|
|
||||||
raise RuntimeError("both status snapshots are required for a drop delta")
|
|
||||||
if before is not None:
|
|
||||||
validate_drop_counters(before)
|
|
||||||
validate_drop_counters(after)
|
|
||||||
deltas = {name: after[name] - before[name] if before is not None else None
|
|
||||||
for name in DROP_COUNTER_FIELDS}
|
|
||||||
if any(value is not None and value < 0 for value in deltas.values()):
|
|
||||||
raise RuntimeError("cumulative drop counter decreased during measured interval")
|
|
||||||
fields = {field: deltas[name] for name, field in DROP_COUNTER_FIELDS.items()}
|
|
||||||
fields["history_drop_percent_of_completed_requests"] = (
|
|
||||||
100 * deltas["logs_dropped"] / requests if before is not None else None)
|
|
||||||
return {"applicable": before is not None, "counter_deltas": deltas, "row_fields": fields,
|
|
||||||
"completed_requests_denominator": requests, "denominator_is_approximate": True,
|
|
||||||
"interval_note": DROP_INTERVAL_NOTE}
|
|
||||||
|
|
||||||
|
|
||||||
def analytics_counter_report(before, after, requests):
|
|
||||||
if requests <= 0 or (before is None) != (after is None):
|
|
||||||
raise ValueError("analytics delta requires paired snapshots and completed requests")
|
|
||||||
delta = traffic_analytics_delta(before, after) if before is not None else {
|
|
||||||
"queue_dropped_delta": None, "recorded_requests_delta": None, "interval_seconds": None}
|
|
||||||
fields = {"analytics_queue_dropped_delta": delta["queue_dropped_delta"],
|
|
||||||
"analytics_recorded_requests_delta": delta["recorded_requests_delta"],
|
|
||||||
"analytics_queue_drop_percent_of_completed_requests": (
|
|
||||||
100 * delta["queue_dropped_delta"] / requests if before is not None else None)}
|
|
||||||
return {"applicable": before is not None, **delta, "row_fields": fields,
|
|
||||||
"completed_requests_denominator": requests, "denominator_is_approximate": True,
|
|
||||||
"interval_note": DROP_INTERVAL_NOTE,
|
|
||||||
"recorded_requests_note": "Asynchronous analytics records; not HTTP success or completion counts."}
|
|
||||||
|
|
||||||
|
|
||||||
def persist_rows(out, rows):
|
|
||||||
write_json(out / "runs.json", rows)
|
|
||||||
if not rows:
|
|
||||||
return
|
|
||||||
with (out / "runs.csv").open("w", newline="") as stream:
|
|
||||||
writer = csv.DictWriter(stream, fieldnames=list(rows[0]))
|
|
||||||
writer.writeheader()
|
|
||||||
writer.writerows(rows)
|
|
||||||
groups = {}
|
|
||||||
for row in rows:
|
|
||||||
groups.setdefault((row["mode"], row["size"], row["concurrency"], row["variant"]), []).append(row)
|
|
||||||
summary = []
|
|
||||||
for (mode, size, concurrency, variant), group in sorted(groups.items()):
|
|
||||||
rps = [row["rps"] for row in group]
|
|
||||||
summary.append({"mode": mode, "size": size, "concurrency": concurrency, "variant": variant,
|
|
||||||
"samples": len(group), "rps_median": statistics.median(rps),
|
|
||||||
"rps_min": min(rps), "rps_max": max(rps),
|
|
||||||
"rps_cv_percent": statistics.stdev(rps) / statistics.mean(rps) * 100 if len(rps) > 1 else None,
|
|
||||||
"p50_us_median": statistics.median(row["p50_us"] for row in group),
|
|
||||||
"p95_us_median": statistics.median(row["p95_us"] for row in group),
|
|
||||||
"p99_us_median": statistics.median(row["p99_us"] for row in group),
|
|
||||||
"server_cpu_us_per_request_median": statistics.median(row["server_cpu_us_per_request"] for row in group),
|
|
||||||
"server_rss_peak_bytes": max(row["server_rss_peak_bytes"] for row in group),
|
|
||||||
"server_fd_peak": max(row["server_fd_peak"] for row in group),
|
|
||||||
"requests": sum(row["requests"] for row in group),
|
|
||||||
"cgroup_throttled_samples": sum(row["cgroup_nr_throttled_delta"] > 0 for row in group),
|
|
||||||
"cgroup_nr_throttled_delta": sum(row["cgroup_nr_throttled_delta"] for row in group),
|
|
||||||
"cgroup_throttled_usec_delta": sum(row["cgroup_throttled_usec_delta"] for row in group),
|
|
||||||
"validation_failures": sum(not (row["payload_before_passed"] and row["payload_after_passed"]
|
|
||||||
and row["transfer_sanity_passed"]) for row in group),
|
|
||||||
"errors": sum(row["error_total"] for row in group)})
|
|
||||||
write_json(out / "summary.json", summary)
|
|
||||||
with (out / "summary.csv").open("w", newline="") as stream:
|
|
||||||
writer = csv.DictWriter(stream, fieldnames=list(summary[0]))
|
|
||||||
writer.writeheader()
|
|
||||||
writer.writerows(summary)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--max-idle", action="append", default=[], metavar="LABEL=COUNT")
|
|
||||||
parser.add_argument("--nginx", type=Path, required=True)
|
|
||||||
parser.add_argument("--wrk", type=Path, required=True)
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
parser.add_argument("--workers", type=int, default=4)
|
|
||||||
parser.add_argument("--origin-workers", type=int, default=4)
|
|
||||||
parser.add_argument("--client-threads", type=int, default=4)
|
|
||||||
parser.add_argument("--server-cpus")
|
|
||||||
parser.add_argument("--origin-cpus")
|
|
||||||
parser.add_argument("--client-cpus")
|
|
||||||
parser.add_argument("--duration", type=int, default=15)
|
|
||||||
parser.add_argument("--warmup", type=int, default=3)
|
|
||||||
parser.add_argument("--repeats", type=int, default=5)
|
|
||||||
parser.add_argument("--concurrency", default="64")
|
|
||||||
parser.add_argument("--sizes", default="small,large")
|
|
||||||
parser.add_argument("--modes", default="proxy,static")
|
|
||||||
parser.add_argument("--resource-interval", type=float, default=.5)
|
|
||||||
parser.add_argument("--exploratory", action="store_true", help="explicitly allow <5 repetitions or <15s samples")
|
|
||||||
parser.add_argument("--prepare-only", action="store_true", help="start processes and verify hashes; run no load generator")
|
|
||||||
args = parser.parse_args()
|
|
||||||
if (args.duration < 15 or args.repeats < 5) and not args.exploratory:
|
|
||||||
parser.error("acceptance requires >=15-second runs and >=5 repetitions; use --exploratory for labelled short diagnostics")
|
|
||||||
if args.out.exists() and any(args.out.iterdir()):
|
|
||||||
parser.error("--out must be a new or empty directory so raw evidence is never overwritten")
|
|
||||||
modes, sizes = args.modes.split(","), args.sizes.split(",")
|
|
||||||
concurrencies = [int(value) for value in args.concurrency.split(",")]
|
|
||||||
if any(mode not in ("proxy", "static") for mode in modes) or any(size not in SIZES for size in sizes):
|
|
||||||
parser.error("supported modes: proxy,static; sizes: small,large,huge")
|
|
||||||
if min(concurrencies) < 1 or max(concurrencies) > 4096:
|
|
||||||
parser.error("concurrency must be 1..4096")
|
|
||||||
if args.client_threads < 1:
|
|
||||||
parser.error("--client-threads must be positive")
|
|
||||||
if args.workers < 1 or args.origin_workers < 1:
|
|
||||||
parser.error("--workers and --origin-workers must be positive")
|
|
||||||
try:
|
|
||||||
cpu_sets = validate_cpu_sets({"server": args.server_cpus, "origin": args.origin_cpus,
|
|
||||||
"client": args.client_cpus}, set(os.sched_getaffinity(0)))
|
|
||||||
except ValueError as error:
|
|
||||||
parser.error(str(error))
|
|
||||||
if args.resource_interval <= 0:
|
|
||||||
parser.error("--resource-interval must be positive")
|
|
||||||
args.nginx, args.wrk, args.out = args.nginx.resolve(), args.wrk.resolve(), args.out.resolve()
|
|
||||||
variants = dict(item.split("=", 1) for item in args.binary)
|
|
||||||
if len(variants) != len(args.binary) or "nginx" in variants:
|
|
||||||
parser.error("variant names must be unique and must not be nginx")
|
|
||||||
max_idle = dict(item.split("=", 1) for item in args.max_idle)
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
environment = {"kind": "same-host real HTTP/1.1 process benchmark", "simulation": False,
|
|
||||||
"exploratory": args.exploratory, "prepare_only": args.prepare_only,
|
|
||||||
"acceptance_sampling": args.duration >= 15 and args.repeats >= 5 and not args.prepare_only,
|
|
||||||
"platform": platform.platform(), "python": sys.version,
|
|
||||||
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
|
||||||
"cpu_affinity_available": sorted(os.sched_getaffinity(0)),
|
|
||||||
"cpu_sets": {label: sorted(cpus) for label, cpus in cpu_sets.items()},
|
|
||||||
"resource_process_discovery": "PPid map from /proc/*/stat; root and configured nginx process counts must be readable",
|
|
||||||
"expected_nginx_processes": {"frontend": args.workers + 1, "origin": args.origin_workers + 1},
|
|
||||||
"generator_plan": [{"connections": concurrency, "requested_threads": args.client_threads,
|
|
||||||
"effective_threads": effective_threads(args.client_threads, concurrency)}
|
|
||||||
for concurrency in concurrencies],
|
|
||||||
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
|
|
||||||
"wrk_sha256": sha256(args.wrk), "nginx_sha256": sha256(args.nginx),
|
|
||||||
"limitations": ["Generator, upstream and frontend share one host and kernel; not production capacity",
|
|
||||||
"Closed-loop wrk latency includes its histogram correction; no open-loop arrival-rate SLA claim",
|
|
||||||
"RSS sums processes and may double-count shared pages in nginx",
|
|
||||||
"RSS and FD peaks are sampled, not absolute instantaneous peaks",
|
|
||||||
"Exact HTTP 200 and payload hashes are checked before/after each measured run, not for every response during load",
|
|
||||||
"wrk status errors count only status >399; aggregate bytes/request is a gross sanity check, not response integrity proof",
|
|
||||||
"cgroup CPU-throttling deltas cover all processes in the shared cgroup, not only the measured server",
|
|
||||||
DROP_INTERVAL_NOTE,
|
|
||||||
"No TLS/H2/H3 speed inference; no CPU cycles/allocation profiler claim",
|
|
||||||
"nginx has a smaller feature set; request logging/ACL/analytics work is not feature-matched"]}
|
|
||||||
for path in ["/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max", "/proc/version"]:
|
|
||||||
try:
|
|
||||||
environment[path] = Path(path).read_text().strip()
|
|
||||||
except OSError:
|
|
||||||
environment[path] = None
|
|
||||||
write_json(args.out / "environment.json", environment)
|
|
||||||
expected_samples = len(modes) * len(sizes) * len(concurrencies) * args.repeats * (len(variants) + 1)
|
|
||||||
write_json(args.out / "completion.json", {"completed": False, "samples": 0,
|
|
||||||
"expected_samples": expected_samples,
|
|
||||||
"simulation": False, "prepare_only": args.prepare_only})
|
|
||||||
rows, error = [], None
|
|
||||||
try:
|
|
||||||
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
|
|
||||||
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
|
|
||||||
nginx = fixture.nginx(args.nginx)
|
|
||||||
servers = {"nginx": {"process": nginx["process"], "proxy": nginx["reverse_port"],
|
|
||||||
"static": nginx["static_port"], "rust": None}}
|
|
||||||
for label, binary in variants.items():
|
|
||||||
rust = fixture.rust(Path(binary), label, max_idle=int(max_idle[label]) if label in max_idle else None)
|
|
||||||
rust.login()
|
|
||||||
servers[label] = {"process": rust.process, "proxy": rust.reverse_port, "static": rust.static_port, "rust": rust}
|
|
||||||
quiescence = {label: server["rust"].wait_quiescent()
|
|
||||||
for label, server in servers.items() if server["rust"] is not None}
|
|
||||||
write_json(args.out / "quiescence-preflight.json", quiescence)
|
|
||||||
initial_affinity = {"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], args.origin_workers + 1)}
|
|
||||||
initial_affinity.update({label: affinity_snapshot(server["process"].pid, cpu_sets["server"],
|
|
||||||
args.workers + 1 if label == "nginx" else 1)
|
|
||||||
for label, server in servers.items()})
|
|
||||||
write_json(args.out / "affinity-preflight.json", initial_affinity)
|
|
||||||
# The fixture has only master + configured workers. Refuse a
|
|
||||||
# master-only snapshot even if HTTP payload checks would succeed.
|
|
||||||
prepare_monitor = Monitor({"server": nginx["process"].pid, "origin": nginx["origin"].pid},
|
|
||||||
args.resource_interval,
|
|
||||||
{"server": args.workers + 1, "origin": args.origin_workers + 1})
|
|
||||||
prepare_monitor.sample()
|
|
||||||
write_json(args.out / "resources-preflight.json", prepare_monitor.samples)
|
|
||||||
telemetry_preflight = {label: {"status": status_counter_snapshot(server["rust"]),
|
|
||||||
"traffic": traffic_analytics_snapshot(server["rust"]) if server["rust"] else None}
|
|
||||||
for label, server in servers.items()}
|
|
||||||
write_json(args.out / "telemetry-preflight.json", telemetry_preflight)
|
|
||||||
for snapshot in telemetry_preflight.values():
|
|
||||||
if snapshot["status"] is not None:
|
|
||||||
validate_drop_counters(snapshot["status"])
|
|
||||||
preflight = []
|
|
||||||
for label, server in servers.items():
|
|
||||||
for mode, size in itertools.product(modes, sizes):
|
|
||||||
payload = (fixture.site / SIZES[size]).read_bytes()
|
|
||||||
check = payload_check(server[mode], SIZES[size], payload)
|
|
||||||
preflight.append({"variant": label, "mode": mode, "size": size, **check})
|
|
||||||
write_json(args.out / "preflight.json", preflight)
|
|
||||||
if not check["passed"]:
|
|
||||||
raise AssertionError(f"payload preflight failed: {label} {mode} {size}: {check}")
|
|
||||||
write_json(args.out / "preflight.json", preflight)
|
|
||||||
labels = [*variants, "nginx"]
|
|
||||||
schedule = []
|
|
||||||
for mode, size, concurrency in itertools.product(modes, sizes, concurrencies):
|
|
||||||
for repetition in range(args.repeats):
|
|
||||||
offset = repetition % len(labels)
|
|
||||||
order = labels[offset:] + labels[:offset]
|
|
||||||
if repetition % 2:
|
|
||||||
order.reverse()
|
|
||||||
for variant in order:
|
|
||||||
schedule.append({"sequence": len(schedule) + 1, "mode": mode, "size": size,
|
|
||||||
"concurrency": concurrency, "repetition": repetition + 1, "variant": variant})
|
|
||||||
write_json(args.out / "schedule.json", schedule)
|
|
||||||
if not args.prepare_only:
|
|
||||||
for item in schedule:
|
|
||||||
server = servers[item["variant"]]
|
|
||||||
expected_processes = {"origin": args.origin_workers + 1,
|
|
||||||
"server": args.workers + 1 if item["variant"] == "nginx" else 1}
|
|
||||||
stem = f"{item['sequence']:04}-{item['mode']}-{item['size']}-c{item['concurrency']}-{item['variant']}"
|
|
||||||
if args.warmup:
|
|
||||||
warmup = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.warmup,
|
|
||||||
item["concurrency"], args.out / "warmup" / stem)
|
|
||||||
if warmup["error_total"]:
|
|
||||||
raise RuntimeError(f"warmup errors in {stem}; raw evidence retained")
|
|
||||||
payload = (fixture.site / SIZES[item["size"]]).read_bytes()
|
|
||||||
before = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
|
|
||||||
write_json(args.out / "checks" / (stem + "-before.json"), before)
|
|
||||||
if not before["passed"]:
|
|
||||||
raise RuntimeError(f"exact payload check failed before {stem}: {before}")
|
|
||||||
counters_before = status_counter_snapshot(server["rust"])
|
|
||||||
write_json(args.out / "status" / (stem + "-before.json"), counters_before)
|
|
||||||
if counters_before is not None:
|
|
||||||
validate_drop_counters(counters_before)
|
|
||||||
analytics_before = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
|
|
||||||
write_json(args.out / "status" / (stem + "-traffic-before.json"), analytics_before)
|
|
||||||
cpu_before = cgroup_cpu_stat()
|
|
||||||
try:
|
|
||||||
result = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.duration,
|
|
||||||
item["concurrency"], args.out / "raw" / stem,
|
|
||||||
{"server": server["process"].pid, "origin": nginx["origin"].pid},
|
|
||||||
expected_processes)
|
|
||||||
finally:
|
|
||||||
cpu_after = cgroup_cpu_stat()
|
|
||||||
cpu_delta = cgroup_cpu_delta(cpu_before, cpu_after)
|
|
||||||
throttling = {"before": cpu_before, "after": cpu_after, "delta": cpu_delta}
|
|
||||||
write_json(args.out / "checks" / (stem + "-cgroup-cpu.json"), throttling)
|
|
||||||
after = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
|
|
||||||
write_json(args.out / "checks" / (stem + "-after.json"), after)
|
|
||||||
write_json(args.out / "checks" / (stem + "-affinity-after.json"), {
|
|
||||||
"server": affinity_snapshot(server["process"].pid, cpu_sets["server"], expected_processes.get("server")),
|
|
||||||
"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], expected_processes["origin"])})
|
|
||||||
counters_after = status_counter_snapshot(server["rust"])
|
|
||||||
write_json(args.out / "status" / (stem + ".json"), counters_after)
|
|
||||||
analytics_after = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
|
|
||||||
write_json(args.out / "status" / (stem + "-traffic-after.json"), analytics_after)
|
|
||||||
drops = drop_counter_report(counters_before, counters_after, result["requests"])
|
|
||||||
write_json(args.out / "status" / (stem + "-delta.json"), drops)
|
|
||||||
analytics = analytics_counter_report(analytics_before, analytics_after, result["requests"])
|
|
||||||
write_json(args.out / "status" / (stem + "-traffic-delta.json"), analytics)
|
|
||||||
sanity = transfer_sanity(result, len(payload), before["approximate_header_bytes"], item["concurrency"])
|
|
||||||
result.update({"payload_before": before, "payload_after": after, "transfer_sanity": sanity,
|
|
||||||
"cgroup_cpu": throttling, "observability": drops, "traffic_analytics": analytics})
|
|
||||||
write_json((args.out / "raw" / stem).with_suffix(".json"), result)
|
|
||||||
row = {**item, **{key: result[key] for key in ["requests", "bytes", "duration_us", "rps", "p50_us", "p95_us", "p99_us",
|
|
||||||
"client_threads_requested", "client_threads", "requested_connections", "effective_connections",
|
|
||||||
"error_total", "errors_connect", "errors_read", "errors_write", "errors_timeout", "errors_status", "generator_cpu_seconds"]}}
|
|
||||||
row.update({"payload_before_passed": before["passed"], "payload_after_passed": after["passed"],
|
|
||||||
"transfer_sanity_passed": sanity["passed"], "wire_bytes_per_request": sanity["wire_bytes_per_request"],
|
|
||||||
"cgroup_nr_throttled_delta": cpu_delta["nr_throttled"],
|
|
||||||
"cgroup_throttled_usec_delta": cpu_delta["throttled_usec"]})
|
|
||||||
row.update(drops["row_fields"])
|
|
||||||
row.update(analytics["row_fields"])
|
|
||||||
for group in ["server", "origin"]:
|
|
||||||
row.update({f"{group}_{key}": value for key, value in result["resources"][group].items()})
|
|
||||||
rows.append(row)
|
|
||||||
persist_rows(args.out, rows)
|
|
||||||
print(json.dumps({"sequence": item["sequence"], "total": len(schedule), "variant": item["variant"],
|
|
||||||
"scenario": f"{item['mode']}_{item['size']}_c{item['concurrency']}",
|
|
||||||
"rps": round(result["rps"], 2), "p95_ms": result["p95_us"] / 1000,
|
|
||||||
"errors": result["error_total"]}), flush=True)
|
|
||||||
if result["error_total"]:
|
|
||||||
raise RuntimeError(f"benchmark errors in {stem}; raw evidence retained")
|
|
||||||
if not after["passed"] or not sanity["passed"]:
|
|
||||||
raise RuntimeError(f"payload/byte-count validation failed in {stem}; raw evidence retained")
|
|
||||||
except (Exception, KeyboardInterrupt) as exc:
|
|
||||||
error = {"error": str(exc) or type(exc).__name__, "type": type(exc).__name__,
|
|
||||||
"interrupted": isinstance(exc, KeyboardInterrupt), "traceback": traceback.format_exc()}
|
|
||||||
write_json(args.out / "failure.json", error)
|
|
||||||
print(json.dumps(error), flush=True)
|
|
||||||
finally:
|
|
||||||
persist_rows(args.out, rows)
|
|
||||||
completed = error is None and (args.prepare_only or len(rows) == expected_samples)
|
|
||||||
write_json(args.out / "completion.json", {"completed": completed, "samples": len(rows),
|
|
||||||
"expected_samples": expected_samples,
|
|
||||||
"simulation": False, "prepare_only": args.prepare_only,
|
|
||||||
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
|
|
||||||
seal_results(args.out)
|
|
||||||
return (130 if error.get("interrupted") else 1) if error else (0 if completed else 1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,180 +0,0 @@
|
|||||||
"""Cache and response-processing paths must survive disabled-path optimization."""
|
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
|
||||||
import copy
|
|
||||||
import gzip
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
import time
|
|
||||||
import zlib
|
|
||||||
|
|
||||||
from common import http
|
|
||||||
|
|
||||||
|
|
||||||
def run(check, rust, peer):
|
|
||||||
port = rust.reverse_port
|
|
||||||
before = peer.count("GET", "/fresh")
|
|
||||||
results = [http(port, "/fresh") for _ in range(2)]
|
|
||||||
check("cache.global_off", all(s == 200 and h.get("x-cache") != "HIT" for s, h, b in results)
|
|
||||||
and peer.count("GET", "/fresh") - before == 2)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["cache"] = {"enabled": True, "expose_status": True, "expose_hits": True,
|
|
||||||
"expose_cache_status": True, "http_invalidation_allowed_cidrs": ["127.0.0.1/32"]}
|
|
||||||
config["web_services"][0]["cache_control"] = "public, max-age=60"
|
|
||||||
rust.apply(config, "cache-enabled")
|
|
||||||
first = http(port, "/fresh")
|
|
||||||
second = http(port, "/fresh")
|
|
||||||
check("cache.miss_then_hit", first[0] == 200 and first[1].get("x-cache") == "MISS"
|
|
||||||
and second[1].get("x-cache") == "HIT" and first[2] == second[2],
|
|
||||||
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
|
|
||||||
results = {}
|
|
||||||
for language in ["en", "pl", "en", "pl"]:
|
|
||||||
response = http(port, "/vary", headers={"Accept-Language": language})
|
|
||||||
previous = results.get(language)
|
|
||||||
if previous:
|
|
||||||
check("cache.vary_hit." + language, response[1].get("x-cache") == "HIT" and response[2] == previous[2],
|
|
||||||
cache=response[1].get("x-cache"))
|
|
||||||
results[language] = response
|
|
||||||
check("cache.vary_isolation", results["en"][2] != results["pl"][2])
|
|
||||||
for path in ["/private", "/no-store", "/cookie"]:
|
|
||||||
before = peer.count("GET", path)
|
|
||||||
first, second = http(port, path), http(port, path)
|
|
||||||
check("cache.nonpublic_bypass." + path, first[0] == second[0] == 200
|
|
||||||
and first[1].get("x-cache") != "HIT" and second[1].get("x-cache") != "HIT"
|
|
||||||
and peer.count("GET", path) - before == 2,
|
|
||||||
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
|
|
||||||
for name, headers in [("authorization", {"Authorization": "Bearer fixture-value"}),
|
|
||||||
("cookie", {"Cookie": "application_session=fixture"})]:
|
|
||||||
before = peer.count("GET", "/fresh")
|
|
||||||
s, h, b = http(port, "/fresh", headers=headers)
|
|
||||||
check("cache.request_bypass." + name, s == 200 and h.get("x-cache") != "HIT"
|
|
||||||
and peer.count("GET", "/fresh") - before == 1, cache=h.get("x-cache"))
|
|
||||||
before = peer.count("GET", "/fresh")
|
|
||||||
s, h, b = http(port, "/fresh", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
|
|
||||||
check("cache.nonempty_get_bypass_and_preserve", s == 200 and h.get("x-cache") != "HIT"
|
|
||||||
and h.get("x-received-body-bytes") == "17" and peer.count("GET", "/fresh") - before == 1,
|
|
||||||
cache=h.get("x-cache"), origin_body_bytes=h.get("x-received-body-bytes"))
|
|
||||||
s, h, b = http(port, "/fresh", "PURGE")
|
|
||||||
after = http(port, "/fresh")
|
|
||||||
check("cache.http_purge", s in (200, 202, 204) and after[1].get("x-cache") == "MISS", status=s)
|
|
||||||
with ThreadPoolExecutor(max_workers=8) as pool:
|
|
||||||
results = list(pool.map(lambda _: http(port, "/concurrent"), range(8)))
|
|
||||||
check("cache.concurrent_fill", all(item[0] == 200 for item in results)
|
|
||||||
and len({item[2] for item in results}) == 1, origin_requests=peer.count("GET", "/concurrent"))
|
|
||||||
first = http(port, "/revalidate")
|
|
||||||
time.sleep(1.2)
|
|
||||||
second = http(port, "/revalidate")
|
|
||||||
check("cache.expired_validator_revalidation", first[0] == second[0] == 200 and first[2] == second[2]
|
|
||||||
and peer.count("GET", "/revalidate") >= 2, cache=second[1].get("x-cache"))
|
|
||||||
|
|
||||||
first, second = http(rust.static_port), http(rust.static_port)
|
|
||||||
check("cache.static_hit", first[0] == second[0] == 200 and second[1].get("x-cache") == "HIT"
|
|
||||||
and first[2] == second[2], cache=second[1].get("x-cache"))
|
|
||||||
empty_first, empty_second = http(rust.static_port, "/empty.txt"), http(rust.static_port, "/empty.txt")
|
|
||||||
check("cache.static_empty_hit", empty_first[0] == empty_second[0] == 200
|
|
||||||
and empty_second[1].get("x-cache") == "HIT" and empty_second[2] == b"",
|
|
||||||
first=empty_first[1].get("x-cache"), second=empty_second[1].get("x-cache"))
|
|
||||||
hs, hh, hb = http(rust.static_port, "/download.txt", "HEAD")
|
|
||||||
gs, gh, gb = http(rust.static_port, "/download.txt")
|
|
||||||
check("cache.static_head_cannot_fill_empty_get", hs == gs == 200 and hb == b""
|
|
||||||
and gb == (rust.fixture.site / "download.txt").read_bytes(), head_status=hs, get_status=gs)
|
|
||||||
s, h, b = http(rust.static_port, headers={"Range": "bytes=0-9"})
|
|
||||||
check("cache.static_range_survives", s == 206 and b == first[2][:10], status=s)
|
|
||||||
s, h, b = http(rust.static_port, headers={"If-None-Match": first[1]["etag"]})
|
|
||||||
check("cache.static_conditional_survives", s == 304 and b == b"", status=s)
|
|
||||||
|
|
||||||
config["vhosts"][0]["cache"] = {"enabled": False}
|
|
||||||
rust.apply(config, "cache-vhost-disabled")
|
|
||||||
first, second = http(port, "/fresh"), http(port, "/fresh")
|
|
||||||
check("cache.vhost_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
|
|
||||||
and first[2] != second[2])
|
|
||||||
config["vhosts"][0]["cache"] = {"enabled": True}
|
|
||||||
config["vhosts"][0]["routes"] = [{"id": "fresh-no-cache", "path_prefix": "/fresh", "cache": False}]
|
|
||||||
rust.apply(config, "cache-route-disabled")
|
|
||||||
first, second = http(port, "/fresh"), http(port, "/fresh")
|
|
||||||
check("cache.route_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
|
|
||||||
and first[2] != second[2])
|
|
||||||
config["cache"]["enabled"] = False
|
|
||||||
config["vhosts"][0]["routes"] = []
|
|
||||||
rust.apply(config, "cache-global-disabled-vhost-enabled")
|
|
||||||
first, second = http(port, "/fresh"), http(port, "/fresh")
|
|
||||||
check("cache.vhost_cannot_override_global_off", first[2] != second[2]
|
|
||||||
and second[1].get("x-cache") != "HIT")
|
|
||||||
|
|
||||||
|
|
||||||
def transforms(check, rust, peer):
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
original = (rust.fixture.site / "app.json").read_bytes()
|
|
||||||
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
|
|
||||||
s, h, b = http(port, "/app.json")
|
|
||||||
check("transform.off." + source, s == 200 and b == original, status=s)
|
|
||||||
config["transform"] = {"enabled": True, "json": True, "html": True}
|
|
||||||
for vhost in config["vhosts"]:
|
|
||||||
vhost["transform"] = {"enabled": True, "json": True, "html": True}
|
|
||||||
rust.apply(config, "transform-enabled-cache-off")
|
|
||||||
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
|
|
||||||
s, h, b = http(port, "/app.json")
|
|
||||||
check("transform.on." + source, s == 200 and json.loads(b) == json.loads(original)
|
|
||||||
and len(b) < len(original), original_bytes=len(original), result_bytes=len(b), status=s)
|
|
||||||
s, h, b = http(port, "/large.bin")
|
|
||||||
check("transform.binary_unchanged." + source, s == 200 and b == (rust.fixture.site / "large.bin").read_bytes())
|
|
||||||
s, h, b = http(port, "/app.json", headers={"Range": "bytes=0-9"})
|
|
||||||
check("transform.range_exclusion." + source, s == 206 and b == original[:10], status=s)
|
|
||||||
config["cache"] = {"enabled": True, "expose_status": True}
|
|
||||||
rust.apply(config, "transform-enabled-cache-on")
|
|
||||||
first, second = http(rust.reverse_port, "/app.json"), http(rust.reverse_port, "/app.json")
|
|
||||||
check("transform.cache_combination", second[0] == 200 and second[1].get("x-cache") == "HIT"
|
|
||||||
and first[2] == second[2] and json.loads(second[2]) == json.loads(original),
|
|
||||||
cache=second[1].get("x-cache"))
|
|
||||||
|
|
||||||
rust.reset()
|
|
||||||
expected = (rust.fixture.site / "compress.txt").read_bytes()
|
|
||||||
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
|
|
||||||
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
|
|
||||||
check("compression.gzip." + source, s == 200 and h.get("content-encoding") == "gzip"
|
|
||||||
and gzip.decompress(b) == expected and len(b) < len(expected), bytes=len(b))
|
|
||||||
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip;q=0, identity"})
|
|
||||||
check("compression.identity_qvalue." + source, s == 200 and "content-encoding" not in h and b == expected)
|
|
||||||
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
|
|
||||||
check("compression.range_exclusion." + source, s == 206 and "content-encoding" not in h and b == expected[:10])
|
|
||||||
s, h, b = http(rust.reverse_port, "/no-transform", headers={"Accept-Encoding": "gzip"})
|
|
||||||
check("compression.no_transform_exclusion", s == 200 and "content-encoding" not in h)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["compression"] = {"enabled": False}
|
|
||||||
rust.apply(config, "compression-vhost-disabled")
|
|
||||||
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
|
|
||||||
check("compression.vhost_off", s == 200 and "content-encoding" not in h and b == expected)
|
|
||||||
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
|
|
||||||
config["vhosts"][0]["compression"] = {"enabled": True, "gzip": True}
|
|
||||||
rust.apply(config, "compression-global-disabled")
|
|
||||||
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
|
|
||||||
check("compression.global_codec_ceiling", s == 200 and "content-encoding" not in h and b == expected)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["compression"] = {"gzip": True, "br": True, "zstd": True, "deflate": True}
|
|
||||||
config["cache"] = {"enabled": True, "expose_status": True}
|
|
||||||
rust.apply(config, "compression-cache-enabled")
|
|
||||||
for codec in ["gzip", "deflate", "br", "zstd"]:
|
|
||||||
decoder = None
|
|
||||||
if codec == "gzip":
|
|
||||||
decoder = gzip.decompress
|
|
||||||
elif codec == "deflate":
|
|
||||||
decoder = zlib.decompress
|
|
||||||
elif codec == "br":
|
|
||||||
try:
|
|
||||||
import brotli
|
|
||||||
decoder = brotli.decompress
|
|
||||||
except ImportError:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
import zstandard
|
|
||||||
decoder = lambda data: zstandard.ZstdDecompressor().stream_reader(io.BytesIO(data)).read()
|
|
||||||
except ImportError:
|
|
||||||
pass
|
|
||||||
if decoder is None:
|
|
||||||
check("compression.codec." + codec, None, skip_reason="optional Python decoder is not installed")
|
|
||||||
continue
|
|
||||||
first = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
|
|
||||||
second = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
|
|
||||||
check("compression.codec." + codec, second[0] == 200 and second[1].get("content-encoding") == codec
|
|
||||||
and decoder(first[2]) == decoder(second[2]) == expected, cache=second[1].get("x-cache"))
|
|
||||||
@@ -1,184 +0,0 @@
|
|||||||
"""Live ACL, authentication, atomic reload and logging regressions."""
|
|
||||||
import base64
|
|
||||||
import copy
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import time
|
|
||||||
|
|
||||||
from common import http, write_json
|
|
||||||
|
|
||||||
|
|
||||||
DENY = {"default": "deny", "rules": []}
|
|
||||||
ALLOW_LOCAL = {"default": "deny", "rules": [{"network": "127.0.0.1/32", "action": "allow"}]}
|
|
||||||
|
|
||||||
|
|
||||||
def run(check, rust, peer):
|
|
||||||
port = rust.reverse_port
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["host_policies"] = [{"hosts": ["fixture.test"], "acl": DENY}]
|
|
||||||
rust.apply(config, "policy-global-host")
|
|
||||||
count = peer.count("GET", "/echo")
|
|
||||||
s, h, b = http(port, "/echo", headers={"Host": "fixture.test"})
|
|
||||||
check("acl.global_host_without_vhost_acl", s == 403 and peer.count("GET", "/echo") == count,
|
|
||||||
status=s, upstream_requests=peer.count("GET", "/echo") - count)
|
|
||||||
s, h, b = http(port, "/echo", headers={"Host": "localhost"})
|
|
||||||
check("acl.global_other_host_unchanged", s == 200, status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["acl"] = DENY
|
|
||||||
rust.apply(config, "policy-vhost-deny")
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("acl.vhost_deny", s == 403, status=s)
|
|
||||||
s, h, b = http(rust.static_port)
|
|
||||||
check("acl.vhost_isolation", s == 200, status=s)
|
|
||||||
config["vhosts"][0]["acl"] = ALLOW_LOCAL
|
|
||||||
rust.apply(config, "policy-vhost-allow")
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("acl.hot_enable_allow", s == 200, status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["acl_profiles"] = [{"id": "restricted", "name": "Restricted", "acl": DENY}]
|
|
||||||
config["vhosts"][0]["acl_profile_id"] = "restricted"
|
|
||||||
config["vhosts"][0]["acl"] = ALLOW_LOCAL
|
|
||||||
rust.apply(config, "policy-profile-deny")
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("acl.profile_and_vhost_stack", s == 403, status=s)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["routes"] = [{"id": "denied", "path_prefix": "/echo", "acl": DENY}]
|
|
||||||
rust.apply(config, "policy-location-deny")
|
|
||||||
count = peer.count("GET", "/echo")
|
|
||||||
s, h, b = http(port, "/echo")
|
|
||||||
check("acl.location_before_upstream", s == 403 and peer.count("GET", "/echo") == count, status=s)
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("acl.location_other_path_unchanged", s == 200, status=s)
|
|
||||||
config["vhosts"][0]["acl"] = DENY
|
|
||||||
config["vhosts"][0]["routes"][0]["acl"] = ALLOW_LOCAL
|
|
||||||
rust.apply(config, "policy-location-allow-vhost-deny")
|
|
||||||
s, h, b = http(port, "/echo")
|
|
||||||
check("acl.location_cannot_override_vhost_deny", s == 403, status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["acl"] = {"default": "deny", "rules": [{"network": "203.0.113.22/32", "action": "allow"}]}
|
|
||||||
rust.apply(config, "policy-untrusted-forwarded")
|
|
||||||
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
|
|
||||||
check("acl.spoofed_forwarded_ip_denied", s == 403, status=s)
|
|
||||||
config["forwarding"] = {"trusted_proxies": ["127.0.0.1/32"]}
|
|
||||||
rust.apply(config, "policy-trusted-forwarded")
|
|
||||||
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22"})
|
|
||||||
check("acl.trusted_forwarded_ip_allowed", s == 200, status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["rate_limit"] = {"country_access": {"mode": "allow", "countries": ["PL"]}}
|
|
||||||
config["vhosts"][0]["cloudflare_client_ip"] = True
|
|
||||||
rust.apply(config, "policy-country")
|
|
||||||
s, h, b = http(port, headers={"CF-IPCountry": "PL", "CF-Connecting-IP": "203.0.113.22"})
|
|
||||||
check("acl.untrusted_country_header_cannot_allow", s == 403, status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["method_policy"] = {"mode": "allow", "methods": ["GET", "HEAD"]}
|
|
||||||
rust.apply(config, "policy-method")
|
|
||||||
s, h, b = http(port, "/echo", "POST", body=b"denied method")
|
|
||||||
check("policy.method_allowlist", s in (403, 405), status=s)
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("policy.allowed_method_survives", s == 200, status=s)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["rate_limit"] = {"enabled": True, "requests": 2, "window_seconds": 60, "burst": 0}
|
|
||||||
rust.apply(config, "policy-rate-limit")
|
|
||||||
statuses = [http(port)[0] for _ in range(3)]
|
|
||||||
check("policy.rate_limit", statuses[:2] == [200, 200] and statuses[2] == 429, statuses=statuses)
|
|
||||||
rust.reset()
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("policy.hot_disable_restores_request", s == 200, status=s)
|
|
||||||
|
|
||||||
# Every route check uses the real admin socket and no bearer/backdoor token.
|
|
||||||
for path in ["/api/config", "/api/status", "/api/logs", "/api/events", "/api/openapi.json", "/assets/app.js"]:
|
|
||||||
s, h, b = http(rust.admin_port, path)
|
|
||||||
check("admin.private." + path, s in (401, 403), status=s)
|
|
||||||
check("admin.login", rust.login() == 200)
|
|
||||||
s, h, b = rust.admin("/api/config")
|
|
||||||
envelope = json.loads(b)
|
|
||||||
check("admin.authenticated_config", s == 200 and "config" in envelope and "revision" in envelope, status=s)
|
|
||||||
s, h, b = rust.admin("/api/config", "PUT", {"expected_revision": envelope["revision"], "config": rust.base}, csrf=False)
|
|
||||||
check("admin.csrf_required", s == 403, status=s)
|
|
||||||
s, h, b = rust.admin("/api/config", origin="http://foreign.example")
|
|
||||||
check("admin.foreign_origin_denied", s == 403, status=s)
|
|
||||||
|
|
||||||
# Generate the disposable Basic Auth hash through the existing secured API.
|
|
||||||
s, h, b = rust.admin("/api/basic-auth/hash", "POST", {"username": "fixtureuser", "password": rust.password})
|
|
||||||
hashed = json.loads(b)
|
|
||||||
check("basic.hash_api", s == 200 and "password_hash" in hashed, status=s)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["basic_auth"] = {"enabled": True, "require_tls": False,
|
|
||||||
"users": [{"username": "fixtureuser", "password_hash": hashed["password_hash"]}]}
|
|
||||||
rust.apply(config, "policy-basic-auth")
|
|
||||||
s, h, b = http(port, "/echo")
|
|
||||||
check("basic.challenge", s == 401 and "Basic" in h.get("www-authenticate", ""), status=s)
|
|
||||||
credentials = base64.b64encode(("fixtureuser:" + rust.password).encode()).decode()
|
|
||||||
s, h, b = http(port, "/echo", headers={"Authorization": "Basic " + credentials})
|
|
||||||
check("basic.correct_password_and_strip", s == 200 and "authorization" not in json.loads(b)["headers"], status=s)
|
|
||||||
s, h, b = http(port, "/echo", headers={"Authorization": "Basic Zml4dHVyZXVzZXI6d3Jvbmc="})
|
|
||||||
check("basic.wrong_password_denied", s == 401, status=s)
|
|
||||||
config["vhosts"][0]["acl"] = DENY
|
|
||||||
rust.apply(config, "policy-acl-before-basic")
|
|
||||||
s, h, b = http(port, "/echo")
|
|
||||||
check("basic.acl_precedes_auth", s == 403, status=s)
|
|
||||||
rust.reset()
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["frontend"]["listener_ids"] = ["does-not-exist"]
|
|
||||||
rust.apply(config, "invalid-config-retains-snapshot", expected_success=False)
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("reload.invalid_document_retains_previous_snapshot", s == 200, status=s)
|
|
||||||
s, h, b = http(rust.static_port)
|
|
||||||
check("reload.invalid_document_keeps_other_vhost", s == 200, status=s)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["vhosts"][0]["backends"][0]["allow_private"] = False
|
|
||||||
# Explicit private-address rejection is checked through apply, preserving the
|
|
||||||
# active configuration if compile-time validation rejects the backend.
|
|
||||||
rust.apply(config, "ssrf-private-origin-disallowed", expected_success=False)
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("ssrf.rejected_origin_does_not_replace_snapshot", s == 200, status=s)
|
|
||||||
|
|
||||||
query = "/echo?observability=" + rust.label
|
|
||||||
s, h, b = http(port, query)
|
|
||||||
request_id = json.loads(b)["headers"]["x-request-id"]
|
|
||||||
log = None
|
|
||||||
for _ in range(20):
|
|
||||||
ls, lh, lb = rust.admin("/api/logs?vhost=reverse&limit=512")
|
|
||||||
logs = json.loads(lb)
|
|
||||||
log = next((entry for entry in logs.get("entries", []) if entry.get("request_id") == request_id), None)
|
|
||||||
if log:
|
|
||||||
break
|
|
||||||
time.sleep(.05)
|
|
||||||
check("observability.log_fields_and_body_bytes", log is not None and log.get("status") == 200
|
|
||||||
and log.get("response_bytes") == len(b) and log.get("request_bytes") == 0
|
|
||||||
and log.get("cache_effective_enabled") is False and log.get("transform_effective_enabled") is False
|
|
||||||
and bool(re.fullmatch("[0-9a-f]{64}", log.get("request_id", ""))),
|
|
||||||
recorded_status=log.get("status") if log else None,
|
|
||||||
recorded_response_bytes=log.get("response_bytes") if log else None, response_bytes=len(b))
|
|
||||||
if log:
|
|
||||||
write_json(rust.fixture.results / f"{rust.label}-sample-exchange.json", log)
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["diagnostic_headers"] = {"request_id": True}
|
|
||||||
rust.apply(config, "observability-static-eos")
|
|
||||||
s, h, b = http(rust.static_port)
|
|
||||||
static_request_id = h.get("x-request-id")
|
|
||||||
static_log = None
|
|
||||||
for _ in range(20):
|
|
||||||
ls, lh, lb = rust.admin("/api/logs?vhost=static&limit=64")
|
|
||||||
static_log = next((entry for entry in json.loads(lb).get("entries", [])
|
|
||||||
if entry.get("request_id") == static_request_id), None)
|
|
||||||
if static_log:
|
|
||||||
break
|
|
||||||
time.sleep(.05)
|
|
||||||
check("observability.static_complete_after_content_length", s == 200 and static_log is not None
|
|
||||||
and static_log.get("outcome") == "complete" and static_log.get("response_bytes") == len(b),
|
|
||||||
outcome=static_log.get("outcome") if static_log else None,
|
|
||||||
recorded_response_bytes=static_log.get("response_bytes") if static_log else None,
|
|
||||||
response_bytes=len(b))
|
|
||||||
if static_log:
|
|
||||||
write_json(rust.fixture.results / f"{rust.label}-sample-static-exchange.json", static_log)
|
|
||||||
s, h, b = rust.admin("/api/status")
|
|
||||||
status = json.loads(b)
|
|
||||||
write_json(rust.fixture.results / f"{rust.label}-runtime-status.json", status)
|
|
||||||
check("observability.status_available", s == 200, status=s)
|
|
||||||
@@ -1,135 +0,0 @@
|
|||||||
"""Proxy streaming, status, header, request-ID and WebSocket tests."""
|
|
||||||
import copy
|
|
||||||
import hashlib
|
|
||||||
from http.client import HTTPConnection, HTTPResponse
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import socket
|
|
||||||
import time
|
|
||||||
|
|
||||||
from common import http
|
|
||||||
|
|
||||||
|
|
||||||
def run(check, rust, peer):
|
|
||||||
port = rust.reverse_port
|
|
||||||
for name in ["small.html", "large.bin", "huge.bin"]:
|
|
||||||
expected = (rust.fixture.site / name).read_bytes()
|
|
||||||
s, h, b = http(port, "/" + name)
|
|
||||||
check("proxy.get." + name, s == 200 and b == expected, status=s, bytes=len(b))
|
|
||||||
s, h, b = http(port, "/" + name, "HEAD")
|
|
||||||
check("proxy.head." + name, s == 200 and b == b"" and h.get("content-length") == str(len(expected)), status=s)
|
|
||||||
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-127"})
|
|
||||||
check("proxy.range", s == 206 and b == bytes(range(128)), status=s, bytes=len(b))
|
|
||||||
s, h, b = http(port)
|
|
||||||
s, h, b = http(port, headers={"If-None-Match": h["etag"]})
|
|
||||||
check("proxy.conditional304", s == 304 and b == b"", status=s)
|
|
||||||
for status in [201, 400, 404, 500, 503]:
|
|
||||||
s, h, b = http(port, f"/status/{status}")
|
|
||||||
check(f"proxy.origin_status.{status}", s == status and b == b"origin status payload\n"
|
|
||||||
and h.get("x-origin-fixture") == "yes", status=s)
|
|
||||||
s, h, b = http(port, "/redirect")
|
|
||||||
check("proxy.redirect", s == 302 and h.get("location") == "/small.html", status=s)
|
|
||||||
|
|
||||||
payload = bytes(range(256)) * 4096
|
|
||||||
ids = []
|
|
||||||
for method in ["POST", "PUT"]:
|
|
||||||
s, h, b = http(port, "/echo?query=preserved", method,
|
|
||||||
{"Content-Type": "application/octet-stream", "X-Request-ID": "untrusted-client-id"}, payload)
|
|
||||||
data = json.loads(b)
|
|
||||||
request_id = data["headers"].get("x-request-id", "")
|
|
||||||
ids.append(request_id)
|
|
||||||
check("proxy.upload." + method, s == 200 and data["method"] == method
|
|
||||||
and data["path"] == "/echo?query=preserved" and data["body_bytes"] == len(payload)
|
|
||||||
and data["body_sha256"] == hashlib.sha256(payload).hexdigest(), status=s, bytes=data["body_bytes"])
|
|
||||||
check("proxy.request_id." + method, bool(re.fullmatch("[0-9a-f]{64}", request_id)),
|
|
||||||
valid_64_hex=bool(re.fullmatch("[0-9a-f]{64}", request_id)))
|
|
||||||
check("proxy.request_id_unique", len(set(ids)) == 2)
|
|
||||||
s, h, b = http(port, "/echo", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
|
|
||||||
data = json.loads(b)
|
|
||||||
check("proxy.nonempty_get_body_preserved", s == 200 and data["body_bytes"] == 17
|
|
||||||
and data["body_sha256"] == hashlib.sha256(b"nonempty-get-body").hexdigest(), status=s)
|
|
||||||
|
|
||||||
peer.started_upload.clear()
|
|
||||||
first, last = b"first streamed chunk" * 64, b"final streamed chunk" * 64
|
|
||||||
with socket.create_connection(("127.0.0.1", port), timeout=5) as sock:
|
|
||||||
sock.sendall((f"POST /stream-upload HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\n"
|
|
||||||
"Transfer-Encoding: chunked\r\nContent-Type: application/octet-stream\r\n\r\n").encode())
|
|
||||||
sock.sendall(f"{len(first):x}\r\n".encode() + first + b"\r\n")
|
|
||||||
forwarded_before_end = peer.started_upload.wait(2)
|
|
||||||
sock.sendall(f"{len(last):x}\r\n".encode() + last + b"\r\n0\r\n\r\n")
|
|
||||||
response = HTTPResponse(sock)
|
|
||||||
response.begin()
|
|
||||||
data = json.loads(response.read())
|
|
||||||
check("proxy.chunked_streaming_upload", forwarded_before_end and response.status == 200
|
|
||||||
and data["body_sha256"] == hashlib.sha256(first + last).hexdigest(),
|
|
||||||
first_chunk_arrived_before_end=forwarded_before_end, status=response.status)
|
|
||||||
|
|
||||||
peer.release_stream.clear()
|
|
||||||
connection = HTTPConnection("127.0.0.1", port, timeout=3)
|
|
||||||
connection.request("GET", "/stream-download")
|
|
||||||
started = time.monotonic()
|
|
||||||
response = connection.getresponse()
|
|
||||||
first = response.read(5)
|
|
||||||
first_seconds = time.monotonic() - started
|
|
||||||
peer.release_stream.set()
|
|
||||||
last = response.read()
|
|
||||||
connection.close()
|
|
||||||
check("proxy.streaming_response", response.status == 200 and first == b"first" and last == b"last"
|
|
||||||
and first_seconds < 2, first_chunk_seconds=first_seconds)
|
|
||||||
peer.release_stream.clear()
|
|
||||||
connection = HTTPConnection("127.0.0.1", port, timeout=3)
|
|
||||||
connection.request("GET", "/stream-download")
|
|
||||||
response = connection.getresponse()
|
|
||||||
response.read(5)
|
|
||||||
connection.close()
|
|
||||||
peer.release_stream.set()
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("proxy.client_disconnect_survives", s == 200, status=s)
|
|
||||||
|
|
||||||
before = peer.count("POST", "/no-replay")
|
|
||||||
s, h, b = http(port, "/no-replay", "POST", {"Content-Type": "application/octet-stream"}, b"write-once")
|
|
||||||
after = peer.count("POST", "/no-replay")
|
|
||||||
check("proxy.failed_post_not_replayed", s in (502, 503, 504) and after - before == 1,
|
|
||||||
status=s, upstream_writes=after - before)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Global", "value": "global"}],
|
|
||||||
"response": [{"op": "set", "name": "X-Fixture-Global", "value": "response"}]}
|
|
||||||
config["vhosts"][0]["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Vhost", "value": "vhost"}]}
|
|
||||||
config["vhosts"][0]["routes"] = [{"id": "echo", "path_prefix": "/echo", "headers": {
|
|
||||||
"request": [{"op": "set", "name": "X-Fixture-Route", "value": "route"}],
|
|
||||||
"response": [{"op": "set", "name": "X-Fixture-Route", "value": "response"}]}}]
|
|
||||||
rust.apply(config, "proxy-headers")
|
|
||||||
s, h, b = http(port, "/echo", headers={"Connection": "keep-alive, x-hop-fixture", "X-Hop-Fixture": "must-be-stripped"})
|
|
||||||
observed = json.loads(b)["headers"]
|
|
||||||
check("proxy.header_scopes", s == 200 and observed.get("x-fixture-global") == "global"
|
|
||||||
and observed.get("x-fixture-vhost") == "vhost" and observed.get("x-fixture-route") == "route"
|
|
||||||
and h.get("x-fixture-global") == "response" and h.get("x-fixture-route") == "response", status=s)
|
|
||||||
check("proxy.hop_by_hop_removed", "x-hop-fixture" not in observed)
|
|
||||||
s, h, b = http(port, "/echo", headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
|
|
||||||
observed = json.loads(b)["headers"]
|
|
||||||
check("proxy.untrusted_forwarded_ip", observed.get("x-real-ip") == "127.0.0.1"
|
|
||||||
and "203.0.113.22" not in observed.get("x-forwarded-for", ""),
|
|
||||||
observed_x_real_ip=observed.get("x-real-ip"), observed_x_forwarded_for=observed.get("x-forwarded-for"))
|
|
||||||
|
|
||||||
rust.reset()
|
|
||||||
with socket.create_connection(("127.0.0.1", port), timeout=4) as sock:
|
|
||||||
sock.sendall((f"GET /ws HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\nUpgrade: websocket\r\n"
|
|
||||||
"Connection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
|
|
||||||
"Sec-WebSocket-Version: 13\r\n\r\n").encode())
|
|
||||||
buffer = b""
|
|
||||||
while b"\r\n\r\n" not in buffer:
|
|
||||||
piece = sock.recv(4096)
|
|
||||||
if not piece:
|
|
||||||
break
|
|
||||||
buffer += piece
|
|
||||||
upgraded = buffer.startswith(b"HTTP/1.1 101")
|
|
||||||
echoed = b""
|
|
||||||
if upgraded:
|
|
||||||
message, mask = b"hello", b"\x11\x22\x33\x44"
|
|
||||||
sock.sendall(bytes([0x81, 0x80 | len(message)]) + mask +
|
|
||||||
bytes(c ^ mask[i % 4] for i, c in enumerate(message)))
|
|
||||||
frame = sock.recv(2)
|
|
||||||
if len(frame) == 2:
|
|
||||||
echoed = sock.recv(frame[1] & 127)
|
|
||||||
check("proxy.websocket_echo", upgraded and echoed == b"hello", upgraded=upgraded, bytes=len(echoed))
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
"""Live static-file regression cases against RustProxy's network listener."""
|
|
||||||
import copy
|
|
||||||
from email.parser import BytesParser
|
|
||||||
from email.policy import default
|
|
||||||
import gzip
|
|
||||||
|
|
||||||
from common import http
|
|
||||||
|
|
||||||
|
|
||||||
def run(check, rust, peer):
|
|
||||||
port, fixture = rust.static_port, rust.fixture
|
|
||||||
for name in ["small.html", "large.bin", "huge.bin", "empty.txt"]:
|
|
||||||
expected = (fixture.site / name).read_bytes()
|
|
||||||
status, headers, body = http(port, "/" + name)
|
|
||||||
check(f"static.get.{name}", status == 200 and body == expected,
|
|
||||||
status=status, bytes=len(body))
|
|
||||||
head_status, head_headers, head_body = http(port, "/" + name, "HEAD")
|
|
||||||
check(f"static.head.{name}", head_status == 200 and head_body == b""
|
|
||||||
and head_headers.get("content-length") == str(len(expected)),
|
|
||||||
status=head_status, content_length=head_headers.get("content-length"))
|
|
||||||
|
|
||||||
status, headers, body = http(port)
|
|
||||||
tag = headers.get("etag")
|
|
||||||
modified = headers.get("last-modified")
|
|
||||||
check("static.validators_present", bool(tag and modified), etag=tag, last_modified=modified)
|
|
||||||
for label, condition in [("etag", {"If-None-Match": tag}),
|
|
||||||
("modified", {"If-Modified-Since": modified})]:
|
|
||||||
s, h, b = http(port, headers=condition)
|
|
||||||
check("static.conditional304." + label, s == 304 and b == b"", status=s, bytes=len(b))
|
|
||||||
s, h, b = http(port, headers={"If-None-Match": '"missing"'})
|
|
||||||
check("static.conditional_miss", s == 200 and b == body, status=s)
|
|
||||||
|
|
||||||
data = (fixture.site / "large.bin").read_bytes()
|
|
||||||
cases = [("prefix", "bytes=0-127", data[:128], "bytes 0-127/65536"),
|
|
||||||
("suffix", "bytes=-37", data[-37:], "bytes 65499-65535/65536"),
|
|
||||||
("open", "bytes=65500-", data[65500:], "bytes 65500-65535/65536")]
|
|
||||||
for name, range_header, expected, content_range in cases:
|
|
||||||
s, h, b = http(port, "/large.bin", headers={"Range": range_header})
|
|
||||||
check("static.range." + name, s == 206 and b == expected
|
|
||||||
and h.get("content-range") == content_range,
|
|
||||||
status=s, bytes=len(b), content_range=h.get("content-range"))
|
|
||||||
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=999999-"})
|
|
||||||
check("static.range.unsatisfiable", s == 416 and h.get("content-range") == "bytes */65536",
|
|
||||||
status=s, content_range=h.get("content-range"))
|
|
||||||
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-15,100-115"})
|
|
||||||
mime = BytesParser(policy=default).parsebytes(
|
|
||||||
b"Content-Type: " + h.get("content-type", "").encode() + b"\r\n\r\n" + b)
|
|
||||||
parts = list(mime.iter_parts())
|
|
||||||
check("static.range.multipart", s == 206 and len(parts) == 2
|
|
||||||
and parts[0].get_payload(decode=True) == data[:16]
|
|
||||||
and parts[1].get_payload(decode=True) == data[100:116], status=s, parts=len(parts))
|
|
||||||
for name, value, expected_status in [("match", tag, 206), ("mismatch", '"other"', 200)]:
|
|
||||||
s, h, b = http(port, headers={"Range": "bytes=0-15", "If-Range": value})
|
|
||||||
expected = body[:16] if expected_status == 206 else body
|
|
||||||
check("static.if_range." + name, s == expected_status and b == expected,
|
|
||||||
status=s, bytes=len(b))
|
|
||||||
|
|
||||||
s, h, b = http(port, "/directory")
|
|
||||||
check("static.directory_slash", s in (301, 308) and h.get("location", "").endswith("/directory/"),
|
|
||||||
status=s, location=h.get("location"))
|
|
||||||
s, h, b = http(port, "/directory/")
|
|
||||||
check("static.directory_index", s == 200 and b == b"directory index\n", status=s)
|
|
||||||
s, h, b = http(port, "/listing/")
|
|
||||||
check("static.directory_listing_denied", s == 403, status=s)
|
|
||||||
for path in ["/escape-link.txt", "/inside-link.txt"]:
|
|
||||||
s, h, b = http(port, path)
|
|
||||||
check("static.symlink_deny." + path, s == 403 and b"MUST-NOT" not in b, status=s)
|
|
||||||
for path in ["/%2e%2e/outside-secret.txt", "/%2fetc/passwd", "/%252e%252e/outside-secret.txt",
|
|
||||||
"/%00", "/..%5coutside-secret.txt"]:
|
|
||||||
s, h, b = http(port, path)
|
|
||||||
check("static.invalid_path." + path, s in (400, 403, 404) and b"MUST-NOT" not in b,
|
|
||||||
status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
service = config["web_services"][0]
|
|
||||||
service["symlink_policy"] = "inside_root"
|
|
||||||
service["directory_listing"] = True
|
|
||||||
rust.apply(config, "static-inside-root")
|
|
||||||
s, h, b = http(port, "/inside-link.txt")
|
|
||||||
check("static.symlink_inside_root_allowed", s == 200 and b == b"download fixture\n", status=s)
|
|
||||||
s, h, b = http(port, "/escape-link.txt")
|
|
||||||
check("static.symlink_inside_root_escape_denied", s == 403 and b"MUST-NOT" not in b, status=s)
|
|
||||||
s, h, b = http(port, "/listing/")
|
|
||||||
check("static.directory_listing_enabled", s == 200 and b"visible.txt" in b, status=s)
|
|
||||||
# Alternate a symlink target between valid and outside-root files. The file
|
|
||||||
# read must stay confined even when a pathname changes between requests.
|
|
||||||
link = fixture.site / "changing-link.txt"
|
|
||||||
race_statuses = []
|
|
||||||
escaped = False
|
|
||||||
for index in range(12):
|
|
||||||
if link.is_symlink():
|
|
||||||
link.unlink()
|
|
||||||
link.symlink_to(fixture.outside if index % 2 else fixture.site / "download.txt")
|
|
||||||
s, h, b = http(port, "/changing-link.txt")
|
|
||||||
race_statuses.append(s)
|
|
||||||
escaped = escaped or b"MUST-NOT" in b
|
|
||||||
check("static.changing_symlink_confinement", not escaped and all(s in (200, 403) for s in race_statuses),
|
|
||||||
statuses=race_statuses, note="Sequential target replacement; not a concurrent rename-race proof")
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["web_services"][0]["precompressed"] = {"enabled": True, "gzip": True, "brotli": False}
|
|
||||||
rust.apply(config, "static-precompressed")
|
|
||||||
s, h, b = http(port, headers={"Accept-Encoding": "gzip"})
|
|
||||||
sidecar = (fixture.site / "small.html.gz").read_bytes()
|
|
||||||
check("static.precompressed_gzip", s == 200 and h.get("content-encoding") == "gzip"
|
|
||||||
and b == sidecar and gzip.decompress(b) == body, status=s, bytes=len(b))
|
|
||||||
s, h, b = http(port, headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
|
|
||||||
check("static.precompressed_range", s == 206 and b == sidecar[:10], status=s, bytes=len(b))
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["web_services"][0]["try_files"] = ["$uri", "/small.html"]
|
|
||||||
config["web_services"][0]["rewrites"] = [{"pattern": "^/old$", "replacement": "/small.html", "status": 302}]
|
|
||||||
config["web_services"][0]["download_extensions"] = ["txt"]
|
|
||||||
config["web_services"][0]["mime_types"] = {"bin": "application/x-fixture"}
|
|
||||||
rust.apply(config, "static-routing")
|
|
||||||
s, h, b = http(port, "/spa/deep/link")
|
|
||||||
check("static.try_files_spa", s == 200 and b == body, status=s)
|
|
||||||
s, h, b = http(port, "/old")
|
|
||||||
check("static.rewrite_redirect", s == 302 and h.get("location") == "/small.html", status=s)
|
|
||||||
s, h, b = http(port, "/download.txt")
|
|
||||||
check("static.download_header", s == 200 and "attachment" in h.get("content-disposition", ""), status=s)
|
|
||||||
s, h, b = http(port, "/large.bin")
|
|
||||||
check("static.mime_override", s == 200 and h.get("content-type") == "application/x-fixture", status=s)
|
|
||||||
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
(fixture.site / ".htaccess").write_text("Header set X-Fixture-Htaccess yes\nDirectoryIndex small.html\n")
|
|
||||||
config["web_services"][0]["htaccess"] = {"enabled": True, "strict": True}
|
|
||||||
rust.apply(config, "static-htaccess")
|
|
||||||
s, h, b = http(port)
|
|
||||||
check("static.htaccess_header", s == 200 and h.get("x-fixture-htaccess") == "yes", status=s)
|
|
||||||
s, h, b = http(port, "/.htaccess")
|
|
||||||
check("static.htaccess_not_served", s in (403, 404) and b"Header set" not in b, status=s)
|
|
||||||
@@ -1,448 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Disposable, loopback-only fixtures for testing actual RustProxy processes.
|
|
||||||
|
|
||||||
Runtime databases and credentials live in TemporaryDirectory, never in results.
|
|
||||||
No pre-existing SQLite database, fixed port, or historical /mnt/data path is used.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import copy
|
|
||||||
import gzip
|
|
||||||
import hashlib
|
|
||||||
from http.client import HTTPConnection, HTTPException
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import pwd
|
|
||||||
import secrets
|
|
||||||
import shutil
|
|
||||||
import signal
|
|
||||||
import socket
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
def sha256(path: Path) -> str:
|
|
||||||
result = hashlib.sha256()
|
|
||||||
with path.open("rb") as stream:
|
|
||||||
for block in iter(lambda: stream.read(1024 * 1024), b""):
|
|
||||||
result.update(block)
|
|
||||||
return result.hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def write_json(path: Path, value) -> None:
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n")
|
|
||||||
|
|
||||||
|
|
||||||
def seal_results(directory: Path) -> None:
|
|
||||||
"""Checksums cover every regular evidence file, except the checksum manifest."""
|
|
||||||
paths = sorted(p for p in directory.rglob("*") if p.is_file()
|
|
||||||
and p.name != "SHA256SUMS")
|
|
||||||
(directory / "SHA256SUMS").write_text("".join(
|
|
||||||
f"{sha256(path)} {path.relative_to(directory)}\n" for path in paths))
|
|
||||||
|
|
||||||
|
|
||||||
def http(port: int, path="/small.html", method="GET", headers=None, body=None,
|
|
||||||
timeout=10, source_address=None):
|
|
||||||
connection = HTTPConnection("127.0.0.1", port, timeout=timeout,
|
|
||||||
source_address=source_address)
|
|
||||||
try:
|
|
||||||
connection.request(method, path, body, headers or {})
|
|
||||||
response = connection.getresponse()
|
|
||||||
payload = response.read()
|
|
||||||
return response.status, {k.lower(): v for k, v in response.getheaders()}, payload
|
|
||||||
finally:
|
|
||||||
connection.close()
|
|
||||||
|
|
||||||
|
|
||||||
def free_port() -> int:
|
|
||||||
with socket.socket() as sock:
|
|
||||||
sock.bind(("127.0.0.1", 0))
|
|
||||||
return sock.getsockname()[1]
|
|
||||||
|
|
||||||
|
|
||||||
def wait_ready(port: int, process: subprocess.Popen, timeout=30):
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
if process.poll() is not None:
|
|
||||||
raise RuntimeError(f"process {process.pid} exited with {process.returncode}")
|
|
||||||
try:
|
|
||||||
http(port, "/", timeout=.5)
|
|
||||||
return
|
|
||||||
except (OSError, HTTPException):
|
|
||||||
time.sleep(.05)
|
|
||||||
raise RuntimeError(f"loopback listener {port} did not start")
|
|
||||||
|
|
||||||
|
|
||||||
class Fixture:
|
|
||||||
def __init__(self, results: Path, workers=4, temp_parent: Path | None = None,
|
|
||||||
server_cpus=None, origin_cpus=None, origin_workers=None):
|
|
||||||
self.results = results.resolve()
|
|
||||||
self.results.mkdir(parents=True, exist_ok=True)
|
|
||||||
# Keep live state outside the evidence tree. Even an interrupted run or
|
|
||||||
# workspace synchronization must never put databases/keys in results.
|
|
||||||
self.temp = tempfile.TemporaryDirectory(prefix="rustproxy-perf-", dir=temp_parent)
|
|
||||||
self.root = Path(self.temp.name)
|
|
||||||
self.root.chmod(0o755) # nginx workers must traverse the public fixture root
|
|
||||||
self.site = self.root / "site"
|
|
||||||
self.site.mkdir(mode=0o755)
|
|
||||||
self.workers = workers
|
|
||||||
self.server_cpus = server_cpus
|
|
||||||
self.origin_cpus = origin_cpus
|
|
||||||
self.origin_workers = origin_workers or workers
|
|
||||||
self.children = []
|
|
||||||
self.handles = []
|
|
||||||
self.ports = set()
|
|
||||||
self.origin_port = self.port()
|
|
||||||
self.make_site()
|
|
||||||
|
|
||||||
def port(self):
|
|
||||||
while True:
|
|
||||||
port = free_port()
|
|
||||||
if port not in self.ports:
|
|
||||||
self.ports.add(port)
|
|
||||||
return port
|
|
||||||
|
|
||||||
def make_site(self):
|
|
||||||
# Exact, reproducible payload sizes match the original report's cases.
|
|
||||||
small = b"<!doctype html><html><body>RustProxy benchmark "
|
|
||||||
small += b"x" * (1040 - len(small) - len(b"</body></html>"))
|
|
||||||
small += b"</body></html>"
|
|
||||||
files = {"small.html": small,
|
|
||||||
"large.bin": bytes(range(256)) * 256,
|
|
||||||
"huge.bin": bytes(range(256)) * 4096,
|
|
||||||
"empty.txt": b"",
|
|
||||||
"app.json": (b'{\n "message": "stable fixture",\n "values": [1, 2, 3]\n}\n'),
|
|
||||||
"app.css": b"/* fixture comment */\nbody { color: red; padding: 0px; }\n",
|
|
||||||
"compress.txt": (b"compressible RustProxy text fixture. " * 4096),
|
|
||||||
"download.txt": b"download fixture\n"}
|
|
||||||
for name, body in files.items():
|
|
||||||
(self.site / name).write_bytes(body)
|
|
||||||
(self.site / "small.html.gz").write_bytes(gzip.compress(small, mtime=0))
|
|
||||||
(self.site / "directory").mkdir()
|
|
||||||
(self.site / "directory" / "index.html").write_bytes(b"directory index\n")
|
|
||||||
(self.site / "listing").mkdir()
|
|
||||||
(self.site / "listing" / "visible.txt").write_bytes(b"listing fixture\n")
|
|
||||||
(self.site / "inside-link.txt").symlink_to(self.site / "download.txt")
|
|
||||||
self.outside = self.root / "outside-secret.txt"
|
|
||||||
self.outside.write_bytes(b"MUST-NOT-BE-SERVED-OUTSIDE-ROOT\n")
|
|
||||||
(self.site / "escape-link.txt").symlink_to(self.outside)
|
|
||||||
write_json(self.results / "payloads.json", {
|
|
||||||
name: {"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest()}
|
|
||||||
for name, body in files.items()})
|
|
||||||
|
|
||||||
def spawn(self, argv, name, env=None, affinity=None):
|
|
||||||
log = (self.results / f"{name}.log").open("w")
|
|
||||||
self.handles.append(log)
|
|
||||||
if affinity:
|
|
||||||
argv = ["taskset", "-c", affinity, *argv]
|
|
||||||
child = subprocess.Popen([str(x) for x in argv], stdout=log,
|
|
||||||
stderr=subprocess.STDOUT, env=env, start_new_session=True)
|
|
||||||
self.children.append(child)
|
|
||||||
return child
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def stop(process):
|
|
||||||
if process.poll() is None:
|
|
||||||
os.killpg(process.pid, signal.SIGTERM)
|
|
||||||
try:
|
|
||||||
process.wait(timeout=12)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
os.killpg(process.pid, signal.SIGKILL)
|
|
||||||
process.wait(timeout=3)
|
|
||||||
|
|
||||||
def close(self):
|
|
||||||
for process in reversed(self.children):
|
|
||||||
self.stop(process)
|
|
||||||
for handle in self.handles:
|
|
||||||
handle.close()
|
|
||||||
self.temp.cleanup()
|
|
||||||
write_json(self.results / "fixture-cleanup.json", {
|
|
||||||
"temporary_state_removed": not self.root.exists(),
|
|
||||||
"state_was_outside_results": not self.root.is_relative_to(self.results)})
|
|
||||||
if self.root.exists():
|
|
||||||
raise RuntimeError("temporary runtime state was not removed; do not package the fixture directory")
|
|
||||||
|
|
||||||
def __enter__(self):
|
|
||||||
return self
|
|
||||||
|
|
||||||
def __exit__(self, *_):
|
|
||||||
self.close()
|
|
||||||
|
|
||||||
def base_config(self, reverse_port, static_port):
|
|
||||||
endpoints = []
|
|
||||||
for label, port in [("reverse", reverse_port), ("static", static_port)]:
|
|
||||||
endpoints.append({"id": label, "name": label, "scope": "internal",
|
|
||||||
"enabled": True, "binding": {
|
|
||||||
"address": f"127.0.0.1:{port}", "tls": False,
|
|
||||||
"http2": False, "http3": None}})
|
|
||||||
return {
|
|
||||||
"cache": {"enabled": False},
|
|
||||||
"geoip_updates": {"enabled": False},
|
|
||||||
"listeners": {"endpoints": endpoints},
|
|
||||||
"web_services": [{"id": "fixture-static", "name": "Fixture static",
|
|
||||||
"document_root": str(self.site), "index": ["index.html", "small.html"],
|
|
||||||
"directory_listing": False}],
|
|
||||||
"vhosts": [{"id": "reverse", "frontend": {
|
|
||||||
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
|
|
||||||
"listener_ids": ["reverse"]},
|
|
||||||
"backends": [{"id": "origin", "url": f"http://127.0.0.1:{self.origin_port}",
|
|
||||||
"allow_private": True, "preserve_host": False}],
|
|
||||||
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []},
|
|
||||||
{"id": "static", "frontend": {
|
|
||||||
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
|
|
||||||
"listener_ids": ["static"]},
|
|
||||||
"source": {"type": "web_server", "web_service_id": "fixture-static"},
|
|
||||||
"routes": []}]
|
|
||||||
}
|
|
||||||
|
|
||||||
def rust(self, binary: Path, label: str, max_idle=None):
|
|
||||||
return RustInstance(self, binary.resolve(), label, max_idle=max_idle)
|
|
||||||
|
|
||||||
def nginx(self, executable: Path, frontend=True):
|
|
||||||
"""Use a separate nginx master for the shared origin and comparator."""
|
|
||||||
executable = executable.resolve()
|
|
||||||
result = subprocess.run([str(executable), "-V"], capture_output=True, text=True, timeout=10)
|
|
||||||
(self.results / "nginx-version.txt").write_text(result.stdout + result.stderr)
|
|
||||||
origin_conf = self.nginx_config("nginx-origin", f"""
|
|
||||||
server {{ listen 127.0.0.1:{self.origin_port}; server_name _;
|
|
||||||
root "{self.site}"; location / {{ add_header Cache-Control "public, max-age=60"; }}
|
|
||||||
}}
|
|
||||||
""", workers=self.origin_workers)
|
|
||||||
origin = self.spawn([executable, "-p", self.root, "-c", origin_conf], "nginx-origin",
|
|
||||||
affinity=self.origin_cpus)
|
|
||||||
wait_ready(self.origin_port, origin)
|
|
||||||
result = {"origin": origin, "origin_port": self.origin_port}
|
|
||||||
if frontend:
|
|
||||||
reverse_port, static_port = self.port(), self.port()
|
|
||||||
conf = self.nginx_config("nginx-frontend", f"""
|
|
||||||
upstream fixture_origin {{ server 127.0.0.1:{self.origin_port}; keepalive 64; }}
|
|
||||||
server {{ listen 127.0.0.1:{reverse_port}; server_name _;
|
|
||||||
location / {{ proxy_pass http://fixture_origin; proxy_http_version 1.1;
|
|
||||||
proxy_set_header Connection ""; proxy_buffering off; }}
|
|
||||||
}}
|
|
||||||
server {{ listen 127.0.0.1:{static_port}; server_name _; root "{self.site}"; }}
|
|
||||||
""")
|
|
||||||
process = self.spawn([executable, "-p", self.root, "-c", conf], "nginx-frontend",
|
|
||||||
affinity=self.server_cpus)
|
|
||||||
wait_ready(reverse_port, process)
|
|
||||||
wait_ready(static_port, process)
|
|
||||||
result.update({"process": process, "reverse_port": reverse_port,
|
|
||||||
"static_port": static_port})
|
|
||||||
return result
|
|
||||||
|
|
||||||
def nginx_config(self, name, servers, workers=None):
|
|
||||||
path = self.root / f"{name}.conf"
|
|
||||||
text = f"""daemon off;
|
|
||||||
master_process on;
|
|
||||||
user {pwd.getpwuid(os.geteuid()).pw_name};
|
|
||||||
worker_processes {workers or self.workers};
|
|
||||||
pid {self.root}/{name}.pid;
|
|
||||||
error_log stderr warn;
|
|
||||||
events {{ worker_connections 8192; }}
|
|
||||||
http {{
|
|
||||||
access_log off;
|
|
||||||
sendfile on;
|
|
||||||
tcp_nopush on;
|
|
||||||
keepalive_timeout 65;
|
|
||||||
keepalive_requests 1000000;
|
|
||||||
default_type application/octet-stream;
|
|
||||||
types {{ text/html html; text/css css; application/json json; text/plain txt; }}
|
|
||||||
client_body_temp_path {self.root}/{name}-body;
|
|
||||||
proxy_temp_path {self.root}/{name}-proxy;
|
|
||||||
{servers}
|
|
||||||
}}
|
|
||||||
"""
|
|
||||||
path.write_text(text)
|
|
||||||
(self.results / path.name).write_text(text)
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
class RustInstance:
|
|
||||||
def __init__(self, fixture, binary, label, max_idle=None):
|
|
||||||
self.fixture, self.binary, self.label = fixture, binary, label
|
|
||||||
if not os.access(binary, os.X_OK):
|
|
||||||
target = fixture.root / f"{label}-rust-proxy"
|
|
||||||
shutil.copyfile(binary, target)
|
|
||||||
target.chmod(0o700)
|
|
||||||
self.binary = target
|
|
||||||
self.admin_port, self.reverse_port, self.static_port = (fixture.port() for _ in range(3))
|
|
||||||
self.data = fixture.root / f"{label}-data"
|
|
||||||
self.base = fixture.base_config(self.reverse_port, self.static_port)
|
|
||||||
if max_idle is not None:
|
|
||||||
self.base["vhosts"][0]["load_balancing"]["keepalive"] = {"max_idle_per_host": max_idle}
|
|
||||||
self.password = "Fixture!" + secrets.token_hex(20)
|
|
||||||
self.username = "performance"
|
|
||||||
self.env = os.environ.copy()
|
|
||||||
self.env["TOKIO_WORKER_THREADS"] = str(fixture.workers)
|
|
||||||
self.env["RUST_PROXY_PROXY_LISTEN"] = f"127.0.0.1:{self.reverse_port}"
|
|
||||||
self.env["RUST_PROXY_UPDATE_CHECK_INTERVAL_SECS"] = "0"
|
|
||||||
self.env.pop("RUST_PROXY_TLS_LISTEN", None)
|
|
||||||
self.applies = []
|
|
||||||
self.cli("admin", "init", "--username", self.username, "--password-stdin",
|
|
||||||
input=self.password + "\n")
|
|
||||||
self.start("bootstrap")
|
|
||||||
self.apply(self.base, "base")
|
|
||||||
fixture.stop(self.process)
|
|
||||||
self.start("runtime")
|
|
||||||
wait_ready(self.static_port, self.process)
|
|
||||||
version = self.cli("--version").stdout.strip()
|
|
||||||
write_json(fixture.results / f"{label}-binary.json", {
|
|
||||||
"label": label, "path": str(binary), "sha256": sha256(binary),
|
|
||||||
"version": version, "worker_threads": fixture.workers,
|
|
||||||
"pid": self.process.pid, "protocol": "HTTP/1.1 plaintext loopback",
|
|
||||||
"cpu_affinity": fixture.server_cpus,
|
|
||||||
"max_idle_per_host_override": max_idle})
|
|
||||||
|
|
||||||
def cli(self, *args, input=None, check=True):
|
|
||||||
result = subprocess.run([str(self.binary), "--data-dir", str(self.data),
|
|
||||||
"--admin-listen", f"127.0.0.1:{self.admin_port}", *args],
|
|
||||||
input=input, text=True, capture_output=True,
|
|
||||||
env=self.env, timeout=40)
|
|
||||||
if check and result.returncode:
|
|
||||||
raise RuntimeError(f"{self.label} CLI {' '.join(args[:2])}: "
|
|
||||||
f"{result.stdout[-2000:]} {result.stderr[-3000:]}")
|
|
||||||
return result
|
|
||||||
|
|
||||||
def start(self, phase):
|
|
||||||
self.process = self.fixture.spawn([
|
|
||||||
self.binary, "--data-dir", self.data, "--admin-listen", f"127.0.0.1:{self.admin_port}",
|
|
||||||
"--update-check-interval-secs", "0", "serve", "--proxy-listen",
|
|
||||||
f"127.0.0.1:{self.reverse_port}", "--max-connections", "4096", "--max-inflight", "4096"],
|
|
||||||
f"{self.label}-{phase}", env=self.env, affinity=self.fixture.server_cpus)
|
|
||||||
wait_ready(self.admin_port, self.process)
|
|
||||||
wait_ready(self.reverse_port, self.process)
|
|
||||||
|
|
||||||
def apply(self, config, name, expected_success=True):
|
|
||||||
config_path = self.fixture.root / f"{self.label}-{name}.json"
|
|
||||||
write_json(config_path, config)
|
|
||||||
result = self.cli("config", "apply", str(config_path), check=False)
|
|
||||||
# Configurations with ephemeral test passwords/hashes are intentionally not copied.
|
|
||||||
safe = copy.deepcopy(config)
|
|
||||||
for vhost in safe.get("vhosts", []):
|
|
||||||
if "basic_auth" in vhost:
|
|
||||||
vhost["basic_auth"]["users"] = "REDACTED fixture users"
|
|
||||||
write_json(self.fixture.results / "configs" / f"{self.label}-{name}.json", safe)
|
|
||||||
event = {"name": name, "sha256": sha256(config_path), "returncode": result.returncode,
|
|
||||||
"expected_success": expected_success, "stdout": result.stdout[-2000:],
|
|
||||||
"stderr": result.stderr[-3000:]}
|
|
||||||
self.applies.append(event)
|
|
||||||
write_json(self.fixture.results / f"{self.label}-config-applies.json", self.applies)
|
|
||||||
if (result.returncode == 0) != expected_success:
|
|
||||||
raise RuntimeError(f"config {name}: {result.stdout} {result.stderr}")
|
|
||||||
return result
|
|
||||||
|
|
||||||
def reset(self):
|
|
||||||
return self.apply(self.base, "restored-base")
|
|
||||||
|
|
||||||
def login(self):
|
|
||||||
status, headers, body = http(self.admin_port, "/api/auth/login", "POST",
|
|
||||||
{"Origin": f"http://127.0.0.1:{self.admin_port}", "Content-Type": "application/json",
|
|
||||||
"X-RustProxy-Request": "1"},
|
|
||||||
json.dumps({"username": self.username, "password": self.password}))
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"fixture login failed: HTTP {status}: {body[:200]!r}")
|
|
||||||
self.cookie = headers["set-cookie"].split(";", 1)[0]
|
|
||||||
self.csrf = json.loads(body)["csrf_token"]
|
|
||||||
return status
|
|
||||||
|
|
||||||
def wait_quiescent(self, timeout=180, quiet_seconds=5, poll_seconds=.5):
|
|
||||||
"""Wait for the real, one-time ACME bootstrap before any timed work.
|
|
||||||
|
|
||||||
This does not disable ACME or change the process environment. The
|
|
||||||
private fixture has no issuers/jobs; a terminal unavailable state is
|
|
||||||
valid for HTTP tests, but does not establish ACME issuance support.
|
|
||||||
"""
|
|
||||||
from resources import descendants, process_stats
|
|
||||||
|
|
||||||
if timeout <= 0 or quiet_seconds <= 0 or poll_seconds <= 0 or timeout <= quiet_seconds:
|
|
||||||
raise ValueError("quiescence timeout must exceed the positive quiet period")
|
|
||||||
started = time.monotonic()
|
|
||||||
deadline = started + timeout
|
|
||||||
output = self.fixture.results / f"{self.label}-quiescence.json"
|
|
||||||
report = {"completed": False, "label": self.label, "pid": self.process.pid,
|
|
||||||
"timeout_seconds": timeout, "required_quiet_seconds": quiet_seconds,
|
|
||||||
"poll_seconds": poll_seconds, "observations": [],
|
|
||||||
"acme_issuance_tested": False}
|
|
||||||
|
|
||||||
def read_admin(path):
|
|
||||||
remaining = deadline - time.monotonic()
|
|
||||||
if remaining <= 0:
|
|
||||||
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
|
|
||||||
if self.process.poll() is not None:
|
|
||||||
raise RuntimeError(f"{self.label} exited during quiescence")
|
|
||||||
status, _, body = self.admin(path, timeout=min(1, remaining))
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"{self.label} quiescence {path}: HTTP {status}")
|
|
||||||
return json.loads(body)
|
|
||||||
|
|
||||||
try:
|
|
||||||
root = process_stats(self.process.pid)
|
|
||||||
if root is None:
|
|
||||||
raise RuntimeError(f"{self.label} has no live root process")
|
|
||||||
identity = (root["pid"], root["start_ticks"])
|
|
||||||
report["start_ticks"] = root["start_ticks"]
|
|
||||||
envelope = read_admin("/api/config")
|
|
||||||
if not isinstance(envelope, dict) or not isinstance(envelope.get("config"), dict):
|
|
||||||
raise RuntimeError("quiescence requires the actual configuration envelope")
|
|
||||||
certificates = envelope["config"].get("tls", {}).get("certificates", [])
|
|
||||||
if not isinstance(certificates, list) or any(not isinstance(cert, dict) for cert in certificates):
|
|
||||||
raise RuntimeError("unexpected certificate configuration during quiescence")
|
|
||||||
report["configured_issuers"] = sum(cert.get("issuer") is not None for cert in certificates)
|
|
||||||
if report["configured_issuers"]:
|
|
||||||
raise RuntimeError("performance fixture must have no configured ACME issuers")
|
|
||||||
quiet_started, previous_terminal = None, None
|
|
||||||
while True:
|
|
||||||
payload = read_admin("/api/acme")
|
|
||||||
if not isinstance(payload, dict) or not {"runtime", "job", "renewals"} <= payload.keys():
|
|
||||||
raise RuntimeError("quiescence requires explicit ACME runtime/job/renewal state")
|
|
||||||
if payload["job"] is not None or payload["renewals"] != {}:
|
|
||||||
raise RuntimeError("performance fixture has an unexpected ACME job or renewal")
|
|
||||||
runtime = payload["runtime"]
|
|
||||||
if not isinstance(runtime, dict) or runtime.get("status") not in {
|
|
||||||
"checking", "initializing", "ready", "unavailable"}:
|
|
||||||
raise RuntimeError("unexpected ACME runtime state during quiescence")
|
|
||||||
pids = descendants(self.process.pid)
|
|
||||||
root = process_stats(self.process.pid)
|
|
||||||
if root is None or (root["pid"], root["start_ticks"]) != identity:
|
|
||||||
raise RuntimeError(f"{self.label} root process identity changed during quiescence")
|
|
||||||
now = time.monotonic()
|
|
||||||
if now >= deadline:
|
|
||||||
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
|
|
||||||
terminal = tuple(runtime.get(key) for key in ("status", "mode", "message"))
|
|
||||||
quiet = runtime["status"] in {"ready", "unavailable"} and pids == [self.process.pid]
|
|
||||||
if not quiet:
|
|
||||||
quiet_started, previous_terminal = None, None
|
|
||||||
elif quiet_started is None or terminal != previous_terminal:
|
|
||||||
quiet_started, previous_terminal = now, terminal
|
|
||||||
elapsed_quiet = now - quiet_started if quiet_started is not None else 0
|
|
||||||
report["observations"].append({
|
|
||||||
"elapsed_seconds": now - started, "process_count": len(pids),
|
|
||||||
"acme_status": runtime["status"], "acme_mode": runtime.get("mode"),
|
|
||||||
"acme_message": runtime.get("message"), "quiet_seconds": elapsed_quiet})
|
|
||||||
if elapsed_quiet >= quiet_seconds:
|
|
||||||
report.update({"completed": True, "elapsed_seconds": now - started,
|
|
||||||
"terminal_status": runtime["status"], "process_count": len(pids)})
|
|
||||||
write_json(output, report)
|
|
||||||
return report
|
|
||||||
write_json(output, report)
|
|
||||||
time.sleep(min(poll_seconds, max(0, deadline - time.monotonic())))
|
|
||||||
except (Exception, KeyboardInterrupt) as error:
|
|
||||||
report.update({"elapsed_seconds": time.monotonic() - started,
|
|
||||||
"error_type": type(error).__name__, "error": str(error)})
|
|
||||||
write_json(output, report)
|
|
||||||
raise
|
|
||||||
|
|
||||||
def admin(self, path, method="GET", body=None, csrf=True, cookie=True, origin=None,
|
|
||||||
timeout=10):
|
|
||||||
headers = {"Origin": origin or f"http://127.0.0.1:{self.admin_port}", "X-RustProxy-Request": "1"}
|
|
||||||
if cookie:
|
|
||||||
headers["Cookie"] = self.cookie
|
|
||||||
if csrf:
|
|
||||||
headers["X-CSRF-Token"] = self.csrf
|
|
||||||
if body is not None:
|
|
||||||
headers["Content-Type"] = "application/json"
|
|
||||||
body = json.dumps(body)
|
|
||||||
return http(self.admin_port, path, method, headers, body, timeout=timeout)
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Finite live HTTP concurrency correctness stress; no throughput/latency claim."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, http, seal_results, sha256, write_json
|
|
||||||
from resources import process_stats
|
|
||||||
|
|
||||||
|
|
||||||
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
|
|
||||||
|
|
||||||
|
|
||||||
def active_backends(rust):
|
|
||||||
status, _, body = rust.admin("/api/status")
|
|
||||||
if status != 200:
|
|
||||||
return {"error": f"HTTP {status}"}
|
|
||||||
values = []
|
|
||||||
|
|
||||||
def walk(value):
|
|
||||||
if isinstance(value, dict):
|
|
||||||
if "id" in value and "active" in value and "status" in value and "requests" in value:
|
|
||||||
values.append({key: value[key] for key in ("id", "active", "status", "requests")})
|
|
||||||
for child in value.values():
|
|
||||||
walk(child)
|
|
||||||
elif isinstance(value, list):
|
|
||||||
for child in value:
|
|
||||||
walk(child)
|
|
||||||
walk(json.loads(body))
|
|
||||||
return values
|
|
||||||
|
|
||||||
|
|
||||||
def batch(rust, port, name, expected, concurrency):
|
|
||||||
count = max(32, concurrency * 2)
|
|
||||||
expected_sha = hashlib.sha256(expected).hexdigest()
|
|
||||||
ready = threading.Barrier(concurrency, timeout=15)
|
|
||||||
lock = threading.Lock()
|
|
||||||
running = peak = 0
|
|
||||||
|
|
||||||
def fetch(index):
|
|
||||||
nonlocal running, peak
|
|
||||||
with lock:
|
|
||||||
running += 1
|
|
||||||
peak = max(peak, running)
|
|
||||||
try:
|
|
||||||
if index < concurrency:
|
|
||||||
ready.wait()
|
|
||||||
status, headers, body = http(port, "/" + name, timeout=15)
|
|
||||||
digest = hashlib.sha256(body).hexdigest()
|
|
||||||
return {"index": index, "status": status, "bytes": len(body), "sha256": digest,
|
|
||||||
"passed": status == 200 and len(body) == len(expected) and digest == expected_sha}
|
|
||||||
except Exception as error:
|
|
||||||
return {"index": index, "passed": False, "error": str(error)}
|
|
||||||
finally:
|
|
||||||
with lock:
|
|
||||||
running -= 1
|
|
||||||
|
|
||||||
before = process_stats(rust.process.pid)
|
|
||||||
started = time.monotonic()
|
|
||||||
with ThreadPoolExecutor(max_workers=concurrency) as executor:
|
|
||||||
responses = list(executor.map(fetch, range(count)))
|
|
||||||
elapsed = time.monotonic() - started
|
|
||||||
# Completion means every client consumed or failed its full response. Give
|
|
||||||
# asynchronous server bookkeeping a bounded interval to release its leases.
|
|
||||||
for _ in range(40):
|
|
||||||
active = active_backends(rust)
|
|
||||||
if isinstance(active, list) and active and all(row["active"] == 0 for row in active):
|
|
||||||
break
|
|
||||||
time.sleep(.05)
|
|
||||||
after = process_stats(rust.process.pid)
|
|
||||||
return {"configured_concurrent_clients": concurrency, "peak_running_client_tasks": peak,
|
|
||||||
"client_tasks_include_barrier": True, "requests": count, "elapsed_seconds": elapsed,
|
|
||||||
"expected_bytes": len(expected), "expected_sha256": expected_sha,
|
|
||||||
"response_failures": sum(not response["passed"] for response in responses),
|
|
||||||
"passed": all(response["passed"] for response in responses)
|
|
||||||
and isinstance(active, list) and bool(active) and all(row["active"] == 0 for row in active),
|
|
||||||
"responses": responses, "active_backends_after_drain": active,
|
|
||||||
"server_before": before, "server_after_drain": after}
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--nginx", type=Path, required=True)
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.out.exists() and any(args.out.iterdir()):
|
|
||||||
parser.error("--out must be new or empty")
|
|
||||||
reports = {}
|
|
||||||
for specification in args.binary:
|
|
||||||
label, binary = specification.split("=", 1)
|
|
||||||
cases, error = [], None
|
|
||||||
try:
|
|
||||||
with Fixture(args.out / label, workers=4, origin_workers=2) as fixture:
|
|
||||||
fixture.nginx(args.nginx, frontend=False)
|
|
||||||
rust = fixture.rust(Path(binary), label)
|
|
||||||
rust.login()
|
|
||||||
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
|
|
||||||
for name in PAYLOADS:
|
|
||||||
expected = (fixture.site / name).read_bytes()
|
|
||||||
for concurrency in [1, 16, 128]:
|
|
||||||
result = batch(rust, port, name, expected, concurrency)
|
|
||||||
case = {"case": f"concurrency.{source}.{name}.c{concurrency}", **result}
|
|
||||||
cases.append(case)
|
|
||||||
write_json(args.out / label / "cases" / f"{source}-{name}-c{concurrency}.json", case)
|
|
||||||
print(f"{label} {'PASS' if case['passed'] else 'FAIL'} {case['case']}", flush=True)
|
|
||||||
if rust.process.poll() is not None:
|
|
||||||
raise RuntimeError("RustProxy exited during concurrency regression")
|
|
||||||
except Exception as exc:
|
|
||||||
error = {"error": str(exc), "traceback": traceback.format_exc()}
|
|
||||||
write_json(args.out / label / "failure.json", error)
|
|
||||||
report = {"kind": "live HTTP/1.1 finite concurrency functional stress", "simulation": False,
|
|
||||||
"binary_sha256": sha256(Path(binary)), "throughput_measurement": False,
|
|
||||||
"passed_cases": sum(case["passed"] for case in cases),
|
|
||||||
"failed_cases": sum(not case["passed"] for case in cases),
|
|
||||||
"requests": sum(case["requests"] for case in cases),
|
|
||||||
"response_failures": sum(case["response_failures"] for case in cases),
|
|
||||||
"completed": error is None and len(cases) == 18,
|
|
||||||
"cases": [{key: case[key] for key in ("case", "passed", "requests", "response_failures")} for case in cases],
|
|
||||||
"limitations": ["configured client concurrency is not proof of simultaneous active server exchanges",
|
|
||||||
"Python client is not a throughput or latency benchmark",
|
|
||||||
"FD/RSS are before/after snapshots, not peaks or a long-term leak test"]}
|
|
||||||
reports[label] = report
|
|
||||||
write_json(args.out / label / "report.json", report)
|
|
||||||
write_json(args.out / "comparison.json", reports)
|
|
||||||
seal_results(args.out)
|
|
||||||
return 0 if all(report["completed"] and not report["failed_cases"] for report in reports.values()) else 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,223 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Small live two-origin failover and active-exchange regression, not a benchmark."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from collections import Counter
|
|
||||||
import copy
|
|
||||||
import hashlib
|
|
||||||
from http.client import HTTPConnection
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, http, seal_results, write_json
|
|
||||||
|
|
||||||
|
|
||||||
class Origin:
|
|
||||||
def __init__(self, port, name):
|
|
||||||
self.name, self.port = name, port
|
|
||||||
self.fail_get = False
|
|
||||||
self.counts, self.post_hashes = Counter(), []
|
|
||||||
self.release, self.lock = threading.Event(), threading.Lock()
|
|
||||||
origin = self
|
|
||||||
|
|
||||||
class Handler(BaseHTTPRequestHandler):
|
|
||||||
protocol_version = "HTTP/1.1"
|
|
||||||
|
|
||||||
def log_message(self, *_):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def do_GET(self):
|
|
||||||
self.respond()
|
|
||||||
|
|
||||||
def do_POST(self):
|
|
||||||
self.respond()
|
|
||||||
|
|
||||||
def respond(self):
|
|
||||||
body = self.rfile.read(int(self.headers.get("Content-Length", "0")))
|
|
||||||
with origin.lock:
|
|
||||||
origin.counts[(self.command, self.path)] += 1
|
|
||||||
if self.command == "POST":
|
|
||||||
origin.post_hashes.append(hashlib.sha256(body).hexdigest())
|
|
||||||
hold = self.path == "/hold"
|
|
||||||
status = 503 if self.command == "POST" or origin.fail_get else 200
|
|
||||||
payload = b"firstlast" if hold else origin.name.encode()
|
|
||||||
try:
|
|
||||||
self.send_response(status)
|
|
||||||
self.send_header("Content-Length", str(len(payload)))
|
|
||||||
self.send_header("Content-Type", "application/octet-stream")
|
|
||||||
self.send_header("Cache-Control", "no-store")
|
|
||||||
# Fresh connections make refused-listener tests independent
|
|
||||||
# of already accepted upstream keepalive connections.
|
|
||||||
self.send_header("Connection", "close")
|
|
||||||
self.end_headers()
|
|
||||||
if hold:
|
|
||||||
self.wfile.write(payload[:5])
|
|
||||||
self.wfile.flush()
|
|
||||||
origin.release.wait(15)
|
|
||||||
self.wfile.write(payload[5:])
|
|
||||||
else:
|
|
||||||
self.wfile.write(payload)
|
|
||||||
self.wfile.flush()
|
|
||||||
except (BrokenPipeError, ConnectionResetError):
|
|
||||||
pass
|
|
||||||
self.close_connection = True
|
|
||||||
|
|
||||||
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
|
|
||||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
|
||||||
self.thread.start()
|
|
||||||
self.closed = False
|
|
||||||
|
|
||||||
def close(self):
|
|
||||||
self.release.set()
|
|
||||||
if not self.closed:
|
|
||||||
self.server.shutdown()
|
|
||||||
self.server.server_close()
|
|
||||||
self.thread.join(timeout=3)
|
|
||||||
self.closed = True
|
|
||||||
|
|
||||||
|
|
||||||
def backend_views(rust):
|
|
||||||
status, _, body = rust.admin("/api/status")
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"status unavailable: {status}")
|
|
||||||
found = {}
|
|
||||||
|
|
||||||
def walk(value):
|
|
||||||
if isinstance(value, dict):
|
|
||||||
if value.get("id") in ("a", "b") and "active" in value and "status" in value:
|
|
||||||
found[value["id"]] = {key: value[key] for key in value
|
|
||||||
if key in ("id", "status", "active", "requests", "errors", "bytes_out")
|
|
||||||
or key.startswith("retry_") or key.startswith("connect_errors")}
|
|
||||||
for child in value.values():
|
|
||||||
walk(child)
|
|
||||||
elif isinstance(value, list):
|
|
||||||
for child in value:
|
|
||||||
walk(child)
|
|
||||||
walk(json.loads(body))
|
|
||||||
return found
|
|
||||||
|
|
||||||
|
|
||||||
def run(check, fixture, binary, label):
|
|
||||||
a, b = Origin(fixture.origin_port, "a"), Origin(fixture.port(), "b")
|
|
||||||
try:
|
|
||||||
rust = fixture.rust(binary, label)
|
|
||||||
rust.login()
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
vhost = config["vhosts"][0]
|
|
||||||
vhost["backends"] = [{"id": origin.name, "url": f"http://127.0.0.1:{origin.port}",
|
|
||||||
"allow_private": True, "preserve_host": False} for origin in [a, b]]
|
|
||||||
vhost["load_balancing"] = {"algorithm": "round_robin", "health_check": {"mode": "off"},
|
|
||||||
"slow_start_seconds": 0, "passive_health": {"enabled": False},
|
|
||||||
"retry": {"max_retries": 1, "retry_statuses": [503]}}
|
|
||||||
vhost["routes"] = [{"id": "hold-a", "path_prefix": "/hold", "backend_id": "a"}]
|
|
||||||
rust.apply(config, "lb-two-origins")
|
|
||||||
replies = [http(rust.reverse_port, "/rr") for _ in range(8)]
|
|
||||||
counts = Counter(body.decode() for status, _, body in replies if status == 200)
|
|
||||||
check("lb.round_robin_both_origins", counts == {"a": 4, "b": 4}, counts=dict(counts))
|
|
||||||
|
|
||||||
a.fail_get = True
|
|
||||||
replies = [http(rust.reverse_port, "/retry-status") for _ in range(6)]
|
|
||||||
check("lb.get_503_failover", all(status == 200 and body == b"b" for status, _, body in replies)
|
|
||||||
and a.counts[("GET", "/retry-status")] > 0,
|
|
||||||
failed_origin_attempts=a.counts[("GET", "/retry-status")])
|
|
||||||
views = backend_views(rust)
|
|
||||||
check("lb.retry_observations_recorded", sum(view.get("retry_attempts_total", 0) for view in views.values()) > 0,
|
|
||||||
backends=views)
|
|
||||||
a.fail_get = False
|
|
||||||
payload = b"one POST must be attempted once"
|
|
||||||
replies = [http(rust.reverse_port, "/post-no-replay", "POST", body=payload) for _ in range(4)]
|
|
||||||
attempts = sum(origin.counts[("POST", "/post-no-replay")] for origin in [a, b])
|
|
||||||
check("lb.post_503_never_replayed", all(status == 503 for status, _, _ in replies) and attempts == 4,
|
|
||||||
client_requests=4, origin_attempts=attempts)
|
|
||||||
check("lb.post_body_preserved", len(a.post_hashes + b.post_hashes) == 4 and
|
|
||||||
all(digest == hashlib.sha256(payload).hexdigest() for digest in a.post_hashes + b.post_hashes))
|
|
||||||
|
|
||||||
for mode in ["drain", "disable"]:
|
|
||||||
vhost["backends"][0].update({"enabled": True, "drain": False})
|
|
||||||
rust.apply(config, f"lb-{mode}-before")
|
|
||||||
a.release.clear()
|
|
||||||
connection = HTTPConnection("127.0.0.1", rust.reverse_port, timeout=10)
|
|
||||||
try:
|
|
||||||
connection.request("GET", "/hold")
|
|
||||||
response = connection.getresponse()
|
|
||||||
first = response.read(5)
|
|
||||||
before = backend_views(rust)
|
|
||||||
check(f"lb.{mode}.lease_spans_response_body", response.status == 200 and first == b"first"
|
|
||||||
and before.get("a", {}).get("active") == 1, backends=before)
|
|
||||||
vhost["backends"][0].update({"drain": mode == "drain", "enabled": mode != "disable"})
|
|
||||||
rust.apply(config, f"lb-{mode}-active")
|
|
||||||
replies = [http(rust.reverse_port, "/new") for _ in range(4)]
|
|
||||||
check(f"lb.{mode}.new_requests_use_healthy_member",
|
|
||||||
all(status == 200 and body == b"b" for status, _, body in replies))
|
|
||||||
during = backend_views(rust)
|
|
||||||
check(f"lb.{mode}.active_lease_survives_hot_edit", during.get("a", {}).get("active") == 1,
|
|
||||||
backends=during)
|
|
||||||
a.release.set()
|
|
||||||
last = response.read()
|
|
||||||
check(f"lb.{mode}.admitted_body_finishes", first + last == b"firstlast")
|
|
||||||
for _ in range(20):
|
|
||||||
after = backend_views(rust)
|
|
||||||
if after.get("a", {}).get("active") == 0:
|
|
||||||
break
|
|
||||||
time.sleep(.05)
|
|
||||||
check(f"lb.{mode}.lease_released_after_body", after.get("a", {}).get("active") == 0,
|
|
||||||
backends=after)
|
|
||||||
finally:
|
|
||||||
a.release.set()
|
|
||||||
connection.close()
|
|
||||||
|
|
||||||
vhost["backends"][0].update({"enabled": True, "drain": False})
|
|
||||||
rust.apply(config, "lb-connect-refused")
|
|
||||||
a.close()
|
|
||||||
replies = [http(rust.reverse_port, "/retry-connect") for _ in range(6)]
|
|
||||||
views = backend_views(rust)
|
|
||||||
check("lb.connect_refused_failover", all(status == 200 and body == b"b" for status, _, body in replies)
|
|
||||||
and views.get("a", {}).get("connect_errors_total", 0) > 0, backends=views)
|
|
||||||
rust.reset()
|
|
||||||
check("lb.static_vhost_survives_reload", http(rust.static_port)[0] == 200)
|
|
||||||
finally:
|
|
||||||
a.close()
|
|
||||||
b.close()
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.out.exists() and any(args.out.iterdir()):
|
|
||||||
parser.error("--out must be new or empty")
|
|
||||||
reports = {}
|
|
||||||
for specification in args.binary:
|
|
||||||
label, binary = specification.split("=", 1)
|
|
||||||
cases = []
|
|
||||||
|
|
||||||
def check(case, passed, **details):
|
|
||||||
cases.append({"case": case, "passed": passed, **details})
|
|
||||||
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
|
|
||||||
write_json(args.out / label / "checks.json", cases)
|
|
||||||
|
|
||||||
try:
|
|
||||||
with Fixture(args.out / label) as fixture:
|
|
||||||
run(check, fixture, Path(binary), label)
|
|
||||||
except Exception as exc:
|
|
||||||
check("lb.runner", False, error=str(exc), traceback=traceback.format_exc())
|
|
||||||
report = {"passed": sum(case["passed"] for case in cases),
|
|
||||||
"failed": sum(not case["passed"] for case in cases), "cases": cases,
|
|
||||||
"simulation": False, "kind": "live RustProxy two-origin HTTP/1.1 regression",
|
|
||||||
"not_covered": ["sticky affinity", "active health probes", "long-running connection soak",
|
|
||||||
"TLS origin failures", "DNS rotation"]}
|
|
||||||
reports[label] = report
|
|
||||||
write_json(args.out / label / "report.json", report)
|
|
||||||
write_json(args.out / "comparison.json", reports)
|
|
||||||
seal_results(args.out)
|
|
||||||
return 1 if any(report["failed"] for report in reports.values()) else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,223 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""A protocol peer, not a RustProxy emulator; used only as the test upstream.
|
|
||||||
|
|
||||||
Performance comparisons use nginx as the upstream, never this Python fixture.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import base64
|
|
||||||
import collections
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import socket
|
|
||||||
import struct
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
from urllib.parse import urlsplit, parse_qs
|
|
||||||
|
|
||||||
|
|
||||||
class Peer:
|
|
||||||
def __init__(self, port, site):
|
|
||||||
self.site = site
|
|
||||||
self.counts = collections.Counter()
|
|
||||||
self.received = {}
|
|
||||||
self.lock = threading.Lock()
|
|
||||||
self.started_upload = threading.Event()
|
|
||||||
self.release_stream = threading.Event()
|
|
||||||
peer = self
|
|
||||||
|
|
||||||
class Handler(BaseHTTPRequestHandler):
|
|
||||||
protocol_version = "HTTP/1.1"
|
|
||||||
|
|
||||||
def log_message(self, *_):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def emit(self, status, body=b"", headers=None):
|
|
||||||
self.send_response(status)
|
|
||||||
self.send_header("Content-Length", str(len(body)))
|
|
||||||
if "Content-Type" not in (headers or {}):
|
|
||||||
self.send_header("Content-Type", "application/octet-stream")
|
|
||||||
for key, value in (headers or {}).items():
|
|
||||||
self.send_header(key, value)
|
|
||||||
self.end_headers()
|
|
||||||
if self.command != "HEAD" and body:
|
|
||||||
self.wfile.write(body)
|
|
||||||
|
|
||||||
def upload(self):
|
|
||||||
chunks = []
|
|
||||||
if self.headers.get("Transfer-Encoding", "").lower() == "chunked":
|
|
||||||
while True:
|
|
||||||
length = int(self.rfile.readline().split(b";", 1)[0], 16)
|
|
||||||
if length == 0:
|
|
||||||
while self.rfile.readline().strip():
|
|
||||||
pass
|
|
||||||
break
|
|
||||||
payload = self.rfile.read(length)
|
|
||||||
self.rfile.read(2)
|
|
||||||
chunks.append(payload)
|
|
||||||
peer.started_upload.set()
|
|
||||||
else:
|
|
||||||
remaining = int(self.headers.get("Content-Length", 0))
|
|
||||||
while remaining:
|
|
||||||
payload = self.rfile.read(min(remaining, 1024))
|
|
||||||
if not payload:
|
|
||||||
break
|
|
||||||
chunks.append(payload)
|
|
||||||
remaining -= len(payload)
|
|
||||||
peer.started_upload.set()
|
|
||||||
return b"".join(chunks)
|
|
||||||
|
|
||||||
def do_POST(self):
|
|
||||||
self.handle_method()
|
|
||||||
|
|
||||||
def do_PUT(self):
|
|
||||||
self.handle_method()
|
|
||||||
|
|
||||||
def do_HEAD(self):
|
|
||||||
self.handle_method()
|
|
||||||
|
|
||||||
def do_GET(self):
|
|
||||||
self.handle_method()
|
|
||||||
|
|
||||||
def handle_method(self):
|
|
||||||
try:
|
|
||||||
self.route()
|
|
||||||
except (BrokenPipeError, ConnectionResetError, socket.timeout):
|
|
||||||
self.close_connection = True
|
|
||||||
|
|
||||||
def route(self):
|
|
||||||
parsed = urlsplit(self.path)
|
|
||||||
path, query = parsed.path, parse_qs(parsed.query)
|
|
||||||
with peer.lock:
|
|
||||||
peer.counts[(self.command, path)] += 1
|
|
||||||
count = peer.counts[(self.command, path)]
|
|
||||||
body = self.upload() if self.command in ("POST", "PUT") or self.headers.get("Content-Length") or self.headers.get("Transfer-Encoding") else b""
|
|
||||||
if path == "/no-replay":
|
|
||||||
self.close_connection = True
|
|
||||||
self.connection.shutdown(socket.SHUT_RDWR)
|
|
||||||
self.connection.close()
|
|
||||||
return
|
|
||||||
if path in ("/echo", "/stream-upload"):
|
|
||||||
data = {"method": self.command, "path": self.path,
|
|
||||||
"headers": {key.lower(): value for key, value in self.headers.items()},
|
|
||||||
"body_bytes": len(body), "body_sha256": hashlib.sha256(body).hexdigest()}
|
|
||||||
with peer.lock:
|
|
||||||
peer.received[path] = data
|
|
||||||
self.emit(200, json.dumps(data).encode(), {"Content-Type": "application/json",
|
|
||||||
"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
|
|
||||||
return
|
|
||||||
if path == "/stream-download":
|
|
||||||
self.send_response(200)
|
|
||||||
self.send_header("Transfer-Encoding", "chunked")
|
|
||||||
self.send_header("Content-Type", "application/octet-stream")
|
|
||||||
self.send_header("Cache-Control", "no-store")
|
|
||||||
self.end_headers()
|
|
||||||
self.wfile.write(b"5\r\nfirst\r\n")
|
|
||||||
self.wfile.flush()
|
|
||||||
peer.release_stream.wait(5)
|
|
||||||
self.wfile.write(b"4\r\nlast\r\n0\r\n\r\n")
|
|
||||||
self.wfile.flush()
|
|
||||||
return
|
|
||||||
if path == "/ws":
|
|
||||||
key = self.headers.get("Sec-WebSocket-Key", "")
|
|
||||||
accept = base64.b64encode(hashlib.sha1(
|
|
||||||
(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").encode()).digest()).decode()
|
|
||||||
self.send_response(101)
|
|
||||||
self.send_header("Upgrade", "websocket")
|
|
||||||
self.send_header("Connection", "Upgrade")
|
|
||||||
self.send_header("Sec-WebSocket-Accept", accept)
|
|
||||||
self.end_headers()
|
|
||||||
self.wfile.flush()
|
|
||||||
frame = self.rfile.read(2)
|
|
||||||
if len(frame) == 2:
|
|
||||||
length = frame[1] & 127
|
|
||||||
if length == 126:
|
|
||||||
length = struct.unpack("!H", self.rfile.read(2))[0]
|
|
||||||
mask = self.rfile.read(4) if frame[1] & 128 else b"\0\0\0\0"
|
|
||||||
data = self.rfile.read(length)
|
|
||||||
data = bytes(byte ^ mask[i % 4] for i, byte in enumerate(data))
|
|
||||||
self.wfile.write(bytes([0x81, len(data)]) + data)
|
|
||||||
self.wfile.flush()
|
|
||||||
self.close_connection = True
|
|
||||||
return
|
|
||||||
if path.startswith("/status/"):
|
|
||||||
self.emit(int(path.rsplit("/", 1)[1]), b"origin status payload\n",
|
|
||||||
{"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
|
|
||||||
return
|
|
||||||
if path == "/redirect":
|
|
||||||
self.emit(302, b"redirect body\n", {"Location": "/small.html", "Cache-Control": "no-store"})
|
|
||||||
return
|
|
||||||
if path in ("/private", "/cookie", "/no-store", "/vary", "/fresh", "/revalidate", "/concurrent"):
|
|
||||||
cache_control = "public, max-age=60"
|
|
||||||
headers = {"Content-Type": "text/plain"}
|
|
||||||
if body:
|
|
||||||
headers["X-Received-Body-Bytes"] = str(len(body))
|
|
||||||
if path == "/private":
|
|
||||||
cache_control = "private, max-age=60"
|
|
||||||
if path == "/no-store":
|
|
||||||
cache_control = "no-store"
|
|
||||||
if path == "/cookie":
|
|
||||||
headers["Set-Cookie"] = "application-session=fixture; Path=/"
|
|
||||||
if path == "/vary":
|
|
||||||
headers["Vary"] = "Accept-Language"
|
|
||||||
if path == "/concurrent":
|
|
||||||
time.sleep(.08)
|
|
||||||
if path == "/revalidate":
|
|
||||||
cache_control = "public, max-age=1, stale-if-error=10"
|
|
||||||
headers["ETag"] = '"fixture-v1"'
|
|
||||||
if self.headers.get("If-None-Match") == headers["ETag"]:
|
|
||||||
self.emit(304, b"", {**headers, "Cache-Control": cache_control})
|
|
||||||
return
|
|
||||||
data = f"{path}:count={count}:lang={self.headers.get('Accept-Language', '')}".encode()
|
|
||||||
if path == "/revalidate":
|
|
||||||
data = b"stable revalidated object"
|
|
||||||
self.emit(200, data, {**headers, "Cache-Control": cache_control})
|
|
||||||
return
|
|
||||||
if path == "/no-transform":
|
|
||||||
self.emit(200, b"unchanged compression fixture " * 1024,
|
|
||||||
{"Content-Type": "text/plain", "Cache-Control": "no-transform, public, max-age=60"})
|
|
||||||
return
|
|
||||||
file = peer.site / path.lstrip("/")
|
|
||||||
if not file.is_file() or ".." in file.parts:
|
|
||||||
self.emit(404, b"fixture not found")
|
|
||||||
return
|
|
||||||
payload = file.read_bytes()
|
|
||||||
tag = '"' + hashlib.sha256(payload).hexdigest() + '"'
|
|
||||||
headers = {"Cache-Control": "public, max-age=60", "ETag": tag,
|
|
||||||
"Accept-Ranges": "bytes", "X-Origin-Fixture": "yes"}
|
|
||||||
headers["Content-Type"] = {".html": "text/html", ".txt": "text/plain",
|
|
||||||
".css": "text/css", ".json": "application/json"}.get(file.suffix, "application/octet-stream")
|
|
||||||
if self.headers.get("If-None-Match") == tag:
|
|
||||||
self.emit(304, b"", headers)
|
|
||||||
return
|
|
||||||
range_value = self.headers.get("Range")
|
|
||||||
if range_value and self.headers.get("If-Range", tag) == tag:
|
|
||||||
try:
|
|
||||||
first, last = range_value.removeprefix("bytes=").split("-", 1)
|
|
||||||
start = int(first) if first else max(0, len(payload) - int(last))
|
|
||||||
end = min(int(last), len(payload) - 1) if first and last else len(payload) - 1
|
|
||||||
if start >= len(payload) or end < start:
|
|
||||||
self.emit(416, b"", {**headers, "Content-Range": f"bytes */{len(payload)}"})
|
|
||||||
else:
|
|
||||||
self.emit(206, payload[start:end + 1], {**headers,
|
|
||||||
"Content-Range": f"bytes {start}-{end}/{len(payload)}"})
|
|
||||||
return
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
self.emit(200, payload, headers)
|
|
||||||
|
|
||||||
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
|
|
||||||
self.server.daemon_threads = True
|
|
||||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
|
||||||
self.thread.start()
|
|
||||||
|
|
||||||
def count(self, method, path):
|
|
||||||
with self.lock:
|
|
||||||
return self.counts[(method, path)]
|
|
||||||
|
|
||||||
def close(self):
|
|
||||||
self.release_stream.set()
|
|
||||||
self.server.shutdown()
|
|
||||||
self.server.server_close()
|
|
||||||
self.thread.join(timeout=2)
|
|
||||||
@@ -1,284 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Profile selected server syscalls separately from throughput measurements.
|
|
||||||
|
|
||||||
Runs each RustProxy binary sequentially against a disposable loopback fixture.
|
|
||||||
The measured operation is one persistent HTTP/1.1 connection requesting the
|
|
||||||
64-KiB static file with cache disabled and Accept-Encoding: identity.
|
|
||||||
Every response is validated; syscall counts include background server work.
|
|
||||||
strace elapsed/CPU times must not be interpreted as uninstrumented performance.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import hashlib
|
|
||||||
from http.client import HTTPConnection
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import platform
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import signal
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, seal_results, sha256, write_json
|
|
||||||
from resources import descendants
|
|
||||||
|
|
||||||
|
|
||||||
SYSCALLS = ("read", "pread64", "write", "writev", "openat", "close", "fcntl", "lseek", "futex")
|
|
||||||
REQUEST_PATH = "/large.bin"
|
|
||||||
|
|
||||||
|
|
||||||
def thread_tracers(pid: int) -> dict[int, int]:
|
|
||||||
result = {}
|
|
||||||
for task in Path(f"/proc/{pid}/task").iterdir():
|
|
||||||
try:
|
|
||||||
status = (task / "status").read_text()
|
|
||||||
except FileNotFoundError:
|
|
||||||
continue # A thread may exit between the directory read and status.
|
|
||||||
match = re.search(r"^TracerPid:\s+(\d+)$", status, re.MULTILINE)
|
|
||||||
if match:
|
|
||||||
result[int(task.name)] = int(match.group(1))
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def wait_attached(tracer, server, timeout=10):
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
if tracer.poll() is not None:
|
|
||||||
raise RuntimeError("strace exited before attachment; see strace diagnostic log (ptrace may be unavailable)")
|
|
||||||
if server.poll() is not None:
|
|
||||||
raise RuntimeError("server exited during strace attachment")
|
|
||||||
traced = thread_tracers(server.pid)
|
|
||||||
if traced and all(traced.values()):
|
|
||||||
return sorted(traced)
|
|
||||||
time.sleep(.01)
|
|
||||||
raise RuntimeError("strace did not attach to every server thread within 10 seconds")
|
|
||||||
|
|
||||||
|
|
||||||
def detach(tracer, server):
|
|
||||||
# Do not signal the process group: only the independently started tracer.
|
|
||||||
if tracer.poll() is None:
|
|
||||||
tracer.send_signal(signal.SIGINT)
|
|
||||||
try:
|
|
||||||
tracer.wait(timeout=10)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
tracer.terminate()
|
|
||||||
try:
|
|
||||||
tracer.wait(timeout=3)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
tracer.kill()
|
|
||||||
tracer.wait(timeout=3)
|
|
||||||
raise RuntimeError("strace failed to finish on SIGINT; profile is incomplete")
|
|
||||||
if server.poll() is not None:
|
|
||||||
raise RuntimeError("server exited while the tracer was detached")
|
|
||||||
traced = thread_tracers(server.pid)
|
|
||||||
if not traced or any(traced.values()):
|
|
||||||
raise RuntimeError("server still has traced threads after strace exited")
|
|
||||||
return {"signal": "SIGINT to strace PID only", "strace_returncode": tracer.returncode,
|
|
||||||
"server_alive": True, "remaining_traced_threads": 0}
|
|
||||||
|
|
||||||
|
|
||||||
def parse_summary(path: Path, requests: int):
|
|
||||||
rows = {name: {"calls": 0, "errors": 0, "calls_per_request": 0.0}
|
|
||||||
for name in SYSCALLS}
|
|
||||||
total = None
|
|
||||||
for line in path.read_text().splitlines():
|
|
||||||
fields = line.split()
|
|
||||||
if len(fields) not in (5, 6) or fields[-1] not in (*SYSCALLS, "total"):
|
|
||||||
continue
|
|
||||||
percent, seconds, usecs, calls = map(float, fields[:4])
|
|
||||||
errors = int(fields[4]) if len(fields) == 6 else 0
|
|
||||||
row = {"calls": int(calls), "errors": errors,
|
|
||||||
"calls_per_request": int(calls) / requests,
|
|
||||||
"strace_system_seconds": seconds, "strace_percent_time": percent,
|
|
||||||
"strace_us_per_call": usecs}
|
|
||||||
if fields[-1] == "total":
|
|
||||||
total = row
|
|
||||||
else:
|
|
||||||
rows[fields[-1]] = row
|
|
||||||
if total is None:
|
|
||||||
raise RuntimeError(f"no complete strace summary found in {path}")
|
|
||||||
if sum(row["calls"] for row in rows.values()) != total["calls"]:
|
|
||||||
raise RuntimeError("parsed syscall counts do not match the raw strace total")
|
|
||||||
return {"selected_syscalls": rows, "selected_total": total}
|
|
||||||
|
|
||||||
|
|
||||||
def request_checked(connection, expected_socket, expected_sha, phase, sequence, evidence):
|
|
||||||
if connection.sock is not expected_socket:
|
|
||||||
raise RuntimeError("keepalive connection was replaced before the next request")
|
|
||||||
connection.request("GET", REQUEST_PATH,
|
|
||||||
headers={"Accept-Encoding": "identity", "Connection": "keep-alive"})
|
|
||||||
response = connection.getresponse()
|
|
||||||
payload = response.read()
|
|
||||||
digest = hashlib.sha256(payload).hexdigest()
|
|
||||||
headers = {key.lower(): value for key, value in response.getheaders()}
|
|
||||||
valid = (response.status == 200 and len(payload) == 65536 and digest == expected_sha
|
|
||||||
and headers.get("content-length") == "65536"
|
|
||||||
and headers.get("content-encoding", "identity") == "identity"
|
|
||||||
and connection.sock is expected_socket and not response.will_close)
|
|
||||||
result = {"phase": phase, "sequence": sequence, "status": response.status,
|
|
||||||
"bytes": len(payload), "sha256": digest,
|
|
||||||
"content_length": headers.get("content-length"),
|
|
||||||
"content_encoding": headers.get("content-encoding"),
|
|
||||||
"connection_reused": connection.sock is expected_socket,
|
|
||||||
"will_close": response.will_close, "passed": valid}
|
|
||||||
evidence.write(json.dumps(result, sort_keys=True) + "\n")
|
|
||||||
evidence.flush()
|
|
||||||
if not valid:
|
|
||||||
raise RuntimeError(f"response verification failed in {phase} request {sequence}: {result}")
|
|
||||||
|
|
||||||
|
|
||||||
def profile_variant(fixture, label, binary, args):
|
|
||||||
rust = fixture.rust(binary, label)
|
|
||||||
rust.login()
|
|
||||||
rust.wait_quiescent()
|
|
||||||
raw = args.out / "raw"
|
|
||||||
raw.mkdir(exist_ok=True)
|
|
||||||
summary_path = raw / f"{label}.strace.txt"
|
|
||||||
diagnostic_path = raw / f"{label}.strace.log"
|
|
||||||
responses_path = raw / f"{label}.responses.jsonl"
|
|
||||||
expected_sha = sha256(fixture.site / "large.bin")
|
|
||||||
quiescence_path = fixture.results / f"{label}-quiescence.json"
|
|
||||||
config_path = fixture.results / "configs" / f"{label}-base.json"
|
|
||||||
config_sha = sha256(config_path)
|
|
||||||
if config_sha != rust.applies[-1]["sha256"]:
|
|
||||||
raise RuntimeError("saved fixture configuration differs from the applied configuration")
|
|
||||||
command = [str(args.strace), "-f", "-c", "-e", "trace=" + ",".join(SYSCALLS),
|
|
||||||
"-p", str(rust.process.pid), "-o", str(summary_path)]
|
|
||||||
connection = HTTPConnection("127.0.0.1", rust.static_port, timeout=args.timeout)
|
|
||||||
tracer = None
|
|
||||||
result = {"variant": label, "binary": str(binary), "binary_sha256": sha256(binary),
|
|
||||||
"config": str(config_path.relative_to(args.out)), "config_sha256": config_sha,
|
|
||||||
"quiescence": str(quiescence_path.relative_to(args.out)),
|
|
||||||
"quiescence_sha256": sha256(quiescence_path),
|
|
||||||
"pid": rust.process.pid, "request_path": REQUEST_PATH, "payload_bytes": 65536,
|
|
||||||
"payload_sha256": expected_sha, "cache_enabled": False,
|
|
||||||
"request_encoding": "identity", "concurrency": 1,
|
|
||||||
"warmup_requests": args.warmup, "profile_requests": args.requests,
|
|
||||||
"strace_command": command, "raw_summary": str(summary_path.relative_to(args.out)),
|
|
||||||
"raw_diagnostics": str(diagnostic_path.relative_to(args.out)),
|
|
||||||
"responses": str(responses_path.relative_to(args.out)), "passed": False}
|
|
||||||
try:
|
|
||||||
with responses_path.open("w") as responses, diagnostic_path.open("w") as diagnostics:
|
|
||||||
connection.connect()
|
|
||||||
expected_socket = connection.sock
|
|
||||||
result["client_local_address"] = expected_socket.getsockname()
|
|
||||||
for index in range(args.warmup):
|
|
||||||
request_checked(connection, expected_socket, expected_sha, "warmup", index + 1, responses)
|
|
||||||
result["processes_before_attach"] = descendants(rust.process.pid)
|
|
||||||
if result["processes_before_attach"] != [rust.process.pid]:
|
|
||||||
raise RuntimeError("server spawned a child after quiescence; refusing contaminated syscall trace")
|
|
||||||
environment = os.environ.copy()
|
|
||||||
environment["LC_ALL"] = "C"
|
|
||||||
tracer = subprocess.Popen(command, stdout=diagnostics, stderr=diagnostics,
|
|
||||||
env=environment, start_new_session=True)
|
|
||||||
result["initial_traced_threads"] = wait_attached(tracer, rust.process)
|
|
||||||
started = time.monotonic()
|
|
||||||
try:
|
|
||||||
for index in range(args.requests):
|
|
||||||
request_checked(connection, expected_socket, expected_sha, "profile", index + 1, responses)
|
|
||||||
if tracer.poll() is not None:
|
|
||||||
raise RuntimeError("strace exited before all profile requests completed")
|
|
||||||
finally:
|
|
||||||
result["instrumented_request_loop_seconds"] = time.monotonic() - started
|
|
||||||
result["detach"] = detach(tracer, rust.process)
|
|
||||||
tracer = None
|
|
||||||
# An actual HTTP response after detaching proves the server stayed usable.
|
|
||||||
request_checked(connection, expected_socket, expected_sha, "after_detach", 1, responses)
|
|
||||||
result["processes_after_detach"] = descendants(rust.process.pid)
|
|
||||||
if result["processes_after_detach"] != [rust.process.pid]:
|
|
||||||
raise RuntimeError("server has a child after profiling; syscall trace needs review")
|
|
||||||
result.update(parse_summary(summary_path, args.requests))
|
|
||||||
result["verified_responses"] = args.requests + args.warmup + 1
|
|
||||||
result["passed"] = True
|
|
||||||
return result
|
|
||||||
finally:
|
|
||||||
if tracer is not None:
|
|
||||||
detach(tracer, rust.process)
|
|
||||||
connection.close()
|
|
||||||
fixture.stop(rust.process)
|
|
||||||
write_json(args.out / f"{label}.json", result)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--nginx", type=Path, required=True, help="nginx executable for the shared fixture origin")
|
|
||||||
parser.add_argument("--strace", type=Path, default=shutil.which("strace"))
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
parser.add_argument("--requests", type=int, default=500)
|
|
||||||
parser.add_argument("--warmup", type=int, default=20)
|
|
||||||
parser.add_argument("--workers", type=int, default=4)
|
|
||||||
parser.add_argument("--origin-workers", type=int, default=4)
|
|
||||||
parser.add_argument("--server-cpus")
|
|
||||||
parser.add_argument("--origin-cpus")
|
|
||||||
parser.add_argument("--timeout", type=float, default=20)
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.requests < 1 or args.warmup < 0 or args.workers < 1 or args.origin_workers < 1 or args.timeout <= 0:
|
|
||||||
parser.error("requests/workers/timeout must be positive, warmup must be nonnegative")
|
|
||||||
if args.strace is None:
|
|
||||||
parser.error("strace is required; pass --strace /absolute/path")
|
|
||||||
if args.out.exists() and (not args.out.is_dir() or any(args.out.iterdir())):
|
|
||||||
parser.error("--out must be a new or empty directory; existing evidence is never overwritten")
|
|
||||||
variants = {}
|
|
||||||
for value in args.binary:
|
|
||||||
label, separator, filename = value.partition("=")
|
|
||||||
if not separator or not re.fullmatch(r"[A-Za-z][A-Za-z0-9_-]{0,48}", label) or label in variants:
|
|
||||||
parser.error("each --binary must use a unique filesystem-safe LABEL=/ABS/PATH")
|
|
||||||
binary = Path(filename).resolve()
|
|
||||||
if not binary.is_file():
|
|
||||||
parser.error(f"binary does not exist: {binary}")
|
|
||||||
variants[label] = binary
|
|
||||||
args.out, args.nginx, args.strace = args.out.resolve(), args.nginx.resolve(), args.strace.resolve()
|
|
||||||
for executable in (args.nginx, args.strace):
|
|
||||||
if not executable.is_file() or not os.access(executable, os.X_OK):
|
|
||||||
parser.error(f"executable is missing or is not executable: {executable}")
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
version = subprocess.run([str(args.strace), "-V"], capture_output=True, text=True, timeout=10)
|
|
||||||
environment = {"kind": "selected syscall counts with ptrace instrumentation", "simulation": False,
|
|
||||||
"platform": platform.platform(), "python": sys.version, "strace_version": version.stdout + version.stderr,
|
|
||||||
"strace_sha256": sha256(args.strace), "nginx_sha256": sha256(args.nginx),
|
|
||||||
"variant_order": list(variants), "selected_syscalls": SYSCALLS,
|
|
||||||
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
|
||||||
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
|
|
||||||
"limitations": ["strace changes scheduling and latency; do not use instrumented times as RPS evidence",
|
|
||||||
"Counts cover selected syscalls in every traced server thread, including background work",
|
|
||||||
"Normalized counts divide the whole trace by verified profile requests; they do not isolate per-request causality",
|
|
||||||
"This is a warm-file-cache, c1, HTTP/1.1 plaintext static-file scenario; no general capacity inference",
|
|
||||||
"Syscall counts do not directly count Tokio task dispatches or memory allocations"]}
|
|
||||||
for path in ("/proc/sys/kernel/yama/ptrace_scope", "/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max"):
|
|
||||||
try:
|
|
||||||
environment[path] = Path(path).read_text().strip()
|
|
||||||
except OSError:
|
|
||||||
environment[path] = None
|
|
||||||
write_json(args.out / "environment.json", environment)
|
|
||||||
rows, error = [], None
|
|
||||||
try:
|
|
||||||
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
|
|
||||||
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
|
|
||||||
fixture.nginx(args.nginx, frontend=False)
|
|
||||||
for label, binary in variants.items():
|
|
||||||
result = profile_variant(fixture, label, binary, args)
|
|
||||||
rows.append(result)
|
|
||||||
write_json(args.out / "profiles.json", rows)
|
|
||||||
print(json.dumps({"variant": label, "requests": args.requests, "passed": result["passed"],
|
|
||||||
"selected_syscall_calls": result["selected_total"]["calls"]}), flush=True)
|
|
||||||
except Exception as exc:
|
|
||||||
error = {"error": str(exc), "traceback": traceback.format_exc()}
|
|
||||||
write_json(args.out / "failure.json", error)
|
|
||||||
print(json.dumps(error), flush=True)
|
|
||||||
finally:
|
|
||||||
write_json(args.out / "profiles.json", rows)
|
|
||||||
write_json(args.out / "completion.json", {"completed": error is None, "profiles": len(rows),
|
|
||||||
"simulation": False, "throughput_measurement": False,
|
|
||||||
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
|
|
||||||
seal_results(args.out)
|
|
||||||
return 1 if error else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Run black-box acceptance tests against one or more real RustProxy binaries.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
python3 scripts/performance/regression.py --binary before=/abs/before \
|
|
||||||
--binary after=/abs/after --out /abs/results/regression
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import platform
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, seal_results, write_json
|
|
||||||
from peer import Peer
|
|
||||||
import checks_cache
|
|
||||||
import checks_policy
|
|
||||||
import checks_proxy
|
|
||||||
import checks_static
|
|
||||||
|
|
||||||
|
|
||||||
GROUPS = {"static": checks_static.run, "proxy": checks_proxy.run,
|
|
||||||
"policy": checks_policy.run, "cache": checks_cache.run,
|
|
||||||
"transform": checks_cache.transforms}
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--out", required=True, type=Path)
|
|
||||||
parser.add_argument("--workers", type=int, default=4)
|
|
||||||
parser.add_argument("--groups", default=",".join(GROUPS))
|
|
||||||
args = parser.parse_args()
|
|
||||||
groups = args.groups.split(",")
|
|
||||||
if any(group not in GROUPS for group in groups):
|
|
||||||
parser.error(f"groups must be selected from {','.join(GROUPS)}")
|
|
||||||
args.out = args.out.resolve()
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
metadata = {"kind": "live RustProxy black-box HTTP acceptance", "simulation": False,
|
|
||||||
"python": sys.version, "platform": platform.platform(),
|
|
||||||
"cpu_affinity": sorted(os.sched_getaffinity(0)) if hasattr(os, "sched_getaffinity") else None,
|
|
||||||
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
|
||||||
"groups": groups,
|
|
||||||
"not_covered": ["TLS/ALPN", "HTTP/2", "HTTP/3", "PHP-FPM",
|
|
||||||
"Tinyauth verifier", "real DNS rebinding", "24-hour soak",
|
|
||||||
"concurrent filesystem rename-race proof"]}
|
|
||||||
write_json(args.out / "environment.json", metadata)
|
|
||||||
reports = {}
|
|
||||||
for binary_arg in args.binary:
|
|
||||||
label, separator, name = binary_arg.partition("=")
|
|
||||||
if not separator or not label.replace("-", "").replace("_", "").isalnum():
|
|
||||||
parser.error("--binary must be LABEL=/ABS/PATH with a simple unique label")
|
|
||||||
if label in reports:
|
|
||||||
parser.error("binary labels must be unique")
|
|
||||||
cases = []
|
|
||||||
started = time.monotonic()
|
|
||||||
output = args.out / label
|
|
||||||
|
|
||||||
def check(case, passed, **details):
|
|
||||||
row = {"case": case, "passed": passed, **details}
|
|
||||||
cases.append(row)
|
|
||||||
print(f"{label} {'SKIP' if passed is None else 'PASS' if passed else 'FAIL'} {case}", flush=True)
|
|
||||||
write_json(output / "checks.json", cases)
|
|
||||||
|
|
||||||
try:
|
|
||||||
with Fixture(output, workers=args.workers) as fixture:
|
|
||||||
peer = Peer(fixture.origin_port, fixture.site)
|
|
||||||
try:
|
|
||||||
rust = fixture.rust(Path(name), label)
|
|
||||||
check("runtime.started", True)
|
|
||||||
for group in groups:
|
|
||||||
try:
|
|
||||||
rust.reset()
|
|
||||||
GROUPS[group](check, rust, peer)
|
|
||||||
except Exception as exc:
|
|
||||||
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
|
|
||||||
finally:
|
|
||||||
rust.reset()
|
|
||||||
check("runtime.still_alive", rust.process.poll() is None)
|
|
||||||
finally:
|
|
||||||
peer.close()
|
|
||||||
except Exception as exc:
|
|
||||||
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
|
|
||||||
report = {"label": label, "kind": metadata["kind"], "simulation": False,
|
|
||||||
"seconds": time.monotonic() - started, "passed": sum(c["passed"] is True for c in cases),
|
|
||||||
"failed": sum(c["passed"] is False for c in cases),
|
|
||||||
"skipped": sum(c["passed"] is None for c in cases), "cases": cases}
|
|
||||||
reports[label] = report
|
|
||||||
write_json(output / "report.json", report)
|
|
||||||
print(f"{label}: {report['passed']} passed, {report['failed']} failed, {report['skipped']} skipped", flush=True)
|
|
||||||
comparison = {"variants": {name: {key: report[key] for key in ["passed", "failed", "skipped", "seconds"]}
|
|
||||||
for name, report in reports.items()},
|
|
||||||
"cases": [{"case": case, **{name: next((entry["passed"] for entry in report["cases"]
|
|
||||||
if entry["case"] == case), "not_run") for name, report in reports.items()}}
|
|
||||||
for case in sorted({entry["case"] for report in reports.values() for entry in report["cases"]})]}
|
|
||||||
write_json(args.out / "comparison.json", comparison)
|
|
||||||
seal_results(args.out)
|
|
||||||
return 1 if any(report["failed"] for report in reports.values()) else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,228 +0,0 @@
|
|||||||
"""Linux /proc sampling outside the server process; no synthetic resource data."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
def cgroup_cpu_stat(path=Path("/sys/fs/cgroup/cpu.stat")):
|
|
||||||
values = {key: int(value) for key, value in (line.split() for line in path.read_text().splitlines())}
|
|
||||||
if not {"nr_throttled", "throttled_usec"} <= values.keys():
|
|
||||||
raise RuntimeError(f"{path} lacks cgroup v2 throttling counters")
|
|
||||||
return {"path": str(path), "observed_monotonic": time.monotonic(), "values": values}
|
|
||||||
|
|
||||||
|
|
||||||
def cgroup_cpu_delta(before, after):
|
|
||||||
delta = {key: after["values"][key] - value for key, value in before["values"].items()
|
|
||||||
if key in after["values"]}
|
|
||||||
if any(value < 0 for value in delta.values()):
|
|
||||||
raise RuntimeError("cgroup CPU counters reset during the measured run")
|
|
||||||
return delta
|
|
||||||
|
|
||||||
|
|
||||||
def parse_cpu_set(value):
|
|
||||||
if not value:
|
|
||||||
raise ValueError("an explicit, nonempty CPU set is required")
|
|
||||||
result = set()
|
|
||||||
for part in value.split(","):
|
|
||||||
bounds = part.strip().split("-")
|
|
||||||
if len(bounds) not in (1, 2) or any(not item.isascii() or not item.isdigit() for item in bounds):
|
|
||||||
raise ValueError(f"invalid CPU set {value!r}; use comma-separated numbers or ranges")
|
|
||||||
start, end = int(bounds[0]), int(bounds[-1])
|
|
||||||
if start > end or end - start > 65536:
|
|
||||||
raise ValueError(f"invalid CPU range {part!r}")
|
|
||||||
result.update(range(start, end + 1))
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def validate_cpu_sets(values, allowed):
|
|
||||||
groups = {label: parse_cpu_set(value) for label, value in values.items()}
|
|
||||||
for label, cpus in groups.items():
|
|
||||||
if not cpus <= allowed:
|
|
||||||
raise ValueError(f"{label} CPUs {sorted(cpus - allowed)} are outside the allowed affinity")
|
|
||||||
for other, other_cpus in groups.items():
|
|
||||||
if label < other and cpus & other_cpus:
|
|
||||||
raise ValueError(f"{label} and {other} CPU sets overlap: {sorted(cpus & other_cpus)}")
|
|
||||||
return groups
|
|
||||||
|
|
||||||
|
|
||||||
def affinity_snapshot(root, expected, expected_processes=None):
|
|
||||||
"""Verify every currently visible thread in the root process and children."""
|
|
||||||
tasks = []
|
|
||||||
for pid in descendants(root):
|
|
||||||
try:
|
|
||||||
threads = list(Path(f"/proc/{pid}/task").iterdir())
|
|
||||||
except FileNotFoundError:
|
|
||||||
continue
|
|
||||||
for thread in threads:
|
|
||||||
try:
|
|
||||||
actual = set(os.sched_getaffinity(int(thread.name)))
|
|
||||||
except ProcessLookupError:
|
|
||||||
continue
|
|
||||||
if actual != expected:
|
|
||||||
raise RuntimeError(f"affinity mismatch for PID {pid} TID {thread.name}: "
|
|
||||||
f"expected {sorted(expected)}, got {sorted(actual)}")
|
|
||||||
tasks.append({"pid": pid, "tid": int(thread.name), "cpus": sorted(actual)})
|
|
||||||
if not tasks or not any(task["pid"] == root for task in tasks):
|
|
||||||
raise RuntimeError(f"cannot inspect affinity of live process {root}")
|
|
||||||
processes = len({task["pid"] for task in tasks})
|
|
||||||
if expected_processes is not None and processes != expected_processes:
|
|
||||||
raise RuntimeError(f"process count mismatch for PID {root}: "
|
|
||||||
f"expected {expected_processes}, inspected {processes}")
|
|
||||||
return {"root_pid": root, "processes": processes,
|
|
||||||
"threads": len(tasks), "tasks": tasks}
|
|
||||||
|
|
||||||
|
|
||||||
def wait_for_affinity(process, expected, timeout=2):
|
|
||||||
"""Allow taskset to execute before inspecting the generator's inherited mask."""
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
while True:
|
|
||||||
if process.poll() is not None:
|
|
||||||
raise RuntimeError(f"generator exited before affinity verification: {process.returncode}")
|
|
||||||
try:
|
|
||||||
return affinity_snapshot(process.pid, expected)
|
|
||||||
except RuntimeError:
|
|
||||||
if time.monotonic() >= deadline:
|
|
||||||
raise
|
|
||||||
time.sleep(.01)
|
|
||||||
|
|
||||||
|
|
||||||
def descendants(pid, proc_root=Path("/proc")):
|
|
||||||
"""Find the process tree through stat PPids, including nginx workers.
|
|
||||||
|
|
||||||
/proc/PID/task/PID/children is optional on some Linux kernels and may also
|
|
||||||
omit children created by another thread. Missing that file is not evidence
|
|
||||||
that a live process is a leaf. A single PPid-map scan avoids either case.
|
|
||||||
"""
|
|
||||||
children, seen = {}, set()
|
|
||||||
for entry in proc_root.iterdir():
|
|
||||||
if not entry.name.isdecimal():
|
|
||||||
continue
|
|
||||||
current = int(entry.name)
|
|
||||||
try:
|
|
||||||
fields = (entry / "stat").read_text().rsplit(")", 1)[1].split()
|
|
||||||
parent = int(fields[1])
|
|
||||||
except (FileNotFoundError, ProcessLookupError):
|
|
||||||
# A non-root process may finish while the snapshot is assembled.
|
|
||||||
continue
|
|
||||||
except (OSError, ValueError, IndexError) as error:
|
|
||||||
raise RuntimeError(f"cannot discover process tree from PID {current}: {error}") from error
|
|
||||||
if fields[0] in {"Z", "X", "x"}:
|
|
||||||
# A dead/zombie worker cannot satisfy the configured live count.
|
|
||||||
continue
|
|
||||||
seen.add(current)
|
|
||||||
children.setdefault(parent, []).append(current)
|
|
||||||
if pid not in seen:
|
|
||||||
raise RuntimeError(f"no readable live root process for PID {pid}")
|
|
||||||
result, pending = [], [pid]
|
|
||||||
while pending:
|
|
||||||
current = pending.pop()
|
|
||||||
if current in result:
|
|
||||||
continue
|
|
||||||
result.append(current)
|
|
||||||
pending.extend(sorted(children.get(current, []), reverse=True))
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def process_stats(pid):
|
|
||||||
try:
|
|
||||||
data = Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()
|
|
||||||
if data[0] in {"Z", "X", "x"}:
|
|
||||||
return None
|
|
||||||
ticks = os.sysconf("SC_CLK_TCK")
|
|
||||||
pages = os.sysconf("SC_PAGE_SIZE")
|
|
||||||
return {"pid": pid, "start_ticks": int(data[19]),
|
|
||||||
"cpu_seconds": (int(data[11]) + int(data[12])) / ticks,
|
|
||||||
"rss_bytes": int(data[21]) * pages,
|
|
||||||
"fd": len(list(Path(f"/proc/{pid}/fd").iterdir())),
|
|
||||||
"threads": int(data[17])}
|
|
||||||
except (FileNotFoundError, ProcessLookupError):
|
|
||||||
return None
|
|
||||||
except (OSError, ValueError, IndexError) as error:
|
|
||||||
raise RuntimeError(f"cannot read resource counters for PID {pid}: {error}") from error
|
|
||||||
|
|
||||||
|
|
||||||
class Monitor:
|
|
||||||
def __init__(self, roots, interval=.5, expected_processes=None):
|
|
||||||
self.roots = roots
|
|
||||||
self.expected_processes = expected_processes or {}
|
|
||||||
self.process_identities = {}
|
|
||||||
self.interval = interval
|
|
||||||
if interval <= 0:
|
|
||||||
raise ValueError("resource interval must be positive")
|
|
||||||
self.samples = []
|
|
||||||
self.errors = []
|
|
||||||
self.stop_event = threading.Event()
|
|
||||||
self.started = time.monotonic()
|
|
||||||
self.thread = threading.Thread(target=self.run, daemon=True)
|
|
||||||
|
|
||||||
def sample(self):
|
|
||||||
groups = {}
|
|
||||||
for label, root in self.roots.items():
|
|
||||||
processes = [stats for pid in descendants(root) if (stats := process_stats(pid)) is not None]
|
|
||||||
if not any(process["pid"] == root for process in processes):
|
|
||||||
raise RuntimeError(f"no readable live root process for {label} (PID {root})")
|
|
||||||
expected = self.expected_processes.get(label)
|
|
||||||
if expected is not None and len(processes) != expected:
|
|
||||||
raise RuntimeError(f"resource process count mismatch for {label} (PID {root}): "
|
|
||||||
f"expected {expected}, read {len(processes)}")
|
|
||||||
identities = {(process["pid"], process["start_ticks"]) for process in processes}
|
|
||||||
if expected is not None:
|
|
||||||
original = self.process_identities.setdefault(label, identities)
|
|
||||||
if identities != original:
|
|
||||||
raise RuntimeError(f"process identities changed while measuring {label}: "
|
|
||||||
f"expected {sorted(original)}, read {sorted(identities)}")
|
|
||||||
groups[label] = {"processes": processes,
|
|
||||||
"rss_bytes": sum(p["rss_bytes"] for p in processes),
|
|
||||||
"fd": sum(p["fd"] for p in processes),
|
|
||||||
"threads": sum(p["threads"] for p in processes)}
|
|
||||||
self.samples.append({"seconds": time.monotonic() - self.started, "groups": groups})
|
|
||||||
|
|
||||||
def run(self):
|
|
||||||
while not self.stop_event.wait(self.interval):
|
|
||||||
try:
|
|
||||||
self.sample()
|
|
||||||
except Exception as error:
|
|
||||||
self.errors.append(str(error))
|
|
||||||
self.stop_event.set()
|
|
||||||
|
|
||||||
def start(self):
|
|
||||||
self.sample()
|
|
||||||
self.thread.start()
|
|
||||||
|
|
||||||
def stop(self):
|
|
||||||
self.stop_event.set()
|
|
||||||
self.thread.join(timeout=3)
|
|
||||||
if self.thread.is_alive():
|
|
||||||
self.errors.append("resource sampler did not stop")
|
|
||||||
try:
|
|
||||||
self.sample()
|
|
||||||
except Exception as error:
|
|
||||||
self.errors.append(str(error))
|
|
||||||
|
|
||||||
def summary(self, requests):
|
|
||||||
if self.errors:
|
|
||||||
raise RuntimeError("resource sampling failed: " + "; ".join(self.errors))
|
|
||||||
result = {}
|
|
||||||
for label in self.roots:
|
|
||||||
observations = [sample["groups"][label] for sample in self.samples]
|
|
||||||
totals = {}
|
|
||||||
for group in observations:
|
|
||||||
for process in group["processes"]:
|
|
||||||
key = (process["pid"], process["start_ticks"])
|
|
||||||
low, high = totals.get(key, (process["cpu_seconds"], process["cpu_seconds"]))
|
|
||||||
totals[key] = (min(low, process["cpu_seconds"]), max(high, process["cpu_seconds"]))
|
|
||||||
if not totals:
|
|
||||||
raise RuntimeError(f"no readable process samples for {label}; refusing zero resource metrics")
|
|
||||||
seconds = sum(high - low for low, high in totals.values())
|
|
||||||
result[label] = {"cpu_seconds": seconds,
|
|
||||||
"cpu_us_per_request": seconds * 1_000_000 / requests if requests else None,
|
|
||||||
"rss_peak_bytes": max(group["rss_bytes"] for group in observations),
|
|
||||||
"rss_start_bytes": observations[0]["rss_bytes"],
|
|
||||||
"rss_end_bytes": observations[-1]["rss_bytes"],
|
|
||||||
"fd_peak": max(group["fd"] for group in observations),
|
|
||||||
"fd_start": observations[0]["fd"], "fd_end": observations[-1]["fd"],
|
|
||||||
"processes_seen": len(totals)}
|
|
||||||
return result
|
|
||||||
@@ -1,112 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Deterministic analytical/synthetic cache-retention scenario, not a RustProxy load test.
|
|
||||||
|
|
||||||
Model: 20 vhosts with 100 HOT + 100 COLD uniformly requested objects each.
|
|
||||||
A new configuration is saved 1,000 times. Eighty percent of saves edit only
|
|
||||||
keepalive/LB operational controls, and 20% edit content policy on one vhost.
|
|
||||||
Every save is followed by 500 uniform GETs; every object is public/cacheable and
|
|
||||||
has infinite TTL. A miss immediately inserts the object in its original tier.
|
|
||||||
Capacity is assumed adequate and there is no origin latency or concurrent IO.
|
|
||||||
The baseline flushes both tiers globally on every save, as in 0.33.8.
|
|
||||||
The candidate changes only the affected vhost after content edits.
|
|
||||||
"""
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import random
|
|
||||||
|
|
||||||
|
|
||||||
def simulate(vhosts=20, objects_per_tier=100, saves=1000, requests_per_save=500, seed=3400):
|
|
||||||
if min(vhosts, objects_per_tier, saves, requests_per_save) < 1:
|
|
||||||
raise ValueError("all counts must be positive")
|
|
||||||
rng = random.Random(seed)
|
|
||||||
objects = vhosts * objects_per_tier * 2
|
|
||||||
old = set(range(objects))
|
|
||||||
new = set(range(objects))
|
|
||||||
old_hits = new_hits = old_misses = new_misses = 0
|
|
||||||
old_evicted = new_evicted = 0
|
|
||||||
edits_operational = edits_content = 0
|
|
||||||
for step in range(saves):
|
|
||||||
changed_vhost = rng.randrange(vhosts)
|
|
||||||
content_edit = step % 5 == 0
|
|
||||||
if content_edit:
|
|
||||||
edits_content += 1
|
|
||||||
else:
|
|
||||||
edits_operational += 1
|
|
||||||
old_evicted += len(old)
|
|
||||||
old.clear()
|
|
||||||
if content_edit:
|
|
||||||
low = changed_vhost * objects_per_tier * 2
|
|
||||||
high = low + objects_per_tier * 2
|
|
||||||
removed = set(range(low, high)) & new
|
|
||||||
new_evicted += len(removed)
|
|
||||||
new.difference_update(removed)
|
|
||||||
for _ in range(requests_per_save):
|
|
||||||
object_id = rng.randrange(objects)
|
|
||||||
if object_id in old:
|
|
||||||
old_hits += 1
|
|
||||||
else:
|
|
||||||
old_misses += 1
|
|
||||||
old.add(object_id)
|
|
||||||
if object_id in new:
|
|
||||||
new_hits += 1
|
|
||||||
else:
|
|
||||||
new_misses += 1
|
|
||||||
new.add(object_id)
|
|
||||||
total = saves * requests_per_save
|
|
||||||
assert old_hits + old_misses == total
|
|
||||||
assert new_hits + new_misses == total
|
|
||||||
assert new_evicted <= old_evicted
|
|
||||||
assert new_misses <= old_misses
|
|
||||||
return {
|
|
||||||
"scenario": {
|
|
||||||
"vhosts": vhosts,
|
|
||||||
"hot_objects_per_vhost": objects_per_tier,
|
|
||||||
"cold_objects_per_vhost": objects_per_tier,
|
|
||||||
"saves": saves,
|
|
||||||
"operational_saves": edits_operational,
|
|
||||||
"single_vhost_content_saves": edits_content,
|
|
||||||
"gets_per_save": requests_per_save,
|
|
||||||
"total_gets": total,
|
|
||||||
"seed": seed,
|
|
||||||
"assumptions": "All objects public/cacheable, infinite TTL, instantly refilled, uniform GETs; no bandwidth/CPU/origin timings",
|
|
||||||
},
|
|
||||||
"legacy_global_purge": {
|
|
||||||
"invalidated_objects": old_evicted,
|
|
||||||
"misses": old_misses,
|
|
||||||
"hits": old_hits,
|
|
||||||
"hit_ratio_percent": round(100 * old_hits / total, 3),
|
|
||||||
},
|
|
||||||
"scoped_cache": {
|
|
||||||
"invalidated_objects": new_evicted,
|
|
||||||
"misses": new_misses,
|
|
||||||
"hits": new_hits,
|
|
||||||
"hit_ratio_percent": round(100 * new_hits / total, 3),
|
|
||||||
},
|
|
||||||
"improvement": {
|
|
||||||
"fewer_invalidations_percent": round(100 * (1 - new_evicted / old_evicted), 3) if old_evicted else 0,
|
|
||||||
"fewer_cache_misses_percent": round(100 * (1 - new_misses / old_misses), 3) if old_misses else 0,
|
|
||||||
"hit_ratio_gain_percentage_points": round(100 * (new_hits - old_hits) / total, 3),
|
|
||||||
"observed_rps_gain_percent": None,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--vhosts", type=int, default=20)
|
|
||||||
parser.add_argument("--objects-per-tier", type=int, default=100)
|
|
||||||
parser.add_argument("--saves", type=int, default=1000)
|
|
||||||
parser.add_argument("--gets-per-save", type=int, default=500)
|
|
||||||
parser.add_argument("--seed", type=int, default=3400)
|
|
||||||
parser.add_argument("--output", help="optional JSON results file")
|
|
||||||
args = parser.parse_args()
|
|
||||||
result = simulate(args.vhosts, args.objects_per_tier, args.saves, args.gets_per_save, args.seed)
|
|
||||||
encoded = json.dumps(result, indent=2, sort_keys=True)
|
|
||||||
if args.output:
|
|
||||||
from pathlib import Path
|
|
||||||
Path(args.output).write_text(encoded + "\n")
|
|
||||||
print(encoded)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Amdahl sensitivity study only: 0.33.8 measured baseline, 0.34.0 not measured.
|
|
||||||
|
|
||||||
No synthetic RPS is described as benchmarked RustProxy throughput. Amdahl uses
|
|
||||||
RPS_new = RPS_old / (1 - eligible_cost_fraction * fraction_eliminated).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# Verified medians reproduced from the supplied 0.33.8 evidence. Cache OFF.
|
|
||||||
BASELINE = [
|
|
||||||
("reverse_proxy", 1040, 29334.12, 170610.16, False),
|
|
||||||
("reverse_proxy", 65536, 20735.07, 35393.29, False),
|
|
||||||
("reverse_proxy", 1048576, 4461.19, 2730.23, False),
|
|
||||||
("www", 1040, 24906.20, 351605.13, False),
|
|
||||||
("www", 65536, 20856.67, 203709.91, True),
|
|
||||||
("www", 1048576, 2833.71, 12611.49, True),
|
|
||||||
]
|
|
||||||
|
|
||||||
# WWW scenarios: share of per-request service time due to file copy/read;
|
|
||||||
# assume sendfile removes HALF of that part, not the full service time.
|
|
||||||
WWW_SHARES = {"low": 0.20, "middle": 0.40, "high": 0.60}
|
|
||||||
COPY_COST_REMOVED = 0.50
|
|
||||||
# Proxy independent optimization: hypothetical reduction in *total* request
|
|
||||||
# service time, modeling removal of redundant response-frame mutex, nothing else.
|
|
||||||
PROXY_TIME_SAVED = {"low": 0.01, "middle": 0.03, "high": 0.05}
|
|
||||||
|
|
||||||
|
|
||||||
def run() -> dict:
|
|
||||||
items = []
|
|
||||||
for kind, payload_bytes, rust_rps, nginx_rps, eligible in BASELINE:
|
|
||||||
baseline_ratio = rust_rps / nginx_rps
|
|
||||||
row = {
|
|
||||||
"kind": kind,
|
|
||||||
"bytes": payload_bytes,
|
|
||||||
"baseline_version": "0.33.8",
|
|
||||||
"baseline_rust_rps": rust_rps,
|
|
||||||
"baseline_nginx_rps": nginx_rps,
|
|
||||||
"baseline_of_nginx_pct": round(100 * baseline_ratio, 2),
|
|
||||||
"factor_to_match_nginx": round(nginx_rps / rust_rps, 2),
|
|
||||||
"sendfile_eligible_only_if_plain_linux_http1_1_no_cache_no_compression": eligible,
|
|
||||||
"scenarios": {},
|
|
||||||
}
|
|
||||||
for level in ("low", "middle", "high"):
|
|
||||||
if kind == "www" and eligible:
|
|
||||||
saved_fraction = WWW_SHARES[level] * COPY_COST_REMOVED
|
|
||||||
elif kind == "reverse_proxy":
|
|
||||||
saved_fraction = PROXY_TIME_SAVED[level]
|
|
||||||
else:
|
|
||||||
saved_fraction = 0.0
|
|
||||||
simulated_rps = rust_rps / (1.0 - saved_fraction)
|
|
||||||
row["scenarios"][level] = {
|
|
||||||
"modeled_total_time_saved_pct": round(100 * saved_fraction, 2),
|
|
||||||
"simulated_rps": round(simulated_rps),
|
|
||||||
"simulated_rps_gain_pct": round(100 * (simulated_rps / rust_rps - 1), 2),
|
|
||||||
"simulated_of_nginx_pct": round(100 * simulated_rps / nginx_rps, 2),
|
|
||||||
"simulated_remaining_gap_to_nginx_pct_points": round(
|
|
||||||
100 * (1 - simulated_rps / nginx_rps), 2
|
|
||||||
),
|
|
||||||
}
|
|
||||||
items.append(row)
|
|
||||||
return {
|
|
||||||
"kind": "HYPOTHETICAL_AMDAHL_SENSITIVITY_NOT_A_BENCHMARK",
|
|
||||||
"comparison": "Existing 0.33.8 measurements (5 runs, cache off) vs measured nginx",
|
|
||||||
"not_measured": "0.34.0 server performance and sendfile syscall/runtime correctness",
|
|
||||||
"conditions": [
|
|
||||||
"Identical CPU and network assumptions to supplied 0.33.8 baseline",
|
|
||||||
"WWW zero-copy is limited to Linux, cleartext HTTP/1.1, GET >=64KiB, no cache, range, transform, compression or TLS",
|
|
||||||
"WWW low/middle/high: file-copy contributes 20%/40%/60% of total request time; modeled sendfile eliminates 50% of that component",
|
|
||||||
"Reverse proxy low/middle/high: removing response-frame mutex saves a hypothetical 1%/3%/5% of total service time",
|
|
||||||
"These fractions are chosen sensitivity parameters; not measured or inferred from profiling",
|
|
||||||
"No change is assumed for WWW 1040B or out-of-scope protocol paths",
|
|
||||||
"No network saturation, TLS record processing, queueing, origin latency, filesystem cache misses, CPU contention or concurrency effects modeled",
|
|
||||||
"No additional speedup due to cache-retention changes is assumed",
|
|
||||||
],
|
|
||||||
"formula": "RPS_sim = RPS_0.33.8 / (1 - fraction_of_total_service_time_eliminated)",
|
|
||||||
"entries": items,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--output", type=Path, help="write JSON to this file")
|
|
||||||
args = parser.parse_args()
|
|
||||||
payload = json.dumps(run(), indent=2, ensure_ascii=False) + "\n"
|
|
||||||
if args.output:
|
|
||||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
args.output.write_text(payload)
|
|
||||||
print(f"wrote {args.output}")
|
|
||||||
else:
|
|
||||||
print(payload)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,325 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Fixed-rate mixed traffic, controlled client closes and cache hot-toggle soak.
|
|
||||||
|
|
||||||
The default is a 30-minute live run. Short invocations must be explicitly marked
|
|
||||||
--exploratory and cannot satisfy the release soak gate. No run is auto-scheduled.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from collections import Counter
|
|
||||||
from concurrent.futures import ThreadPoolExecutor, wait, FIRST_COMPLETED
|
|
||||||
import copy
|
|
||||||
import hashlib
|
|
||||||
from http.client import HTTPConnection
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import platform
|
|
||||||
import statistics
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from benchmark import counter_values
|
|
||||||
from common import Fixture, http, seal_results, sha256, write_json
|
|
||||||
from resources import Monitor
|
|
||||||
from telemetry import traffic_analytics_snapshot, traffic_analytics_delta
|
|
||||||
|
|
||||||
|
|
||||||
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
|
|
||||||
|
|
||||||
|
|
||||||
def fetch(index, rust, expected, abort_every):
|
|
||||||
name = PAYLOADS[index % len(PAYLOADS)]
|
|
||||||
source = "proxy" if (index // len(PAYLOADS)) % 2 else "static"
|
|
||||||
port = rust.reverse_port if source == "proxy" else rust.static_port
|
|
||||||
started = time.monotonic()
|
|
||||||
case = {"index": index, "source": source, "file": name, "method": "GET", "intentional_abort": False}
|
|
||||||
try:
|
|
||||||
if abort_every and index % abort_every == 0:
|
|
||||||
case.update({"file": "huge.bin", "intentional_abort": True})
|
|
||||||
connection = HTTPConnection("127.0.0.1", port, timeout=10)
|
|
||||||
try:
|
|
||||||
connection.request("GET", "/huge.bin", headers={"Accept-Encoding": "identity"})
|
|
||||||
response = connection.getresponse()
|
|
||||||
body = response.read(128)
|
|
||||||
case.update({"status": response.status, "bytes": len(body),
|
|
||||||
"cache": response.getheader("X-Cache"),
|
|
||||||
"passed": response.status == 200 and body == expected["huge.bin"][:128]})
|
|
||||||
finally:
|
|
||||||
connection.close()
|
|
||||||
else:
|
|
||||||
headers = {"Accept-Encoding": "identity"}
|
|
||||||
method = "HEAD" if index % 29 == 0 else "GET"
|
|
||||||
if method == "GET" and index % 37 == 0:
|
|
||||||
headers["Range"] = "bytes=0-127"
|
|
||||||
expected_status, expected_body = 206, expected[name][:128]
|
|
||||||
else:
|
|
||||||
expected_status, expected_body = 200, b"" if method == "HEAD" else expected[name]
|
|
||||||
status, response_headers, body = http(port, "/" + name, method, headers)
|
|
||||||
case.update({"method": method, "status": status, "bytes": len(body),
|
|
||||||
"cache": response_headers.get("x-cache"),
|
|
||||||
"passed": status == expected_status and body == expected_body})
|
|
||||||
if not case["passed"]:
|
|
||||||
case.update({"expected_status": expected_status, "expected_bytes": len(expected_body),
|
|
||||||
"body_sha256": hashlib.sha256(body).hexdigest()})
|
|
||||||
except Exception as exc:
|
|
||||||
case.update({"passed": False, "error": str(exc)})
|
|
||||||
case["seconds"] = time.monotonic() - started
|
|
||||||
return case
|
|
||||||
|
|
||||||
|
|
||||||
def probe_cache_mode(rust, expected, enabled, phase, out, observations):
|
|
||||||
"""Record two exact GETs per source, separate from the paced workload."""
|
|
||||||
phase_observations = []
|
|
||||||
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
|
|
||||||
replies = []
|
|
||||||
for attempt in [1, 2]:
|
|
||||||
try:
|
|
||||||
status, headers, body = http(port, "/small.html?soak_cache_probe=1",
|
|
||||||
headers={"Accept-Encoding": "identity"})
|
|
||||||
replies.append({"attempt": attempt, "status": status, "cache": headers.get("x-cache"),
|
|
||||||
"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
|
|
||||||
"body_passed": status == 200 and body == expected["small.html"]})
|
|
||||||
except Exception as error:
|
|
||||||
replies.append({"attempt": attempt, "body_passed": False, "error": str(error)})
|
|
||||||
mode_passed = replies[1].get("cache") == "HIT" if enabled else all(
|
|
||||||
reply.get("cache") is None for reply in replies)
|
|
||||||
phase_observations.append({"phase": phase, "source": source, "cache_enabled": enabled,
|
|
||||||
"responses": replies,
|
|
||||||
"passed": all(reply["body_passed"] for reply in replies) and mode_passed})
|
|
||||||
observations.extend(phase_observations)
|
|
||||||
write_json(out / "cache-probes.json", observations)
|
|
||||||
if not all(observation["passed"] for observation in phase_observations):
|
|
||||||
raise RuntimeError(f"cache mode/body probe failed after {phase}; see cache-probes.json")
|
|
||||||
|
|
||||||
|
|
||||||
def backend_drain_snapshot(rust):
|
|
||||||
status, _, body = rust.admin("/api/status")
|
|
||||||
if status != 200:
|
|
||||||
return {"passed": False, "error": f"admin status HTTP {status}"}
|
|
||||||
backends = []
|
|
||||||
|
|
||||||
def walk(value):
|
|
||||||
if isinstance(value, dict):
|
|
||||||
if "id" in value and "active" in value and "status" in value and "requests" in value:
|
|
||||||
backends.append({key: value[key] for key in ("id", "active", "status", "requests")})
|
|
||||||
for child in value.values():
|
|
||||||
walk(child)
|
|
||||||
elif isinstance(value, list):
|
|
||||||
for child in value:
|
|
||||||
walk(child)
|
|
||||||
walk(json.loads(body))
|
|
||||||
return {"passed": bool(backends) and all(backend["active"] == 0 for backend in backends),
|
|
||||||
"backends": backends}
|
|
||||||
|
|
||||||
|
|
||||||
def memory_trend(samples, label="server"):
|
|
||||||
observations = [(sample["seconds"], sample["groups"][label]["rss_bytes"], sample["groups"][label]["fd"])
|
|
||||||
for sample in samples if sample["seconds"] >= 60]
|
|
||||||
if len(observations) < 3:
|
|
||||||
return {"available": False, "reason": "less than three samples after the first minute"}
|
|
||||||
tail_seconds = observations[-1][0]
|
|
||||||
early = [row[1] for row in observations if row[0] < observations[0][0] + 60]
|
|
||||||
late = [row[1] for row in observations if row[0] >= tail_seconds - 60]
|
|
||||||
x_mean = statistics.mean(row[0] for row in observations)
|
|
||||||
y_mean = statistics.mean(row[1] for row in observations)
|
|
||||||
denominator = sum((row[0] - x_mean) ** 2 for row in observations)
|
|
||||||
slope = sum((row[0] - x_mean) * (row[1] - y_mean) for row in observations) / denominator if denominator else 0
|
|
||||||
return {"available": True, "rss_first_minute_after_warmup_median": statistics.median(early),
|
|
||||||
"rss_last_minute_median": statistics.median(late),
|
|
||||||
"rss_growth_bytes": statistics.median(late) - statistics.median(early),
|
|
||||||
"rss_linear_bytes_per_hour": slope * 3600, "fd_first": observations[0][2],
|
|
||||||
"fd_last": observations[-1][2], "fd_peak": max(row[2] for row in observations),
|
|
||||||
"interpretation": "Descriptive only: cache toggles and allocator retention can change RSS without a leak"}
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", type=Path, required=True)
|
|
||||||
parser.add_argument("--label", default="after")
|
|
||||||
parser.add_argument("--nginx", type=Path, required=True)
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
parser.add_argument("--duration", type=int, default=1800)
|
|
||||||
parser.add_argument("--rate", type=float, default=100, help="offered requests per second, not a saturation benchmark")
|
|
||||||
parser.add_argument("--clients", type=int, default=8)
|
|
||||||
parser.add_argument("--workers", type=int, default=4)
|
|
||||||
parser.add_argument("--origin-workers", type=int, default=2)
|
|
||||||
parser.add_argument("--server-cpus")
|
|
||||||
parser.add_argument("--origin-cpus")
|
|
||||||
parser.add_argument("--toggle-seconds", type=int, default=120)
|
|
||||||
parser.add_argument("--sample-seconds", type=int, default=5)
|
|
||||||
parser.add_argument("--abort-every", type=int, default=101)
|
|
||||||
parser.add_argument("--cooldown-seconds", type=int, default=15)
|
|
||||||
parser.add_argument("--exploratory", action="store_true")
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.duration < 1800 and not args.exploratory:
|
|
||||||
parser.error("the soak gate requires >=1800 seconds; label short diagnostics with --exploratory")
|
|
||||||
if args.rate <= 0 or not 1 <= args.clients <= 128 or args.toggle_seconds < 1 or args.sample_seconds < 1:
|
|
||||||
parser.error("rate, sample/toggle intervals and clients must be positive; clients <=128")
|
|
||||||
args.out = args.out.resolve()
|
|
||||||
if args.out.exists() and any(args.out.iterdir()):
|
|
||||||
parser.error("--out must be a new or empty evidence directory")
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
settings = {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()}
|
|
||||||
write_json(args.out / "environment.json", {
|
|
||||||
"kind": "actual RustProxy fixed-rate mixed-traffic resource soak", "simulation": False,
|
|
||||||
"settings": settings, "platform": platform.platform(), "binary_sha256": sha256(args.binary),
|
|
||||||
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
|
||||||
"frontend_connection_policy": "A new HTTP/1.1 TCP connection for each workload request and probe",
|
|
||||||
"controlled_close_interpretation": "Client closes after reading 128 bytes; the server may already have sent the full body. No server-cancellation count is inferred.",
|
|
||||||
"traffic_analytics_observation_policy": "Global counters are read before traffic, periodically with status diagnostics, and after final drain. Deltas include cache-mode probes and cooldown, and are separate from HTTP errors.",
|
|
||||||
"not_covered": ["24-hour soak", "TLS/H2/H3 soak", "multi-backend failures", "PHP-FPM",
|
|
||||||
"real application write load", "origin DNS/TLS rotation", "production traffic",
|
|
||||||
"long-lived frontend keepalive connections"]})
|
|
||||||
totals = {"submitted": 0, "completed": 0, "passed": 0, "failed": 0, "controlled_client_closes": 0,
|
|
||||||
"response_bytes": 0, "scheduler_late_events": 0, "peak_pending": 0}
|
|
||||||
failure_examples, progress, toggles, counter_samples = [], [], [], []
|
|
||||||
traffic_samples, traffic_delta = [], None
|
|
||||||
cache_probes, cache_status_counts = [], {"proxy": Counter(), "static": Counter()}
|
|
||||||
drain_snapshot, configuration_restored = None, False
|
|
||||||
samples, resources = [], {}
|
|
||||||
fatal, runtime_seconds = None, 0
|
|
||||||
try:
|
|
||||||
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
|
|
||||||
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
|
|
||||||
nginx = fixture.nginx(args.nginx, frontend=False)
|
|
||||||
rust = fixture.rust(args.binary, args.label)
|
|
||||||
rust.login()
|
|
||||||
rust.wait_quiescent()
|
|
||||||
expected = {name: (fixture.site / name).read_bytes() for name in PAYLOADS}
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
config["cache"].update({"expose_status": True})
|
|
||||||
config["web_services"][0]["cache_control"] = "public, max-age=60"
|
|
||||||
rust.apply(config, "soak-initial")
|
|
||||||
probe_cache_mode(rust, expected, False, "initial-off", args.out, cache_probes)
|
|
||||||
traffic_before = traffic_analytics_snapshot(rust)
|
|
||||||
traffic_samples.append({"phase": "before", "seconds": 0, "snapshot": traffic_before})
|
|
||||||
write_json(args.out / "traffic-analytics.json", traffic_samples)
|
|
||||||
monitor = Monitor({"server": rust.process.pid, "origin": nginx["origin"].pid}, args.sample_seconds)
|
|
||||||
monitor.start()
|
|
||||||
start = time.monotonic()
|
|
||||||
deadline = start + args.duration
|
|
||||||
next_toggle, next_snapshot, next_submit = start + args.toggle_seconds, start, start
|
|
||||||
pending = set()
|
|
||||||
|
|
||||||
def consume(done):
|
|
||||||
for future in done:
|
|
||||||
result = future.result()
|
|
||||||
totals["completed"] += 1
|
|
||||||
totals["passed" if result["passed"] else "failed"] += 1
|
|
||||||
totals["controlled_client_closes"] += int(result["intentional_abort"])
|
|
||||||
totals["response_bytes"] += result.get("bytes", 0)
|
|
||||||
cache_status = (result.get("cache") or "absent") if "status" in result else "no_response"
|
|
||||||
cache_status_counts[result["source"]][cache_status] += 1
|
|
||||||
if not result["passed"] and len(failure_examples) < 100:
|
|
||||||
failure_examples.append(result)
|
|
||||||
|
|
||||||
try:
|
|
||||||
with ThreadPoolExecutor(max_workers=args.clients) as pool:
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
now = time.monotonic()
|
|
||||||
done = {future for future in pending if future.done()}
|
|
||||||
consume(done)
|
|
||||||
pending -= done
|
|
||||||
if now >= next_toggle:
|
|
||||||
config["cache"]["enabled"] = not config["cache"]["enabled"]
|
|
||||||
rust.apply(config, f"soak-toggle-{len(toggles) + 1}")
|
|
||||||
toggles.append({"seconds": time.monotonic() - start, "cache_enabled": config["cache"]["enabled"]})
|
|
||||||
write_json(args.out / "cache-toggles.json", toggles)
|
|
||||||
probe_cache_mode(rust, expected, config["cache"]["enabled"],
|
|
||||||
f"toggle-{len(toggles)}", args.out, cache_probes)
|
|
||||||
next_toggle += args.toggle_seconds
|
|
||||||
if now >= next_snapshot:
|
|
||||||
status, headers, body = rust.admin("/api/status")
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"admin status became unavailable: HTTP {status}")
|
|
||||||
counter_samples.append({"seconds": time.monotonic() - start,
|
|
||||||
"cache_enabled": config["cache"]["enabled"],
|
|
||||||
"counters": counter_values(json.loads(body))})
|
|
||||||
traffic_now = traffic_analytics_snapshot(rust)
|
|
||||||
traffic_delta = traffic_analytics_delta(traffic_before, traffic_now)
|
|
||||||
traffic_samples.append({"phase": "during", "seconds": time.monotonic() - start,
|
|
||||||
"snapshot": traffic_now, "delta_from_start": traffic_delta})
|
|
||||||
row = {"seconds": time.monotonic() - start, **totals,
|
|
||||||
"cache_enabled": config["cache"]["enabled"], "pending": len(pending),
|
|
||||||
"observed_cache_status_counts": {source: dict(counts)
|
|
||||||
for source, counts in cache_status_counts.items()}}
|
|
||||||
progress.append(row)
|
|
||||||
write_json(args.out / "progress.json", progress)
|
|
||||||
write_json(args.out / "status-counters.json", counter_samples)
|
|
||||||
write_json(args.out / "traffic-analytics.json", traffic_samples)
|
|
||||||
write_json(args.out / "resources.json", list(monitor.samples))
|
|
||||||
print(json.dumps(row), flush=True)
|
|
||||||
next_snapshot = time.monotonic() + args.sample_seconds
|
|
||||||
now = time.monotonic()
|
|
||||||
if len(pending) >= args.clients * 2:
|
|
||||||
done, pending = wait(pending, timeout=.1, return_when=FIRST_COMPLETED)
|
|
||||||
consume(done)
|
|
||||||
continue
|
|
||||||
if now < next_submit:
|
|
||||||
time.sleep(min(.02, next_submit - now))
|
|
||||||
continue
|
|
||||||
if now > next_submit + .25:
|
|
||||||
totals["scheduler_late_events"] += 1
|
|
||||||
next_submit = now
|
|
||||||
totals["submitted"] += 1
|
|
||||||
pending.add(pool.submit(fetch, totals["submitted"], rust, expected, args.abort_every))
|
|
||||||
totals["peak_pending"] = max(totals["peak_pending"], len(pending))
|
|
||||||
next_submit += 1 / args.rate
|
|
||||||
done, pending = wait(pending, timeout=30)
|
|
||||||
consume(done)
|
|
||||||
if pending:
|
|
||||||
raise RuntimeError("requests failed to settle during bounded soak drain")
|
|
||||||
runtime_seconds = time.monotonic() - start
|
|
||||||
rust.reset()
|
|
||||||
configuration_restored = True
|
|
||||||
cooldown_until = time.monotonic() + args.cooldown_seconds
|
|
||||||
while time.monotonic() < cooldown_until:
|
|
||||||
time.sleep(max(0, min(.5, cooldown_until - time.monotonic())))
|
|
||||||
probe_cache_mode(rust, expected, False, "restored-off", args.out, cache_probes)
|
|
||||||
for _ in range(40):
|
|
||||||
drain_snapshot = backend_drain_snapshot(rust)
|
|
||||||
if drain_snapshot["passed"]:
|
|
||||||
break
|
|
||||||
time.sleep(.05)
|
|
||||||
write_json(args.out / "backend-drain.json", drain_snapshot)
|
|
||||||
if not drain_snapshot["passed"]:
|
|
||||||
raise RuntimeError("backend active exchanges did not drain to zero")
|
|
||||||
traffic_after = traffic_analytics_snapshot(rust)
|
|
||||||
traffic_delta = traffic_analytics_delta(traffic_before, traffic_after)
|
|
||||||
traffic_samples.append({"phase": "after_drain", "seconds": time.monotonic() - start,
|
|
||||||
"snapshot": traffic_after, "delta_from_start": traffic_delta})
|
|
||||||
finally:
|
|
||||||
monitor.stop()
|
|
||||||
samples = monitor.samples
|
|
||||||
resources = monitor.summary(totals["completed"])
|
|
||||||
write_json(args.out / "resources.json", samples)
|
|
||||||
write_json(args.out / "status-counters.json", counter_samples)
|
|
||||||
write_json(args.out / "traffic-analytics.json", traffic_samples)
|
|
||||||
write_json(args.out / "failure-examples.json", failure_examples)
|
|
||||||
except Exception as exc:
|
|
||||||
fatal = {"error": str(exc), "traceback": traceback.format_exc()}
|
|
||||||
write_json(args.out / "failure.json", fatal)
|
|
||||||
report = {"completed": fatal is None, "simulation": False, "exploratory": args.exploratory,
|
|
||||||
"duration_seconds": runtime_seconds, "soak_duration_gate": runtime_seconds >= 1800,
|
|
||||||
"totals": totals, "resources": resources, "memory_trend": memory_trend(samples) if samples else None,
|
|
||||||
"cache_toggles": len(toggles), "errors": failure_examples,
|
|
||||||
"traffic_analytics": {"observations": len(traffic_samples), "delta": traffic_delta,
|
|
||||||
"includes_cache_mode_probes_and_cooldown": True},
|
|
||||||
"cache_mode_probes": {"source_checks": len(cache_probes),
|
|
||||||
"requests": sum(len(probe["responses"]) for probe in cache_probes),
|
|
||||||
"passed": bool(cache_probes) and all(probe["passed"] for probe in cache_probes)},
|
|
||||||
"observed_cache_status_counts": cache_status_counts,
|
|
||||||
"configuration_restored": configuration_restored, "backend_drain": drain_snapshot,
|
|
||||||
"actual_completed_rps": totals["completed"] / runtime_seconds if runtime_seconds else None,
|
|
||||||
"passed": fatal is None and totals["failed"] == 0 and runtime_seconds >= args.duration}
|
|
||||||
write_json(args.out / "report.json", report)
|
|
||||||
seal_results(args.out)
|
|
||||||
print(json.dumps(report), flush=True)
|
|
||||||
return 0 if report["passed"] else 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
"""Selected counters from existing authenticated APIs, without traffic records."""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
def traffic_analytics_snapshot(rust, timeout=2):
|
|
||||||
"""Read global analytics queue loss; never retain paths, IPs or user agents.
|
|
||||||
|
|
||||||
queue_dropped is process-global even when the API's vhost query is used.
|
|
||||||
recorded_requests is asynchronous analytics work, not an HTTP success count.
|
|
||||||
Call outside a timed load interval so this control-plane read is not charged
|
|
||||||
to the request path being measured.
|
|
||||||
"""
|
|
||||||
status, _, body = rust.admin("/api/traffic", timeout=timeout)
|
|
||||||
if status != 200:
|
|
||||||
raise RuntimeError(f"traffic analytics counter capture failed: HTTP {status}")
|
|
||||||
payload = json.loads(body)
|
|
||||||
if not isinstance(payload, dict) or payload.get("scope") != "*" or payload.get("since") != "process_start":
|
|
||||||
raise RuntimeError("traffic analytics requires the global process-start scope")
|
|
||||||
analytics = payload.get("analytics")
|
|
||||||
if not isinstance(analytics, dict) or analytics.get("scope") != "*":
|
|
||||||
raise RuntimeError("traffic analytics response lacks the global queue view")
|
|
||||||
data = analytics.get("data")
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
raise RuntimeError("traffic analytics response lacks the recorded counters")
|
|
||||||
values = {"queue_capacity": analytics.get("queue_capacity"),
|
|
||||||
"queue_dropped": analytics.get("queue_dropped"),
|
|
||||||
"recorded_requests": data.get("requests")}
|
|
||||||
if any(type(value) is not int or value < 0 for value in values.values()) or not values["queue_capacity"]:
|
|
||||||
raise RuntimeError("traffic analytics queue counters are absent or malformed")
|
|
||||||
return {"endpoint": "/api/traffic", "scope": "*", "since": "process_start",
|
|
||||||
"pid": rust.process.pid, "observed_monotonic": time.monotonic(), **values}
|
|
||||||
|
|
||||||
|
|
||||||
def traffic_analytics_delta(before, after):
|
|
||||||
"""Keep telemetry loss separate from failed HTTP requests and log rotation."""
|
|
||||||
for key in ["pid", "endpoint", "scope", "since", "queue_capacity"]:
|
|
||||||
if before[key] != after[key]:
|
|
||||||
raise RuntimeError(f"traffic analytics identity changed during capture: {key}")
|
|
||||||
result = {"queue_dropped_delta": after["queue_dropped"] - before["queue_dropped"],
|
|
||||||
"recorded_requests_delta": after["recorded_requests"] - before["recorded_requests"],
|
|
||||||
"interval_seconds": after["observed_monotonic"] - before["observed_monotonic"]}
|
|
||||||
if any(value < 0 for value in result.values()):
|
|
||||||
raise RuntimeError("traffic analytics counters or clock reset during capture")
|
|
||||||
return result
|
|
||||||
@@ -1,191 +0,0 @@
|
|||||||
"""Strict H2/TLS and H3/QUIC clients for local RustProxy acceptance tests.
|
|
||||||
|
|
||||||
The generated fixture certificate is explicitly trusted. Neither transport
|
|
||||||
silently falls back to HTTP/1.1 or disables certificate verification.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from contextlib import asynccontextmanager
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from pathlib import Path
|
|
||||||
import ssl
|
|
||||||
|
|
||||||
import httpx
|
|
||||||
from aioquic.asyncio import QuicConnectionProtocol, connect
|
|
||||||
from aioquic.h3.connection import H3Connection, MessageError
|
|
||||||
from aioquic.h3.events import DataReceived, HeadersReceived
|
|
||||||
from aioquic.quic.configuration import QuicConfiguration
|
|
||||||
from aioquic.quic.events import ConnectionTerminated, ProtocolNegotiated, StreamReset
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class Response:
|
|
||||||
status: int
|
|
||||||
headers: dict[str, str]
|
|
||||||
body: bytes
|
|
||||||
protocol: str
|
|
||||||
alpn: str | None
|
|
||||||
stream_id: int | None = None
|
|
||||||
|
|
||||||
|
|
||||||
class H2Client:
|
|
||||||
def __init__(self, client: httpx.AsyncClient, port: int):
|
|
||||||
self.client, self.port = client, port
|
|
||||||
|
|
||||||
async def request(self, method, path, headers=None, body=None):
|
|
||||||
async def chunks():
|
|
||||||
for offset in range(0, len(body), 16384):
|
|
||||||
yield body[offset:offset + 16384]
|
|
||||||
await asyncio.sleep(0)
|
|
||||||
|
|
||||||
response = await self.client.request(
|
|
||||||
method, f"https://127.0.0.1:{self.port}{path}",
|
|
||||||
headers=headers or {}, content=chunks() if body is not None else None,
|
|
||||||
)
|
|
||||||
network = response.extensions.get("network_stream")
|
|
||||||
tls = network.get_extra_info("ssl_object") if network else None
|
|
||||||
alpn = tls.selected_alpn_protocol() if tls else None
|
|
||||||
if response.http_version != "HTTP/2" or alpn != "h2":
|
|
||||||
raise AssertionError(f"H2 required, got {response.http_version}, ALPN={alpn!r}")
|
|
||||||
return Response(response.status_code, dict(response.headers), response.content,
|
|
||||||
"HTTP/2", alpn, response.extensions.get("stream_id"))
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class Pending:
|
|
||||||
future: asyncio.Future
|
|
||||||
status: int = 0
|
|
||||||
headers: dict[str, str] = field(default_factory=dict)
|
|
||||||
body: bytearray = field(default_factory=bytearray)
|
|
||||||
|
|
||||||
|
|
||||||
class HeadAwareH3Connection(H3Connection):
|
|
||||||
"""Supply request-method context missing from aioquic 1.7.0's length check.
|
|
||||||
|
|
||||||
RFC 9110 section 8.6 permits HEAD's Content-Length to describe a GET body.
|
|
||||||
Still reject any actual HEAD DATA; keep aioquic's exact length check for all
|
|
||||||
other requests. This changes only the test client, never the target server.
|
|
||||||
"""
|
|
||||||
def __init__(self, *args, **kwargs):
|
|
||||||
super().__init__(*args, **kwargs)
|
|
||||||
self.head_streams = set()
|
|
||||||
|
|
||||||
def _check_content_length(self, stream):
|
|
||||||
if stream.stream_id in self.head_streams:
|
|
||||||
if stream.content_length:
|
|
||||||
raise MessageError("HEAD response contained DATA")
|
|
||||||
return
|
|
||||||
super()._check_content_length(stream)
|
|
||||||
|
|
||||||
|
|
||||||
def validate_head_adapter():
|
|
||||||
"""Keep adapter checks separate from the real-server acceptance totals."""
|
|
||||||
from types import SimpleNamespace
|
|
||||||
|
|
||||||
connection = HeadAwareH3Connection.__new__(HeadAwareH3Connection)
|
|
||||||
connection.head_streams = {0}
|
|
||||||
for stream_id, declared, received, valid in [(0, 1040, 0, True), (0, 1040, 1, False),
|
|
||||||
(4, 1040, 1040, True), (4, 1040, 0, False)]:
|
|
||||||
stream = SimpleNamespace(stream_id=stream_id, expected_content_length=declared,
|
|
||||||
content_length=received)
|
|
||||||
try:
|
|
||||||
connection._check_content_length(stream)
|
|
||||||
accepted = True
|
|
||||||
except MessageError:
|
|
||||||
accepted = False
|
|
||||||
assert accepted == valid, "H3 fixture HEAD adapter changed ordinary framing validation"
|
|
||||||
return {"cases": 4, "passed": 4, "kind": "test-client self-test, not RustProxy coverage"}
|
|
||||||
|
|
||||||
|
|
||||||
class H3Client(QuicConnectionProtocol):
|
|
||||||
def __init__(self, *args, **kwargs):
|
|
||||||
super().__init__(*args, **kwargs)
|
|
||||||
self.http = HeadAwareH3Connection(self._quic)
|
|
||||||
self.pending: dict[int, Pending] = {}
|
|
||||||
self.alpn = None
|
|
||||||
self.authority = None
|
|
||||||
|
|
||||||
def quic_event_received(self, event):
|
|
||||||
if isinstance(event, ProtocolNegotiated):
|
|
||||||
self.alpn = event.alpn_protocol
|
|
||||||
if isinstance(event, ConnectionTerminated):
|
|
||||||
for item in self.pending.values():
|
|
||||||
if not item.future.done():
|
|
||||||
item.future.set_exception(ConnectionError(
|
|
||||||
f"QUIC closed code={event.error_code}: {event.reason_phrase}"))
|
|
||||||
if isinstance(event, StreamReset):
|
|
||||||
item = self.pending.get(event.stream_id)
|
|
||||||
if item is not None and not item.future.done():
|
|
||||||
item.future.set_exception(ConnectionError(
|
|
||||||
f"H3 stream reset code={event.error_code}"))
|
|
||||||
for http_event in self.http.handle_event(event):
|
|
||||||
if not isinstance(http_event, (HeadersReceived, DataReceived)):
|
|
||||||
continue
|
|
||||||
item = self.pending.get(http_event.stream_id)
|
|
||||||
if item is None or item.future.done():
|
|
||||||
continue
|
|
||||||
if isinstance(http_event, HeadersReceived):
|
|
||||||
for name, value in http_event.headers:
|
|
||||||
if name == b":status":
|
|
||||||
item.status = int(value)
|
|
||||||
elif not name.startswith(b":"):
|
|
||||||
item.headers[name.decode("ascii").lower()] = value.decode("latin-1")
|
|
||||||
else:
|
|
||||||
item.body.extend(http_event.data)
|
|
||||||
if len(item.body) > 4 * 1024 * 1024:
|
|
||||||
item.future.set_exception(ValueError("fixture response exceeded 4 MiB bound"))
|
|
||||||
continue
|
|
||||||
if http_event.stream_ended:
|
|
||||||
item.future.set_result(Response(item.status, item.headers, bytes(item.body),
|
|
||||||
"HTTP/3", self.alpn, http_event.stream_id))
|
|
||||||
|
|
||||||
async def request(self, method, path, headers=None, body=None):
|
|
||||||
if self.alpn != "h3":
|
|
||||||
raise AssertionError(f"H3 required, got ALPN={self.alpn!r}")
|
|
||||||
stream_id = self._quic.get_next_available_stream_id()
|
|
||||||
item = Pending(asyncio.get_running_loop().create_future())
|
|
||||||
self.pending[stream_id] = item
|
|
||||||
if method == "HEAD":
|
|
||||||
self.http.head_streams.add(stream_id)
|
|
||||||
fields = [(b":method", method.encode("ascii")), (b":scheme", b"https"),
|
|
||||||
(b":authority", self.authority.encode("ascii")),
|
|
||||||
(b":path", path.encode("ascii"))]
|
|
||||||
fields += [(name.lower().encode("ascii"), value.encode("latin-1"))
|
|
||||||
for name, value in (headers or {}).items()]
|
|
||||||
try:
|
|
||||||
self.http.send_headers(stream_id, fields, end_stream=body is None)
|
|
||||||
self.transmit()
|
|
||||||
if body is not None:
|
|
||||||
if not body:
|
|
||||||
self.http.send_data(stream_id, b"", end_stream=True)
|
|
||||||
self.transmit()
|
|
||||||
for offset in range(0, len(body), 16384):
|
|
||||||
end = min(offset + 16384, len(body))
|
|
||||||
self.http.send_data(stream_id, body[offset:end], end_stream=end == len(body))
|
|
||||||
self.transmit()
|
|
||||||
await asyncio.sleep(0)
|
|
||||||
return await asyncio.wait_for(item.future, timeout=10)
|
|
||||||
finally:
|
|
||||||
self.pending.pop(stream_id, None)
|
|
||||||
self.http.head_streams.discard(stream_id)
|
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def client(protocol: str, port: int, certificate: Path):
|
|
||||||
if protocol == "h2":
|
|
||||||
context = ssl.create_default_context(cafile=str(certificate))
|
|
||||||
async with httpx.AsyncClient(http2=True, verify=context, trust_env=False,
|
|
||||||
timeout=10, limits=httpx.Limits(max_connections=1)) as session:
|
|
||||||
yield H2Client(session, port)
|
|
||||||
elif protocol == "h3":
|
|
||||||
config = QuicConfiguration(is_client=True, alpn_protocols=["h3"],
|
|
||||||
server_name="localhost", idle_timeout=10)
|
|
||||||
config.verify_mode = ssl.CERT_REQUIRED
|
|
||||||
config.load_verify_locations(cafile=str(certificate))
|
|
||||||
async with connect("127.0.0.1", port, configuration=config,
|
|
||||||
create_protocol=H3Client) as session:
|
|
||||||
session.authority = f"127.0.0.1:{port}"
|
|
||||||
yield session
|
|
||||||
else:
|
|
||||||
raise ValueError(f"unsupported fixture protocol {protocol!r}")
|
|
||||||
@@ -1,281 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Real, bounded RustProxy H2/TLS, H3/QUIC and H1 WebSocket acceptance.
|
|
||||||
|
|
||||||
Install Python dependencies aioquic, httpx[http2] and websockets. Example:
|
|
||||||
python3 scripts/performance/transport_regression.py --binary before=/abs/binary \
|
|
||||||
--binary after=/abs/binary --out /abs/results --python-deps /abs/optional/pydeps
|
|
||||||
|
|
||||||
Only fresh temporary databases and loopback listeners are used. Certificates
|
|
||||||
are generated by RustProxy and verified by clients; no protocol fallback is used.
|
|
||||||
This is an interoperability/functional suite, not a throughput benchmark.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import asyncio
|
|
||||||
import copy
|
|
||||||
import hashlib
|
|
||||||
import importlib.metadata
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
import platform
|
|
||||||
import socket
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import traceback
|
|
||||||
|
|
||||||
from common import Fixture, seal_results, sha256, write_json
|
|
||||||
from peer import Peer
|
|
||||||
|
|
||||||
|
|
||||||
def response_details(response):
|
|
||||||
return {"status": response.status, "protocol": response.protocol,
|
|
||||||
"alpn": response.alpn, "stream_id": response.stream_id,
|
|
||||||
"bytes": len(response.body), "sha256": hashlib.sha256(response.body).hexdigest(),
|
|
||||||
"cache": response.headers.get("x-cache"),
|
|
||||||
"cache_reason": response.headers.get("x-cache-reason")}
|
|
||||||
|
|
||||||
|
|
||||||
def tls_port(fixture):
|
|
||||||
while True:
|
|
||||||
port = fixture.port()
|
|
||||||
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as udp:
|
|
||||||
try:
|
|
||||||
udp.bind(("127.0.0.1", port))
|
|
||||||
return port
|
|
||||||
except OSError:
|
|
||||||
continue
|
|
||||||
|
|
||||||
|
|
||||||
def transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin):
|
|
||||||
config = copy.deepcopy(rust.base)
|
|
||||||
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
|
|
||||||
identifier = f"{source}-tls"
|
|
||||||
config["listeners"]["endpoints"].append({
|
|
||||||
"id": identifier, "name": identifier, "scope": "internal", "enabled": True,
|
|
||||||
"binding": {"address": f"127.0.0.1:{port}", "tls": True, "http2": True,
|
|
||||||
"http3": {"max_concurrent_connections": 16,
|
|
||||||
"max_concurrent_bidistream": 16,
|
|
||||||
"max_concurrent_unistream": 16, "max_idle_timeout": 10}}})
|
|
||||||
vhost = next(v for v in config["vhosts"] if v["id"] == source)
|
|
||||||
vhost["frontend"]["listener_ids"].append(identifier)
|
|
||||||
vhost["http2_enabled"] = True
|
|
||||||
vhost["http3_enabled"] = True
|
|
||||||
config["listeners"]["endpoints"].append({
|
|
||||||
"id": "websocket", "name": "WebSocket", "scope": "internal", "enabled": True,
|
|
||||||
"binding": {"address": f"127.0.0.1:{websocket_port}", "tls": False,
|
|
||||||
"http2": False, "http3": None}})
|
|
||||||
config["vhosts"].append({
|
|
||||||
"id": "websocket", "frontend": {"hosts": ["127.0.0.1", "localhost"],
|
|
||||||
"listener_ids": ["websocket"]},
|
|
||||||
"websockets": True,
|
|
||||||
"backends": [{"id": "echo", "url": f"http://127.0.0.1:{websocket_origin}",
|
|
||||||
"allow_private": True}],
|
|
||||||
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []})
|
|
||||||
config["cache"].update({"enabled": False, "expose_status": True, "expose_hits": True,
|
|
||||||
"expose_cache_status": True, "expose_only_hits": False})
|
|
||||||
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
|
|
||||||
config["web_services"][0]["cache_control"] = "public, max-age=60"
|
|
||||||
return config
|
|
||||||
|
|
||||||
|
|
||||||
async def http_cases(check, session, protocol, source, cache, fixture):
|
|
||||||
prefix = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
|
|
||||||
expected = (fixture.site / "small.html").read_bytes()
|
|
||||||
path = f"/transport-{protocol}.html"
|
|
||||||
first = await session.request("GET", path)
|
|
||||||
check(prefix + ".get", first.status == 200 and first.body == expected,
|
|
||||||
**response_details(first))
|
|
||||||
second = await session.request("GET", path)
|
|
||||||
check(prefix + ".cache_delivery", second.status == 200 and second.body == expected
|
|
||||||
and ((first.headers.get("x-cache") == "MISS" and second.headers.get("x-cache") == "HIT")
|
|
||||||
if cache else second.headers.get("x-cache") not in ("MISS", "HIT")),
|
|
||||||
first_cache=first.headers.get("x-cache"), **response_details(second))
|
|
||||||
head = await session.request("HEAD", path)
|
|
||||||
check(prefix + ".head", head.status == 200 and head.body == b""
|
|
||||||
and head.headers.get("content-length") == str(len(expected)), **response_details(head))
|
|
||||||
etag = second.headers.get("etag")
|
|
||||||
check(prefix + ".validator", bool(etag), etag=etag)
|
|
||||||
ranged = await session.request("GET", path, {"range": "bytes=11-40", "if-range": etag or ""})
|
|
||||||
check(prefix + ".range", ranged.status == 206 and ranged.body == expected[11:41]
|
|
||||||
and ranged.headers.get("content-range") == f"bytes 11-40/{len(expected)}",
|
|
||||||
**response_details(ranged))
|
|
||||||
mismatch = await session.request("GET", path, {"range": "bytes=11-40", "if-range": '"different"'})
|
|
||||||
check(prefix + ".if_range_mismatch", mismatch.status == 200 and mismatch.body == expected,
|
|
||||||
**response_details(mismatch))
|
|
||||||
conditional = await session.request("GET", path, {"if-none-match": etag or ""})
|
|
||||||
check(prefix + ".conditional_304", conditional.status == 304 and conditional.body == b"",
|
|
||||||
**response_details(conditional))
|
|
||||||
invalid = await session.request("GET", path, {"range": "bytes=999999-"})
|
|
||||||
check(prefix + ".range_416", invalid.status == 416 and invalid.body == b""
|
|
||||||
and invalid.headers.get("content-range") == f"bytes */{len(expected)}",
|
|
||||||
**response_details(invalid))
|
|
||||||
for name in ["large.bin", "huge.bin"]:
|
|
||||||
expected_file = (fixture.site / name).read_bytes()
|
|
||||||
response = await session.request("GET", "/" + name)
|
|
||||||
check(prefix + ".get." + name, response.status == 200 and response.body == expected_file,
|
|
||||||
**response_details(response))
|
|
||||||
parallel = await asyncio.gather(*(session.request("GET", "/large.bin") for _ in range(4)))
|
|
||||||
expected_large = (fixture.site / "large.bin").read_bytes()
|
|
||||||
check(prefix + ".multiplex_four_streams", all(r.status == 200 and r.body == expected_large for r in parallel)
|
|
||||||
and len({r.stream_id for r in parallel}) == 4,
|
|
||||||
stream_ids=[r.stream_id for r in parallel], response_bytes=[len(r.body) for r in parallel])
|
|
||||||
if source == "reverse":
|
|
||||||
for payload in [b"", (bytes(range(256)) * 512) + b"streamed-end"]:
|
|
||||||
response = await session.request("POST", "/echo?transport=preserved",
|
|
||||||
{"content-type": "application/octet-stream"}, payload)
|
|
||||||
data = json.loads(response.body) if response.status == 200 else {}
|
|
||||||
check(prefix + f".post_stream_{len(payload)}", response.status == 200
|
|
||||||
and data.get("method") == "POST" and data.get("body_bytes") == len(payload)
|
|
||||||
and data.get("body_sha256") == hashlib.sha256(payload).hexdigest()
|
|
||||||
and data.get("path") == "/echo?transport=preserved",
|
|
||||||
upload_bytes=len(payload), **response_details(response))
|
|
||||||
else:
|
|
||||||
response = await session.request("POST", path, body=b"static-method-gate")
|
|
||||||
check(prefix + ".post_static_method_gate", response.status == 405, **response_details(response))
|
|
||||||
|
|
||||||
|
|
||||||
async def websocket_cases(check, port, cache):
|
|
||||||
import websockets
|
|
||||||
|
|
||||||
prefix = f"h1.websocket.cache_{'on' if cache else 'off'}"
|
|
||||||
async with websockets.connect(f"ws://127.0.0.1:{port}/echo", open_timeout=5,
|
|
||||||
close_timeout=3, ping_interval=None, compression=None) as session:
|
|
||||||
text = "RustProxy echo: Zażółć gęślą jaźń"
|
|
||||||
await session.send(text)
|
|
||||||
returned = await asyncio.wait_for(session.recv(), timeout=5)
|
|
||||||
check(prefix + ".text_echo", returned == text)
|
|
||||||
binary = bytes(range(256)) * 256
|
|
||||||
await session.send(binary)
|
|
||||||
returned = await asyncio.wait_for(session.recv(), timeout=5)
|
|
||||||
check(prefix + ".binary_echo", returned == binary, bytes=len(returned))
|
|
||||||
pong = await session.ping(b"rustproxy-transport")
|
|
||||||
await asyncio.wait_for(pong, timeout=5)
|
|
||||||
check(prefix + ".ping_pong", True)
|
|
||||||
await session.close(code=1000, reason="fixture complete")
|
|
||||||
check(prefix + ".normal_close", session.close_code == 1000, close_code=session.close_code)
|
|
||||||
|
|
||||||
|
|
||||||
async def run_variant(binary, label, output, workers):
|
|
||||||
import websockets
|
|
||||||
from transport_clients import client
|
|
||||||
|
|
||||||
cases = []
|
|
||||||
started = time.monotonic()
|
|
||||||
|
|
||||||
def check(case, passed, **details):
|
|
||||||
cases.append({"case": case, "passed": bool(passed), **details})
|
|
||||||
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
|
|
||||||
write_json(output / "checks.json", cases)
|
|
||||||
|
|
||||||
try:
|
|
||||||
with Fixture(output, workers=workers) as fixture:
|
|
||||||
plain = (fixture.site / "small.html").read_bytes()
|
|
||||||
for name in ["transport-h2.html", "transport-h3.html", "transport-shared.html"]:
|
|
||||||
(fixture.site / name).write_bytes(plain)
|
|
||||||
peer = Peer(fixture.origin_port, fixture.site)
|
|
||||||
websocket_origin, websocket_port = fixture.port(), fixture.port()
|
|
||||||
|
|
||||||
async def echo(session, _path=None):
|
|
||||||
async for message in session:
|
|
||||||
await session.send(message)
|
|
||||||
|
|
||||||
try:
|
|
||||||
async with websockets.serve(echo, "127.0.0.1", websocket_origin,
|
|
||||||
ping_interval=None, compression=None, close_timeout=3):
|
|
||||||
rust = fixture.rust(binary, label)
|
|
||||||
reverse_tls, static_tls = tls_port(fixture), tls_port(fixture)
|
|
||||||
config = transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin)
|
|
||||||
rust.apply(config, "transport-listeners")
|
|
||||||
fixture.stop(rust.process)
|
|
||||||
rust.start("transport")
|
|
||||||
certificate = rust.data / "certs" / "default.pem"
|
|
||||||
write_json(output / "transport.json", {
|
|
||||||
"certificate_sha256": sha256(certificate), "certificate_verification": True,
|
|
||||||
"h2_alpn_required": "h2", "h3_alpn_required": "h3",
|
|
||||||
"reverse_tls_tcp_udp": reverse_tls, "static_tls_tcp_udp": static_tls,
|
|
||||||
"websocket_h1_tcp": websocket_port, "fallback_allowed": False})
|
|
||||||
for cache in [False, True]:
|
|
||||||
if cache:
|
|
||||||
config["cache"]["enabled"] = True
|
|
||||||
rust.apply(config, "transport-cache-on")
|
|
||||||
for protocol in ["h2", "h3"]:
|
|
||||||
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
|
|
||||||
group = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
|
|
||||||
try:
|
|
||||||
async with client(protocol, port, certificate) as session:
|
|
||||||
await http_cases(check, session, protocol, source, cache, fixture)
|
|
||||||
except Exception as exc:
|
|
||||||
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
|
|
||||||
try:
|
|
||||||
await websocket_cases(check, websocket_port, cache)
|
|
||||||
except Exception as exc:
|
|
||||||
check(f"h1.websocket.cache_{'on' if cache else 'off'}.runner", False,
|
|
||||||
error=str(exc), traceback=traceback.format_exc())
|
|
||||||
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
|
|
||||||
try:
|
|
||||||
async with client("h2", port, certificate) as h2:
|
|
||||||
warm = await h2.request("GET", "/transport-shared.html")
|
|
||||||
async with client("h3", port, certificate) as h3:
|
|
||||||
hit = await h3.request("GET", "/transport-shared.html")
|
|
||||||
check(f"cache_cross_protocol.{source}.h2_to_h3",
|
|
||||||
warm.status == 200 and hit.status == 200 and hit.body == plain
|
|
||||||
and hit.headers.get("x-cache") == "HIT",
|
|
||||||
first_cache=warm.headers.get("x-cache"), **response_details(hit))
|
|
||||||
except Exception as exc:
|
|
||||||
check(f"cache_cross_protocol.{source}.runner", False,
|
|
||||||
error=str(exc), traceback=traceback.format_exc())
|
|
||||||
check("runtime.still_alive", rust.process.poll() is None)
|
|
||||||
finally:
|
|
||||||
peer.close()
|
|
||||||
except Exception as exc:
|
|
||||||
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
|
|
||||||
report = {"label": label, "simulation": False, "seconds": time.monotonic() - started,
|
|
||||||
"passed": sum(case["passed"] for case in cases),
|
|
||||||
"failed": sum(not case["passed"] for case in cases), "cases": cases}
|
|
||||||
write_json(output / "report.json", report)
|
|
||||||
print(f"{label}: {report['passed']} passed, {report['failed']} failed", flush=True)
|
|
||||||
return report
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
|
|
||||||
parser.add_argument("--out", type=Path, required=True)
|
|
||||||
parser.add_argument("--workers", type=int, default=4)
|
|
||||||
parser.add_argument("--python-deps", type=Path)
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.python_deps:
|
|
||||||
sys.path.insert(0, str(args.python_deps.resolve()))
|
|
||||||
variants = {}
|
|
||||||
for value in args.binary:
|
|
||||||
label, separator, path = value.partition("=")
|
|
||||||
if not separator or not label.replace("-", "").replace("_", "").isalnum() or label in variants:
|
|
||||||
parser.error("--binary must be unique SIMPLE_LABEL=/ABS/PATH")
|
|
||||||
variants[label] = Path(path).resolve()
|
|
||||||
args.out = args.out.resolve()
|
|
||||||
args.out.mkdir(parents=True, exist_ok=True)
|
|
||||||
from transport_clients import validate_head_adapter
|
|
||||||
write_json(args.out / "environment.json", {
|
|
||||||
"kind": "real RustProxy transport acceptance", "simulation": False,
|
|
||||||
"python": sys.version, "platform": platform.platform(),
|
|
||||||
"dependencies": {name: importlib.metadata.version(name) for name in ["aioquic", "httpx", "h2", "websockets"]},
|
|
||||||
"aioquic_head_adapter": validate_head_adapter(),
|
|
||||||
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
|
||||||
"not_covered": ["production certificates", "TLS upstreams", "H3 upstreams", "H2/H3 WebSocket extended CONNECT",
|
|
||||||
"network loss or migration", "24-hour soak", "real-browser navigation"]})
|
|
||||||
reports = {}
|
|
||||||
for label, binary in variants.items():
|
|
||||||
reports[label] = asyncio.run(run_variant(binary, label, args.out / label, args.workers))
|
|
||||||
write_json(args.out / "comparison.json", {
|
|
||||||
"variants": {label: {key: report[key] for key in ["passed", "failed", "seconds"]}
|
|
||||||
for label, report in reports.items()},
|
|
||||||
"cases": [{"case": case, **{label: next((row["passed"] for row in report["cases"] if row["case"] == case), "not_run")
|
|
||||||
for label, report in reports.items()}}
|
|
||||||
for case in sorted({row["case"] for report in reports.values() for row in report["cases"]})]})
|
|
||||||
seal_results(args.out)
|
|
||||||
return 1 if any(report["failed"] for report in reports.values()) else 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
-- No response hook: benchmarks must not be limited by Lua body processing.
|
|
||||||
-- Known payload/status/headers are verified separately before measurement.
|
|
||||||
done = function(summary, latency, requests)
|
|
||||||
io.write("RUSTPROXY_BENCH_JSON {")
|
|
||||||
io.write(string.format('"requests":%d,"duration_us":%d,"bytes":%d,',
|
|
||||||
summary.requests, summary.duration, summary.bytes))
|
|
||||||
io.write(string.format('"p50_us":%.3f,"p95_us":%.3f,"p99_us":%.3f,',
|
|
||||||
latency:percentile(50), latency:percentile(95), latency:percentile(99)))
|
|
||||||
io.write(string.format('"latency_mean_us":%.3f,"latency_max_us":%.3f,', latency.mean, latency.max))
|
|
||||||
io.write(string.format('"errors_connect":%d,"errors_read":%d,"errors_write":%d,"errors_timeout":%d,"errors_status":%d',
|
|
||||||
summary.errors.connect, summary.errors.read, summary.errors.write,
|
|
||||||
summary.errors.timeout, summary.errors.status))
|
|
||||||
io.write("}\n")
|
|
||||||
end
|
|
||||||
@@ -74,7 +74,7 @@ def check(root: Path, repair_known: bool = False) -> list[str]:
|
|||||||
flat = module.parent.with_suffix(".rs")
|
flat = module.parent.with_suffix(".rs")
|
||||||
if flat.is_file():
|
if flat.is_file():
|
||||||
raise ValueError(f"Conflicting Rust modules: {flat.relative_to(root)} and {module.relative_to(root)}")
|
raise ValueError(f"Conflicting Rust modules: {flat.relative_to(root)} and {module.relative_to(root)}")
|
||||||
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/TODO.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
|
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
|
||||||
if not (root / path).is_file():
|
if not (root / path).is_file():
|
||||||
raise ValueError(f"Incomplete RustProxy 0.31.9 source tree: missing {path}")
|
raise ValueError(f"Incomplete RustProxy 0.31.9 source tree: missing {path}")
|
||||||
# The Reader uses compile-time include_str! for its bundled Markdown. A
|
# The Reader uses compile-time include_str! for its bundled Markdown. A
|
||||||
|
|||||||
@@ -1,62 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Deterministic oracle for HTTP Age / local freshness-cap algebra.
|
|
||||||
No binary execution, no claim of measured throughput. Run with Python 3 only.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import random
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def old_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
|
|
||||||
age = max(header_age, apparent_age) + response_delay
|
|
||||||
return max(0, min(max_age, local_limit) - age)
|
|
||||||
|
|
||||||
|
|
||||||
def new_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
|
|
||||||
# RFC 9111 corrected_initial_age = max(apparent_age, age_value + response_delay)
|
|
||||||
age = max(apparent_age, header_age + response_delay)
|
|
||||||
lifetime = min(max_age, age + local_limit)
|
|
||||||
return max(0, lifetime - age)
|
|
||||||
|
|
||||||
|
|
||||||
def run():
|
|
||||||
cases = [
|
|
||||||
('user_cdn_example', 2_592_000, 1_553_483, 0, 0, 86_400),
|
|
||||||
('user_cdn_example_7d', 2_592_000, 1_553_483, 0, 0, 604_800),
|
|
||||||
('origin_nearly_stale', 2_592_000, 2_591_990, 0, 0, 86_400),
|
|
||||||
('origin_stale', 2_592_000, 2_592_010, 0, 0, 86_400),
|
|
||||||
('normal_small', 3_600, 0, 0, 0, 86_400),
|
|
||||||
('date_is_older', 3_600, 10, 600, 2, 86_400),
|
|
||||||
('rfc_delay_case', 3_600, 3, 10, 2, 86_400),
|
|
||||||
]
|
|
||||||
report = {'reference': 'RFC 9111 4.2.3 (Age) and 4.2.1 (freshness lifetime)',
|
|
||||||
'kind': 'algebra simulation; not RustProxy RPS benchmark',
|
|
||||||
'scenarios': [], 'randomized_checks': 0, 'false_miss_cases_old': 0}
|
|
||||||
for name, max_age, age, apparent, delay, local in cases:
|
|
||||||
old = old_freshness(max_age, age, apparent, delay, local)
|
|
||||||
new = new_freshness(max_age, age, apparent, delay, local)
|
|
||||||
report['scenarios'].append({'scenario': name, 'origin_max_age_seconds': max_age,
|
|
||||||
'received_age_seconds': age, 'apparent_age_seconds': apparent,
|
|
||||||
'response_delay_seconds': delay, 'local_max_ttl_seconds': local,
|
|
||||||
'old_freshness_seconds': old, 'new_freshness_seconds': new})
|
|
||||||
rng = random.Random(0x3400)
|
|
||||||
for _ in range(100_000):
|
|
||||||
origin = rng.randrange(0, 1 << 24)
|
|
||||||
header_age = rng.randrange(0, 1 << 24)
|
|
||||||
apparent = rng.randrange(0, 1 << 24)
|
|
||||||
delay = rng.randrange(0, 300)
|
|
||||||
local = rng.randrange(0, 604_801)
|
|
||||||
corrected_age = max(apparent, header_age + delay)
|
|
||||||
new = new_freshness(origin, header_age, apparent, delay, local)
|
|
||||||
expected = min(max(0, origin - corrected_age), local)
|
|
||||||
assert new == expected, (origin, header_age, apparent, delay, local, new, expected)
|
|
||||||
assert new <= local and new <= max(0, origin - corrected_age)
|
|
||||||
old = old_freshness(origin, header_age, apparent, delay, local)
|
|
||||||
report['false_miss_cases_old'] += old == 0 < new
|
|
||||||
report['randomized_checks'] += 1
|
|
||||||
return report
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
report = run()
|
|
||||||
print(json.dumps(report, ensure_ascii=False, indent=2))
|
|
||||||
@@ -1,212 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Loopback integration regression: transformed Age, HOT/COLD, diagnostics, isolation.
|
|
||||||
|
|
||||||
Run after compiling:
|
|
||||||
python3 scripts/tests/cache_age_transform_034_live.py --binary target/release/rust-proxy
|
|
||||||
To reproduce the original regression with the user-supplied 0.34.0 binary:
|
|
||||||
python3 scripts/tests/cache_age_transform_034_live.py --binary /path/to/old/rust-proxy --expect-old-age-bug
|
|
||||||
No internet and no user data; test config/data/passwords are in a disposable temporary dir.
|
|
||||||
"""
|
|
||||||
import argparse
|
|
||||||
import collections
|
|
||||||
import copy
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'performance'))
|
|
||||||
from common import Fixture, http, sha256, write_json
|
|
||||||
|
|
||||||
BODY = b'/* test cache */\nbody { color: red; padding: 0px; }\n'
|
|
||||||
# Fixed sample from Brotli encoder, embedded to avoid Python dependency on brotli.
|
|
||||||
BROTLI_BODY = b"\x1b3\x00\xe8\x1d\x07\xee\xbe\r\xa2\xbc3'\x8cP\xe9\xad\xa0\xf2\xb2\xa7\xe4\xfa\x12\x84\xac\x0c\x1cr\xc0\xfe\xc0>\x0fR*a\x9c\x9e2\xe8\x08\xcb\x1cP\x881\xac\xdd\x88s5(\x9d\x01"
|
|
||||||
COUNTS = collections.Counter()
|
|
||||||
|
|
||||||
class Origin(BaseHTTPRequestHandler):
|
|
||||||
def log_message(self, *_):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def do_GET(self):
|
|
||||||
path = self.path.split('?', 1)[0]
|
|
||||||
if path not in {'/aged.css', '/normal.css', '/private.css', '/no-store.css', '/other.css', '/br.css', '/aged-br.css'}:
|
|
||||||
self.send_error(404)
|
|
||||||
return
|
|
||||||
COUNTS[path] += 1
|
|
||||||
self.send_response(200)
|
|
||||||
if path in ('/aged.css', '/aged-br.css'):
|
|
||||||
policy = 'public, max-age=2592000'
|
|
||||||
elif path == '/private.css':
|
|
||||||
policy = 'private, max-age=2592000'
|
|
||||||
elif path == '/no-store.css':
|
|
||||||
policy = 'no-store'
|
|
||||||
else:
|
|
||||||
policy = 'public, max-age=3600'
|
|
||||||
self.send_header('Cache-Control', policy)
|
|
||||||
self.send_header('Content-Type', 'text/css')
|
|
||||||
payload = BROTLI_BODY if path in ('/br.css', '/aged-br.css') else BODY
|
|
||||||
self.send_header('Content-Length', str(len(payload)))
|
|
||||||
if path in ('/br.css', '/aged-br.css'):
|
|
||||||
self.send_header('Content-Encoding', 'br')
|
|
||||||
self.send_header('ETag', '"origin-css"')
|
|
||||||
self.send_header('Last-Modified', 'Thu, 07 Jan 2016 20:04:59 GMT')
|
|
||||||
self.send_header('Vary', 'Accept-Encoding')
|
|
||||||
if path in ('/aged.css', '/aged-br.css'):
|
|
||||||
self.send_header('Age', '1553483')
|
|
||||||
self.end_headers()
|
|
||||||
self.wfile.write(payload)
|
|
||||||
|
|
||||||
|
|
||||||
def request(port, path, encoding='identity'):
|
|
||||||
status, headers, body = http(port, path, headers={'Accept-Encoding': encoding})
|
|
||||||
return {'status': status, 'x_cache': headers.get('x-cache'),
|
|
||||||
'x_cache_hits': headers.get('x-cache-hits'), 'x_cache_resident_age': headers.get('x-cache-resident-age'),
|
|
||||||
'cache_status': headers.get('cache-status'), 'age': headers.get('age'),
|
|
||||||
'body': body.decode('utf-8', 'replace'), 'content_length': headers.get('content-length'),
|
|
||||||
'etag': headers.get('etag'), 'encoding': headers.get('content-encoding'), 'origin_count': COUNTS[path.split('?', 1)[0]]}
|
|
||||||
|
|
||||||
|
|
||||||
def dash(rust):
|
|
||||||
status, _, body = rust.admin('/api/cache?limit=200')
|
|
||||||
assert status == 200, ('cache dashboard', status, body[:200])
|
|
||||||
data = json.loads(body)['dashboard']
|
|
||||||
return {'objects': [{k: row.get(k) for k in ('vhost','path','tier','transformed',
|
|
||||||
'hits','age_seconds','resident_age_seconds','remaining_ttl_seconds')} for row in data['objects']],
|
|
||||||
'domains': data['domains']}
|
|
||||||
|
|
||||||
|
|
||||||
def find(d, path):
|
|
||||||
matches = [item for item in d['objects'] if item['path'] == path and item['vhost'] == 'reverse']
|
|
||||||
assert len(matches) == 1, (path, matches)
|
|
||||||
return matches[0]
|
|
||||||
|
|
||||||
|
|
||||||
def run(binary: Path, output: Path, expect_old_age_bug: bool):
|
|
||||||
outcome = {'binary_sha256': sha256(binary), 'expect_old_age_bug': expect_old_age_bug,
|
|
||||||
'tests': [], 'passed': False}
|
|
||||||
output.mkdir(parents=True, exist_ok=True)
|
|
||||||
with Fixture(output, workers=2) as fixture:
|
|
||||||
origin = ThreadingHTTPServer(('127.0.0.1', fixture.origin_port), Origin)
|
|
||||||
thread = threading.Thread(target=origin.serve_forever, daemon=True)
|
|
||||||
thread.start()
|
|
||||||
try:
|
|
||||||
rust = fixture.rust(binary, 'cache-age-034')
|
|
||||||
rust.login()
|
|
||||||
cfg = copy.deepcopy(rust.base)
|
|
||||||
cfg['cache'] = {'enabled': True, 'max_ttl_seconds': 86_400,
|
|
||||||
'expose_status': True, 'expose_hits': True,
|
|
||||||
'expose_cache_status': True, 'expose_resident_age': True}
|
|
||||||
cfg['transform'] = {'enabled': True, 'css': True}
|
|
||||||
for vhost in cfg['vhosts']:
|
|
||||||
vhost['transform'] = {'enabled': True, 'css': True}
|
|
||||||
rust.apply(cfg, 'fresh-cache-transform')
|
|
||||||
for path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css', '/private.css', '/no-store.css'):
|
|
||||||
before = COUNTS[path]
|
|
||||||
encoding = 'br' if path in ('/br.css', '/aged-br.css') else 'identity'
|
|
||||||
first, second = request(rust.reverse_port, path, encoding), request(rust.reverse_port, path, encoding)
|
|
||||||
row = {'path': path, 'first': first, 'second': second,
|
|
||||||
'origin_delta': COUNTS[path] - before}
|
|
||||||
outcome['tests'].append(row)
|
|
||||||
if path in ('/aged.css', '/aged-br.css') and expect_old_age_bug:
|
|
||||||
assert first['x_cache'] == second['x_cache'] == 'MISS', row
|
|
||||||
assert row['origin_delta'] == 2, row
|
|
||||||
elif path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css'):
|
|
||||||
assert first['x_cache'] == 'MISS' and second['x_cache'] == 'HIT', row
|
|
||||||
assert row['origin_delta'] == 1, row
|
|
||||||
assert second['cache_status'] and 'transformed' in second['cache_status'], row
|
|
||||||
assert first['etag'] == second['etag'] and first['body'] == second['body'], row
|
|
||||||
else:
|
|
||||||
assert first['x_cache'] == second['x_cache'] == 'MISS', row
|
|
||||||
print(path, first['x_cache'], second['x_cache'], 'origin+', row['origin_delta'], flush=True)
|
|
||||||
snapshot = dash(rust)
|
|
||||||
outcome['tests'].append({'dashboard': snapshot})
|
|
||||||
assert find(snapshot, '/normal.css')['transformed'], snapshot
|
|
||||||
aged = find(snapshot, '/aged.css')
|
|
||||||
if expect_old_age_bug:
|
|
||||||
assert aged['remaining_ttl_seconds'] == 0, aged
|
|
||||||
else:
|
|
||||||
assert aged['transformed'] and aged['remaining_ttl_seconds'] >= 86_000, aged
|
|
||||||
assert aged['age_seconds'] >= 1_553_483, aged
|
|
||||||
assert aged['resident_age_seconds'] < 10, aged
|
|
||||||
aged_br = find(snapshot, '/aged-br.css')
|
|
||||||
assert aged_br['transformed'] and aged_br['remaining_ttl_seconds'] >= 86_000, aged_br
|
|
||||||
|
|
||||||
# Query-string bypass is an explicit administrator rule. Even a
|
|
||||||
# transformed CSS with public max-age stays BYPASS while it is ON,
|
|
||||||
# then the *same* URL must become MISS -> HIT after disabling it.
|
|
||||||
# This is not a transform regression: it is a request policy.
|
|
||||||
cfg['cache']['bypass_query'] = True
|
|
||||||
rust.apply(cfg, 'query-bypass-enabled')
|
|
||||||
query_path = '/normal.css?v=transformed-public'
|
|
||||||
before = COUNTS['/normal.css']
|
|
||||||
q_bypass = [request(rust.reverse_port, query_path) for _ in range(2)]
|
|
||||||
assert [row['x_cache'] for row in q_bypass] == ['BYPASS', 'BYPASS'], q_bypass
|
|
||||||
assert COUNTS['/normal.css'] - before == 2, q_bypass
|
|
||||||
cfg['cache']['bypass_query'] = False
|
|
||||||
rust.apply(cfg, 'query-bypass-disabled')
|
|
||||||
q_cached = [request(rust.reverse_port, query_path) for _ in range(2)]
|
|
||||||
assert [row['x_cache'] for row in q_cached] == ['MISS', 'HIT'], q_cached
|
|
||||||
assert q_cached[1]['cache_status'] and 'transformed' in q_cached[1]['cache_status']
|
|
||||||
assert COUNTS['/normal.css'] - before == 3, q_cached
|
|
||||||
outcome['tests'].append({'query_bypass_policy': {'bypass': q_bypass, 'cached': q_cached}})
|
|
||||||
|
|
||||||
# Operational LB tuning must not invalidate a cached representation.
|
|
||||||
before = COUNTS['/normal.css']
|
|
||||||
cfg['vhosts'][0]['load_balancing']['keepalive'] = {'max_idle_per_host': 16}
|
|
||||||
rust.apply(cfg, 'only-backend-keepalive-changed')
|
|
||||||
normal = request(rust.reverse_port, '/normal.css')
|
|
||||||
assert normal['x_cache'] == 'HIT' and COUNTS['/normal.css'] == before
|
|
||||||
outcome['tests'].append({'keepalive_update_without_purge': normal})
|
|
||||||
|
|
||||||
# Diagnostics can be intentionally hidden on MISS and remain visible on HIT.
|
|
||||||
cfg['cache']['expose_only_hits'] = True
|
|
||||||
rust.apply(cfg, 'only-show-hit-headers')
|
|
||||||
before = COUNTS['/other.css']
|
|
||||||
first, second = request(rust.reverse_port, '/other.css'), request(rust.reverse_port, '/other.css')
|
|
||||||
assert first['x_cache'] is None and second['x_cache'] == 'HIT'
|
|
||||||
assert COUNTS['/other.css'] - before == 1
|
|
||||||
outcome['tests'].append({'only_hits': [first, second]})
|
|
||||||
|
|
||||||
if not expect_old_age_bug:
|
|
||||||
# Cold path on a 1-entry HOT tier: demotion and on-disk reuse.
|
|
||||||
cfg['cache'].update({'max_entries': 1, 'cold_enabled': True, 'cold_on_lru': True,
|
|
||||||
'cold_path': 'cache/age-regression-cold'})
|
|
||||||
cfg['cache'].pop('expose_only_hits', None)
|
|
||||||
rust.apply(cfg, 'cold-enabled-one-hot-entry')
|
|
||||||
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'MISS'
|
|
||||||
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'HIT'
|
|
||||||
assert request(rust.reverse_port, '/other.css')['x_cache'] == 'MISS'
|
|
||||||
deadline = time.monotonic() + 4
|
|
||||||
while True:
|
|
||||||
snapshot = dash(rust)
|
|
||||||
matches = [x for x in snapshot['objects'] if x['path'] == '/aged.css' and x['tier'] == 'cold']
|
|
||||||
if matches:
|
|
||||||
assert matches[0]['remaining_ttl_seconds'] >= 86_000, matches
|
|
||||||
break
|
|
||||||
if time.monotonic() > deadline:
|
|
||||||
raise AssertionError('aged transformed CSS never demoted to COLD: ' + str(snapshot))
|
|
||||||
time.sleep(.05)
|
|
||||||
cold_hit = request(rust.reverse_port, '/aged.css')
|
|
||||||
assert cold_hit['x_cache'] == 'HIT' and 'transformed' in cold_hit['cache_status'], cold_hit
|
|
||||||
outcome['tests'].append({'cold': {'row': matches[0], 'hit': cold_hit}})
|
|
||||||
outcome['passed'] = True
|
|
||||||
finally:
|
|
||||||
origin.shutdown()
|
|
||||||
origin.server_close()
|
|
||||||
write_json(output / 'cache-age-transform-result.json', outcome)
|
|
||||||
return outcome
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument('--binary', type=Path, required=True)
|
|
||||||
parser.add_argument('--output', type=Path, default=Path('/tmp/rustproxy-cache-age-034-live'))
|
|
||||||
parser.add_argument('--expect-old-age-bug', action='store_true')
|
|
||||||
args = parser.parse_args()
|
|
||||||
try:
|
|
||||||
report = run(args.binary.resolve(), args.output.resolve(), args.expect_old_age_bug)
|
|
||||||
except Exception as error:
|
|
||||||
print('FAIL', type(error).__name__, str(error)[:1000], file=sys.stderr)
|
|
||||||
raise
|
|
||||||
print('PASS', len(report['tests']), 'scenarios; see', args.output / 'cache-age-transform-result.json')
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Source-only hot-cache safety guards; NOT a Rust compile, concurrency test or RPS result."""
|
|
||||||
from pathlib import Path
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
read = lambda path: (ROOT / path).read_text()
|
|
||||||
hot = read("src/cache/lookup/hot.rs")
|
|
||||||
entry = read("src/cache/model/cache.rs")
|
|
||||||
model = read("src/cache/model/maintenance.rs")
|
|
||||||
inv = read("src/cache/invalidation/invalidate.rs")
|
|
||||||
dash = read("src/cache/dashboard.rs")
|
|
||||||
assert 'headers: Arc<HeaderMap>' in entry
|
|
||||||
assert 'Arc::clone(&entry.headers)' in hot
|
|
||||||
assert 'drop(store);' in hot
|
|
||||||
assert hot.index('drop(store);\n\n let mut response') < hot.index('headers.as_ref().clone()')
|
|
||||||
assert 'self.domain_stats.saved_body_hit(&ticket.scope, &ticket.domain, body_bytes)' in hot
|
|
||||||
assert 'store.lru.remove(&old_sequence)' in hot and 'store.lru.insert(sequence, storage_key)' in hot
|
|
||||||
assert '.sweep()' not in hot
|
|
||||||
assert 'fn sweep(' not in entry
|
|
||||||
stats = read('src/cache/stats.rs')
|
|
||||||
assert 'fn saved_body_hit(' in stats
|
|
||||||
assert 'const DOMAIN_STATS_LIMIT: usize = 4096' in stats
|
|
||||||
assert 'ArcSwap<PublishedDomains>' in stats
|
|
||||||
assert 'fn record_reason(&self, reason:' in stats
|
|
||||||
assert 'self.store.lock()' not in stats
|
|
||||||
assert 'if self.ban_rule_count.load(Ordering::Acquire) == 0' in inv
|
|
||||||
assert 'bans.iter().any(|rule| rule.matches_hot' in inv
|
|
||||||
assert 'fn matching_hot_key(' in model and 'entry.vary.is_empty()' in model
|
|
||||||
assert 'self.domain_stats.clear()' in dash
|
|
||||||
assert 'self.domain_stats.record(scope, domain, outcome, reason);' in dash
|
|
||||||
assert 'include!("tests/hot_hit_034_tests.rs")' in read('src/cache/tests.rs')
|
|
||||||
for path in (ROOT / 'src').rglob('*.rs'):
|
|
||||||
if '/tests/' not in path.as_posix() and path.name != 'tests.rs':
|
|
||||||
assert len(path.read_text().splitlines()) <= 500, path
|
|
||||||
print('PASS HOT path static invariants (no Rust compilation or throughput claim)')
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""0.34.0 source-wiring checks only; do not mistake these for Rust tests."""
|
|
||||||
from pathlib import Path
|
|
||||||
import tomllib
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parents[2]
|
|
||||||
read = lambda path: (root / path).read_text()
|
|
||||||
assert tomllib.loads(read("Cargo.toml"))["package"]["version"] == "0.34.0"
|
|
||||||
assert 'version = "0.34.0"' in read("Cargo.lock")
|
|
||||||
app = read("src/runtime/updates.rs")
|
|
||||||
assert "CacheInvalidation::Scopes(scopes)" in app
|
|
||||||
assert "this.cache.purge_scope(&scope)" in app
|
|
||||||
assert "this.cache.set_control_revision(envelope.revision)" in app
|
|
||||||
assert app.index("this.cache.set_control_revision(envelope.revision)") < app.index("match invalidation")
|
|
||||||
source = read("src/runtime/types.rs")
|
|
||||||
assert "view.load_balancing = Default::default();" in source
|
|
||||||
assert "member.max_concurrent_requests = 0;" in source
|
|
||||||
assert "fn vhost_cache_revision" in source
|
|
||||||
assert "&web_services" in source
|
|
||||||
assert "service.sendfile = crate::web_server::SendfileMode::Off;" in source
|
|
||||||
assert "&config.php_fpm_profiles" in source
|
|
||||||
for filename in ("src/proxy/forward/route_cache.rs", "src/proxy/forward/web_source.rs"):
|
|
||||||
assert "ticket_scoped_for_config(" in read(filename)
|
|
||||||
assert "snapshot.envelope.revision" in read(filename)
|
|
||||||
assert "expected_control_revision" in read("src/cache/tickets.rs")
|
|
||||||
assert "file.scope_revision != expected_scope_revision" in read("src/cache/cold/startup.rs")
|
|
||||||
assert "scope_revision: draft.scope_revision" in read("src/cache/cold/worker.rs")
|
|
||||||
body = read("src/proxy/body.rs")
|
|
||||||
assert "response_backend: Option<Arc<crate::balance::Lease>>" in body
|
|
||||||
assert "lease.bytes(count, false);" in body
|
|
||||||
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
|
|
||||||
assert "lease.bytes(count, true);" in body
|
|
||||||
assert "cold_scope_fingerprint_preserves_unmodified_vhost" in read("src/cache/tests/cold_tier_tests.rs")
|
|
||||||
assert "stale_snapshot_cannot_issue_cache_ticket" in read("src/cache/tests/scope_tests.rs")
|
|
||||||
for file in (root / 'src').rglob('*.rs'):
|
|
||||||
if '/tests/' in file.as_posix() or file.name == 'tests.rs':
|
|
||||||
continue
|
|
||||||
if len(file.read_text().splitlines()) > 500:
|
|
||||||
raise AssertionError(f"production Rust source exceeds 500 lines: {file}")
|
|
||||||
print('PASS source wiring for scoped invalidation, cold fingerprints, stale-snapshot barrier, response accounting and <=500-line modules (STATIC ONLY)')
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Overview traffic/connection heading cleanup against Chromium and the bundled JS.
|
|
||||||
|
|
||||||
Runs a simulated authenticated DOM rather than a live RustProxy service.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert match
|
|
||||||
body = ROUTER[match.end():]
|
|
||||||
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
|
|
||||||
result = []
|
|
||||||
for line in body.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
result.append((ROOT / 'src/admin' / include.group(1)).read_text(encoding='utf-8'))
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n' + '\n'.join(
|
|
||||||
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text(encoding='utf-8')
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
bundle = app_bundle()
|
|
||||||
for width in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 920})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=bundle)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
currentPanel = 'overview';
|
|
||||||
samples.splice(0); samples.push(1.5, 85.5, 12.3);
|
|
||||||
redrawOverviewHistoryCharts();
|
|
||||||
}''')
|
|
||||||
for density in ('normal', 'compact', 'ultra'):
|
|
||||||
for span in (3, 6):
|
|
||||||
data = page.evaluate('''({density,span}) => {
|
|
||||||
document.documentElement.dataset.dashboardDensity = density;
|
|
||||||
const visible = ['traffic_chart', 'connection_chart'];
|
|
||||||
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !visible.includes(id));
|
|
||||||
applyDashboardLayout([...visible,...hidden], hidden,
|
|
||||||
{traffic_chart:span,connection_chart:span});
|
|
||||||
const traffic = document.querySelector('[data-dashboard-widget="traffic_chart"]');
|
|
||||||
const connection = document.querySelector('[data-dashboard-widget="connection_chart"]');
|
|
||||||
const heading = traffic.querySelector('.card-heading');
|
|
||||||
const peak = traffic.querySelector('#chart-peak');
|
|
||||||
const peakRect = peak.getBoundingClientRect();
|
|
||||||
const cardRect = traffic.getBoundingClientRect();
|
|
||||||
const headingRect = heading.getBoundingClientRect();
|
|
||||||
return {
|
|
||||||
title: heading.querySelector('h2')?.textContent.trim(),
|
|
||||||
peak: peak.textContent,
|
|
||||||
peakInHeading: peak.parentElement === heading,
|
|
||||||
peakNotInFooter: !traffic.querySelector('.chart-footer #chart-peak'),
|
|
||||||
noTrafficEyebrow: !traffic.querySelector('.small-label'),
|
|
||||||
noTrafficLiveBadge: !traffic.querySelector('.pill.live'),
|
|
||||||
noConnectionLiveBadge: !connection.querySelector('.pill.live'),
|
|
||||||
connectionTitleIntact: !!connection.querySelector('.card-heading h3.overview-chart-heading[data-i18n="connectionActivity"]'),
|
|
||||||
peakInsideCard: peakRect.left >= cardRect.left - 1 && peakRect.right <= cardRect.right + 1,
|
|
||||||
peakInsideHeading: peakRect.top >= headingRect.top - 1 && peakRect.bottom <= headingRect.bottom + 1,
|
|
||||||
browserOverflow: document.documentElement.scrollWidth - innerWidth,
|
|
||||||
};
|
|
||||||
}''', {'density':density,'span':span})
|
|
||||||
assert data['title'] == 'Requests per second', (width, density, span, data)
|
|
||||||
assert data['peak'] == 'max 85.5 req/s', (width, density, span, data)
|
|
||||||
for key in ('peakInHeading', 'peakNotInFooter', 'noTrafficEyebrow',
|
|
||||||
'noTrafficLiveBadge', 'noConnectionLiveBadge',
|
|
||||||
'connectionTitleIntact', 'peakInsideCard', 'peakInsideHeading'):
|
|
||||||
assert data[key], (width, density, span, key, data)
|
|
||||||
assert data['browserOverflow'] <= 2, (width, density, span, data)
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print(f'PASS traffic/connection chart headers at {width}px (all densities, spans 3 and 6)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS Overview chart heading cleanup')
|
|
||||||
@@ -1,168 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""All chart titles and series labels in simulated Chromium (no live Rust server).
|
|
||||||
|
|
||||||
Covers Overview, diagnostics, Web Server, Traffic, Cache/Transform, Vhosts,
|
|
||||||
Streams, and DOM-created chart/legend variants. Preserve localized DOM text.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
CHECK = r'''(settings) => {
|
|
||||||
const { density, compactBody, theme } = settings;
|
|
||||||
document.documentElement.dataset.dashboardDensity = density;
|
|
||||||
document.documentElement.dataset.theme = theme;
|
|
||||||
document.body.dataset.density = compactBody ? 'compact' : 'comfortable';
|
|
||||||
|
|
||||||
const titleSelectors = [
|
|
||||||
'#panel-overview .overview-wide-chart .card-heading h2',
|
|
||||||
'#panel-overview .activity-card .card-heading h2',
|
|
||||||
'#panel-overview :is(.system-card, .connection-card) .overview-chart-heading',
|
|
||||||
'#panel-overview :is(.stat-chart, .stat-donut) > span:first-child',
|
|
||||||
'.mini-live-chart > .mini-live-chart-head > span:first-child',
|
|
||||||
'.traffic-chart-card > h3.subheading',
|
|
||||||
'.function-live-chart-heading > strong',
|
|
||||||
'.vhost-chart-head > span:first-child',
|
|
||||||
];
|
|
||||||
const monoSelectors = [
|
|
||||||
'.overview-wide-summary', '.overview-wide-legend',
|
|
||||||
'.overview-donut-legend', '.www-method-legend', '.stat-chart-legend',
|
|
||||||
'.mini-chart-legend', '.connection-legend',
|
|
||||||
'.traffic-protocol-legend', '.traffic-chart-legend',
|
|
||||||
'.function-live-chart-legend', '.stream-chart-legend',
|
|
||||||
'.chart-footer', '.vhost-chart-head > :not(:first-child)',
|
|
||||||
'.mini-live-chart-head > b', '.mini-live-chart-head > .mini-live-chart-actions',
|
|
||||||
'.network-interface-rate',
|
|
||||||
'.function-live-chart-summary > b', '.function-live-chart-heading small',
|
|
||||||
'.function-live-chart-samples',
|
|
||||||
'#panel-overview .stat-chart > small',
|
|
||||||
'#panel-overview .overview-wide-chart > .fine',
|
|
||||||
'#panel-overview .www-top-row', '.traffic-bar-row',
|
|
||||||
'#panel-overview .system-card .card-heading h2',
|
|
||||||
'#panel-overview .system-card .card-heading .meta',
|
|
||||||
'#panel-overview .system-card .memory-line',
|
|
||||||
'#panel-overview .connection-card .card-heading h2',
|
|
||||||
];
|
|
||||||
const legendSelectors = [
|
|
||||||
'.overview-wide-legend', '.overview-donut-legend', '.www-ring-legend',
|
|
||||||
'.www-method-legend', '.stat-chart-legend', '.mini-chart-legend',
|
|
||||||
'.connection-legend', '.traffic-protocol-legend', '.traffic-chart-legend',
|
|
||||||
'.function-live-chart-legend', '.stream-chart-legend', '.stream-traffic-legend'
|
|
||||||
];
|
|
||||||
const legendNodes = [...document.querySelectorAll(legendSelectors.join(','))];
|
|
||||||
const requestLegend = document.querySelector('[data-dashboard-widget="traffic_chart"] .traffic-protocol-legend');
|
|
||||||
const referenceLegendSize = getComputedStyle(requestLegend).fontSize;
|
|
||||||
const titleNodes = Array.from(document.querySelectorAll(titleSelectors.join(',')));
|
|
||||||
const monoNodes = Array.from(document.querySelectorAll(monoSelectors.join(',')));
|
|
||||||
const base = document.createElement('div');
|
|
||||||
base.style.fontFamily = 'var(--mono)';
|
|
||||||
document.body.append(base);
|
|
||||||
const mono = getComputedStyle(base).fontFamily;
|
|
||||||
base.style.fontFamily = 'var(--sans)';
|
|
||||||
const sans = getComputedStyle(base).fontFamily;
|
|
||||||
base.remove();
|
|
||||||
|
|
||||||
const titleErrors = titleNodes.flatMap(node => {
|
|
||||||
const css = getComputedStyle(node);
|
|
||||||
const inOverview = !!node.closest('#panel-overview');
|
|
||||||
const isSmallCaption = inOverview && !!node.closest('.mini-live-chart-head');
|
|
||||||
const wantedFont = isSmallCaption ? mono : sans;
|
|
||||||
const wantedSize = isSmallCaption ? '10px' : '12px';
|
|
||||||
const wantedCase = inOverview ? 'none' : 'uppercase';
|
|
||||||
const problems = [];
|
|
||||||
if (css.fontFamily !== wantedFont) problems.push(`font=${css.fontFamily}`);
|
|
||||||
if (css.fontSize !== wantedSize) problems.push(`size=${css.fontSize}`);
|
|
||||||
if (css.textTransform !== wantedCase) problems.push(`case=${css.textTransform}`);
|
|
||||||
return problems.map(problem => `${node.outerHTML.slice(0, 100)}: ${problem}`);
|
|
||||||
});
|
|
||||||
const monoErrors = monoNodes.flatMap(node => {
|
|
||||||
const all = [node, ...node.querySelectorAll('*')].filter(el => el.tagName.toLowerCase() !== 'svg');
|
|
||||||
return all.filter(el => getComputedStyle(el).fontFamily !== mono)
|
|
||||||
.map(el => `${el.outerHTML.slice(0, 100)}: ${getComputedStyle(el).fontFamily}`);
|
|
||||||
});
|
|
||||||
const legendErrors = legendNodes.flatMap(root => {
|
|
||||||
const children = [root, ...root.querySelectorAll('span, b, strong, small, li, em')];
|
|
||||||
return children.flatMap(node => {
|
|
||||||
const style = getComputedStyle(node);
|
|
||||||
const issues = [];
|
|
||||||
if (style.fontSize !== referenceLegendSize) issues.push(`size=${style.fontSize}`);
|
|
||||||
if (style.fontFamily !== mono) issues.push(`font=${style.fontFamily}`);
|
|
||||||
return issues.map(issue => `${node.outerHTML.slice(0, 95)} ${issue}`);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
// Keep localized Overview titles in sentence case; other sections retain
|
|
||||||
// their independent uppercase chart-heading convention.
|
|
||||||
const compression = document.querySelector('[data-dashboard-widget="wide_compression_activity"]');
|
|
||||||
const compressionHeading = compression.querySelector('.card-heading h2');
|
|
||||||
const compressionLegend = compression.querySelector('.overview-wide-legend');
|
|
||||||
const originalTranslation = compressionHeading.textContent.trim() === 'Aktywność kompresji';
|
|
||||||
const legendFonts = [...compressionLegend.querySelectorAll('span, b')]
|
|
||||||
.map(el => getComputedStyle(el).fontFamily);
|
|
||||||
const unrelatedHeading = document.querySelector('#panel-traffic .card-heading h2');
|
|
||||||
const numericReading = document.querySelector('#system-cpu');
|
|
||||||
return {
|
|
||||||
counts: {titles: titleNodes.length, mono: monoNodes.length, legends: legendNodes.length},
|
|
||||||
titleErrors, monoErrors, legendErrors, referenceLegendSize, mono, sans,
|
|
||||||
originalTranslation, compressionTitleSize: getComputedStyle(compressionHeading).fontSize,
|
|
||||||
compressionTitleCase: getComputedStyle(compressionHeading).textTransform,
|
|
||||||
compressionLegendIsMono: legendFonts.every(font => font === mono),
|
|
||||||
unrelatedTitleSize: getComputedStyle(unrelatedHeading).fontSize,
|
|
||||||
numericReadingIsMono: getComputedStyle(numericReading).fontFamily === mono,
|
|
||||||
};
|
|
||||||
}'''
|
|
||||||
|
|
||||||
with sync_playwright() as api:
|
|
||||||
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 960})
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
// Representative dynamically rendered Vhost/Web Service/Stream charts.
|
|
||||||
for (const panel of ['panel-vhosts','panel-web-server','panel-streams']) {
|
|
||||||
document.getElementById(panel).insertAdjacentHTML('beforeend', `
|
|
||||||
<div class="vhost-chart"><div class="vhost-chart-head">
|
|
||||||
<span class="small-label">Connection history</span>
|
|
||||||
<span class="fine">42 req/s</span></div>
|
|
||||||
<svg viewBox="0 0 300 54"><path d=""></path></svg>
|
|
||||||
<div class="stream-chart-legend"><span>RX 2 MB/s</span><span>TX 1 MB/s</span></div>
|
|
||||||
</div>`);
|
|
||||||
}
|
|
||||||
const comp = document.querySelector('[data-dashboard-widget="wide_compression_activity"] .card-heading h2');
|
|
||||||
comp.textContent = 'Aktywność kompresji';
|
|
||||||
// The traffic renderer inserts these series dynamically.
|
|
||||||
document.querySelectorAll('.traffic-chart-legend').forEach(root => {
|
|
||||||
root.innerHTML = `<li><span class="traffic-chart-label">GET requests</span><b>12</b></li>`;
|
|
||||||
});
|
|
||||||
const sample = document.createElement('div');
|
|
||||||
sample.innerHTML = `<div class="www-top-row"><span>php-worker</span><b>11/s</b></div>
|
|
||||||
<div class="traffic-bar-row"><span class="traffic-bar-label">GET</span><b class="traffic-bar-value">22</b></div>
|
|
||||||
<div class="network-interface-rate">100 Mb/s</div>`;
|
|
||||||
document.getElementById('panel-overview').append(sample);
|
|
||||||
}''')
|
|
||||||
for density in ('normal','compact','ultra'):
|
|
||||||
for compact_body in (False, True):
|
|
||||||
for theme in ('dark', 'light'):
|
|
||||||
result = page.evaluate(CHECK, {'density': density, 'compactBody': compact_body, 'theme': theme})
|
|
||||||
description = (width, density, compact_body, theme)
|
|
||||||
assert result['counts']['titles'] >= 43, (description, result['counts'])
|
|
||||||
assert result['counts']['mono'] >= 90, (description, result['counts'])
|
|
||||||
assert not result['titleErrors'], (description, result['titleErrors'][:10])
|
|
||||||
assert not result['monoErrors'], (description, result['monoErrors'][:10])
|
|
||||||
assert result['counts']['legends'] >= 39, (description, result['counts'])
|
|
||||||
assert result['referenceLegendSize'] == '9px', description
|
|
||||||
assert not result['legendErrors'], (description, result['legendErrors'][:10])
|
|
||||||
assert result['originalTranslation'], description
|
|
||||||
assert result['compressionTitleSize'] == '12px', description
|
|
||||||
assert result['compressionTitleCase'] == 'none', description
|
|
||||||
assert result['compressionLegendIsMono'], description
|
|
||||||
assert result['numericReadingIsMono'], description
|
|
||||||
assert result['unrelatedTitleSize'] != '12px', (description, result['unrelatedTitleSize'])
|
|
||||||
print(f'PASS all chart families at {width}px (3 dashboard densities, 2 interface densities, 2 themes)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Real-browser regression for Overview chart content when card width changes.
|
|
||||||
|
|
||||||
Runs the production HTML/CSS/assembled admin JS against a simulated DOM and
|
|
||||||
fake network-interface samples. This does not exercise Rust HTTP/SSE.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert match
|
|
||||||
body = ROUTER[match.end():]
|
|
||||||
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
|
|
||||||
scripts = []
|
|
||||||
for line in body.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
scripts.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
scripts.append('(() => {\n' + '\n'.join(
|
|
||||||
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
scripts.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(scripts)
|
|
||||||
|
|
||||||
|
|
||||||
def inspect(page):
|
|
||||||
return page.evaluate('''() => {
|
|
||||||
const selectors = {
|
|
||||||
'websocket_protocol':'[data-dashboard-widget="websocket_protocol"] svg',
|
|
||||||
'wide_transform_activity':'[data-dashboard-widget="wide_transform_activity"] svg',
|
|
||||||
'www_requests':'[data-dashboard-widget="www_requests"] svg',
|
|
||||||
'traffic_chart':'[data-dashboard-widget="traffic_chart"] svg',
|
|
||||||
'system_chart':'[data-dashboard-widget="system_chart"] svg',
|
|
||||||
'cache_diagnostics':'[data-dashboard-widget="cache_diagnostics"] .mini-live-chart svg',
|
|
||||||
'network_diagnostics':'[data-dashboard-widget="network_diagnostics"] .mini-live-chart svg',
|
|
||||||
'network_interface':'[data-dashboard-widget="network_diagnostics"] .network-interface-row svg',
|
|
||||||
};
|
|
||||||
return Object.fromEntries(Object.entries(selectors).map(([name, selector]) => {
|
|
||||||
const svg = document.querySelector(selector);
|
|
||||||
if (!svg) throw new Error('Missing SVG: ' + name);
|
|
||||||
const box = svg.getBoundingClientRect();
|
|
||||||
const matrix = svg.getScreenCTM();
|
|
||||||
const viewBox = svg.viewBox.baseVal;
|
|
||||||
const contentWidth = Math.hypot(matrix.a, matrix.b) * viewBox.width;
|
|
||||||
const series = svg.querySelector('.overview-wide-series');
|
|
||||||
return [name, {width:box.width, contentWidth,
|
|
||||||
preservation:svg.getAttribute('preserveAspectRatio'),
|
|
||||||
lineStroke: series ? getComputedStyle(series).vectorEffect : null}];
|
|
||||||
}));
|
|
||||||
}''')
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as browser_api:
|
|
||||||
browser = browser_api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for screen_width in (320, 375, 800, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': screen_width, 'height': 920})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=app_bundle())
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
currentPanel = 'overview';
|
|
||||||
renderNetworkInterfaceCharts([{name:'eth0',rx_bytes_per_second:12500,tx_bytes_per_second:5200}], true);
|
|
||||||
}''')
|
|
||||||
ids = ['websocket_protocol', 'wide_transform_activity', 'www_requests', 'traffic_chart',
|
|
||||||
'system_chart', 'cache_diagnostics', 'network_diagnostics']
|
|
||||||
results = []
|
|
||||||
for span in (3, 12):
|
|
||||||
page.evaluate('''({ids,span}) => {
|
|
||||||
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id));
|
|
||||||
const sizes = Object.fromEntries(ids.map(id => [id, span]));
|
|
||||||
applyDashboardLayout([...ids, ...hidden], hidden, sizes);
|
|
||||||
}''', {'ids': ids, 'span': span})
|
|
||||||
measured = inspect(page)
|
|
||||||
results.append(measured)
|
|
||||||
for chart, value in measured.items():
|
|
||||||
assert value['width'] > 50, (screen_width, span, chart, value)
|
|
||||||
assert abs(value['width'] - value['contentWidth']) <= 2, (
|
|
||||||
'chart not filling its own SVG viewport', screen_width, span, chart, value)
|
|
||||||
assert value['preservation'] == 'none', (screen_width, chart, value)
|
|
||||||
if value['lineStroke'] is not None:
|
|
||||||
assert value['lineStroke'] == 'non-scaling-stroke', (chart, value)
|
|
||||||
if screen_width >= 1440:
|
|
||||||
for name in results[0]:
|
|
||||||
assert results[1][name]['width'] > results[0][name]['width'] * 1.4, (
|
|
||||||
'chart does not grow when widening card', screen_width, name, results)
|
|
||||||
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, screen_width
|
|
||||||
assert not errors, (screen_width, errors)
|
|
||||||
print('PASS SVG chart uses full width at spans 3 and 12:', screen_width)
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS resizable Overview chart content regression')
|
|
||||||
@@ -1,128 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Legacy CPU plot regression: low and high real values on a fixed 0-100% axis.
|
|
||||||
|
|
||||||
Autoscaling was intentionally replaced by 0.33.5 fixed scaling. This test
|
|
||||||
retains the original history/reset and label checks using the current scale.
|
|
||||||
Chromium is simulated, not a live RustProxy server.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
|
|
||||||
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
|
|
||||||
def inspect(page):
|
|
||||||
return page.evaluate('''() => {
|
|
||||||
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
|
|
||||||
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
|
|
||||||
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
|
|
||||||
const backendSvg = get('backend_traffic').querySelector('svg');
|
|
||||||
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
|
|
||||||
const bounds = cpu.getBoundingClientRect();
|
|
||||||
const matrix = cpuSvg.getScreenCTM();
|
|
||||||
return {
|
|
||||||
cpuHeight: cpuSvg.getBoundingClientRect().height,
|
|
||||||
protocolHeight: protocolSvg.getBoundingClientRect().height,
|
|
||||||
backendHeight: backendSvg.getBoundingClientRect().height,
|
|
||||||
svgWidth: cpuSvg.getBoundingClientRect().width,
|
|
||||||
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
|
|
||||||
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
|
|
||||||
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
|
|
||||||
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
|
|
||||||
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
|
|
||||||
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
|
|
||||||
overflow: document.documentElement.scrollWidth - innerWidth,
|
|
||||||
};
|
|
||||||
}''')
|
|
||||||
|
|
||||||
|
|
||||||
def render_cpu(page, revision, system, process):
|
|
||||||
return page.evaluate('''({revision,system,process}) => {
|
|
||||||
window.rustProxyWideOverview.render({revision,live_system:{
|
|
||||||
cpu_percent:system, process_cpu_percent:process
|
|
||||||
}});
|
|
||||||
const read = name => {
|
|
||||||
const prefix = `overview-wide-wide_cpu-${name}`;
|
|
||||||
return {
|
|
||||||
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
|
|
||||||
label: document.getElementById(`${prefix}-value`).textContent,
|
|
||||||
};
|
|
||||||
};
|
|
||||||
return {system:read('system'),process:read('process')};
|
|
||||||
}''', {'revision': revision, 'system': system, 'process': process})
|
|
||||||
|
|
||||||
|
|
||||||
def ys(path):
|
|
||||||
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for viewport in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': viewport, 'height': 950})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
const grid = document.getElementById('dashboard-overview-grid');
|
|
||||||
grid.classList.add('is-aligned');
|
|
||||||
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
|
|
||||||
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic'].includes(card.dataset.dashboardWidget);
|
|
||||||
}
|
|
||||||
window.currentPanel = 'overview';
|
|
||||||
window.$ = id => document.getElementById(id);
|
|
||||||
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
|
|
||||||
window.fmtBytes = value => `${value} B`;
|
|
||||||
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
|
|
||||||
}''')
|
|
||||||
page.add_script_tag(content=SCALE_JS)
|
|
||||||
page.add_script_tag(content=WIDE_JS)
|
|
||||||
|
|
||||||
for density in ('compact', 'normal', 'ultra'):
|
|
||||||
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
|
|
||||||
result = inspect(page)
|
|
||||||
assert result['protocolHeight'] >= 124, (viewport, density, result)
|
|
||||||
assert abs(result['cpuHeight'] - result['protocolHeight']) <= 1, (
|
|
||||||
'CPU must use standard chart height', viewport, density, result)
|
|
||||||
assert result['backendHeight'] > 0, (viewport, density, result)
|
|
||||||
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
|
|
||||||
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
|
|
||||||
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
|
|
||||||
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
|
|
||||||
|
|
||||||
# Measured 0.5-0.8% stay numerically visible without axis auto-expansion.
|
|
||||||
first = render_cpu(page, 1, 0.5, 0.2)
|
|
||||||
second = render_cpu(page, 1, 0.8, 0.6)
|
|
||||||
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
|
|
||||||
assert ys(second['system']['path']) == [136, 136], (viewport, second)
|
|
||||||
assert ys(second['process']['path']) == [137, 136], (viewport, second)
|
|
||||||
|
|
||||||
# Multi-core values display accurately but clip at the fixed 100% ceiling.
|
|
||||||
third = render_cpu(page, 1, 3.4, 2.2)
|
|
||||||
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
|
|
||||||
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
|
|
||||||
high = render_cpu(page, 1, 250, 123)
|
|
||||||
assert high['system']['label'] == '250.0%'
|
|
||||||
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
|
|
||||||
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
|
|
||||||
|
|
||||||
# Revision changes reset the history. Zero stays at the plot baseline.
|
|
||||||
idle = render_cpu(page, 2, 0, 0)
|
|
||||||
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
|
|
||||||
missing = render_cpu(page, 3, None, None)
|
|
||||||
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
|
|
||||||
assert not errors, (viewport, errors)
|
|
||||||
print(f'PASS legacy CPU labels/history on fixed Y-scale at {viewport}px (all densities)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""0.33.5 fixed 0-100% CPU plot and height regression with production HTML/CSS/JS.
|
|
||||||
|
|
||||||
Uses Chromium with simulated authenticated DOM and status samples, not a live
|
|
||||||
RustProxy server, SSE stream or real CPU telemetry.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
|
|
||||||
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
|
|
||||||
def inspect(page):
|
|
||||||
return page.evaluate('''() => {
|
|
||||||
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
|
|
||||||
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
|
|
||||||
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
|
|
||||||
const connection = get('connection_chart');
|
|
||||||
const backendSvg = get('backend_traffic').querySelector('svg');
|
|
||||||
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
|
|
||||||
const bounds = cpu.getBoundingClientRect();
|
|
||||||
const matrix = cpuSvg.getScreenCTM();
|
|
||||||
return {
|
|
||||||
cpuHeight: cpuSvg.getBoundingClientRect().height,
|
|
||||||
cpuCardHeight: bounds.height,
|
|
||||||
connectionCardHeight: connection.getBoundingClientRect().height,
|
|
||||||
protocolHeight: protocolSvg.getBoundingClientRect().height,
|
|
||||||
backendHeight: backendSvg.getBoundingClientRect().height,
|
|
||||||
svgWidth: cpuSvg.getBoundingClientRect().width,
|
|
||||||
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
|
|
||||||
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
|
|
||||||
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
|
|
||||||
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
|
|
||||||
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
|
|
||||||
gridPath: cpuSvg.querySelector('.chart-grid').getAttribute('d'),
|
|
||||||
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
|
|
||||||
overflow: document.documentElement.scrollWidth - innerWidth,
|
|
||||||
};
|
|
||||||
}''')
|
|
||||||
|
|
||||||
|
|
||||||
def render_cpu(page, revision, system, process):
|
|
||||||
return page.evaluate('''({revision,system,process}) => {
|
|
||||||
window.rustProxyWideOverview.render({revision,live_system:{
|
|
||||||
cpu_percent:system, process_cpu_percent:process
|
|
||||||
}});
|
|
||||||
const read = name => {
|
|
||||||
const prefix = `overview-wide-wide_cpu-${name}`;
|
|
||||||
return {
|
|
||||||
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
|
|
||||||
label: document.getElementById(`${prefix}-value`).textContent,
|
|
||||||
};
|
|
||||||
};
|
|
||||||
return {system:read('system'),process:read('process')};
|
|
||||||
}''', {'revision': revision, 'system': system, 'process': process})
|
|
||||||
|
|
||||||
|
|
||||||
def ys(path):
|
|
||||||
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for viewport in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': viewport, 'height': 950})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
const grid = document.getElementById('dashboard-overview-grid');
|
|
||||||
grid.classList.add('is-aligned');
|
|
||||||
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
|
|
||||||
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic', 'connection_chart'].includes(card.dataset.dashboardWidget);
|
|
||||||
}
|
|
||||||
window.currentPanel = 'overview';
|
|
||||||
window.$ = id => document.getElementById(id);
|
|
||||||
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
|
|
||||||
window.fmtBytes = value => `${value} B`;
|
|
||||||
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
|
|
||||||
}''')
|
|
||||||
page.add_script_tag(content=SCALE_JS)
|
|
||||||
page.add_script_tag(content=WIDE_JS)
|
|
||||||
|
|
||||||
for density in ('compact', 'normal', 'ultra'):
|
|
||||||
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
|
|
||||||
result = inspect(page)
|
|
||||||
assert result['protocolHeight'] >= 124, (viewport, density, result)
|
|
||||||
assert abs(result['cpuHeight'] - 125) <= 1, (
|
|
||||||
'CPU height must remain fixed at 125px', viewport, density, result)
|
|
||||||
assert result['backendHeight'] > 0, (viewport, density, result)
|
|
||||||
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
|
|
||||||
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
|
|
||||||
assert result['gridPath'] == 'M0 47H600 M0 77H600 M0 107H600', result
|
|
||||||
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
|
|
||||||
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
|
|
||||||
|
|
||||||
# Low measured CPU values stay near zero instead of filling the plot.
|
|
||||||
first = render_cpu(page, 1, 0.5, 0.2)
|
|
||||||
second = render_cpu(page, 1, 0.8, 0.6)
|
|
||||||
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
|
|
||||||
assert ys(second['system']['path']) == [136, 136], (viewport, second)
|
|
||||||
assert ys(second['process']['path']) == [137, 136], (viewport, second)
|
|
||||||
|
|
||||||
# The Y axis is always 0-100% regardless of recorded history.
|
|
||||||
third = render_cpu(page, 1, 3.4, 2.2)
|
|
||||||
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
|
|
||||||
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
|
|
||||||
high = render_cpu(page, 1, 250, 123)
|
|
||||||
assert high['system']['label'] == '250.0%' and high['process']['label'] == '123.0%'
|
|
||||||
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
|
|
||||||
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
|
|
||||||
mid = render_cpu(page, 1, 50, 25)
|
|
||||||
assert ys(mid['system']['path'])[-1] == 77, (viewport, mid)
|
|
||||||
assert ys(mid['process']['path'])[-1] == 107, (viewport, mid)
|
|
||||||
|
|
||||||
# Revision changes reset history; missing readings do not plot a zero.
|
|
||||||
idle = render_cpu(page, 2, 0, 0)
|
|
||||||
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
|
|
||||||
full = render_cpu(page, 2, 100, 100)
|
|
||||||
assert ys(full['system']['path'])[-1] == 17 and ys(full['process']['path'])[-1] == 17
|
|
||||||
missing = render_cpu(page, 3, None, None)
|
|
||||||
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
|
|
||||||
|
|
||||||
# Samples may change paths, but not the plot/card or neighboring card sizes.
|
|
||||||
for density in ('compact', 'normal', 'ultra'):
|
|
||||||
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
|
|
||||||
unchanged = inspect(page)
|
|
||||||
assert abs(unchanged['cpuHeight'] - 125) <= 1, (viewport, density, unchanged)
|
|
||||||
for value in (0.1, 99.9, 250, 0.0):
|
|
||||||
before = inspect(page)
|
|
||||||
render_cpu(page, 4, value, value)
|
|
||||||
after = inspect(page)
|
|
||||||
for key in ('cpuHeight', 'cpuCardHeight', 'connectionCardHeight'):
|
|
||||||
assert abs(before[key] - after[key]) < 1, (viewport, density, value, key, before, after)
|
|
||||||
assert not errors, (viewport, errors)
|
|
||||||
print(f'PASS CPU fixed 0-100% Y-scale and 125px plot height at {viewport}px (all densities)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
@@ -1,199 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
|
|
||||||
|
|
||||||
The authenticated app bundle runs against simulated status/preferences; this is
|
|
||||||
not a Rust server integration test.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
|
|
||||||
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert match
|
|
||||||
body = ROUTER[match.end():]
|
|
||||||
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
|
|
||||||
result = []
|
|
||||||
for line in body.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
|
|
||||||
|
|
||||||
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
|
|
||||||
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
|
|
||||||
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
|
|
||||||
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
|
|
||||||
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
|
|
||||||
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
|
|
||||||
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
BUNDLE = app_bundle()
|
|
||||||
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
|
|
||||||
|
|
||||||
|
|
||||||
def load(page):
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=BUNDLE)
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
|
|
||||||
|
|
||||||
|
|
||||||
def inspect(page, ids):
|
|
||||||
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
|
|
||||||
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
|
|
||||||
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
|
|
||||||
}))''', ids)
|
|
||||||
|
|
||||||
|
|
||||||
def assert_visible(page, visible, hidden):
|
|
||||||
result = inspect(page, visible + hidden)
|
|
||||||
for widget in visible:
|
|
||||||
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
|
|
||||||
for widget in hidden:
|
|
||||||
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in [320, 375, 800, 1440]:
|
|
||||||
context = browser.new_context(viewport={'width': width, 'height': 950})
|
|
||||||
page = context.new_page()
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda e: errors.append(str(e)))
|
|
||||||
load(page)
|
|
||||||
result = page.evaluate('''() => {
|
|
||||||
const prefs = ADMIN_APPEARANCE_DEFAULTS;
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
document.getElementById('overview-layout-unlock').hidden = false;
|
|
||||||
currentPanel = 'overview';
|
|
||||||
window.__savedDashboard = JSON.parse(JSON.stringify(prefs));
|
|
||||||
window.rustProxyPreferences.load = async () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.current = () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.update = async patch => {
|
|
||||||
window.__savedDashboard = {...window.__savedDashboard, ...patch};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
return window.__savedDashboard;
|
|
||||||
};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
|
|
||||||
appConfirm = async () => true;
|
|
||||||
return {
|
|
||||||
order: ADMIN_DASHBOARD_WIDGETS,
|
|
||||||
hidden: ADMIN_DEFAULT_HIDDEN_WIDGETS,
|
|
||||||
density: document.documentElement.dataset.dashboardDensity,
|
|
||||||
samples:window.rustProxyOverviewSamples.getLimit(),
|
|
||||||
serviceCount:document.querySelectorAll('#dashboard-overview-grid [data-dashboard-widget]').length,
|
|
||||||
};
|
|
||||||
}''')
|
|
||||||
assert result['order'] == PRESET['dashboard_widgets'], (width, result['order'])
|
|
||||||
assert result['hidden'] == PRESET['dashboard_hidden_widgets'], (width, result['hidden'])
|
|
||||||
assert result['samples'] == 600 and result['density'] == 'compact' and result['serviceCount'] == 78, result
|
|
||||||
assert_visible(page, ['backend_traffic', 'extra_response_latency', 'wide_transform_activity'],
|
|
||||||
['backend_health', 'stream_sessions', 'www_status', 'bandwidth'])
|
|
||||||
|
|
||||||
# Reproduce the previously flaky operation: opt in hidden backend +
|
|
||||||
# Stream cards and opt out a visible backend card, WITHOUT saving yet.
|
|
||||||
page.locator('#overview-layout-unlock').click()
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
outcome = page.evaluate('''() => {
|
|
||||||
const add = id => {
|
|
||||||
const label = dashboardWidgetLabel(id);
|
|
||||||
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
|
|
||||||
.find(node => node.firstElementChild.textContent === label);
|
|
||||||
if (!item) throw new Error('no hidden card to restore: ' + id);
|
|
||||||
item.querySelector('button').click();
|
|
||||||
};
|
|
||||||
add('backend_health');
|
|
||||||
add('stream_sessions');
|
|
||||||
const card = document.querySelector('[data-dashboard-widget="backend_traffic"]');
|
|
||||||
card.querySelector('.overview-inline-hide').click();
|
|
||||||
return window.rustProxyInlineLayout.currentDraft();
|
|
||||||
}''')
|
|
||||||
assert 'backend_traffic' in outcome['hidden'] and 'backend_health' not in outcome['hidden']
|
|
||||||
assert 'stream_sessions' not in outcome['hidden']
|
|
||||||
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
|
|
||||||
for _ in range(5):
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const status = {revision:4,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
|
|
||||||
window.rustProxyBackendDashboard.render(status);
|
|
||||||
window.rustProxyStreamDashboard.render(status);
|
|
||||||
}''')
|
|
||||||
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
|
|
||||||
# Unrelated settings update must not repaint the saved layout over the draft.
|
|
||||||
page.evaluate('''() => applyAdminPreferences({...window.__savedDashboard, ui_scale:105})''')
|
|
||||||
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
|
|
||||||
assert page.evaluate('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
page.locator('#overview-inline-cancel').click()
|
|
||||||
assert not page.evaluate('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions'])
|
|
||||||
|
|
||||||
# Save under live traffic, ensure the in-flight preference application
|
|
||||||
# uses the current draft before the editor exits.
|
|
||||||
page.locator('#overview-layout-unlock').click()
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const label = dashboardWidgetLabel('backend_health');
|
|
||||||
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
|
|
||||||
.find(node => node.firstElementChild.textContent === label);
|
|
||||||
item.querySelector('button').click();
|
|
||||||
document.querySelector('[data-dashboard-widget="backend_traffic"] .overview-inline-hide').click();
|
|
||||||
}''')
|
|
||||||
page.locator('#overview-inline-save').click()
|
|
||||||
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
|
|
||||||
assert_visible(page, ['backend_health'], ['backend_traffic'])
|
|
||||||
saved = page.evaluate('''() => window.__savedDashboard.dashboard_hidden_widgets''')
|
|
||||||
assert 'backend_traffic' in saved and 'backend_health' not in saved
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const status={backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
|
|
||||||
window.rustProxyBackendDashboard.render(status);
|
|
||||||
window.rustProxyStreamDashboard.render(status);
|
|
||||||
}''')
|
|
||||||
assert_visible(page, ['backend_health'], ['backend_traffic'])
|
|
||||||
# Restore factory layout through the new named-presets modal.
|
|
||||||
page.locator('#overview-layout-open').evaluate('(node) => node.hidden = false')
|
|
||||||
page.locator('#overview-layout-open').click()
|
|
||||||
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
|
|
||||||
page.locator('#overview-presets-factory').click()
|
|
||||||
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
|
|
||||||
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions', 'www_status'])
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_hidden_widgets') == PRESET['dashboard_hidden_widgets']
|
|
||||||
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
context.close()
|
|
||||||
print('PASS inline layout defaults, add/hide, live ticks, other preferences, cancel/save', width)
|
|
||||||
# An already-persisted sample choice must override the factory 600.
|
|
||||||
context = browser.new_context()
|
|
||||||
page = context.new_page()
|
|
||||||
# This sandbox blocks all network origins; stub browser storage before
|
|
||||||
# loading the bundle rather than navigating to a real host.
|
|
||||||
page.evaluate("""() => Object.defineProperty(window, 'localStorage', {
|
|
||||||
configurable: true,
|
|
||||||
value: {getItem: key => key === 'rustproxy.overview.liveChartSamples' ? '120' : null,
|
|
||||||
setItem: () => undefined},
|
|
||||||
})""")
|
|
||||||
load(page)
|
|
||||||
assert page.evaluate('window.rustProxyOverviewSamples.getLimit()') == 120
|
|
||||||
context.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS RustProxy 0.33.1 simulated browser layout regression')
|
|
||||||
@@ -1,134 +0,0 @@
|
|||||||
import { renderAdminIndex } from './panel_html.mjs';
|
|
||||||
// Deterministic Overview widget regression; simulated DOM/status, no running API.
|
|
||||||
import assert from 'node:assert/strict';
|
|
||||||
import fs from 'node:fs';
|
|
||||||
import path from 'node:path';
|
|
||||||
import vm from 'node:vm';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
|
||||||
const html = renderAdminIndex(root);
|
|
||||||
const registry = JSON.parse(fs.readFileSync(path.join(root, 'src/admin/web/dashboard-widgets.json'), 'utf8'));
|
|
||||||
const lang = JSON.parse(fs.readFileSync(path.join(root, 'langs/en_US.json'), 'utf8')).translations;
|
|
||||||
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
|
|
||||||
const widgets = new Map(), ids = new Map();
|
|
||||||
class Element {
|
|
||||||
constructor() { this.hidden = false; this.attributes = new Map(); this.style = {}; this.textContent = '-'; }
|
|
||||||
setAttribute(key, value) { this.attributes.set(key, String(value)); }
|
|
||||||
removeAttribute(key) { this.attributes.delete(key); }
|
|
||||||
getAttribute(key) { return this.attributes.get(key) ?? null; }
|
|
||||||
}
|
|
||||||
for (const [, id] of html.matchAll(/\bid="([^"]+)"/g)) {
|
|
||||||
assert.ok(!ids.has(id), `duplicate DOM id: ${id}`);
|
|
||||||
ids.set(id, new Element());
|
|
||||||
}
|
|
||||||
for (const [, id] of html.matchAll(/data-dashboard-widget="([^"]+)"/g)) widgets.set(id, new Element());
|
|
||||||
assert.equal(registry.length, widgets.size, 'registry and DOM widget count must match');
|
|
||||||
for (const entry of registry) assert.ok(widgets.has(entry.id), `registry references missing DOM card ${entry.id}`);
|
|
||||||
let now = 1000;
|
|
||||||
const document = {
|
|
||||||
hidden: false,
|
|
||||||
addEventListener() {},
|
|
||||||
querySelector(query) { return widgets.get(query.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) ?? null; },
|
|
||||||
};
|
|
||||||
const window = { addEventListener() {}, rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
|
|
||||||
const context = vm.createContext({
|
|
||||||
window, document, performance: { now: () => now },
|
|
||||||
currentPanel: 'overview', $: id => ids.get(id) ?? null,
|
|
||||||
fmt: (v, digits = 0) => Number(v).toFixed(digits),
|
|
||||||
fmtBytes: v => `${Number(v)} B`,
|
|
||||||
t: key => lang[key] || key,
|
|
||||||
console,
|
|
||||||
});
|
|
||||||
for (const script of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-extra-metrics.js', 'app/overview-wide-metrics.js']) {
|
|
||||||
vm.runInContext(code(script), context, { filename: script });
|
|
||||||
}
|
|
||||||
const extra = window.rustProxyExtraOverview, wide = window.rustProxyWideOverview;
|
|
||||||
const status = {
|
|
||||||
revision: 9,
|
|
||||||
system: { open_fds: 60, max_fds: 300 },
|
|
||||||
cache: { hits: 100, misses: 50, resident_bytes: 150, hot_limit_bytes: 300,
|
|
||||||
cold_queue_bytes: 25, cold_queue_limit_bytes: 100, cold_bytes: 400, cold_limit_bytes: 1000 },
|
|
||||||
metrics: { requests: 300, http1_requests: 200, http2_requests: 80, http3_requests: 20,
|
|
||||||
compression_gzip: 10, compression_br: 10, compression_zstd: 10, compression_deflate: 1 },
|
|
||||||
transform_metrics: { applied: 10, cache_hits: 10, skipped: 10, errors: 1 },
|
|
||||||
};
|
|
||||||
extra.render(status);
|
|
||||||
wide.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cache_memory-meter-fill').style.width, '50%');
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cold_queue_memory-meter-fill').style.width, '25%');
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '40%');
|
|
||||||
widgets.get('extra_cache_memory').hidden = false;
|
|
||||||
widgets.get('extra_cache_hit_rate').hidden = false;
|
|
||||||
widgets.get('extra_h2_share').hidden = false;
|
|
||||||
widgets.get('extra_h1_share').hidden = false;
|
|
||||||
widgets.get('extra_h3_share').hidden = false;
|
|
||||||
widgets.get('extra_process_fds').hidden = false;
|
|
||||||
extra.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_process_fds-meter-fill').style.width, '20%');
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-'); // first counter sample has no rate
|
|
||||||
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '-');
|
|
||||||
|
|
||||||
now = 4000;
|
|
||||||
status.cache.hits += 30;
|
|
||||||
status.cache.misses += 10;
|
|
||||||
status.metrics.requests += 40;
|
|
||||||
status.metrics.http1_requests += 10;
|
|
||||||
status.metrics.http2_requests += 20;
|
|
||||||
status.metrics.http3_requests += 10;
|
|
||||||
status.metrics.compression_gzip += 3;
|
|
||||||
status.metrics.compression_br += 6;
|
|
||||||
status.metrics.compression_zstd += 9;
|
|
||||||
status.metrics.compression_deflate += 12;
|
|
||||||
status.transform_metrics.applied += 15;
|
|
||||||
status.transform_metrics.cache_hits += 3;
|
|
||||||
status.transform_metrics.skipped += 6;
|
|
||||||
status.transform_metrics.errors += 1;
|
|
||||||
extra.render(status);
|
|
||||||
wide.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '75.0%');
|
|
||||||
assert.equal(ids.get('extra-cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
|
|
||||||
assert.equal(ids.get('extra-cache-ratio-miss-arc').getAttribute('stroke-dashoffset'), '-75');
|
|
||||||
assert.equal(ids.get('extra-cache-ratio-hit-count').textContent, '30');
|
|
||||||
assert.equal(ids.get('extra-cache-ratio-miss-count').textContent, '10');
|
|
||||||
assert.equal(ids.get('overview-extra-extra_h2_share-meter-fill').style.width, '50%');
|
|
||||||
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '8.3/s');
|
|
||||||
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '10.0/s');
|
|
||||||
for (const name of ['applied', 'cache_hits', 'skipped', 'errors'])
|
|
||||||
assert.ok(ids.get(`overview-wide-wide_transform_activity-${name}-line`).getAttribute('d')?.length);
|
|
||||||
for (const name of ['gzip', 'br', 'zstd', 'deflate'])
|
|
||||||
assert.ok(ids.get(`overview-wide-wide_compression_activity-${name}-line`).getAttribute('d')?.length);
|
|
||||||
|
|
||||||
// No hidden-tab spike or fake zero when a long interval is skipped.
|
|
||||||
now = 100000;
|
|
||||||
status.cache.hits += 300;
|
|
||||||
status.cache.misses += 200;
|
|
||||||
status.metrics.compression_gzip += 600;
|
|
||||||
extra.render(status);
|
|
||||||
wide.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-');
|
|
||||||
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
|
|
||||||
now = 103000;
|
|
||||||
status.cache.hits += 5;
|
|
||||||
status.cache.misses += 5;
|
|
||||||
status.metrics.compression_gzip += 3;
|
|
||||||
extra.render(status);
|
|
||||||
wide.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '50.0%');
|
|
||||||
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '1.0/s');
|
|
||||||
|
|
||||||
// Bounded metric with no configured budget remains unknown, not a made-up percentage.
|
|
||||||
status.cache.cold_limit_bytes = 0;
|
|
||||||
extra.render(status);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cold_storage').textContent, '400 B');
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cold_storage-meter').getAttribute('aria-valuenow'), null);
|
|
||||||
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '0%');
|
|
||||||
|
|
||||||
extra.donut('cache-ratio', 75, 25);
|
|
||||||
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
|
|
||||||
assert.equal(ids.get('cache-ratio-chart').getAttribute('aria-label'), 'HIT 75, MISS 25');
|
|
||||||
extra.donut('cache-ratio', 0, 0);
|
|
||||||
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '0 100');
|
|
||||||
assert.ok(code('app/runtime.js').includes('rustProxyExtraOverview?.donut?.("cache-ratio"'),
|
|
||||||
'main HIT/MISS tile must share tested donut rendering');
|
|
||||||
console.log('PASS: bounded meters, COLD tiers, HIT/MISS rings, transform and compression rates, pause/no-limit');
|
|
||||||
@@ -1,176 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Regression: named Overview presets, responsive modal and browser actions.
|
|
||||||
|
|
||||||
Runs the real panel DOM/CSS/bundled JavaScript with mocked preferences/API.
|
|
||||||
This is browser layout testing, not Rust server acceptance.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
PACKS = {
|
|
||||||
name: {'id': name, **json.loads((ROOT / 'langs' / (name + '.json')).read_text())}
|
|
||||||
for name in ('en_US', 'pl_pl')
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert match
|
|
||||||
body = ROUTER[match.end():]
|
|
||||||
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
|
|
||||||
result = []
|
|
||||||
for line in body.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
|
|
||||||
|
|
||||||
BUNDLE = app_bundle()
|
|
||||||
|
|
||||||
|
|
||||||
def load(page):
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=BUNDLE)
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyDashboardLayout')
|
|
||||||
page.evaluate('''packs => {
|
|
||||||
languagePacks = new Map(Object.entries(packs));
|
|
||||||
defaultLanguage = 'en_US';
|
|
||||||
language = 'pl_pl';
|
|
||||||
document.documentElement.lang = 'pl-PL';
|
|
||||||
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
|
|
||||||
}''', PACKS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
document.getElementById('overview-layout-unlock').hidden = false;
|
|
||||||
document.getElementById('overview-layout-open').hidden = false;
|
|
||||||
currentPanel = 'overview';
|
|
||||||
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
|
|
||||||
window.rustProxyPreferences.load = async () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.current = () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.update = async patch => {
|
|
||||||
window.__savedDashboard = {...window.__savedDashboard, ...patch};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
return window.__savedDashboard;
|
|
||||||
};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
language = 'pl_pl';
|
|
||||||
document.documentElement.lang = 'pl-PL';
|
|
||||||
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
|
|
||||||
}''')
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Regression for the replacement named-presets modal, with a simulated SQLite API.
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 800, 1440):
|
|
||||||
context = browser.new_context(viewport={'width': width, 'height': 880})
|
|
||||||
page = context.new_page()
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
load(page)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const saved = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
|
|
||||||
window.__savedDashboard = saved;
|
|
||||||
window.__presets = new Map();
|
|
||||||
window.rustProxyPreferences.load = async () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.current = () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.update = async patch => {
|
|
||||||
window.__savedDashboard = {...window.__savedDashboard, ...patch};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
return window.__savedDashboard;
|
|
||||||
};
|
|
||||||
api = async (path, options={}) => {
|
|
||||||
const method = options.method || 'GET';
|
|
||||||
if (path === '/api/dashboard-presets' && method === 'GET')
|
|
||||||
return [...window.__presets.entries()].map(([name, data]) => ({ name, ...structuredClone(data) }));
|
|
||||||
if (path.startsWith('/api/dashboard-presets/')) {
|
|
||||||
const name = decodeURIComponent(path.split('/').slice(3).join('/'));
|
|
||||||
if (method === 'PUT') { window.__presets.set(name, JSON.parse(options.body)); return {}; }
|
|
||||||
if (method === 'DELETE') { window.__presets.delete(name); return {}; }
|
|
||||||
if (method === 'PATCH') {
|
|
||||||
const changed = JSON.parse(options.body).name;
|
|
||||||
if (window.__presets.has(changed)) throw new Error('already exists');
|
|
||||||
window.__presets.set(changed, window.__presets.get(name));
|
|
||||||
window.__presets.delete(name);
|
|
||||||
return {};
|
|
||||||
}
|
|
||||||
}
|
|
||||||
throw new Error('unexpected API: ' + method + ' ' + path);
|
|
||||||
};
|
|
||||||
appConfirm = async () => true;
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
document.getElementById('overview-layout-open').hidden = false;
|
|
||||||
}''')
|
|
||||||
page.locator('#overview-layout-open').click()
|
|
||||||
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
|
|
||||||
assert page.locator('#overview-presets-list .overview-presets-row').count() == 0
|
|
||||||
widths = page.evaluate('''() => {
|
|
||||||
const d = document.getElementById('overview-presets-dialog');
|
|
||||||
const r = d.getBoundingClientRect();
|
|
||||||
return {left:r.left,right:r.right,overflow:d.scrollWidth - d.clientWidth};
|
|
||||||
}''')
|
|
||||||
assert widths['left'] >= -1 and widths['right'] <= width+1 and widths['overflow'] <= 2, (width, widths)
|
|
||||||
# Persist the factory arrangement as a user preset.
|
|
||||||
page.locator('#overview-presets-name').fill('Daily')
|
|
||||||
page.locator('#overview-presets-save').click()
|
|
||||||
page.wait_for_function("window.__presets.has('Daily')")
|
|
||||||
assert page.locator('#overview-presets-list .overview-presets-row').count() == 1
|
|
||||||
assert page.evaluate("window.__presets.get('Daily').overview_samples") == 600
|
|
||||||
# Create another view, with customized widths, density and history.
|
|
||||||
page.evaluate('''async () => {
|
|
||||||
await window.rustProxyPreferences.update({
|
|
||||||
dashboard_hidden_widgets:['requests'], dashboard_widget_sizes:{requests:3},
|
|
||||||
dashboard_density:'ultra',
|
|
||||||
});
|
|
||||||
window.rustProxyOverviewSamples.setLimit(120, true);
|
|
||||||
}''')
|
|
||||||
page.locator('#overview-presets-name').fill('Detailed')
|
|
||||||
page.locator('#overview-presets-save').click()
|
|
||||||
page.wait_for_function("window.__presets.has('Detailed')")
|
|
||||||
assert page.evaluate('''() => {
|
|
||||||
const p=window.__presets.get('Detailed');
|
|
||||||
return [p.dashboard_widget_sizes.requests,p.dashboard_density,p.overview_samples];
|
|
||||||
}''') == [3,'ultra',120]
|
|
||||||
# Named load must restore full layout and history.
|
|
||||||
page.locator('.overview-presets-row').filter(has_text='Daily').locator('[data-preset-action="dashboardPresetsLoad"]').click()
|
|
||||||
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
|
|
||||||
assert page.evaluate('''() => [window.__savedDashboard.dashboard_density,
|
|
||||||
window.__savedDashboard.dashboard_widget_sizes,
|
|
||||||
window.rustProxyOverviewSamples.getLimit()]''') == ['compact',None,600]
|
|
||||||
# Rename and delete remain administrator-scoped operations.
|
|
||||||
page.locator('#overview-layout-open').click()
|
|
||||||
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
|
|
||||||
page.locator('#overview-presets-name').fill('Nightly')
|
|
||||||
page.locator('.overview-presets-row').filter(has_text='Detailed').locator('[data-preset-action="dashboardPresetsRename"]').click()
|
|
||||||
page.wait_for_function("window.__presets.has('Nightly')")
|
|
||||||
assert not page.evaluate("window.__presets.has('Detailed')")
|
|
||||||
page.locator('.overview-presets-row').filter(has_text='Nightly').locator('[data-preset-action="dashboardPresetsDelete"]').click()
|
|
||||||
page.wait_for_function("!window.__presets.has('Nightly')")
|
|
||||||
assert page.evaluate("window.__presets.has('Daily')")
|
|
||||||
page.locator('#overview-presets-factory').click()
|
|
||||||
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
|
|
||||||
assert page.evaluate("window.__savedDashboard.dashboard_widgets") == json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())['dashboard_widgets']
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print('PASS named Dashboard presets save/load/rename/delete/factory and responsive modal:', width)
|
|
||||||
context.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS Overview preset modal (simulated SQLite API; no running Rust server)')
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""All Overview widget labels and small values in simulated Chromium.
|
|
||||||
|
|
||||||
This is a DOM/CSS regression test, not a live RustProxy integration test.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
PACKS = {name: json.loads((ROOT / f'langs/{name}.json').read_text(encoding='utf-8'))['translations']
|
|
||||||
for name in ('en_US', 'pl_pl')}
|
|
||||||
|
|
||||||
CHECK = r'''({density, lang, translations}) => {
|
|
||||||
document.documentElement.dataset.dashboardDensity = density;
|
|
||||||
document.documentElement.lang = lang === 'pl_pl' ? 'pl-PL' : 'en-US';
|
|
||||||
const grid = document.getElementById('dashboard-overview-grid');
|
|
||||||
grid.querySelectorAll('[data-i18n]').forEach(node => {
|
|
||||||
if (Object.hasOwn(translations, node.dataset.i18n)) node.textContent = translations[node.dataset.i18n];
|
|
||||||
});
|
|
||||||
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
|
|
||||||
const font = value => getComputedStyle(value).fontFamily;
|
|
||||||
const probe = document.createElement('div');
|
|
||||||
grid.append(probe);
|
|
||||||
probe.style.fontFamily = 'var(--sans)'; const sans = font(probe);
|
|
||||||
probe.style.fontFamily = 'var(--mono)'; const mono = font(probe);
|
|
||||||
probe.remove();
|
|
||||||
|
|
||||||
const titleErrors = [];
|
|
||||||
const titles = [];
|
|
||||||
const labelErrors = [];
|
|
||||||
const numericErrors = [];
|
|
||||||
for (const card of cards) {
|
|
||||||
let title;
|
|
||||||
if (card.classList.contains('stat')) title = card.querySelector(':scope > span:first-child');
|
|
||||||
else if (card.classList.contains('system-card') || card.classList.contains('connection-card'))
|
|
||||||
title = card.querySelector('.overview-chart-heading');
|
|
||||||
else if (card.querySelector('.mini-heading > h3')) title = card.querySelector('.mini-heading > h3');
|
|
||||||
else title = card.querySelector('.card-heading h2');
|
|
||||||
if (!title) { titleErrors.push(`${card.dataset.dashboardWidget}: missing title`); continue; }
|
|
||||||
const style = getComputedStyle(title);
|
|
||||||
titles.push(title);
|
|
||||||
if (style.fontFamily !== sans || style.fontSize !== '12px' || style.textTransform !== 'none')
|
|
||||||
titleErrors.push(`${card.dataset.dashboardWidget}: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
|
|
||||||
if (title.textContent.trim() === title.textContent.trim().toLocaleUpperCase('pl') &&
|
|
||||||
/[a-ząćęłńóśżź]{4}/i.test(title.textContent) &&
|
|
||||||
!/^(?:HIT|MISS|HTTP|CPU|RAM|WWW|RX|TX)/.test(title.textContent.trim()))
|
|
||||||
titleErrors.push(`${card.dataset.dashboardWidget}: all-uppercase title ${title.textContent.trim()}`);
|
|
||||||
for (const node of card.querySelectorAll(':scope > small, .mini-metrics dt, .mini-metrics dd, .mini-live-chart-head, .memory-line, .chart-footer, .meta, .small-label')) {
|
|
||||||
if (font(node) !== mono) labelErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
|
|
||||||
}
|
|
||||||
const readings = card.classList.contains('stat')
|
|
||||||
? [...card.querySelectorAll(':scope > strong'), ...card.querySelectorAll(':scope > small')]
|
|
||||||
: [...card.querySelectorAll('.mini-metrics dd')];
|
|
||||||
for (const node of readings) if (font(node) !== mono)
|
|
||||||
numericErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
|
|
||||||
}
|
|
||||||
const nested = [...grid.querySelectorAll('.mini-live-chart > .mini-live-chart-head > span:first-child')];
|
|
||||||
for (const node of nested) {
|
|
||||||
const style = getComputedStyle(node);
|
|
||||||
if (style.fontFamily !== mono || style.fontSize !== '10px' || style.textTransform !== 'none')
|
|
||||||
labelErrors.push(`mini-live-chart: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
|
|
||||||
}
|
|
||||||
const systemNumeric = getComputedStyle(document.querySelector('#system-cpu'));
|
|
||||||
return {
|
|
||||||
cardCount: cards.length, titleCount: titles.length, secondaryCount: nested.length,
|
|
||||||
titleErrors, labelErrors, numericErrors,
|
|
||||||
systemNumericMono: systemNumeric.fontFamily === mono,
|
|
||||||
cpuLabel: document.querySelector('[data-dashboard-widget="system_chart"] .overview-chart-heading').textContent,
|
|
||||||
connectionsLabel: document.querySelector('[data-dashboard-widget="connection_chart"] .overview-chart-heading').textContent,
|
|
||||||
memoryLabel: document.querySelector('[data-dashboard-widget="memory"] > span:first-child').textContent,
|
|
||||||
backendLabel: document.querySelector('[data-dashboard-widget="backend_health"] > span:first-child').textContent,
|
|
||||||
diagnosticsTitle: document.querySelector('[data-dashboard-widget="process_diagnostics"] .mini-heading h3').textContent,
|
|
||||||
};
|
|
||||||
}'''
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
for density in ('normal', 'compact', 'ultra'):
|
|
||||||
for lang in ('en_US', 'pl_pl'):
|
|
||||||
result = page.evaluate(CHECK, {'density': density, 'lang': lang, 'translations': PACKS[lang]})
|
|
||||||
descriptor = (width, density, lang)
|
|
||||||
assert result['cardCount'] == result['titleCount'] >= 75, (descriptor, result)
|
|
||||||
assert result['secondaryCount'] >= 6, descriptor
|
|
||||||
assert not result['titleErrors'], (descriptor, result['titleErrors'][:10])
|
|
||||||
assert not result['labelErrors'], (descriptor, result['labelErrors'][:10])
|
|
||||||
assert not result['numericErrors'], (descriptor, result['numericErrors'][:10])
|
|
||||||
assert result['systemNumericMono'], descriptor
|
|
||||||
expected = ('System load', 'Connection activity') if lang == 'en_US' else ('Obciążenie systemu', 'Aktywność połączeń')
|
|
||||||
assert (result['cpuLabel'], result['connectionsLabel']) == expected, (descriptor, result)
|
|
||||||
assert result['memoryLabel'] and result['backendLabel'] and result['diagnosticsTitle'], descriptor
|
|
||||||
print(f'PASS all Overview titles and small readings at {width}px (3 densities, 2 languages)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Offline SQLite regression for additive admin preset migration and 16-slot upsert."""
|
|
||||||
from pathlib import Path
|
|
||||||
import sqlite3
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
all_scripts = sorted((ROOT / 'migrations').glob('*.sql'))
|
|
||||||
assert [p.name for p in all_scripts][-1] == '0010_dashboard_presets.sql'
|
|
||||||
con = sqlite3.connect(':memory:')
|
|
||||||
con.execute('PRAGMA foreign_keys = ON')
|
|
||||||
for _ in range(2):
|
|
||||||
for script in all_scripts:
|
|
||||||
con.executescript(script.read_text())
|
|
||||||
con.execute("INSERT INTO administrator(id,username,password_hash,generation,created_at) VALUES(1,'owner','hash',1,0)")
|
|
||||||
con.execute("INSERT INTO admin_preferences(administrator_id,document) VALUES(1,'{}')")
|
|
||||||
|
|
||||||
upsert = '''
|
|
||||||
INSERT INTO dashboard_presets(administrator_id, name, document, updated_at)
|
|
||||||
SELECT a.id, ?, ?, ? FROM administrator a WHERE a.id = 1 AND a.username = ?
|
|
||||||
AND ((SELECT count(*) FROM dashboard_presets WHERE administrator_id = a.id) < 16
|
|
||||||
OR EXISTS(SELECT 1 FROM dashboard_presets WHERE administrator_id = a.id AND name = ?))
|
|
||||||
ON CONFLICT(administrator_id, name) DO UPDATE SET document = excluded.document, updated_at = excluded.updated_at
|
|
||||||
'''
|
|
||||||
for i in range(16):
|
|
||||||
name = f'Preset {i}'
|
|
||||||
assert con.execute(upsert, (name,'{}',i,'owner',name)).rowcount == 1
|
|
||||||
assert con.execute(upsert, ('Overflow','{}',16,'owner','Overflow')).rowcount == 0
|
|
||||||
assert con.execute(upsert, ('Preset 0','{"changed":true}',17,'owner','Preset 0')).rowcount == 1
|
|
||||||
assert con.execute(upsert, ('Other account','{}',18,'intruder','Other account')).rowcount == 0
|
|
||||||
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 16
|
|
||||||
assert con.execute('SELECT document FROM dashboard_presets WHERE name="Preset 0"').fetchone()[0] == '{"changed":true}'
|
|
||||||
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Preset 1','Preset 0')).rowcount == 0
|
|
||||||
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Renamed','Preset 0')).rowcount == 1
|
|
||||||
con.execute('DELETE FROM administrator WHERE id=1')
|
|
||||||
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 0
|
|
||||||
assert con.execute('SELECT count(*) FROM admin_preferences').fetchone()[0] == 0
|
|
||||||
print('PASS SQLite migrations repeated, named-preset CRUD, cap, account ownership and cascade')
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
|
|
||||||
|
|
||||||
The authenticated app bundle runs against simulated status/preferences; this is
|
|
||||||
not a Rust server integration test.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
|
|
||||||
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert match
|
|
||||||
body = ROUTER[match.end():]
|
|
||||||
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
|
|
||||||
result = []
|
|
||||||
for line in body.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
|
|
||||||
|
|
||||||
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
|
|
||||||
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
|
|
||||||
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
|
|
||||||
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
|
|
||||||
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
|
|
||||||
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
|
|
||||||
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
BUNDLE = app_bundle()
|
|
||||||
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
|
|
||||||
|
|
||||||
|
|
||||||
def load(page):
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=BUNDLE)
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
|
|
||||||
|
|
||||||
|
|
||||||
def inspect(page, ids):
|
|
||||||
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
|
|
||||||
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
|
|
||||||
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
|
|
||||||
}))''', ids)
|
|
||||||
|
|
||||||
|
|
||||||
def assert_visible(page, visible, hidden):
|
|
||||||
result = inspect(page, visible + hidden)
|
|
||||||
for widget in visible:
|
|
||||||
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
|
|
||||||
for widget in hidden:
|
|
||||||
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Test actual production CSS/JS in a browser with simulated preferences/status.
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in [320, 375, 800, 1440, 1920]:
|
|
||||||
context = browser.new_context(viewport={'width': width, 'height': 1000})
|
|
||||||
page = context.new_page()
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
load(page)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
document.getElementById('overview-layout-unlock').hidden = false;
|
|
||||||
document.getElementById('overview-layout-open').hidden = false;
|
|
||||||
currentPanel = 'overview';
|
|
||||||
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
|
|
||||||
window.rustProxyPreferences.load = async () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.current = () => window.__savedDashboard;
|
|
||||||
window.rustProxyPreferences.update = async patch => {
|
|
||||||
window.__savedDashboard = {...window.__savedDashboard, ...patch};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
return window.__savedDashboard;
|
|
||||||
};
|
|
||||||
applyAdminPreferences(window.__savedDashboard);
|
|
||||||
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
|
|
||||||
appConfirm = async () => true;
|
|
||||||
}''')
|
|
||||||
# Unchanged previous accounts preserve historic grouped rows.
|
|
||||||
assert page.evaluate('''() => ({
|
|
||||||
legacy: document.getElementById('dashboard-overview-grid').dataset.overviewLayoutMode,
|
|
||||||
widths:window.__savedDashboard.dashboard_widget_sizes,
|
|
||||||
})''') == {'legacy':'legacy','widths':None}
|
|
||||||
page.locator('#overview-layout-unlock').click()
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
# Widen a number+sparkline card from 2 to 3 and then 4 columns.
|
|
||||||
selector = '[data-dashboard-widget="http_protocol"]'
|
|
||||||
old_width = page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width')
|
|
||||||
page.locator(selector + ' .overview-inline-widen').click()
|
|
||||||
assert page.locator(selector).get_attribute('data-dashboard-span') == '3'
|
|
||||||
page.locator(selector + ' .overview-inline-widen').click()
|
|
||||||
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
|
|
||||||
assert page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width') >= old_width * 1.7
|
|
||||||
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().sizes.http_protocol') == 4
|
|
||||||
# Minimize a large multi-series chart to the new 3-column option.
|
|
||||||
for _ in range(2):
|
|
||||||
page.locator('[data-dashboard-widget="wide_transform_activity"] .overview-inline-narrow').click()
|
|
||||||
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
|
|
||||||
# Real UI controls allow crossing old groups: send mini-chart to the front
|
|
||||||
# with the upward arrow, then across the original stat/chart boundary.
|
|
||||||
# A direct drag onto the first card also uses the actual event handlers.
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
|
|
||||||
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
|
|
||||||
const transfer = new DataTransfer();
|
|
||||||
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true, cancelable:true, dataTransfer:transfer}));
|
|
||||||
to.dispatchEvent(new DragEvent('drop', {bubbles:true, cancelable:true, dataTransfer:transfer, clientX:0}));
|
|
||||||
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
|
|
||||||
}''')
|
|
||||||
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().order[0]') == 'wide_transform_activity'
|
|
||||||
# Background status and unrelated preference changes cannot reset live sizes.
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const status={revision:5,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
|
|
||||||
window.rustProxyBackendDashboard.render(status);
|
|
||||||
window.rustProxyStreamDashboard.render(status);
|
|
||||||
applyAdminPreferences({...window.__savedDashboard, chart_style:'soft'});
|
|
||||||
}''')
|
|
||||||
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
|
|
||||||
# Cancel should remove both size and layout changes.
|
|
||||||
page.locator('#overview-inline-cancel').click()
|
|
||||||
assert page.locator(selector).get_attribute('data-dashboard-span') == '2'
|
|
||||||
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '6'
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
|
|
||||||
# The actual unlocked dashboard remains the only layout editor.
|
|
||||||
page.locator('#overview-layout-unlock').click()
|
|
||||||
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
|
|
||||||
tile = '[data-dashboard-widget="wide_transform_activity"]'
|
|
||||||
page.locator(tile + ' .overview-inline-narrow').click()
|
|
||||||
assert page.locator(tile).get_attribute('data-dashboard-span') == '4'
|
|
||||||
page.locator(tile + ' .overview-inline-narrow').click()
|
|
||||||
assert page.locator(tile).get_attribute('data-dashboard-span') == '3'
|
|
||||||
page.evaluate('''() => {
|
|
||||||
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
|
|
||||||
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
|
|
||||||
const transfer = new DataTransfer();
|
|
||||||
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true,cancelable:true,dataTransfer:transfer}));
|
|
||||||
to.dispatchEvent(new DragEvent('drop', {bubbles:true,cancelable:true,dataTransfer:transfer,clientX:0}));
|
|
||||||
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
|
|
||||||
}''')
|
|
||||||
page.locator('#overview-inline-save').click()
|
|
||||||
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes.wide_transform_activity') == 3
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widgets[0]') == 'wide_transform_activity'
|
|
||||||
# Reload account settings: new card width persists.
|
|
||||||
page.evaluate('applyAdminPreferences(window.__savedDashboard)')
|
|
||||||
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_density') == 'compact'
|
|
||||||
# Four charts in one row, set through the same layout function used by UI.
|
|
||||||
data = page.evaluate('''() => {
|
|
||||||
const ids=['wide_transform_activity','wide_compression_activity','http_protocol','websocket_protocol'];
|
|
||||||
const order=[...ids,...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
|
|
||||||
const hidden=[...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
|
|
||||||
const sizes=Object.fromEntries(ids.map(id => [id,3]));
|
|
||||||
applyDashboardLayout(order,hidden,sizes);
|
|
||||||
const grid=document.getElementById('dashboard-overview-grid');
|
|
||||||
return { gridWidth:grid.getBoundingClientRect().width,
|
|
||||||
template:getComputedStyle(grid).gridTemplateColumns.split(' ').length,
|
|
||||||
boxes:ids.map(id=>{const b=document.querySelector(`[data-dashboard-widget="${id}"]`).getBoundingClientRect();return {id,x:Math.round(b.x),y:Math.round(b.y),w:Math.round(b.width),scroll:b.width};})
|
|
||||||
};
|
|
||||||
}''')
|
|
||||||
if data['template'] == 12:
|
|
||||||
assert len(set(b['y'] for b in data['boxes'])) == 1, (width, data)
|
|
||||||
assert all(b['w'] > 150 for b in data['boxes']), (width,data)
|
|
||||||
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2, (width, data)
|
|
||||||
page.locator('#overview-layout-open').click()
|
|
||||||
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
|
|
||||||
page.locator('#overview-presets-factory').click()
|
|
||||||
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
|
|
||||||
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
|
|
||||||
assert page.evaluate('document.getElementById("dashboard-overview-grid").dataset.overviewLayoutMode') == 'legacy'
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print('PASS free-order, resize, live refresh, cancel/save, four chart grid', width, data['template'])
|
|
||||||
context.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS 0.33.3 resizable Overview browser regression')
|
|
||||||
@@ -1,115 +0,0 @@
|
|||||||
import { renderAdminIndex } from './panel_html.mjs';
|
|
||||||
// Simulated DOM/status regression: no holes or artificial zeros after tab resume.
|
|
||||||
// This intentionally does not exercise the running Rust server or real browser.
|
|
||||||
import assert from 'node:assert/strict';
|
|
||||||
import fs from 'node:fs';
|
|
||||||
import path from 'node:path';
|
|
||||||
import vm from 'node:vm';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
|
||||||
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
|
|
||||||
const html = renderAdminIndex(root);
|
|
||||||
|
|
||||||
class Element {
|
|
||||||
constructor() { this.attributes = new Map(); this.style = {}; this.textContent = ''; this.children = []; this.hidden = false; }
|
|
||||||
getAttribute(key) { return this.attributes.get(key) ?? null; }
|
|
||||||
setAttribute(key, value) { this.attributes.set(key, String(value)); }
|
|
||||||
removeAttribute(key) { this.attributes.delete(key); }
|
|
||||||
append(...children) { this.children.push(...children); }
|
|
||||||
replaceChildren(...children) { this.children = children; }
|
|
||||||
}
|
|
||||||
const ids = new Map([...html.matchAll(/\bid="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
|
|
||||||
const widgets = new Map([...html.matchAll(/data-dashboard-widget="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
|
|
||||||
const listeners = new Map();
|
|
||||||
const document = {
|
|
||||||
hidden: false,
|
|
||||||
addEventListener: (type, handler) => listeners.set(type, handler),
|
|
||||||
querySelector: selector => widgets.get(selector.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) || null,
|
|
||||||
createElement: () => new Element(),
|
|
||||||
};
|
|
||||||
const samples = {
|
|
||||||
getLimit: () => 60,
|
|
||||||
trim: values => { if (values.length > 60) values.splice(0, values.length - 60); },
|
|
||||||
onChange: () => {},
|
|
||||||
};
|
|
||||||
const window = {
|
|
||||||
addEventListener() {}, rustProxyOverviewSamples: samples,
|
|
||||||
rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) },
|
|
||||||
};
|
|
||||||
let now = 1000;
|
|
||||||
const context = vm.createContext({
|
|
||||||
window, document, performance: { now: () => now }, currentPanel: 'overview', latestStatus: null,
|
|
||||||
$: id => ids.get(id) || null, fmt: (value, n = 0) => Number(value).toFixed(n),
|
|
||||||
fmtBytes: value => `${value} B`, t: value => value,
|
|
||||||
});
|
|
||||||
for (const file of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-wide-metrics.js', 'app/web-overview.js'])
|
|
||||||
vm.runInContext(code(file), context, { filename: file });
|
|
||||||
|
|
||||||
const web = {
|
|
||||||
requests: 200, status_2xx: 190, status_3xx: 2, status_4xx: 5, status_5xx: 3,
|
|
||||||
static_requests: 100, php_requests: 80, other_requests: 20,
|
|
||||||
methods: [160, 20, 10, 5, 5], selected_status: [0, 0, 0, 0, 0, 0, 0, 0],
|
|
||||||
response_headers_us: 300000, response_bytes: 1200000, active_headers: 2,
|
|
||||||
};
|
|
||||||
const php = {
|
|
||||||
metrics: { requests: 80, response_headers_us: 100000, response_bytes: 320000, failures: Array(9).fill(0) },
|
|
||||||
capacity_limit: 6, capacity_used: 1,
|
|
||||||
};
|
|
||||||
const status = {
|
|
||||||
revision: 4,
|
|
||||||
metrics: { requests: 200, http1_requests: 120, http2_requests: 60, http3_requests: 20,
|
|
||||||
compression_gzip: 10, compression_br: 20, compression_zstd: 5, compression_deflate: 8 },
|
|
||||||
cache: { hits: 50, misses: 30 },
|
|
||||||
transform_metrics: { applied: 10, cache_hits: 5, skipped: 1, errors: 0 },
|
|
||||||
web_server_metrics: { demo: { web, php } },
|
|
||||||
};
|
|
||||||
function advance(delta) {
|
|
||||||
for (const key of Object.keys(status.metrics)) status.metrics[key] += delta;
|
|
||||||
for (const key of ['hits', 'misses']) status.cache[key] += delta;
|
|
||||||
for (const key of Object.keys(status.transform_metrics)) status.transform_metrics[key] += delta;
|
|
||||||
for (const key of ['requests', 'status_2xx', 'static_requests', 'php_requests', 'response_headers_us', 'response_bytes']) web[key] += delta;
|
|
||||||
for (const key of ['requests', 'response_headers_us', 'response_bytes']) php.metrics[key] += delta;
|
|
||||||
}
|
|
||||||
function render() { window.rustProxyWideOverview.render(status); window.rustProxyWebOverview.render(status); }
|
|
||||||
const line = id => ids.get(id).getAttribute('d') || '';
|
|
||||||
const compression = 'overview-wide-wide_compression_activity-gzip-line';
|
|
||||||
const website = 'www_requests-all-line';
|
|
||||||
const phpLine = 'www_php_requests-line';
|
|
||||||
const starts = path => (path.match(/M/g) || []).length;
|
|
||||||
|
|
||||||
render(); // no first-interval rate: establish baseline only
|
|
||||||
now = 4000; advance(12); render();
|
|
||||||
const beforeWide = line(compression), beforeWeb = line(website);
|
|
||||||
const beforePhp = ids.get(phpLine).getAttribute('points');
|
|
||||||
assert.equal(starts(beforeWide), 1);
|
|
||||||
assert.equal(starts(beforeWeb), 1);
|
|
||||||
assert.ok(beforePhp);
|
|
||||||
|
|
||||||
// Background tab -> a new generation, regardless of counter increments.
|
|
||||||
document.hidden = true; listeners.get('visibilitychange')();
|
|
||||||
document.hidden = false;
|
|
||||||
now = 100000; advance(600); render();
|
|
||||||
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
|
|
||||||
assert.equal(ids.get('www_requests-value').textContent, '-');
|
|
||||||
assert.equal(line(compression), beforeWide, 'wide rate chart must not receive a null point');
|
|
||||||
assert.equal(line(website), beforeWeb, 'WWW rate chart must not receive a null point');
|
|
||||||
assert.equal(ids.get(phpLine).getAttribute('points'), beforePhp, 'small PHP chart must preserve valid history');
|
|
||||||
|
|
||||||
// A fresh interval then connects to previous valid samples without a gap.
|
|
||||||
now = 103000; advance(9); render();
|
|
||||||
assert.equal(starts(line(compression)), 1, 'wide chart must have one connected path');
|
|
||||||
assert.equal(starts(line(website)), 1, 'WWW chart must have one connected path');
|
|
||||||
assert.ok(line(compression).includes('L'));
|
|
||||||
assert.ok(line(website).includes('L'));
|
|
||||||
|
|
||||||
// Real measured zeros must continue to be plotted, not treated as pauses.
|
|
||||||
now = 106000; render();
|
|
||||||
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '0.0/s');
|
|
||||||
assert.equal(ids.get('www_requests-value').textContent, '0.0/s');
|
|
||||||
assert.equal(starts(line(compression)), 1);
|
|
||||||
assert.equal(starts(line(website)), 1);
|
|
||||||
|
|
||||||
assert.match(code('web-server.js'), /if \(record && rate !== null\) \{ previous\.push\(rate\)/,
|
|
||||||
'Web Server chart must also skip unavailable rates instead of plotting a zero dip');
|
|
||||||
console.log('PASS: resumed wide/WWW/PHP charts stay connected; first invalid rate is unavailable; measured zero persists');
|
|
||||||
@@ -1,129 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""0.33.6 plot size/scale checks in simulated Chromium, not a running RustProxy."""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
SCALES = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
|
|
||||||
RATES = (ROOT / 'src/admin/web/app/rate-sampling.js').read_text(encoding='utf-8')
|
|
||||||
WIDE = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
WIDE_IDS = ('wide_protocol_rates', 'wide_cpu', 'wide_memory', 'wide_cache_activity',
|
|
||||||
'wide_connections', 'wide_backend_health', 'wide_latency',
|
|
||||||
'wide_transform_activity', 'wide_compression_activity',
|
|
||||||
'www_requests', 'www_latency', 'www_failures', 'www_traffic', 'www_status_trend')
|
|
||||||
|
|
||||||
with sync_playwright() as api:
|
|
||||||
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 768, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 920})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading','app-startup-failed');
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
document.getElementById('dashboard-overview-grid').classList.add('is-aligned');
|
|
||||||
for (const card of document.querySelectorAll('#dashboard-overview-grid>[data-dashboard-widget]'))
|
|
||||||
card.hidden = !['wide_protocol_rates','wide_cpu','wide_memory','wide_cache_activity',
|
|
||||||
'wide_connections','wide_backend_health','wide_latency',
|
|
||||||
'wide_transform_activity','wide_compression_activity',
|
|
||||||
'www_requests','www_latency','www_failures','www_traffic','www_status_trend'].includes(card.dataset.dashboardWidget);
|
|
||||||
window.currentPanel='overview';
|
|
||||||
window.$ = id => document.getElementById(id);
|
|
||||||
window.fmt = (n, digits=0) => Number(n).toFixed(digits);
|
|
||||||
window.fmtBytes = n => `${n} B`;
|
|
||||||
window.rustProxyOverviewSamples = {getLimit:()=>60,onChange:()=>{}};
|
|
||||||
window.rustProxyRefresh = {current:()=>({overview_seconds:3})};
|
|
||||||
window.__clock = 1000;
|
|
||||||
Object.defineProperty(performance,'now',{configurable:true,value:()=>window.__clock});
|
|
||||||
}''')
|
|
||||||
for script in (RATES, SCALES, WIDE):
|
|
||||||
page.add_script_tag(content=script)
|
|
||||||
for density in ('normal','compact','ultra'):
|
|
||||||
measured = page.evaluate('''density => {
|
|
||||||
document.documentElement.dataset.dashboardDensity=density;
|
|
||||||
const results={};
|
|
||||||
for (const card of document.querySelectorAll('#dashboard-overview-grid > .overview-wide-chart')) {
|
|
||||||
const svg=card.querySelector('svg.overview-wide-svg');
|
|
||||||
if (!svg) continue;
|
|
||||||
const r=svg.getBoundingClientRect();
|
|
||||||
results[card.dataset.dashboardWidget]={height:r.height, width:r.width,
|
|
||||||
grid:svg.querySelector('.chart-grid').getAttribute('d'),
|
|
||||||
contentWidth:Math.hypot(svg.getScreenCTM().a,svg.getScreenCTM().b)*600};
|
|
||||||
}
|
|
||||||
document.getElementById('panel-transform').hidden=false;
|
|
||||||
document.getElementById('panel-cache').hidden=false;
|
|
||||||
for(const id of ['transform-live-chart','cache-live-chart'])document.getElementById(id).open=true;
|
|
||||||
results.transformDetails=document.getElementById('transform-live-activity-chart').getBoundingClientRect().height;
|
|
||||||
results.cacheDetails=document.getElementById('cache-live-activity-chart').getBoundingClientRect().height;
|
|
||||||
return results;
|
|
||||||
}''', density)
|
|
||||||
assert abs(measured['transformDetails'] - 180) < 1, (width,density,measured)
|
|
||||||
assert abs(measured['cacheDetails'] - 180) < 1, (width,density,measured)
|
|
||||||
for name in WIDE_IDS:
|
|
||||||
data=measured[name]
|
|
||||||
assert abs(data['height']-125)<1, (width,density,name,data)
|
|
||||||
assert data['width']>50 and abs(data['width']-data['contentWidth'])<2, (width,density,name,data)
|
|
||||||
assert data['grid']=='M0 47H600 M0 77H600 M0 107H600', (width,density,name,data)
|
|
||||||
result = page.evaluate('''() => {
|
|
||||||
const c=window.rustProxyChartScale;
|
|
||||||
const first=c.peak('probe:rate','rate',[[.2,.7]]);
|
|
||||||
const count=c.peak('probe:count','count',[[0,1]]);
|
|
||||||
const latency=c.peak('probe:ms','ms',[[.1,1.5]]);
|
|
||||||
const memory=c.peak('probe:bytes','bytes',[[1000]]);
|
|
||||||
const high=c.peak('probe:rate','rate',[[101]]);
|
|
||||||
const held=c.peak('probe:rate','rate',[[.3]]);
|
|
||||||
c.reset('probe:');
|
|
||||||
const reset=c.peak('probe:rate','rate',[[.3]]);
|
|
||||||
const percent=c.peak('probe:cpu','percent',[[.1,500]]);
|
|
||||||
const blank=c.peak('probe:blank','rate',[[null,NaN,Infinity]]);
|
|
||||||
const before={revision:10,live_system:{cpu_percent:0.7,process_cpu_percent:0.2},
|
|
||||||
metrics:{compression_gzip:50,compression_br:100,compression_zstd:10,compression_deflate:5},
|
|
||||||
transform_metrics:{applied:2,cache_hits:10,skipped:1,errors:0}};
|
|
||||||
window.rustProxyWideOverview.render(before);
|
|
||||||
window.__clock=4000;
|
|
||||||
before.metrics.compression_gzip+=3;before.metrics.compression_br+=6;
|
|
||||||
before.metrics.compression_zstd+=0;before.metrics.compression_deflate+=0;
|
|
||||||
before.transform_metrics.applied+=6;before.transform_metrics.cache_hits+=3;
|
|
||||||
window.rustProxyWideOverview.render(before);
|
|
||||||
const get = (id,key) => document.getElementById(`overview-wide-${id}-${key}-line`).getAttribute('d');
|
|
||||||
const compressionLow=get('wide_compression_activity','gzip');
|
|
||||||
const transformLow=get('wide_transform_activity','applied');
|
|
||||||
const cpu=get('wide_cpu','system');
|
|
||||||
const lowLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
|
|
||||||
window.__clock=7000;
|
|
||||||
before.metrics.compression_gzip+=603; // now 201/s => ceiling expands to 500/s
|
|
||||||
before.transform_metrics.applied+=603;
|
|
||||||
window.rustProxyWideOverview.render(before);
|
|
||||||
const highLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
|
|
||||||
const compressionHigh=get('wide_compression_activity','gzip');
|
|
||||||
window.__clock=10000;
|
|
||||||
before.metrics.compression_gzip+=3;
|
|
||||||
before.transform_metrics.applied+=3;
|
|
||||||
window.rustProxyWideOverview.render(before);
|
|
||||||
const afterSpike=get('wide_compression_activity','gzip');
|
|
||||||
return {first,count,latency,memory,high,held,reset,percent,blank,
|
|
||||||
lowLabel,highLabel,compressionLow,transformLow,cpu,compressionHigh,afterSpike};
|
|
||||||
}''')
|
|
||||||
assert result['first']==100 and result['count']==10 and result['latency']==100, (width,result)
|
|
||||||
assert result['memory']>=64*1024*1024, (width,result)
|
|
||||||
assert (result['high'],result['held'],result['reset'],result['percent'],result['blank']) == (200,200,100,100,100), (width,result)
|
|
||||||
assert result['lowLabel']=='1.0/s' and result['highLabel']=='201.0/s', (width,result)
|
|
||||||
def yvalues(line): return [int(n) for n in re.findall(r'[ML]\d+ (\d+)',line)]
|
|
||||||
assert yvalues(result['compressionLow'])[-1]>=135, (width,result)
|
|
||||||
assert yvalues(result['transformLow'])[-1]>=130, (width,result)
|
|
||||||
assert yvalues(result['cpu'])==[136,136], (width,result)
|
|
||||||
assert yvalues(result['compressionHigh'])[-1]<100, (width,result)
|
|
||||||
assert yvalues(result['afterSpike'])[-1]>=135, (width,result)
|
|
||||||
assert not errors, (width,errors)
|
|
||||||
print('PASS all 14 overview-wide plot sizes/grids + full Cache/Transform charts + stable scales:', width)
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Simulated Chromium regression for unified subdued metric/chart titles.
|
|
||||||
|
|
||||||
Tests computed CSS in the assembled panel DOM; not a live Rust server.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
CHECK = r'''({theme, density}) => {
|
|
||||||
document.documentElement.dataset.theme = theme;
|
|
||||||
document.documentElement.dataset.dashboardDensity = density;
|
|
||||||
const grid = document.getElementById('dashboard-overview-grid');
|
|
||||||
const probe = document.createElement('span');
|
|
||||||
probe.style.color = 'var(--muted)';
|
|
||||||
document.body.append(probe);
|
|
||||||
const expected = getComputedStyle(probe).color;
|
|
||||||
probe.style.color = 'var(--text)';
|
|
||||||
const primary = getComputedStyle(probe).color;
|
|
||||||
probe.remove();
|
|
||||||
|
|
||||||
const color = node => getComputedStyle(node).color;
|
|
||||||
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
|
|
||||||
const titles = cards.map(card => {
|
|
||||||
let node;
|
|
||||||
if (card.classList.contains('stat')) node = card.querySelector(':scope > span:first-child');
|
|
||||||
else if (card.matches('.system-card,.connection-card')) node = card.querySelector('.overview-chart-heading');
|
|
||||||
else node = card.querySelector('.mini-heading > h3') || card.querySelector('.card-heading h2');
|
|
||||||
return [card.dataset.dashboardWidget, node];
|
|
||||||
});
|
|
||||||
const otherSelectors = [
|
|
||||||
'#panel-cache .stats-grid .stat > span:first-child',
|
|
||||||
'#panel-transform .stats-grid .stat > span:first-child',
|
|
||||||
'#panel-traffic .stat > span:first-child',
|
|
||||||
'#panel-cache > .card .card-heading h2',
|
|
||||||
'#panel-transform > .card .card-heading h2',
|
|
||||||
'.traffic-chart-card > h3.subheading',
|
|
||||||
'.function-live-chart-heading > strong',
|
|
||||||
'.mini-live-chart-head > span:first-child',
|
|
||||||
'.vhost-chart-head > span:first-child',
|
|
||||||
'.vhost-live-metric > span',
|
|
||||||
];
|
|
||||||
// Include dynamically rendered Vhost/Streams chart and Vhost metric titles.
|
|
||||||
const container = document.getElementById('panel-vhosts');
|
|
||||||
const fixture = document.createElement('div');
|
|
||||||
fixture.innerHTML = `<div class="vhost-chart"><div class="vhost-chart-head">
|
|
||||||
<span class="small-label">Connections history</span><span class="fine">42/s</span>
|
|
||||||
</div></div><div class="vhost-live-metric"><span>Requests</span><strong>42</strong></div>`;
|
|
||||||
container.append(fixture);
|
|
||||||
const otherTitles = otherSelectors.flatMap(selector => [...document.querySelectorAll(selector)]
|
|
||||||
.map(node => [selector, node]));
|
|
||||||
const errors = [...titles, ...otherTitles].flatMap(([name, node]) => {
|
|
||||||
if (!node) return [`${name} missing title`];
|
|
||||||
if (color(node) !== expected) return [`${name}: ${color(node)} != ${expected} (${node.textContent.trim().slice(0, 35)})`];
|
|
||||||
return [];
|
|
||||||
});
|
|
||||||
const numeric = [
|
|
||||||
document.querySelector('#system-cpu'),
|
|
||||||
document.querySelector('#panel-overview .system-card .card-heading h2'),
|
|
||||||
document.querySelector('#panel-overview .connection-card .card-heading h2'),
|
|
||||||
document.querySelector('[data-dashboard-widget="requests"] > strong'),
|
|
||||||
fixture.querySelector('.vhost-live-metric strong'),
|
|
||||||
].map(node => ({node: node && node.outerHTML.slice(0, 80), color: node && color(node)}));
|
|
||||||
fixture.remove();
|
|
||||||
return {expected, primary, count: titles.length, otherCount: otherTitles.length,
|
|
||||||
errors, numeric};
|
|
||||||
}'''
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 768, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
for theme in ('dark', 'light', 'ultra_dark'):
|
|
||||||
for density in ('normal', 'compact', 'ultra'):
|
|
||||||
result = page.evaluate(CHECK, {'theme': theme, 'density': density})
|
|
||||||
descriptor = width, theme, density
|
|
||||||
assert result['count'] == 78, (descriptor, result['count'])
|
|
||||||
assert result['otherCount'] >= 35, (descriptor, result['otherCount'])
|
|
||||||
assert result['primary'] != result['expected'], (descriptor, result)
|
|
||||||
assert not result['errors'], (descriptor, result['errors'][:25])
|
|
||||||
assert all(item['color'] == result['primary'] for item in result['numeric']), (descriptor, result['numeric'])
|
|
||||||
print(f'PASS muted titles, primary numeric readings at {width}px (3 themes, 3 densities)')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS subdued metric/chart title CSS (simulated Chromium, no Rust runtime)')
|
|
||||||
@@ -1,116 +0,0 @@
|
|||||||
import { renderAdminIndex } from './panel_html.mjs';
|
|
||||||
// Simulated browser/status contract: this does not exercise the Rust server.
|
|
||||||
import assert from 'node:assert/strict';
|
|
||||||
import fs from 'node:fs';
|
|
||||||
import path from 'node:path';
|
|
||||||
import vm from 'node:vm';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
|
||||||
const read = file => fs.readFileSync(path.join(root, file), 'utf8');
|
|
||||||
const html = renderAdminIndex(root);
|
|
||||||
const controllerSource = read('src/admin/web/app/overview-samples.js');
|
|
||||||
const store = new Map();
|
|
||||||
|
|
||||||
assert.match(html, /id="overview-layout-unlock"/);
|
|
||||||
// History controls share the connected-core eyebrow row (not a separate
|
|
||||||
// dashboard row), and navigation shows them only on Overview.
|
|
||||||
const metaStart = html.indexOf('class="page-intro-meta-row"');
|
|
||||||
const metaEnd = html.indexOf('class="page-intro-title-row"');
|
|
||||||
const statusPos = html.indexOf('id="connection-label"');
|
|
||||||
const selectPos = html.indexOf('id="overview-live-samples"');
|
|
||||||
assert.ok(metaStart < statusPos && statusPos < selectPos && selectPos < metaEnd);
|
|
||||||
assert.ok(selectPos < html.indexOf('id="panel-overview"'));
|
|
||||||
assert.match(html, /id="overview-samples-toolbar" hidden role="group"/);
|
|
||||||
assert.match(read('src/admin/web/app/core.js'), /\$\("overview-samples-toolbar"\)\.hidden = name !== "overview"/);
|
|
||||||
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="overview-layout-unlock"'));
|
|
||||||
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="dashboard-overview-grid"'));
|
|
||||||
for (const sample of [60, 120, 300, 600]) assert.match(html, new RegExp(`<option value="${sample}">`));
|
|
||||||
|
|
||||||
class Element {
|
|
||||||
constructor() { this.listeners = {}; this.value = ''; this.dataset = {}; this.disabled = false; this.attributes = new Map(); this.textContent = ''; this.hidden = false; }
|
|
||||||
addEventListener(name, callback) { this.listeners[name] = callback; }
|
|
||||||
fire(name) { this.listeners[name](); }
|
|
||||||
setAttribute(name, value) { this.attributes.set(name, String(value)); }
|
|
||||||
getAttribute(name) { return this.attributes.get(name); }
|
|
||||||
}
|
|
||||||
const nodes = new Map();
|
|
||||||
for (const id of ['overview-live-samples', 'overview-samples-save', 'overview-samples-state']) nodes.set(id, new Element());
|
|
||||||
const widgets = new Map([['wide_compression_activity', new Element()]]);
|
|
||||||
const document = {
|
|
||||||
readyState: 'complete', getElementById: id => nodes.get(id) || null,
|
|
||||||
querySelector: query => widgets.get(query.match(/data-dashboard-widget="([^"]+)"/)?.[1]) || null,
|
|
||||||
};
|
|
||||||
const localStorage = {
|
|
||||||
getItem: key => store.get(key) ?? null,
|
|
||||||
setItem: (key, value) => { store.set(key, value); },
|
|
||||||
};
|
|
||||||
let now = 1000;
|
|
||||||
const window = { rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
|
|
||||||
const context = vm.createContext({ document, localStorage, window, queueMicrotask: callback => callback(),
|
|
||||||
t: key => key, performance: { now: () => now },
|
|
||||||
currentPanel: 'overview', latestStatus: null, $: id => { if (!nodes.has(id)) nodes.set(id, new Element()); return nodes.get(id); },
|
|
||||||
fmt: n => String(n), fmtBytes: n => `${n} B`,
|
|
||||||
});
|
|
||||||
vm.runInContext(controllerSource, context);
|
|
||||||
const samples = window.rustProxyOverviewSamples;
|
|
||||||
const select = document.getElementById('overview-live-samples');
|
|
||||||
const save = document.getElementById('overview-samples-save');
|
|
||||||
const state = document.getElementById('overview-samples-state');
|
|
||||||
assert.equal(samples.getLimit(), 60);
|
|
||||||
assert.equal(save.disabled, true);
|
|
||||||
select.value = '600'; select.fire('change');
|
|
||||||
assert.equal(samples.getLimit(), 600);
|
|
||||||
assert.equal(state.dataset.i18n, 'overviewSamplesTemporary');
|
|
||||||
assert.equal(store.size, 0, 'temporary samples must not touch localStorage');
|
|
||||||
assert.equal(save.disabled, false);
|
|
||||||
const range = Array.from({ length: 700 }, (_, i) => i);
|
|
||||||
let calls = 0;
|
|
||||||
samples.onChange(() => { samples.trim(range); calls++; });
|
|
||||||
select.value = '120'; select.fire('change');
|
|
||||||
assert.equal(calls, 1);
|
|
||||||
assert.equal(range.length, 120);
|
|
||||||
assert.equal(range[0], 580);
|
|
||||||
assert.equal(samples.setLimit(999), false);
|
|
||||||
assert.equal(samples.getLimit(), 120);
|
|
||||||
save.fire('click');
|
|
||||||
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '120');
|
|
||||||
assert.equal(state.dataset.i18n, 'overviewSamplesSaved');
|
|
||||||
assert.equal(save.disabled, true);
|
|
||||||
|
|
||||||
// Fresh load reads only the saved choice, not a temporary override.
|
|
||||||
select.value = '300'; select.fire('change');
|
|
||||||
assert.equal(samples.getLimit(), 300);
|
|
||||||
const freshDocument = { readyState: 'loading', addEventListener() {} };
|
|
||||||
const fresh = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
|
|
||||||
vm.runInContext(controllerSource, fresh);
|
|
||||||
assert.equal(fresh.window.rustProxyOverviewSamples.getLimit(), 120);
|
|
||||||
// Permanently saved choice survives a fresh load.
|
|
||||||
save.fire('click');
|
|
||||||
const again = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
|
|
||||||
vm.runInContext(controllerSource, again);
|
|
||||||
assert.equal(again.window.rustProxyOverviewSamples.getLimit(), 300);
|
|
||||||
|
|
||||||
// Wide Overview charts must respect the same limit and redraw immediately when
|
|
||||||
// the limit shrinks, without recording an extra point or hitting the network.
|
|
||||||
vm.runInContext(read('src/admin/web/app/rate-sampling.js'), context);
|
|
||||||
vm.runInContext(read('src/admin/web/app/chart-scales.js'), context);
|
|
||||||
vm.runInContext(read('src/admin/web/app/overview-wide-metrics.js'), context);
|
|
||||||
select.value = '600'; select.fire('change');
|
|
||||||
const status = {
|
|
||||||
revision: 1, metrics: { compression_gzip: 0, compression_br: 0, compression_zstd: 0, compression_deflate: 0 },
|
|
||||||
};
|
|
||||||
context.latestStatus = status;
|
|
||||||
for (let i = 0; i <= 650; i++) {
|
|
||||||
now += 3000;
|
|
||||||
for (const key of ['compression_gzip', 'compression_br', 'compression_zstd', 'compression_deflate']) status.metrics[key] += i % 3 + 1;
|
|
||||||
window.rustProxyWideOverview.render(status);
|
|
||||||
}
|
|
||||||
const line = () => nodes.get('overview-wide-wide_compression_activity-gzip-line').getAttribute('d');
|
|
||||||
assert.equal((line().match(/ L/g) || []).length, 599, 'maximum 600 samples must be retained');
|
|
||||||
select.value = '120'; select.fire('change');
|
|
||||||
assert.equal((line().match(/ L/g) || []).length, 119, 'changing selection must trim and redraw without a new sample');
|
|
||||||
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '300', 'temporary choice must preserve the saved limit');
|
|
||||||
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('fetch('));
|
|
||||||
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('setInterval('));
|
|
||||||
console.log('PASS: Overview samples 60/120/300/600, temporary vs persistent, immediate trim/redraw, no extra requests');
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
import fs from 'node:fs';
|
|
||||||
import path from 'node:path';
|
|
||||||
|
|
||||||
// Same four compile-time substitutions as src/admin/public.rs.
|
|
||||||
export function renderAdminIndex(root) {
|
|
||||||
const web = path.join(root, 'src/admin/web');
|
|
||||||
let html = fs.readFileSync(path.join(web, 'index.html'), 'utf8');
|
|
||||||
const metrics = fs.readFileSync(path.join(web, 'partials/overview-metrics.html'), 'utf8')
|
|
||||||
.split('<!-- RP_METRICS_SPLIT -->');
|
|
||||||
const charts = fs.readFileSync(path.join(web, 'partials/overview-charts.html'), 'utf8')
|
|
||||||
.split('<!-- RP_CHARTS_SPLIT -->');
|
|
||||||
if (metrics.length !== 2 || charts.length !== 2) throw new Error('Invalid Overview partials');
|
|
||||||
for (const [marker, text] of [
|
|
||||||
['METRICS_MAIN', metrics[0]], ['CHARTS_MAIN', charts[0]],
|
|
||||||
['METRICS_WEB', metrics[1]], ['CHARTS_WEB', charts[1]],
|
|
||||||
]) {
|
|
||||||
const token = `<!-- RP_${marker} -->`;
|
|
||||||
if (!html.includes(token)) throw new Error(`Missing ${token}`);
|
|
||||||
html = html.replace(token, text);
|
|
||||||
}
|
|
||||||
return html;
|
|
||||||
}
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
"""Return the actual assembled embedded Overview HTML to simulated-browser tests."""
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def render_admin_index(root: Path) -> str:
|
|
||||||
web = root / 'src/admin/web'
|
|
||||||
html = (web / 'index.html').read_text(encoding='utf-8')
|
|
||||||
metrics = (web / 'partials/overview-metrics.html').read_text(encoding='utf-8').split('<!-- RP_METRICS_SPLIT -->')
|
|
||||||
charts = (web / 'partials/overview-charts.html').read_text(encoding='utf-8').split('<!-- RP_CHARTS_SPLIT -->')
|
|
||||||
assert len(metrics) == len(charts) == 2
|
|
||||||
for key, fragment in (
|
|
||||||
('METRICS_MAIN', metrics[0]), ('CHARTS_MAIN', charts[0]),
|
|
||||||
('METRICS_WEB', metrics[1]), ('CHARTS_WEB', charts[1]),
|
|
||||||
):
|
|
||||||
marker = '<!-- RP_' + key + ' -->'
|
|
||||||
assert html.count(marker) == 1
|
|
||||||
html = html.replace(marker, fragment)
|
|
||||||
return html
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Progressbar Appearance regression using the production CSS in Chromium.
|
|
||||||
|
|
||||||
Browser-only visual/CSS verification. Does not execute RustProxy or authenticate.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from io import BytesIO
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re
|
|
||||||
|
|
||||||
from PIL import Image
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
APP_HTML = render_admin_index(ROOT)
|
|
||||||
assert re.search(r'<div class="overview-meter system-memory-meter" id="memory-meter"', APP_HTML)
|
|
||||||
assert 'id="memory-meter-fill"' in APP_HTML
|
|
||||||
CORNERS = {'default': '3px', 'square': '0px', 'subtle': '2px', 'rounded': '4px', 'extra': '999px'}
|
|
||||||
STYLES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
|
|
||||||
|
|
||||||
HTML = '''<!doctype html><html><body>
|
|
||||||
<section id="panel-cache"><div class="stats-grid cache-stats">
|
|
||||||
<article class="stat"><progress id="cp-hot-entries-meter" value="55" max="100"></progress></article>
|
|
||||||
<article class="stat"><progress id="cp-memory" value="55" max="100"></progress></article>
|
|
||||||
<article class="stat"><progress id="cp-cold-entries-meter" value="55" max="100"></progress></article>
|
|
||||||
<article class="stat"><progress id="cp-cold-disk" value="55" max="100"></progress></article>
|
|
||||||
</div></section>
|
|
||||||
<main id="panel-overview">
|
|
||||||
<div class="stat stat-meter"><div class="overview-meter" id="overview-progress"><span class="overview-meter-fill" id="overview-fill" style="width:55%"></span></div></div>
|
|
||||||
<article class="card system-card"><div class="overview-meter system-memory-meter" id="memory-meter" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="55"><span class="overview-meter-fill" id="memory-meter-fill" style="width:55%"></span></div></article>
|
|
||||||
<div class="www-segmented" id="www-segmented"><span style="width:55%;background:var(--chart-accent)"></span></div>
|
|
||||||
<div class="www-top-meter" id="www-top"><span id="www-top-fill" style="width:55%"></span></div>
|
|
||||||
</main>
|
|
||||||
<div class="traffic-bar-track" id="traffic-track"><span class="traffic-bar-fill" id="traffic-fill" style="width:55%"></span></div>
|
|
||||||
<div id="chart-swatch" style="background:var(--chart-accent);width:4px;height:4px"></div>
|
|
||||||
</body></html>'''
|
|
||||||
|
|
||||||
|
|
||||||
def color_at_progress(page, progress_id: str) -> tuple[int, int, int]:
|
|
||||||
rect = page.locator(f'#{progress_id}').bounding_box()
|
|
||||||
assert rect and rect['width'] > 100
|
|
||||||
screenshot = Image.open(BytesIO(page.screenshot())).convert('RGB')
|
|
||||||
x = round(rect['x'] + rect['width'] * .22)
|
|
||||||
y = round(rect['y'] + rect['height'] / 2)
|
|
||||||
return screenshot.getpixel((x, y))
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (375, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML)
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
for corner, radius in CORNERS.items():
|
|
||||||
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
|
|
||||||
actual = page.evaluate('''() => {
|
|
||||||
const ids = ['cp-hot-entries-meter','cp-memory','cp-cold-entries-meter','cp-cold-disk',
|
|
||||||
'overview-progress','overview-fill','memory-meter','memory-meter-fill','www-segmented','www-top','www-top-fill',
|
|
||||||
'traffic-track','traffic-fill'];
|
|
||||||
return Object.fromEntries(ids.map(id => [id, getComputedStyle(document.getElementById(id)).borderRadius]));
|
|
||||||
}''')
|
|
||||||
assert all(value == radius for value in actual.values()), (corner, radius, actual)
|
|
||||||
actual_pseudo = page.evaluate('''() => {
|
|
||||||
const p = document.getElementById('cp-memory');
|
|
||||||
return ['::-webkit-progress-bar', '::-webkit-progress-value'].map(pseudo => getComputedStyle(p, pseudo).borderRadius);
|
|
||||||
}''')
|
|
||||||
assert actual_pseudo == [radius, radius], (corner, actual_pseudo)
|
|
||||||
|
|
||||||
for theme in ('dark', 'light'):
|
|
||||||
page.evaluate('theme => document.documentElement.dataset.theme = theme', theme)
|
|
||||||
for chart_style in STYLES:
|
|
||||||
page.evaluate('style => document.documentElement.dataset.chartStyle = style', chart_style)
|
|
||||||
observed = page.evaluate('''() => {
|
|
||||||
const color = id => getComputedStyle(document.getElementById(id)).backgroundColor;
|
|
||||||
const progress = document.getElementById('cp-memory');
|
|
||||||
return {
|
|
||||||
chart:color('chart-swatch'), native:getComputedStyle(progress).accentColor,
|
|
||||||
overview:color('overview-fill'), systemRam:color('memory-meter-fill'), www:color('www-top-fill'), traffic:color('traffic-fill'),
|
|
||||||
track:getComputedStyle(progress).backgroundColor,
|
|
||||||
overviewTrack:color('overview-progress'), systemTrack:color('memory-meter'), wwwTrack:color('www-top'),
|
|
||||||
};
|
|
||||||
}''')
|
|
||||||
assert len({observed[key] for key in ('chart', 'native', 'overview', 'systemRam', 'www', 'traffic')}) == 1, (theme, chart_style, observed)
|
|
||||||
assert len({observed[key] for key in ('track', 'overviewTrack', 'systemTrack', 'wwwTrack')}) == 1, (theme, chart_style, observed)
|
|
||||||
# Native progress on Chromium is painted by the WebKit pseudo-element;
|
|
||||||
# a pixel check confirms the color, not just the accent-color property.
|
|
||||||
if chart_style in ('default', 'neon', 'monochrome'):
|
|
||||||
color = color_at_progress(page, 'cp-memory')
|
|
||||||
chart_rgb = tuple(page.evaluate("""() => {
|
|
||||||
const canvas = document.createElement('canvas');
|
|
||||||
canvas.width = canvas.height = 1;
|
|
||||||
const ctx = canvas.getContext('2d');
|
|
||||||
ctx.fillStyle = getComputedStyle(document.getElementById('chart-swatch')).backgroundColor;
|
|
||||||
ctx.fillRect(0, 0, 1, 1);
|
|
||||||
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
|
|
||||||
}"""))
|
|
||||||
assert color == chart_rgb, (theme, chart_style, color, chart_rgb)
|
|
||||||
assert not errors, errors
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
|
|
||||||
print('PASS: native Cache, System RAM, Overview, WWW and Traffic bars respond to 5 corners and 8 palettes (dark/light, 375/1440px).')
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Source-only regression guards for the OPT3 patch (NOT a Rust compile)."""
|
|
||||||
from pathlib import Path
|
|
||||||
root = Path(__file__).resolve().parents[2]
|
|
||||||
|
|
||||||
def read(name):
|
|
||||||
return (root / name).read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
assert 'version = "0.34.0"' in read('Cargo.toml')
|
|
||||||
acl = read('src/acl.rs')
|
|
||||||
assert "pub fn routing_path_view(raw: &str)" in acl
|
|
||||||
assert "return routing_path(raw).map(std::borrow::Cow::Owned)" in acl
|
|
||||||
assert 'raw.split' in acl and 'raw.len() <= 8192' in acl
|
|
||||||
forward = read('src/proxy/forward_entry.rs')
|
|
||||||
assert 'let routing_uri = request.uri().clone();' in forward
|
|
||||||
assert 'crate::acl::routing_path_view(routing_uri.path())' in forward
|
|
||||||
assert 'if context.client_ip != peer.0.ip()' in forward
|
|
||||||
route = read('src/proxy/cache_helpers.rs')
|
|
||||||
assert 'crate::acl::routing_path_view(request.uri().path())' in route
|
|
||||||
for name in ('src/proxy/forward/cache_lookup.rs', 'src/proxy/forward/success_finish.rs', 'src/proxy/forward/web_source.rs'):
|
|
||||||
assert 'path.clone()' not in read(name), name
|
|
||||||
assert 'if !trusted_peer' in read('src/forwarding.rs')
|
|
||||||
assert 'let mut ips = chain(incoming).unwrap_or_default();' in read('src/forwarding.rs')
|
|
||||||
headers = read('src/proxy/headers.rs')
|
|
||||||
assert 'let no_nominations' in headers
|
|
||||||
assert 'let nominated: Vec<HeaderName>' in headers
|
|
||||||
assert 'headers.remove(name)' in headers
|
|
||||||
assert 'String::from_utf8(encoded)' in read('src/proxy/request_id.rs')
|
|
||||||
assert 'if reason == "cache_disabled"' in read('src/cache/stats.rs')
|
|
||||||
assert not read('src/cache/stats.rs').startswith('//!') # previous E0753 build issue
|
|
||||||
for name in ('src/acl/tests.rs', 'src/proxy/headers/tests.rs', 'src/forwarding/tests.rs'):
|
|
||||||
assert (root / name).exists()
|
|
||||||
for source in (root / 'src').rglob('*.rs'):
|
|
||||||
if '/tests/' in source.as_posix() or source.name == 'tests.rs':
|
|
||||||
continue
|
|
||||||
assert len(source.read_text(encoding='utf-8').splitlines()) <= 500, source
|
|
||||||
print('PASS OPT3 source guards (STATIC ONLY; not cargo check, live HTTP or throughput)')
|
|
||||||
@@ -1,52 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""OPT4 source invariants: not compilation or performance proof."""
|
|
||||||
from pathlib import Path
|
|
||||||
r = Path(__file__).resolve().parents[2]
|
|
||||||
|
|
||||||
def source(name):
|
|
||||||
return (r / name).read_text(encoding="utf-8")
|
|
||||||
|
|
||||||
assert 'version = "0.34.0"' in source('Cargo.toml')
|
|
||||||
route = source('src/proxy/forward/route_cache.rs')
|
|
||||||
assert "Cow<'_, str>" in route
|
|
||||||
assert 'routing_uri.path_and_query()' in route
|
|
||||||
assert 'Ok(None) => public_suffix.clone()' in route
|
|
||||||
assert 'std::borrow::Cow::Owned(encoded)' in route
|
|
||||||
for name in ('src/proxy/forward/cache_lookup.rs','src/proxy/forward/success_finish.rs'):
|
|
||||||
assert 'public_suffix: public_suffix.clone()' not in source(name)
|
|
||||||
assert 'public_suffix: public_suffix.to_string()' in source(name)
|
|
||||||
helpers = source('src/proxy/upstream_helpers.rs')
|
|
||||||
assert 'fn wrap_observed_request' in helpers
|
|
||||||
assert 'if body.is_end_stream() { body }' in helpers
|
|
||||||
assert 'let request = request.map(|body| wrap_observed_request' in helpers
|
|
||||||
assert 'fn log_prefix' in helpers
|
|
||||||
assert 'input.is_ascii()' in helpers and 'input.chars().take(max_chars).collect()' in helpers
|
|
||||||
assert 'path: log_prefix(request.uri().path(), 256)' in helpers
|
|
||||||
assert 'pub request_origin_uri: http::Uri' in source('src/config/model.rs')
|
|
||||||
target = source('src/proxy/types.rs')
|
|
||||||
assert 'fn upstream_request_target(\n uri: Uri' in target
|
|
||||||
assert 'return Some(uri);' in target
|
|
||||||
assert 'let mut parts = uri.into_parts();' in target
|
|
||||||
assert 'upstream_request_target(uri, request.headers_mut()' in target
|
|
||||||
assert 'upstream_request_target(\n uri, request.headers_mut()' in source('src/proxy/forward/upstream.rs')
|
|
||||||
forward = source('src/forwarding.rs')
|
|
||||||
assert 'peer_node.as_deref().unwrap_or(&peer_text)' in forward
|
|
||||||
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forward
|
|
||||||
headers = source('src/proxy/headers.rs')
|
|
||||||
assert 'let no_nominations' in headers
|
|
||||||
assert 'eq_ignore_ascii_case(b"close")' in headers
|
|
||||||
assert 'let nominated: Vec<HeaderName>' in headers
|
|
||||||
for name, test in [
|
|
||||||
('src/proxy/body/tests.rs','optimized_empty_request_keeps_exchange_until_response'),
|
|
||||||
('src/proxy/body/tests.rs','optimized_nonempty_request_still_meters_data_and_holds_capacity'),
|
|
||||||
('src/proxy/tests/authority_tests.rs','optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes'),
|
|
||||||
('src/proxy/tests/authority_tests.rs','moved_uri_keeps_h1_origin_and_query'),
|
|
||||||
('src/proxy/headers/tests.rs','absent_and_close_connection_preserve_non_hop_headers'),
|
|
||||||
]:
|
|
||||||
assert test in source(name), (name, test)
|
|
||||||
for p in (r / 'src').rglob('*.rs'):
|
|
||||||
if '/tests/' in p.as_posix() or p.name == 'tests.rs':
|
|
||||||
continue
|
|
||||||
assert len(p.read_text(encoding='utf-8').splitlines()) <= 500, p
|
|
||||||
assert not source('src/cache/stats.rs').startswith('//!')
|
|
||||||
print('PASS OPT4 source guard (STATIC ONLY; no cargo / no OPT4 benchmark)')
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""OPT5 invariants, static-only. Does not compile or benchmark Rust."""
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parents[2]
|
|
||||||
def read(path):
|
|
||||||
return (root / path).read_text(encoding='utf-8')
|
|
||||||
|
|
||||||
assert 'version = "0.34.0"' in read('Cargo.toml')
|
|
||||||
identifier = read('src/proxy/request_id.rs')
|
|
||||||
assert 'rand::random()' in identifier
|
|
||||||
assert 'let mut token: [u8; 32]' in identifier
|
|
||||||
assert 'sequence.to_be_bytes()' in identifier
|
|
||||||
assert '*byte ^= sequence_byte;' in identifier
|
|
||||||
assert 'request_id_hex(&token)' in identifier
|
|
||||||
assert 'let mut encoded = vec![0u8; digest.len() * 2]' in identifier
|
|
||||||
assert 'Sha256::new()' not in identifier
|
|
||||||
forwarding = read('src/forwarding.rs')
|
|
||||||
assert 'self.context_with_original_url(incoming, peer, local, tls, authority, uri, true)' in forwarding
|
|
||||||
assert forwarding.count('if include_original_url {') == 2
|
|
||||||
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forwarding
|
|
||||||
assert 'crate::config::request_host(authority_text)?;' in forwarding
|
|
||||||
assert 'frontend.is_some_and(|frontend| frontend.tinyauth.is_some())' in read('src/proxy/forward_entry.rs')
|
|
||||||
assert '&context.original_url' in read('src/tinyauth.rs')
|
|
||||||
lease = read('src/balance/lease_runtime.rs')
|
|
||||||
assert 'let failure_detail = is_5xx.then(|| format!("status={status}"));' in lease
|
|
||||||
assert 'let detail = failure_detail.as_deref().unwrap_or(reason);' in lease
|
|
||||||
assert 'observe_passive_with_detail(' in lease
|
|
||||||
assert 'lease.response(' in read('src/proxy/forward/upstream.rs') or 'lease.response(' in read('src/proxy/forward/success_prepare.rs')
|
|
||||||
analytics = read('src/traffic.rs')
|
|
||||||
for literal in ['mpsc::channel::<Sample>(QUEUE_CAPACITY)', 'recv_many(&mut batch, 64)',
|
|
||||||
'for sample in batch.drain(..)', 'get_mut(sample.vhost.as_str())',
|
|
||||||
'state.vhosts.len() < MAX_VHOSTS', 'try_reserve()',
|
|
||||||
'self.dropped.fetch_add(1, Ordering::Relaxed)',
|
|
||||||
'candidate.eq_ignore_ascii_case(needle.as_bytes())',
|
|
||||||
'2 => self.statuses.add("2xx", 1)', 'map(Counters::view)']:
|
|
||||||
assert literal in analytics, literal
|
|
||||||
assert 'let lower = value.to_ascii_lowercase();' not in analytics
|
|
||||||
helpers = read('src/proxy/upstream_helpers.rs')
|
|
||||||
assert 'preferred_encoding(&original_headers, global_compression).is_some()' in helpers
|
|
||||||
assert 'compression.resolve(' not in helpers
|
|
||||||
assert 'compression::encode(' in helpers
|
|
||||||
assert 'wrap_observed_request(body, &exchange' in helpers
|
|
||||||
for rel, case in [
|
|
||||||
('src/traffic/tests.rs', 'batched_worker_keeps_exact_counts_and_bounded_vhosts'),
|
|
||||||
('src/traffic/tests.rs', 'status_top_list_preserves_common_and_extension_family_names'),
|
|
||||||
('src/forwarding/tests.rs', 'original_url_is_omitted_only_when_unneeded_and_forwarding_is_identical'),
|
|
||||||
('src/compression/tests.rs', 'unaccepted_codecs_cannot_be_reenabled_by_scoped_overrides'),
|
|
||||||
('src/balance/tests.rs', 'successful_lease_response_keeps_passive_observability_and_failure_detail'),
|
|
||||||
('src/proxy/request_id/tests.rs', 'independent_sequential_ids_keep_full_length_and_unique_values'),
|
|
||||||
]:
|
|
||||||
assert case in read(rel), (rel, case)
|
|
||||||
for path in (root / 'src').rglob('*.rs'):
|
|
||||||
if path.name == 'tests.rs' or '/tests/' in path.as_posix():
|
|
||||||
continue
|
|
||||||
assert len(path.read_text(encoding='utf-8').splitlines()) <= 500, path
|
|
||||||
assert not read('src/cache/stats.rs').startswith('//!')
|
|
||||||
print('PASS OPT5 static source checks; Rust build and runtime acceptance still required')
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Source-only guardrails for the uncompiled 0.34.0 hot-path candidate."""
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
def src(path):
|
|
||||||
return (ROOT / path).read_text()
|
|
||||||
|
|
||||||
snapshot = src("src/config/snapshot.rs")
|
|
||||||
assert "fn host_patterns(host: &str) -> impl Iterator" in snapshot
|
|
||||||
assert "pub fn select_on_arc(" in snapshot
|
|
||||||
assert "self.select_on_arc(host, path, local, tls).map(Arc::as_ref)" in snapshot
|
|
||||||
assert "Vec<Arc<CompiledRoute>>" in src("src/config/model.rs")
|
|
||||||
assert "route: Option<Arc<crate::config::CompiledRoute>>" in src("src/proxy/types.rs")
|
|
||||||
assert "request.extensions_mut().insert(SelectedRouteContext" in src("src/proxy/upstream_helpers.rs")
|
|
||||||
assert "selection.route.as_deref()" in src("src/proxy/forward_entry.rs")
|
|
||||||
assert "let fallback_request = selected" in src("src/proxy/forward_entry.rs")
|
|
||||||
assert "filter(|route| route.error_fallback.is_some())" in src("src/proxy/forward_entry.rs")
|
|
||||||
assert "let Some(next_request) = replay_headers" in src("src/proxy/forward/upstream.rs") or "replay_headers.as_ref().and_then" in src("src/proxy/forward/upstream.rs")
|
|
||||||
assert "let replay_headers = ((pool_retry_possible || h2_replay_possible)" in src("src/proxy/forward/upstream.rs")
|
|
||||||
pool = src("src/balance/pool_core.rs")
|
|
||||||
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
|
|
||||||
assert "let mut scheduling = lock(&self.scheduling);" in pool
|
|
||||||
assert "fn candidates_excluding" in pool
|
|
||||||
assert "health: member.health_settings.clone()" in pool
|
|
||||||
assert "health: selected.health_settings.clone()" in pool
|
|
||||||
assert "health_settings: Arc<HealthCheck>" in src("src/balance/health.rs")
|
|
||||||
assert "healthy_cached: AtomicBool" in src("src/balance/model.rs")
|
|
||||||
assert "self.healthy_cached.store(h.healthy, Ordering::Release)" in src("src/balance/health.rs")
|
|
||||||
exchange = src("src/proxy/body.rs")
|
|
||||||
for field in ("source", "cache_effective_enabled", "transform_effective_enabled"):
|
|
||||||
assert f"pub {field}: AtomicU8" in exchange
|
|
||||||
assert f"pub {field}: Mutex" not in exchange
|
|
||||||
assert "exchange.set_source(" in src("src/proxy/forward/route_cache.rs")
|
|
||||||
assert "exchange.set_cache_enabled(" in src("src/proxy/forward/web_source.rs")
|
|
||||||
assert "self.live.receiver_count() != 0" in src("src/logging/format.rs")
|
|
||||||
assert "fn route_handles_reuse_snapshot_ownership_without_cloning_compiled_routes" in src("src/config/tests.rs")
|
|
||||||
assert "fn single_backend_fast_path_observes_capacity_and_health_without_affinity_cookie" in src("src/balance/tests.rs")
|
|
||||||
assert "fn atomic_exchange_tags_keep_original_observability_fields" in src("src/proxy/body/tests.rs")
|
|
||||||
for path in (ROOT / "src").rglob("*.rs"):
|
|
||||||
if "/tests/" in path.as_posix() or path.name == "tests.rs" or path.is_relative_to(ROOT / "tests"):
|
|
||||||
continue
|
|
||||||
assert len(path.read_text().splitlines()) <= 500, path
|
|
||||||
assert 'version = "0.34.0"' in src("Cargo.toml")
|
|
||||||
assert "max_idle_per_host: 64" in src("src/balance/model.rs")
|
|
||||||
assert "pool_recycle_requests: 25_000" in src("src/balance/model.rs")
|
|
||||||
print("PASS 0.34.0 proxy hot-path wiring, observability, bounded tuned defaults and source-size limits (STATIC ONLY)")
|
|
||||||
@@ -1,129 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Source invariants for 0.34.0 RPS optimizations (NOT a Rust build or RPS test).
|
|
||||||
|
|
||||||
These guards reject accidental removal of important branch/safety conditions.
|
|
||||||
They must be paired with ``cargo test`` and live black-box HTTP acceptance.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
|
|
||||||
|
|
||||||
def source(path):
|
|
||||||
return (ROOT / path).read_text(encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
# Random request identifiers still use the OS-seeded thread-local CSPRNG,
|
|
||||||
# preserve a 256-bit unpredictable 64-hex token and mix the per-process sequence.
|
|
||||||
# OPT5 removes the extra SHA-256 operation, not the entropy or correlation ID.
|
|
||||||
identifier = source("src/proxy/request_id.rs")
|
|
||||||
assert "rand::random()" in identifier
|
|
||||||
assert "sequence.to_be_bytes()" in identifier
|
|
||||||
assert "*byte ^= sequence_byte;" in identifier
|
|
||||||
assert "request_id_hex(&token)" in identifier
|
|
||||||
assert "SysRng.try_fill_bytes" not in identifier
|
|
||||||
|
|
||||||
# Compiled/validated target authority survives request URI construction.
|
|
||||||
config = source("src/config/backend.rs")
|
|
||||||
model = source("src/config/model.rs")
|
|
||||||
request = source("src/proxy/types.rs")
|
|
||||||
assert "let request_origin_uri: http::Uri = request_origin.parse()?;" in config
|
|
||||||
assert "request_origin_uri," in config
|
|
||||||
assert "pub request_origin_uri: http::Uri," in model
|
|
||||||
assert "let path: http::uri::PathAndQuery = suffix.parse().ok()?;" in request
|
|
||||||
assert "let mut parts = backend.request_origin_uri.clone().into_parts();" in request
|
|
||||||
assert "parts.path_and_query = Some(path);" in request
|
|
||||||
assert "Uri::from_parts(parts).ok()" in request
|
|
||||||
|
|
||||||
# No new unbounded caches or disabled LB safety gates.
|
|
||||||
pool = source("src/balance/pool_core.rs")
|
|
||||||
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
|
|
||||||
assert "try_acquire_active(member.definition.max_concurrent_requests)" in pool
|
|
||||||
assert "m.state.healthy()" in pool
|
|
||||||
assert "scheduling.get_mut(member.definition.id.as_str())" in pool
|
|
||||||
assert "scheduling.entry(member.definition.id.clone())" in pool
|
|
||||||
health = source("src/balance/lease_runtime.rs")
|
|
||||||
assert "async fn probe(member: Arc<Member>, settings: Arc<HealthCheck>)" in health
|
|
||||||
assert "checks.spawn(probe(Arc::clone(member), health.clone()))" in health
|
|
||||||
assert "tokio::time::timeout" in health
|
|
||||||
|
|
||||||
# Configurable keep-alive policy, exact recycle-request threshold and age backup.
|
|
||||||
upstream = source("src/upstream/client.rs")
|
|
||||||
assert "self.current.load_full()" in upstream
|
|
||||||
assert "request_due || age_due" in upstream
|
|
||||||
assert "let now = monotonic_millis();" in upstream
|
|
||||||
assert "let age_ms = now.saturating_sub(self.generation_started_ms.load(Ordering::Relaxed));" in upstream
|
|
||||||
assert "fn maintain_rotation_at(" in upstream
|
|
||||||
assert "self.factory.keepalive.pool_recycle_requests" in upstream
|
|
||||||
assert "self.factory.keepalive.pool_recycle_seconds" in upstream
|
|
||||||
assert "pool_max_idle_per_host(if keepalive.enabled" in upstream
|
|
||||||
assert "retry_canceled_requests(false)" in upstream
|
|
||||||
assert "observe_protocol" in upstream
|
|
||||||
|
|
||||||
# Request/response streamed payloads retain accounting and timeout/cancellation.
|
|
||||||
body = source("src/proxy/body.rs")
|
|
||||||
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
|
|
||||||
assert "this.request_backend = exchange.backend" in body
|
|
||||||
assert "lease.bytes(count, true)" in body
|
|
||||||
assert "lease.bytes(count, false)" in body
|
|
||||||
assert "exchange.request_bytes.fetch_add(count" in body
|
|
||||||
assert "exchange.response_bytes.fetch_add(count" in body
|
|
||||||
assert "get_or_insert_with(|| Box::pin(sleep_until(this.deadline)))" in body
|
|
||||||
assert "this.finish();" in body
|
|
||||||
|
|
||||||
# Persist history and preserve live streaming/sink drop accounting.
|
|
||||||
logs = source("src/logging/format.rs")
|
|
||||||
assert "let has_viewers = self.live.receiver_count() != 0;" in logs
|
|
||||||
assert "let has_sink = self.settings.load().enabled();" in logs
|
|
||||||
assert "external.take().expect(\"entry\")" in logs
|
|
||||||
assert "self.live.send(entry.clone())" in logs
|
|
||||||
assert "self.sink.try_send(entry)" in logs
|
|
||||||
assert "self.sink_dropped.fetch_add(1" in logs
|
|
||||||
|
|
||||||
# Never weaken routing, identity, HTTP version and cache eligibility while tuning.
|
|
||||||
route = source("src/proxy/forward/route_cache.rs")
|
|
||||||
assert "admin_session_passthrough" in route
|
|
||||||
assert "cache_plan.request_policy" in route
|
|
||||||
assert "exchange.set_cache_enabled" in route
|
|
||||||
assert "exchange.set_transform_enabled" in route
|
|
||||||
assert "upstream_uri(&backend, &origin_suffix)" in route
|
|
||||||
assert "headers::strip_hop_by_hop" in route
|
|
||||||
assert "pool.select_wait(context.client_ip" in route or ".select_wait(context.client_ip" in route
|
|
||||||
assert "pool.select_member_wait(id)" in route
|
|
||||||
assert "max_idle_per_host: 64" in source("src/balance/model.rs")
|
|
||||||
assert "pool_recycle_requests: 25_000" in source("src/balance/model.rs")
|
|
||||||
assert "(1..=64).contains(&k.max_idle_per_host)" in source("src/balance/model.rs")
|
|
||||||
assert "keepalive.max_idle_per_host ?? 64" in source("src/admin/web/vhosts/editor.js")
|
|
||||||
openapi = json.loads(source("src/admin/openapi.json"))
|
|
||||||
api_keepalive = openapi["components"]["schemas"]
|
|
||||||
assert any(isinstance(schema, dict) and "keepalive" in schema.get("properties", {})
|
|
||||||
and schema["properties"]["keepalive"].get("properties", {})
|
|
||||||
.get("max_idle_per_host", {}).get("default") == 64
|
|
||||||
for schema in api_keepalive.values())
|
|
||||||
assert "| `max_idle_per_host` | 64 | 1..64 |" in source("docs/VHOSTS.md")
|
|
||||||
assert '"keepalive": { "max_idle_per_host": 4 }' in source("src/balance/tests.rs")
|
|
||||||
assert "self.route.get_mut()" in source("src/proxy/body.rs")
|
|
||||||
assert "let client_ip = self" in source("src/proxy/body.rs")
|
|
||||||
assert "if !headers.contains_key(http::header::COOKIE)" in source("src/proxy/headers.rs")
|
|
||||||
assert "if !headers.contains_key(http::header::SET_COOKIE)" in source("src/proxy/headers.rs")
|
|
||||||
assert 'version = "0.34.0"' in source("Cargo.toml")
|
|
||||||
|
|
||||||
# Full exchange duration histogram still exports cumulative Prometheus series,
|
|
||||||
# but only one exclusive bucket is updated per request; per-vhost and Docker
|
|
||||||
# exporters must reconstruct bucket counts from exactly the same interface.
|
|
||||||
histogram = source("src/metrics/vhost.rs")
|
|
||||||
exporter = source("src/prometheus/docker_misc.rs")
|
|
||||||
assert "let bucket = DURATION_BUCKETS_US.partition_point" in histogram
|
|
||||||
assert "self.buckets[bucket].fetch_add(1, Ordering::Relaxed);" in histogram
|
|
||||||
assert "pub fn duration_histogram(&self)" in histogram
|
|
||||||
assert exporter.count("let (buckets, count) = m.duration_histogram();") == 2
|
|
||||||
assert "rust_proxy_vhost_exchange_duration_seconds_bucket" in exporter
|
|
||||||
assert "rust_proxy_docker_vhost_exchange_duration_seconds_bucket" in exporter
|
|
||||||
assert "exclusive_bins_reconstruct_exact_prometheus_cumulative_boundaries" in source("src/metrics/vhost/tests.rs")
|
|
||||||
|
|
||||||
for file in (ROOT / "src").rglob("*.rs"):
|
|
||||||
if "/tests/" in file.as_posix() or file.name == "tests.rs":
|
|
||||||
continue
|
|
||||||
assert len(file.read_text(encoding="utf-8").splitlines()) <= 500, file
|
|
||||||
print("PASS RPS hot-path source invariants (STATIC ONLY; new binary not compiled)")
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Offline Chromium test: sendfile toggle serializes in real admin bundle.
|
|
||||||
|
|
||||||
It uses simulated config and a mocked saveSection; never claims native
|
|
||||||
sendfile execution, Rust compilation or live server acceptance.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
ROUTER = (ROOT / "src/admin/router.rs").read_text()
|
|
||||||
start = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert start
|
|
||||||
array = ROUTER[start.end():]
|
|
||||||
array = array[:re.search(r'\]\s*\.join\("\\n"\)', array).start()]
|
|
||||||
chunks = []
|
|
||||||
for line in array.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
match = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if match:
|
|
||||||
chunks.append((ROOT / "src/admin" / match.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
chunks.append('(() => {\n' + '\n'.join(
|
|
||||||
(ROOT / 'src/admin/web/vhosts' / (f + '.js')).read_text()
|
|
||||||
for f in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
chunks.append(json.loads(line[:-1]))
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (375, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 950})
|
|
||||||
page.set_default_timeout(4500)
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda err: errors.append(str(err)))
|
|
||||||
page.set_content(render_admin_index(ROOT))
|
|
||||||
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
|
|
||||||
page.add_script_tag(content='\n'.join(chunks))
|
|
||||||
page.evaluate('''() => {
|
|
||||||
envelope = {revision: 10, config: {web_services: [{
|
|
||||||
id: 'site-A', name: 'Static Site', document_root: '/srv/static',
|
|
||||||
index: ['index.html'], php: {enabled: false}
|
|
||||||
}], vhosts: [{id: 'vhost-A', keepalive: {max_connections: 8}}], php_fpm_profiles: []}};
|
|
||||||
currentPanel = 'web-server';
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('login').hidden = true;
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-web-server').hidden = false;
|
|
||||||
window.__saved = [];
|
|
||||||
saveSection = async (section, value) => {
|
|
||||||
window.__saved.push({section, value: structuredClone(value)});
|
|
||||||
if (section === 'web-services') envelope.config.web_services = structuredClone(value);
|
|
||||||
return true;
|
|
||||||
};
|
|
||||||
window.rustProxyWebServer.render();
|
|
||||||
}''')
|
|
||||||
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
|
|
||||||
assert page.locator('#web-service-sendfile').input_value() == 'off'
|
|
||||||
page.evaluate("document.getElementById('web-service-sendfile').value = 'auto'")
|
|
||||||
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
|
|
||||||
page.wait_for_function('window.__saved.length === 1')
|
|
||||||
result = page.evaluate('''() => ({
|
|
||||||
saved: window.__saved[0],
|
|
||||||
cacheStillUnrelated: envelope.config.vhosts[0].keepalive.max_connections === 8,
|
|
||||||
closed: !document.getElementById('web-service-dialog').open
|
|
||||||
})''')
|
|
||||||
assert result['saved']['section'] == 'web-services', result
|
|
||||||
assert result['saved']['value'][0]['sendfile'] == 'auto', result
|
|
||||||
assert result['saved']['value'][0]['document_root'] == '/srv/static', result
|
|
||||||
assert result['cacheStillUnrelated'] and result['closed'], result
|
|
||||||
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
|
|
||||||
assert page.locator('#web-service-sendfile').input_value() == 'auto'
|
|
||||||
page.evaluate("document.getElementById('web-service-sendfile').value = 'off'")
|
|
||||||
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
|
|
||||||
page.wait_for_function('window.__saved.length === 2')
|
|
||||||
assert page.evaluate('envelope.config.web_services[0].sendfile') == 'off'
|
|
||||||
assert not errors, errors
|
|
||||||
print(f'PASS simulated panel sendfile off -> auto -> off config save at {width}px')
|
|
||||||
browser.close()
|
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""RustProxy 0.34.0 sendfile source-wiring assertions ONLY, not native tests."""
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
|
|
||||||
root = Path(__file__).resolve().parents[2]
|
|
||||||
get = lambda path: (root / path).read_text()
|
|
||||||
|
|
||||||
model = get('src/web_server/config/model.rs')
|
|
||||||
assert 'pub enum SendfileMode' in model and 'sendfile: SendfileMode::Off' in model
|
|
||||||
assert 'pub sendfile: SendfileMode' in model
|
|
||||||
assert 'sendfile: self.sendfile' in get('src/web_server/config/compile.rs')
|
|
||||||
assert 'pub sendfile: SendfileMode' in get('src/web_server/config/validation.rs')
|
|
||||||
assert 'service.sendfile = crate::web_server::SendfileMode::Off;' in get('src/runtime/types.rs')
|
|
||||||
source = get('src/proxy/forward/web_source.rs')
|
|
||||||
for guard in ('!enabled && identity_request', 'request_method == Method::GET',
|
|
||||||
'response.status() == StatusCode::OK', 'content-encoding',
|
|
||||||
'transfer-encoding', 'content-range', 'candidate.length',
|
|
||||||
'http::Version::HTTP_11'):
|
|
||||||
assert guard in source, guard
|
|
||||||
assert 'content_encoding.is_none()' in get('src/web_server/response/file.rs')
|
|
||||||
assert 'single_range.is_none()' in get('src/web_server/response/file.rs')
|
|
||||||
assert 'opened.file.try_clone()' in get('src/web_server/response/file.rs')
|
|
||||||
bridge = get('src/server/sendfile.rs')
|
|
||||||
for item in ('libc::sendfile(', 'MAX_PENDING: usize = 8', 'MIN_SIZE: u64 = 64 * 1024',
|
|
||||||
'registry.slots.is_empty()', 'native_sendfile_transfers_exact_file',
|
|
||||||
'sendfile placeholder did not match', 'UnexpectedEof'):
|
|
||||||
assert item in bridge, item
|
|
||||||
assert 'pub(crate) mod sendfile' in get('src/server.rs')
|
|
||||||
assert 'SendfileIo::new(stream)' in get('src/server/tls_runtime.rs')
|
|
||||||
assert 'sendfile_slots' in get('src/server/http1_connection.rs')
|
|
||||||
js = get('src/admin/web/web-server.js')
|
|
||||||
html = get('src/admin/web/index.html')
|
|
||||||
assert '"web-service-sendfile"' in js and 'sendfile: "off"' in js
|
|
||||||
assert 'sendfile: $("web-service-sendfile").value' in js
|
|
||||||
assert html.count('id="web-service-sendfile"') == 1
|
|
||||||
assert 'value="auto"' in html
|
|
||||||
for locale in ('en_US', 'pl_pl'):
|
|
||||||
keys = json.loads(get(f'langs/{locale}.json'))['translations']
|
|
||||||
assert all(k in keys for k in ('webServiceSendfile', 'webServiceSendfileHelp', 'webServiceSendfileAuto'))
|
|
||||||
api = json.loads(get('src/admin/openapi.json'))
|
|
||||||
assert 'sendfile' in json.dumps(api)
|
|
||||||
print('PASS 0.34.0 sendfile source guards, default OFF, Linux syscall, UI, translations, API and scoped cache (STATIC ONLY)')
|
|
||||||
@@ -1,116 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Verify sidebar colors use only the UI theme; chart palettes affect charts only.
|
|
||||||
|
|
||||||
Uses actual production HTML/CSS in headless Chromium, not a live RustProxy
|
|
||||||
server or authenticated preference/session roundtrip.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
|
|
||||||
THEMES = ('dark', 'ultra_dark', 'light', 'cream')
|
|
||||||
PALETTES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
|
|
||||||
CORNERS = {'default': 0, 'square': 0, 'subtle': 4, 'rounded': 8, 'extra': 12}
|
|
||||||
|
|
||||||
STYLE_INFO = r'''() => {
|
|
||||||
const color = value => {
|
|
||||||
const c = document.createElement('canvas'); c.width = c.height = 1;
|
|
||||||
const ctx = c.getContext('2d'); ctx.fillStyle = value; ctx.fillRect(0, 0, 1, 1);
|
|
||||||
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
|
|
||||||
};
|
|
||||||
const root = getComputedStyle(document.documentElement);
|
|
||||||
const active = document.querySelector('.sidebar .main-navigation .nav.active');
|
|
||||||
const idle = document.querySelector('.sidebar .main-navigation .nav:not(.active)');
|
|
||||||
const sa = getComputedStyle(active), si = getComputedStyle(idle);
|
|
||||||
return {
|
|
||||||
chartAccent:color(root.getPropertyValue('--chart-accent').trim()),
|
|
||||||
themeSelected:color(root.getPropertyValue('--nav-active-bg').trim()),
|
|
||||||
themeHovered:color(root.getPropertyValue('--nav-hover-bg').trim()),
|
|
||||||
foreground:color(sa.color),
|
|
||||||
background:color(sa.backgroundColor),
|
|
||||||
borderLeft:color(sa.borderLeftColor),
|
|
||||||
borderRight:color(sa.borderRightColor),
|
|
||||||
radius:parseFloat(sa.borderTopLeftRadius),
|
|
||||||
idleBackground:color(si.backgroundColor),
|
|
||||||
mobileBackground:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).backgroundColor),
|
|
||||||
mobileColor:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).color),
|
|
||||||
navWidth:document.querySelector('.main-navigation').scrollWidth,
|
|
||||||
navClientWidth:document.querySelector('.main-navigation').clientWidth,
|
|
||||||
activeWidth:active.scrollWidth,
|
|
||||||
activeClientWidth:active.clientWidth,
|
|
||||||
};
|
|
||||||
}'''
|
|
||||||
|
|
||||||
def contrast(fg, bg):
|
|
||||||
def luminance(rgb):
|
|
||||||
channels = [x / 255 for x in rgb]
|
|
||||||
channels = [x / 12.92 if x <= 0.04045 else ((x + 0.055) / 1.055) ** 2.4 for x in channels]
|
|
||||||
return .2126 * channels[0] + .7152 * channels[1] + .0722 * channels[2]
|
|
||||||
lo, hi = sorted((luminance(fg), luminance(bg)))
|
|
||||||
return (hi + .05) / (lo + .05)
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for viewport in (375, 1200):
|
|
||||||
page = browser.new_page(viewport={'width': viewport, 'height': 820})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', '').replace('/assets/app___RUSTPROXY_ASSET_ID__.js', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.evaluate('''() => {
|
|
||||||
document.documentElement.classList.remove('app-loading');
|
|
||||||
document.getElementById('app-startup').remove();
|
|
||||||
document.querySelector('.sidebar').classList.add('mobile-nav-open');
|
|
||||||
document.querySelector('.mobile-menu-toggle')?.setAttribute('aria-expanded', 'true');
|
|
||||||
}''')
|
|
||||||
theme_colors = []
|
|
||||||
for theme in THEMES:
|
|
||||||
baseline = None
|
|
||||||
chart_accents = set()
|
|
||||||
for palette in PALETTES:
|
|
||||||
page.evaluate('''([theme, palette]) => {
|
|
||||||
document.documentElement.dataset.theme = theme;
|
|
||||||
document.documentElement.dataset.chartStyle = palette;
|
|
||||||
}''', [theme, palette])
|
|
||||||
for corner, expected_radius in CORNERS.items():
|
|
||||||
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
|
|
||||||
style = page.evaluate(STYLE_INFO)
|
|
||||||
assert style['radius'] == expected_radius, (viewport, theme, palette, corner, style['radius'])
|
|
||||||
assert style['borderLeft'] == style['foreground'], (viewport, theme, palette, corner, style)
|
|
||||||
assert style['background'] == style['themeSelected'], (viewport, theme, palette, corner, style)
|
|
||||||
if viewport == 375:
|
|
||||||
assert style['mobileBackground'] == style['background'], (viewport, theme, palette, corner, style)
|
|
||||||
assert style['mobileColor'] == style['foreground'], (viewport, theme, palette, corner, style)
|
|
||||||
assert contrast(style['foreground'], style['background']) >= 4.5, (viewport, theme, palette, corner, style)
|
|
||||||
assert style['navWidth'] <= style['navClientWidth'] + 2, (viewport, theme, palette, corner, style)
|
|
||||||
assert style['activeWidth'] <= style['activeClientWidth'] + 2, (viewport, theme, palette, corner, style)
|
|
||||||
chart_accents.add(tuple(style['chartAccent']))
|
|
||||||
# The entire selected state is unaffected by chart-style changes.
|
|
||||||
nav_colors = (tuple(style['foreground']), tuple(style['background']), tuple(style['borderLeft']))
|
|
||||||
if baseline is None:
|
|
||||||
baseline = nav_colors
|
|
||||||
else:
|
|
||||||
assert baseline == nav_colors, (viewport, theme, palette, baseline, nav_colors)
|
|
||||||
page.locator('.nav[data-panel="vhosts"]').hover()
|
|
||||||
hovered = page.evaluate(STYLE_INFO)
|
|
||||||
assert hovered['idleBackground'] == hovered['themeHovered'], (viewport, theme, palette, hovered)
|
|
||||||
assert hovered['idleBackground'] != style['idleBackground'], (viewport, theme, palette)
|
|
||||||
page.mouse.move(0, 0)
|
|
||||||
if viewport == 1200 and theme == 'ultra_dark' and palette == 'cool':
|
|
||||||
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_desktop.png'))
|
|
||||||
if viewport == 375 and theme == 'dark' and palette == 'soft':
|
|
||||||
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_mobile.png'))
|
|
||||||
assert len(chart_accents) > 1, (viewport, theme, chart_accents)
|
|
||||||
theme_colors.append(baseline)
|
|
||||||
assert len(set(theme_colors)) == len(THEMES), (viewport, theme_colors)
|
|
||||||
page.evaluate("document.documentElement.dir = 'rtl'")
|
|
||||||
rtl = page.evaluate(STYLE_INFO)
|
|
||||||
assert rtl['borderRight'] == rtl['foreground'], (viewport, rtl)
|
|
||||||
assert not errors, errors
|
|
||||||
print('PASS themed navigation independence: viewport', viewport, 'themes', len(THEMES), 'palettes', len(PALETTES), 'corners', len(CORNERS))
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS: sidebar changes with UI theme and corner setting, never with chart style')
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""System Load RAM meter updates using the actual combined panel JavaScript.
|
|
||||||
|
|
||||||
This is a simulated authenticated DOM/status test, not a Rust runtime integration test.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
router = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
def bundle_for(section):
|
|
||||||
match = re.search(r'fn ' + section + r'\([^)]*\)[^{]*\{', router)
|
|
||||||
assert match, section
|
|
||||||
text = router[match.end():]
|
|
||||||
text = text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
|
|
||||||
result = []
|
|
||||||
for line in text.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if 'include_str!("' in line:
|
|
||||||
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
|
|
||||||
result.append((ROOT / 'src/admin' / path).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n' + ''.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() + '\n'
|
|
||||||
for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
|
|
||||||
html = render_admin_index(ROOT)
|
|
||||||
css = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
js = bundle_for('admin_app_javascript')
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (375, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '').replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=css)
|
|
||||||
page.add_script_tag(content=js)
|
|
||||||
page.wait_for_timeout(100)
|
|
||||||
report = page.evaluate('''() => {
|
|
||||||
currentPanel = "overview";
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
const meter = document.getElementById('memory-meter');
|
|
||||||
const fill = document.getElementById('memory-meter-fill');
|
|
||||||
const snapshot = (used, total) => ({system: {source:'measured',cpu_percent:2.1,
|
|
||||||
memory_used_bytes:used, memory_total_bytes:total},
|
|
||||||
live_system:{cpu_percent:2.1}, cache:{}, metrics:{}, listeners:{}});
|
|
||||||
const values = [];
|
|
||||||
for (const [used, total] of [[250,1000],[800,1000],[0,1000],[null,null]]) {
|
|
||||||
renderDiagnostics(snapshot(used,total), false, true);
|
|
||||||
values.push({fill:fill.style.width, now:meter.getAttribute('aria-valuenow'),
|
|
||||||
hidden:meter.hidden, text:meter.getAttribute('aria-valuetext')});
|
|
||||||
}
|
|
||||||
return { values, role:meter.getAttribute('role'), cssHeight:getComputedStyle(meter).height,
|
|
||||||
scrollbar:document.documentElement.scrollWidth>window.innerWidth+2 };
|
|
||||||
}''')
|
|
||||||
assert report['role'] == 'progressbar' and report['cssHeight'] == '5px', report
|
|
||||||
assert report['values'][0]['fill'] == '25%' and report['values'][0]['now'] == '25', report
|
|
||||||
assert report['values'][1]['fill'] == '80%' and report['values'][1]['now'] == '80', report
|
|
||||||
assert report['values'][2]['fill'] == '0%' and report['values'][2]['now'] == '0', report
|
|
||||||
assert report['values'][3]['fill'] == '0%' and report['values'][3]['now'] is None and report['values'][3]['hidden'], report
|
|
||||||
assert not report['scrollbar'] and not errors, (width, errors, report)
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS: System RAM meter live percentage and unknown/zero handling on 375/1440px in bundled panel.')
|
|
||||||
@@ -1,137 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""TCP Stream black-box test: use against a running RustProxy and mock backends.
|
|
||||||
|
|
||||||
The 'selftest' mode validates this harness only; it does NOT run RustProxy.
|
|
||||||
All traffic uses loopback by default. Do not point it at a production service.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from collections import Counter
|
|
||||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
||||||
import secrets
|
|
||||||
import socket
|
|
||||||
import socketserver
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
|
|
||||||
MAX_LINE = 1024
|
|
||||||
|
|
||||||
|
|
||||||
class TCPServer(socketserver.ThreadingTCPServer):
|
|
||||||
allow_reuse_address = True
|
|
||||||
daemon_threads = True
|
|
||||||
request_queue_size = 128
|
|
||||||
|
|
||||||
|
|
||||||
class TCPBackend(socketserver.StreamRequestHandler):
|
|
||||||
def handle(self) -> None:
|
|
||||||
self.request.settimeout(10)
|
|
||||||
backend_id = self.server.backend_id.encode('ascii')
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
line = self.rfile.readline(MAX_LINE + 1)
|
|
||||||
if not line or len(line) > MAX_LINE or not line.endswith(b'\n'):
|
|
||||||
return
|
|
||||||
self.wfile.write(backend_id + b'|' + line)
|
|
||||||
self.wfile.flush()
|
|
||||||
except (OSError, TimeoutError):
|
|
||||||
return
|
|
||||||
|
|
||||||
|
|
||||||
def run_server(host: str, port: int, backend_id: str) -> None:
|
|
||||||
with TCPServer((host, port), TCPBackend) as server:
|
|
||||||
server.backend_id = backend_id
|
|
||||||
print(f'TCP mock {backend_id} listening on {server.server_address}', flush=True)
|
|
||||||
try:
|
|
||||||
server.serve_forever(poll_interval=0.1)
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def check_one(index: int, host: str, port: int, messages: int, timeout: float, hold_ms: float, nonce: str) -> str:
|
|
||||||
backend = None
|
|
||||||
with socket.create_connection((host, port), timeout=timeout) as connection:
|
|
||||||
connection.settimeout(timeout)
|
|
||||||
with connection.makefile('rb') as reader:
|
|
||||||
for seq in range(messages):
|
|
||||||
payload = f'rp-tcp:{nonce}:{index}:{seq}\n'.encode('ascii')
|
|
||||||
connection.sendall(payload)
|
|
||||||
response = reader.readline(MAX_LINE + 1)
|
|
||||||
try:
|
|
||||||
observed, echoed = response.split(b'|', 1)
|
|
||||||
except ValueError as exc:
|
|
||||||
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
|
|
||||||
assert echoed == payload, f'session {index}: response mismatch: {response!r}'
|
|
||||||
assert observed, f'session {index}: empty backend ID'
|
|
||||||
if backend is None:
|
|
||||||
backend = observed
|
|
||||||
assert observed == backend, f'session {index}: TCP backend changed during connection'
|
|
||||||
if hold_ms and seq != messages - 1:
|
|
||||||
time.sleep(hold_ms / 1000)
|
|
||||||
return backend.decode('ascii')
|
|
||||||
|
|
||||||
|
|
||||||
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
|
|
||||||
hold_ms: float, min_backends: int) -> Counter:
|
|
||||||
nonce = secrets.token_hex(8)
|
|
||||||
started = time.monotonic()
|
|
||||||
counts: Counter[str] = Counter()
|
|
||||||
with ThreadPoolExecutor(max_workers=parallel) as pool:
|
|
||||||
futures = [pool.submit(check_one, i, host, port, messages, timeout, hold_ms, nonce)
|
|
||||||
for i in range(sessions)]
|
|
||||||
for future in as_completed(futures):
|
|
||||||
counts[future.result()] += 1
|
|
||||||
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
|
|
||||||
print(f'TCP PASS: {sessions} sessions, {sessions * messages} messages, '
|
|
||||||
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
|
|
||||||
return counts
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
sub = parser.add_subparsers(dest='mode', required=True)
|
|
||||||
server = sub.add_parser('server', help='run a local TCP mock backend')
|
|
||||||
server.add_argument('--host', default='127.0.0.1')
|
|
||||||
server.add_argument('--port', type=int, required=True)
|
|
||||||
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
|
|
||||||
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy listener')
|
|
||||||
client.add_argument('--host', default='127.0.0.1', help='RustProxy TCP listener host')
|
|
||||||
client.add_argument('--port', type=int, required=True, help='RustProxy TCP listener port')
|
|
||||||
client.add_argument('--sessions', type=int, default=24)
|
|
||||||
client.add_argument('--messages', type=int, default=3)
|
|
||||||
client.add_argument('--parallel', type=int, default=8)
|
|
||||||
client.add_argument('--timeout', type=float, default=4.0)
|
|
||||||
client.add_argument('--hold-ms', type=float, default=20.0, help='pause between messages to keep TCP sessions alive')
|
|
||||||
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
|
|
||||||
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.mode == 'server':
|
|
||||||
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
|
|
||||||
parser.error('--backend-id must be nonempty ASCII without | or newline')
|
|
||||||
run_server(args.host, args.port, args.backend_id)
|
|
||||||
elif args.mode == 'client':
|
|
||||||
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.hold_ms < 0:
|
|
||||||
parser.error('invalid port, count, timeout or hold delay')
|
|
||||||
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
|
|
||||||
args.timeout, args.hold_ms, args.min_backends)
|
|
||||||
else:
|
|
||||||
with TCPServer(('127.0.0.1', 0), TCPBackend) as server:
|
|
||||||
server.backend_id = 'local-mock'
|
|
||||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
|
||||||
thread.start()
|
|
||||||
try:
|
|
||||||
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
|
|
||||||
print('SELFTEST PASS: tested mock TCP peer, not RustProxy')
|
|
||||||
finally:
|
|
||||||
server.shutdown()
|
|
||||||
thread.join(timeout=2)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
try:
|
|
||||||
main()
|
|
||||||
except (OSError, AssertionError, UnicodeError) as exc:
|
|
||||||
print(f'TCP FAIL: {exc}', file=sys.stderr)
|
|
||||||
sys.exit(1)
|
|
||||||
@@ -1,135 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""UDP Stream black-box test: sends multiple packets from each stable client socket.
|
|
||||||
|
|
||||||
Start separately labeled mock backends and target an existing RustProxy UDP
|
|
||||||
listener. 'selftest' only validates this Python harness, NOT RustProxy.
|
|
||||||
By default, the server responds PONG to PING for an explicitly configured probe.
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from collections import Counter
|
|
||||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
||||||
import secrets
|
|
||||||
import socket
|
|
||||||
import socketserver
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
|
|
||||||
MAX_DATAGRAM = 1024
|
|
||||||
|
|
||||||
|
|
||||||
class UDPServer(socketserver.UDPServer):
|
|
||||||
allow_reuse_address = True
|
|
||||||
|
|
||||||
|
|
||||||
class UDPBackend(socketserver.BaseRequestHandler):
|
|
||||||
def handle(self) -> None:
|
|
||||||
data, connection = self.request
|
|
||||||
if data == b'PING':
|
|
||||||
reply = b'PONG'
|
|
||||||
elif data and len(data) <= MAX_DATAGRAM:
|
|
||||||
reply = self.server.backend_id.encode('ascii') + b'|' + data
|
|
||||||
else:
|
|
||||||
return
|
|
||||||
connection.sendto(reply, self.client_address)
|
|
||||||
|
|
||||||
|
|
||||||
def run_server(host: str, port: int, backend_id: str) -> None:
|
|
||||||
with UDPServer((host, port), UDPBackend) as server:
|
|
||||||
server.backend_id = backend_id
|
|
||||||
print(f'UDP mock {backend_id} listening on {server.server_address}; health probe PING -> PONG', flush=True)
|
|
||||||
try:
|
|
||||||
server.serve_forever(poll_interval=0.1)
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def check_one(index: int, host: str, port: int, messages: int, timeout: float, pause_ms: float, nonce: str) -> str:
|
|
||||||
info = socket.getaddrinfo(host, port, type=socket.SOCK_DGRAM)[0]
|
|
||||||
family, sock_type, proto, _, address = info
|
|
||||||
backend = None
|
|
||||||
# One socket = one stable client identity for the entire UDP Stream session.
|
|
||||||
with socket.socket(family, sock_type, proto) as connection:
|
|
||||||
connection.settimeout(timeout)
|
|
||||||
for seq in range(messages):
|
|
||||||
payload = f'rp-udp:{nonce}:{index}:{seq}'.encode('ascii')
|
|
||||||
connection.sendto(payload, address)
|
|
||||||
response, _ = connection.recvfrom(MAX_DATAGRAM + 128)
|
|
||||||
try:
|
|
||||||
observed, echoed = response.split(b'|', 1)
|
|
||||||
except ValueError as exc:
|
|
||||||
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
|
|
||||||
assert echoed == payload, f'session {index}: response mismatch {response!r}'
|
|
||||||
assert observed, f'session {index}: empty backend ID'
|
|
||||||
if backend is None:
|
|
||||||
backend = observed
|
|
||||||
assert observed == backend, f'session {index}: UDP backend changed for the same client socket'
|
|
||||||
if pause_ms and seq != messages - 1:
|
|
||||||
time.sleep(pause_ms / 1000)
|
|
||||||
return backend.decode('ascii')
|
|
||||||
|
|
||||||
|
|
||||||
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
|
|
||||||
pause_ms: float, min_backends: int) -> Counter:
|
|
||||||
nonce = secrets.token_hex(8)
|
|
||||||
started = time.monotonic()
|
|
||||||
counts: Counter[str] = Counter()
|
|
||||||
with ThreadPoolExecutor(max_workers=parallel) as pool:
|
|
||||||
futures = [pool.submit(check_one, i, host, port, messages, timeout, pause_ms, nonce)
|
|
||||||
for i in range(sessions)]
|
|
||||||
for future in as_completed(futures):
|
|
||||||
counts[future.result()] += 1
|
|
||||||
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
|
|
||||||
print(f'UDP PASS: {sessions} client sockets, {sessions * messages} datagrams, '
|
|
||||||
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
|
|
||||||
return counts
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
sub = parser.add_subparsers(dest='mode', required=True)
|
|
||||||
server = sub.add_parser('server', help='run local UDP echo and optional PING/PONG probe responder')
|
|
||||||
server.add_argument('--host', default='127.0.0.1')
|
|
||||||
server.add_argument('--port', type=int, required=True)
|
|
||||||
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
|
|
||||||
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy UDP listener')
|
|
||||||
client.add_argument('--host', default='127.0.0.1', help='RustProxy UDP listener host')
|
|
||||||
client.add_argument('--port', type=int, required=True, help='RustProxy UDP listener port')
|
|
||||||
client.add_argument('--sessions', type=int, default=24)
|
|
||||||
client.add_argument('--messages', type=int, default=3)
|
|
||||||
client.add_argument('--parallel', type=int, default=8)
|
|
||||||
client.add_argument('--timeout', type=float, default=4.0)
|
|
||||||
client.add_argument('--pause-ms', type=float, default=20.0, help='pause between datagrams on each stable client socket')
|
|
||||||
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
|
|
||||||
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.mode == 'server':
|
|
||||||
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
|
|
||||||
parser.error('--backend-id must be nonempty ASCII without | or newline')
|
|
||||||
run_server(args.host, args.port, args.backend_id)
|
|
||||||
elif args.mode == 'client':
|
|
||||||
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.pause_ms < 0:
|
|
||||||
parser.error('invalid port, count, timeout or pause delay')
|
|
||||||
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
|
|
||||||
args.timeout, args.pause_ms, args.min_backends)
|
|
||||||
else:
|
|
||||||
with UDPServer(('127.0.0.1', 0), UDPBackend) as server:
|
|
||||||
server.backend_id = 'local-mock'
|
|
||||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
|
||||||
thread.start()
|
|
||||||
try:
|
|
||||||
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
|
|
||||||
print('SELFTEST PASS: tested mock UDP peer, not RustProxy')
|
|
||||||
finally:
|
|
||||||
server.shutdown()
|
|
||||||
thread.join(timeout=2)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
try:
|
|
||||||
main()
|
|
||||||
except (OSError, AssertionError, UnicodeError) as exc:
|
|
||||||
print(f'UDP FAIL: {exc}', file=sys.stderr)
|
|
||||||
sys.exit(1)
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Run source-upgrade preservation/refusal regressions in disposable directories."""
|
|
||||||
from pathlib import Path
|
|
||||||
import shutil
|
|
||||||
import sqlite3
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
sys.path.insert(0, str(ROOT))
|
|
||||||
from scripts.deploy.upgrade import upgrade
|
|
||||||
|
|
||||||
|
|
||||||
def fixture(parent):
|
|
||||||
target = parent / "installed"
|
|
||||||
target.mkdir()
|
|
||||||
(target / "Cargo.toml").write_text('[package]\nname = "rust-proxy"\nversion = "0.33.7"\n')
|
|
||||||
(target / "src").mkdir()
|
|
||||||
(target / "src" / "obsolete.rs").write_text("// old managed source\n")
|
|
||||||
(target / "data").mkdir()
|
|
||||||
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
|
|
||||||
db.execute("CREATE TABLE preserved(value TEXT)")
|
|
||||||
db.execute("INSERT INTO preserved VALUES ('configuration and preferences')")
|
|
||||||
(target / ".env").write_text("LOCAL_FIXTURE=1\n")
|
|
||||||
(target / "langs").mkdir()
|
|
||||||
(target / "langs" / "custom.json").write_text('{"custom": true}\n')
|
|
||||||
return target
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
with tempfile.TemporaryDirectory(prefix="rustproxy-upgrade-test-", dir=ROOT.parent) as temporary:
|
|
||||||
parent = Path(temporary)
|
|
||||||
target = fixture(parent)
|
|
||||||
try:
|
|
||||||
upgrade(ROOT, target, stopped=False)
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
raise AssertionError("upgrade must require stopped confirmation")
|
|
||||||
assert (target / "src" / "obsolete.rs").is_file()
|
|
||||||
print("PASS upgrade refuses missing --stopped without modifying source")
|
|
||||||
|
|
||||||
if sys.platform != "win32":
|
|
||||||
import fcntl
|
|
||||||
with (target / "data" / "process.lock").open("w+") as lock:
|
|
||||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
|
||||||
try:
|
|
||||||
upgrade(ROOT, target, stopped=True)
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
raise AssertionError("upgrade must refuse a live process lock")
|
|
||||||
print("PASS upgrade refuses a data directory locked by a live process")
|
|
||||||
|
|
||||||
backup = upgrade(ROOT, target, stopped=True)
|
|
||||||
assert (target / ".env").read_text() == "LOCAL_FIXTURE=1\n"
|
|
||||||
assert (target / "langs" / "custom.json").read_text() == '{"custom": true}\n'
|
|
||||||
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
|
|
||||||
assert db.execute("SELECT value FROM preserved").fetchone() == ("configuration and preferences",)
|
|
||||||
assert not (target / "src" / "obsolete.rs").exists()
|
|
||||||
assert (backup / "src" / "obsolete.rs").is_file()
|
|
||||||
assert (target / "Cargo.toml").read_bytes() == (ROOT / "Cargo.toml").read_bytes()
|
|
||||||
assert (target / "src" / "proxy" / "cache_plan.rs").is_file()
|
|
||||||
print("PASS upgrade preserves SQLite, environment and custom language; backs up old source and replaces managed tree")
|
|
||||||
|
|
||||||
if sys.platform != "win32":
|
|
||||||
shutil.rmtree(target / "src")
|
|
||||||
(target / "src").symlink_to(backup / "src", target_is_directory=True)
|
|
||||||
try:
|
|
||||||
upgrade(ROOT, target, stopped=True)
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
raise AssertionError("upgrade must refuse a managed symlink")
|
|
||||||
assert (backup / "src" / "obsolete.rs").is_file()
|
|
||||||
print("PASS upgrade refuses managed symlinks without changing their targets")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,268 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Simulated Chromium Vhost backend quick-view, without a Rust server.
|
|
||||||
|
|
||||||
Verify both backend triggers, the shared click-to-open upstream H2 protocol
|
|
||||||
popover without native title flicker, and the two Cache card margins.
|
|
||||||
Exercise live updates, safe text, keyboard/outside dismissal and mobile sizing.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import time
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
PACKS = {lang: {'id': lang, **json.loads((ROOT / 'langs' / f'{lang}.json').read_text())}
|
|
||||||
for lang in ('en_US', 'pl_pl')}
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert found
|
|
||||||
code = ROUTER[found.end():]
|
|
||||||
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
|
|
||||||
fragments = []
|
|
||||||
for line in code.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
fragments.append('(() => {\n' + '\n'.join(
|
|
||||||
(ROOT / 'src/admin/web/vhosts' / f'{name}.js').read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
fragments.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(fragments)
|
|
||||||
|
|
||||||
|
|
||||||
CONFIG = {
|
|
||||||
'vhosts': [{
|
|
||||||
'id': 'api-one', 'enabled': True, 'source': {'type': 'reverse_proxy'},
|
|
||||||
'frontend': {'hosts': ['api.example.test'], 'listener_ids': []},
|
|
||||||
'backends': [
|
|
||||||
{'id': 'origin-http', 'url': 'http://10.12.0.5:8080/api', 'protocol': 'http2', 'enabled': True},
|
|
||||||
{'id': '<origin-dns>', 'url': 'https://hiddenuser:secret@backend.example.test:8443/path', 'protocol': 'auto', 'enabled': True},
|
|
||||||
]
|
|
||||||
}, {'id': 'static-only', 'enabled': True,
|
|
||||||
'source': {'type': 'web_server', 'web_service_id': 'static-test'},
|
|
||||||
'frontend': {'hosts': ['static.example.test'], 'listener_ids': []}, 'backends': []}],
|
|
||||||
'listeners': {'endpoints': []}, 'web_services': [], 'acl_profiles': [],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def snapshot(first=200):
|
|
||||||
return {'backend_pools': [{
|
|
||||||
'id': 'api-one', 'health_mode': 'on',
|
|
||||||
'members': [
|
|
||||||
{'id': 'origin-http', 'status': 'up', 'last_status': first,
|
|
||||||
'requests': 12, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
|
|
||||||
{'id': '<origin-dns>', 'status': 'down', 'last_status': 0,
|
|
||||||
'requests': 0, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
|
|
||||||
]
|
|
||||||
}], 'vhost_metrics': {}}
|
|
||||||
|
|
||||||
|
|
||||||
def protocol_snapshot(h2_count):
|
|
||||||
status = snapshot()
|
|
||||||
status['backend_pools'][0]['members'][0].update({
|
|
||||||
'upstream_protocol_observed': 'h2',
|
|
||||||
'upstream_protocol_last_seen_ms': int(time.time() * 1000),
|
|
||||||
'upstream_protocol_http1_total': 3,
|
|
||||||
'upstream_protocol_http2_total': h2_count,
|
|
||||||
})
|
|
||||||
return status
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 768, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 820})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=app_bundle())
|
|
||||||
page.evaluate('''([config, status, packs]) => {
|
|
||||||
languagePacks = new Map(Object.entries(packs));
|
|
||||||
defaultLanguage = 'en_US'; language = 'pl_pl';
|
|
||||||
document.documentElement.lang = 'pl-PL';
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('login').hidden = true;
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-vhosts').hidden = false;
|
|
||||||
currentPanel = 'vhosts';
|
|
||||||
envelope = {revision: 12, config};
|
|
||||||
latestStatus = status;
|
|
||||||
window.rustProxyVhosts.render();
|
|
||||||
}''', [CONFIG, snapshot(), PACKS])
|
|
||||||
vhost = page.locator('[data-vhost-card="api-one"]')
|
|
||||||
backend = vhost.locator('.vhost-detail.vhost-backend-trigger')
|
|
||||||
health = vhost.locator('.vhost-live-metric.vhost-backend-trigger')
|
|
||||||
assert backend.count() == health.count() == 1, width
|
|
||||||
assert backend.get_attribute('aria-expanded') == 'false'
|
|
||||||
assert backend.get_attribute('aria-haspopup') == 'dialog'
|
|
||||||
# Clickable backend/health fields must match their non-button peers,
|
|
||||||
# including for user-selected interface corner styles.
|
|
||||||
shapes = page.evaluate('''() => {
|
|
||||||
const root = document.querySelector('[data-vhost-card="api-one"]');
|
|
||||||
const peers = [
|
|
||||||
[root.querySelector('.vhost-detail.vhost-backend-trigger'), root.querySelector('.vhost-detail:not(.vhost-backend-trigger)')],
|
|
||||||
[root.querySelector('.vhost-live-metric.vhost-backend-trigger'), root.querySelector('.vhost-live-metric:not(.vhost-backend-trigger)')]
|
|
||||||
];
|
|
||||||
const checks = ['square', 'subtle', 'rounded', 'extra'].map(corner => {
|
|
||||||
document.documentElement.dataset.cornerStyle = corner;
|
|
||||||
return {corner, tiles: peers.map(([trigger, peer]) => {
|
|
||||||
const left = getComputedStyle(trigger), right = getComputedStyle(peer);
|
|
||||||
return {radius: left.borderRadius, peerRadius: right.borderRadius,
|
|
||||||
padding: left.padding, peerPadding: right.padding,
|
|
||||||
font: left.fontFamily, peerFont: right.fontFamily};
|
|
||||||
})};
|
|
||||||
});
|
|
||||||
document.documentElement.removeAttribute('data-corner-style');
|
|
||||||
return checks;
|
|
||||||
}''')
|
|
||||||
for shape in shapes:
|
|
||||||
for tile in shape['tiles']:
|
|
||||||
assert tile['radius'] == tile['peerRadius'] == '0px', (width, shape)
|
|
||||||
assert tile['padding'] == tile['peerPadding'], (width, shape)
|
|
||||||
assert tile['font'] == tile['peerFont'], (width, shape)
|
|
||||||
backend.click()
|
|
||||||
popup = page.locator('#vhost-backend-popup')
|
|
||||||
assert popup.count() == 1, width
|
|
||||||
assert backend.get_attribute('aria-expanded') == 'true'
|
|
||||||
assert popup.get_attribute('role') == 'dialog'
|
|
||||||
# The anchored popup is a surface, not a permanently rounded tooltip.
|
|
||||||
# It must follow live interface corner preference changes without
|
|
||||||
# dismissing/recreating the open backend (or shared H2) popup.
|
|
||||||
corners = page.evaluate('''() => {
|
|
||||||
const node = document.getElementById('vhost-backend-popup');
|
|
||||||
const saved = document.documentElement.getAttribute('data-corner-style');
|
|
||||||
const out = {};
|
|
||||||
for (const style of ['default', 'square', 'subtle', 'rounded', 'extra']) {
|
|
||||||
document.documentElement.dataset.cornerStyle = style;
|
|
||||||
out[style] = {
|
|
||||||
popup: getComputedStyle(node).borderRadius,
|
|
||||||
row: getComputedStyle(node.querySelector('.vhost-backend-popup-row')).borderRadius,
|
|
||||||
close: getComputedStyle(node.querySelector('.vhost-backend-popup-close')).borderRadius,
|
|
||||||
reference: getComputedStyle(document.querySelector('[data-vhost-card="api-one"]')).borderRadius,
|
|
||||||
sameNode: node === document.getElementById('vhost-backend-popup'),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (saved === null) document.documentElement.removeAttribute('data-corner-style');
|
|
||||||
else document.documentElement.dataset.cornerStyle = saved;
|
|
||||||
return out;
|
|
||||||
}''')
|
|
||||||
expected_radii = {'default': ('0px', '0px'), 'square': ('0px', '0px'),
|
|
||||||
'subtle': ('5px', '4px'), 'rounded': ('12px', '8px'),
|
|
||||||
'extra': ('18px', '12px')}
|
|
||||||
for style, (surface, control) in expected_radii.items():
|
|
||||||
actual = corners[style]
|
|
||||||
assert actual == {'popup': surface, 'row': surface, 'close': control,
|
|
||||||
'reference': surface, 'sameNode': True}, (width, style, actual)
|
|
||||||
values = popup.evaluate('''node => ({
|
|
||||||
cards: node.querySelectorAll('.vhost-backend-popup-row').length,
|
|
||||||
text: node.textContent, html: node.innerHTML,
|
|
||||||
box: (() => { const r = node.getBoundingClientRect();
|
|
||||||
return {left: r.left, right: r.right, top:r.top, bottom:r.bottom}; })(),
|
|
||||||
})''')
|
|
||||||
assert values['cards'] == 2, (width, values)
|
|
||||||
assert '10.12.0.5:8080' in values['text'] and 'HTTP 200' in values['text'], (width, values)
|
|
||||||
assert 'backend.example.test:8443' in values['text'], width
|
|
||||||
assert 'secret' not in values['text'], width
|
|
||||||
assert '<origin-dns>' in values['text'] and '<origin-dns>' in values['html'], width
|
|
||||||
assert 'Brak odpowiedzi HTTP' in values['text'], width
|
|
||||||
assert 'HTTPS · AUTO' in values['text'], width
|
|
||||||
assert values['box']['left'] >= -1 and values['box']['right'] <= width + 1, (width, values['box'])
|
|
||||||
assert values['box']['top'] >= -1 and values['box']['bottom'] <= 821, (width, values['box'])
|
|
||||||
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, width
|
|
||||||
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', snapshot(503))
|
|
||||||
assert 'HTTP 503' in popup.inner_text(), width
|
|
||||||
assert popup.locator('.has-error').count() == 1, width
|
|
||||||
health.click()
|
|
||||||
assert popup.count() == 1 and health.get_attribute('aria-expanded') == 'true', width
|
|
||||||
assert backend.get_attribute('aria-expanded') == 'false', width
|
|
||||||
page.keyboard.press('Escape')
|
|
||||||
assert popup.count() == 0 and health.get_attribute('aria-expanded') == 'false', width
|
|
||||||
assert health.evaluate('(node) => document.activeElement === node'), width
|
|
||||||
backend.click()
|
|
||||||
page.locator('#vhost-search').click(force=True)
|
|
||||||
assert popup.count() == 0, width
|
|
||||||
# A static-only vhost presents a translated empty state, not stale pool values.
|
|
||||||
page.locator('[data-vhost-card="static-only"] .vhost-detail.vhost-backend-trigger').click()
|
|
||||||
assert 'Brak skonfigurowanych backendów.' in popup.inner_text(), width
|
|
||||||
popup.locator('.vhost-backend-popup-close').click()
|
|
||||||
assert popup.count() == 0, width
|
|
||||||
backend.click()
|
|
||||||
assert popup.count() == 1, width
|
|
||||||
|
|
||||||
# The existing backend popover is also used by the H2 policy badge.
|
|
||||||
protocol = vhost.locator('[data-upstream-protocol-backend="origin-http"]')
|
|
||||||
assert protocol.count() == 1 and protocol.evaluate('n => n.tagName') == 'BUTTON', width
|
|
||||||
assert protocol.get_attribute('title') is None, width
|
|
||||||
protocol.click()
|
|
||||||
assert backend.get_attribute('aria-expanded') == 'false', width
|
|
||||||
assert protocol.get_attribute('aria-expanded') == 'true', width
|
|
||||||
assert 'Brak odpowiedzi backendu' in popup.inner_text(), width
|
|
||||||
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(8))
|
|
||||||
assert 'origin-http · H2 compatible → H2' in protocol.inner_text(), width
|
|
||||||
assert 'Ostatnia wersja HTTP' in popup.inner_text(), width
|
|
||||||
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '8'], width
|
|
||||||
page.evaluate('''() => {
|
|
||||||
window.__popupReference = document.getElementById('vhost-backend-popup');
|
|
||||||
window.__headingReference = window.__popupReference.querySelector('h4');
|
|
||||||
window.__popupRemoved = 0;
|
|
||||||
window.__popupObserver = new MutationObserver(records => {
|
|
||||||
for (const record of records) for (const item of record.removedNodes) {
|
|
||||||
if (item === window.__popupReference) window.__popupRemoved++;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
window.__popupObserver.observe(document.body, {childList: true});
|
|
||||||
}''')
|
|
||||||
for count in range(9, 19):
|
|
||||||
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(count))
|
|
||||||
unchanged = page.evaluate('''() => ({
|
|
||||||
samePopup: document.getElementById('vhost-backend-popup') === window.__popupReference,
|
|
||||||
sameHeading: window.__popupReference.querySelector('h4') === window.__headingReference,
|
|
||||||
removed: window.__popupRemoved,
|
|
||||||
expanded: document.querySelector('[data-upstream-protocol-backend="origin-http"]').getAttribute('aria-expanded'),
|
|
||||||
})''')
|
|
||||||
assert unchanged == {'samePopup': True, 'sameHeading': True, 'removed': 0, 'expanded': 'true'}, (width, unchanged)
|
|
||||||
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '18'], width
|
|
||||||
page.evaluate('window.__popupObserver.disconnect()')
|
|
||||||
assert protocol.get_attribute('title') is None, width
|
|
||||||
# A stale observation should change the displayed information in
|
|
||||||
# place, not dismiss/recreate the click popover.
|
|
||||||
stale = protocol_snapshot(18)
|
|
||||||
stale['backend_pools'][0]['members'][0]['upstream_protocol_last_seen_ms'] -= 600000
|
|
||||||
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', stale)
|
|
||||||
assert 'H2 compatible → ?' in protocol.inner_text(), width
|
|
||||||
assert popup.locator('.vhost-backend-popup-list').is_hidden(), width
|
|
||||||
assert page.evaluate('document.getElementById("vhost-backend-popup") === window.__popupReference'), width
|
|
||||||
page.keyboard.press('Escape')
|
|
||||||
assert popup.count() == 0 and protocol.get_attribute('aria-expanded') == 'false', width
|
|
||||||
assert protocol.evaluate('node => document.activeElement === node'), width
|
|
||||||
|
|
||||||
# Cache Domains and Stored objects are the only directly adjacent
|
|
||||||
# table cards and need a visible, small gap at any viewport width.
|
|
||||||
cache_gap = page.evaluate('''() => {
|
|
||||||
const panel = document.getElementById('panel-cache');
|
|
||||||
panel.hidden = false;
|
|
||||||
const [domains, stored] = panel.querySelectorAll(':scope > article.card');
|
|
||||||
const gap = stored.getBoundingClientRect().top - domains.getBoundingClientRect().bottom;
|
|
||||||
panel.hidden = true;
|
|
||||||
return gap;
|
|
||||||
}''')
|
|
||||||
assert 9.5 <= cache_gap <= 10.5, (width, cache_gap)
|
|
||||||
page.evaluate('window.rustProxyVhosts.reset()')
|
|
||||||
assert popup.count() == 0, width
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print(f'PASS Vhosts shared backend/H2 popover and Cache gap at {width}px')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS Vhost quick-view with simulated runtime status (not live Rust)')
|
|
||||||
@@ -1,140 +0,0 @@
|
|||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re, json
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
root=Path(__file__).resolve().parents[2]
|
|
||||||
router=(root/'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
def concatenate(section):
|
|
||||||
pattern=r'fn '+section+r'\([^)]*\)[^{]*\{'
|
|
||||||
m=re.search(pattern,router)
|
|
||||||
assert m, section
|
|
||||||
text=router[m.end():]
|
|
||||||
# The array contents are bounded by ].join("\\n") before the next function.
|
|
||||||
text=text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
|
|
||||||
result=[]
|
|
||||||
for line in text.splitlines():
|
|
||||||
line=line.strip()
|
|
||||||
if 'include_str!("' in line:
|
|
||||||
path=re.search(r'include_str!\("([^"]+)"\)',line).group(1)
|
|
||||||
result.append((root/'src/admin'/path).read_text())
|
|
||||||
elif line=='admin_vhosts_javascript(),':
|
|
||||||
result.append('(() => {\n'+''.join((root/'src/admin/web/vhosts'/f'{part}.js').read_text()+'\n' for part in ['core','protocol','health','rates','editor','render'])+'})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
result.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(result)
|
|
||||||
bundle=concatenate('admin_app_javascript')
|
|
||||||
assert 'window.rustProxyOverviewSamples =' in bundle
|
|
||||||
assert bundle.index('window.rustProxyOverviewSamples =') < bundle.index('window.rustProxyOverviewSamples.onChange(')
|
|
||||||
css=(root/'src/admin/web/style.css').read_text()
|
|
||||||
html=render_admin_index(root)
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser=p.chromium.launch(headless=True, executable_path='/usr/bin/chromium',args=['--no-sandbox','--allow-file-access-from-files'])
|
|
||||||
for width in [320,375,800,1440]:
|
|
||||||
page=browser.new_page(viewport={'width':width,'height':900})
|
|
||||||
errors=[]
|
|
||||||
page.on('pageerror',lambda e: errors.append(str(e)))
|
|
||||||
# Execute in a fully populated DOM without disallowed filesystem/network navigation.
|
|
||||||
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js','').replace('/assets/style___RUSTPROXY_ASSET_ID__.css',''))
|
|
||||||
page.add_style_tag(content=css)
|
|
||||||
page.add_script_tag(content=bundle)
|
|
||||||
page.wait_for_timeout(550)
|
|
||||||
state=page.evaluate('''() => ({ controller:typeof window.rustProxyOverviewSamples,
|
|
||||||
handle:typeof window.rustProxyOverviewSamples?.onChange,
|
|
||||||
web:typeof window.rustProxyWebOverview, services:typeof window.rustProxyWebServer,
|
|
||||||
registry: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
|
|
||||||
htmlWidth:document.documentElement.scrollWidth,windowWidth:window.innerWidth })''')
|
|
||||||
print(width,state,'errors',errors)
|
|
||||||
assert state['controller']=='object' and state['handle']=='function' and state['web']=='object'
|
|
||||||
assert state['registry']==12
|
|
||||||
assert not [e for e in errors if 'onChange' in e or 'ReferenceError' in e or 'SyntaxError' in e or 'TypeError' in e],errors
|
|
||||||
assert state['htmlWidth'] <= state['windowWidth'] + 2
|
|
||||||
# The selector belongs to the same header row as the core connection;
|
|
||||||
# hiding it on other panels must not leave an empty dashboard row.
|
|
||||||
layout = page.evaluate('''() => {
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
setPanel('overview', false);
|
|
||||||
const status = document.getElementById('connection-label');
|
|
||||||
const sample = document.getElementById('overview-samples-toolbar');
|
|
||||||
const row = document.querySelector('.page-intro-meta-row');
|
|
||||||
const rect = node => { const r = node.getBoundingClientRect();
|
|
||||||
return {x:r.x,y:r.y,width:r.width,height:r.height,right:r.right}; };
|
|
||||||
const overview = { status:rect(status),sample:rect(sample),row:rect(row),
|
|
||||||
visible: !sample.hidden,inside: row.contains(sample) && row.contains(status),
|
|
||||||
overflow: document.documentElement.scrollWidth > innerWidth + 2 };
|
|
||||||
setPanel('web-server', false);
|
|
||||||
const notOverview = sample.hidden && !status.hidden;
|
|
||||||
setPanel('overview', false);
|
|
||||||
return {overview,notOverview,visibleAgain:!sample.hidden};
|
|
||||||
}''')
|
|
||||||
assert layout['overview']['inside'] and layout['overview']['visible']
|
|
||||||
assert layout['notOverview'] and layout['visibleAgain']
|
|
||||||
assert not layout['overview']['overflow'], (width, layout)
|
|
||||||
if width >= 800:
|
|
||||||
assert abs(layout['overview']['sample']['y'] - layout['overview']['status']['y']) <= 20, (width, layout)
|
|
||||||
assert layout['overview']['sample']['right'] <= layout['overview']['row']['right'] + 2, (width, layout)
|
|
||||||
# A genuinely sampled WWW metrics payload (not a screenshot-only fixture).
|
|
||||||
report = page.evaluate('''() => {
|
|
||||||
Object.defineProperty(window.performance, "now", {configurable:true, value:()=>window.__clock});
|
|
||||||
window.__clock = 1000;
|
|
||||||
currentPanel = "overview";
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('login').hidden = true;
|
|
||||||
document.getElementById('panel-overview').hidden = false;
|
|
||||||
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
|
|
||||||
function snap(k) {
|
|
||||||
return {revision:7, web_server_metrics:{
|
|
||||||
"public-site": {web: {requests:100+k*30, status_2xx:86+k*24, status_3xx:3+k,
|
|
||||||
status_4xx:8+k*2, status_5xx:3+k*3, response_headers_us:4000000+k*1300000,
|
|
||||||
response_bytes:100000+k*20000, static_requests:70+k*20, php_requests:25+k*7,
|
|
||||||
other_requests:5+k*3, methods:[90+k*25,8+k*4,0,0,2+k],
|
|
||||||
selected_status:[80+k*18,2,2,1,5+k,0,0,0], active_headers:0},
|
|
||||||
php: {metrics:{requests:25+k*7, response_headers_us:2000000+k*700000,
|
|
||||||
response_bytes:30000+k*5000,failures:[1+k,1+k,1+k,0,0,0,0,0,0]},
|
|
||||||
capacity_limit:32, capacity_used:8, capacity_available:24}}}
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const before = snap(0), after = snap(1);
|
|
||||||
// Saved custom layout from before 0.33.0 must not enable new cards.
|
|
||||||
const savedLayout = applyDashboardLayout(['requests', 'errors'],
|
|
||||||
normalizeHiddenDashboardWidgets([], ['requests', 'errors']));
|
|
||||||
const newCardsHidden = [...document.querySelectorAll('[data-dashboard-widget^="www_"]')]
|
|
||||||
.every(card=>card.hidden);
|
|
||||||
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
|
|
||||||
window.rustProxyWebOverview.render(before, true);
|
|
||||||
window.__clock = 4000;
|
|
||||||
window.rustProxyWebOverview.render(after, true);
|
|
||||||
let ids = ['www_requests-all-line', 'www_requests-php-line',
|
|
||||||
'www_latency-web-line', 'www_traffic-php-line', 'www_status_trend-5xx-line'];
|
|
||||||
const linePaths = ids.map(id=>document.getElementById(id)?.getAttribute('d'));
|
|
||||||
const statusText = document.getElementById('www_status-value').textContent;
|
|
||||||
const capacityText = document.getElementById('www_capacity-value').textContent;
|
|
||||||
const rateText = document.getElementById('www_php_requests-value').textContent;
|
|
||||||
const bars = document.querySelectorAll('#www_top_services-bars .www-top-row').length;
|
|
||||||
window.rustProxyOverviewSamples.setLimit(120);
|
|
||||||
const historyLimit = window.rustProxyOverviewSamples.getLimit();
|
|
||||||
currentPanel = "web-server";
|
|
||||||
envelope = {config:{web_services:[{id:'public-site', name:'Public site',document_root:'/srv/www/public'}],vhosts:[]}};
|
|
||||||
window.rustProxyWebServer.render();
|
|
||||||
const details = document.querySelector('.www-service-panel');
|
|
||||||
if (!details || details.tagName !== 'SECTION') throw Error('Missing always-visible WWW metrics');
|
|
||||||
window.__clock = 7000;
|
|
||||||
window.rustProxyWebServer.renderLive(before, true);
|
|
||||||
window.__clock = 10000;
|
|
||||||
window.rustProxyWebServer.renderLive(after, true);
|
|
||||||
const value = details.querySelector('[data-web-live-field="requests"]')?.textContent;
|
|
||||||
const phpCapacity = details.querySelector('[data-web-live-field="php"]')?.textContent;
|
|
||||||
return {linePaths,statusText,capacityText,rateText,bars,historyLimit,value,phpCapacity,newCardsHidden,
|
|
||||||
width:document.documentElement.scrollWidth};
|
|
||||||
}''')
|
|
||||||
print('sampled WWW',width,report)
|
|
||||||
assert all(report['linePaths']), report
|
|
||||||
assert report['statusText'] != '-' and report['capacityText'] == '8 / 32', report
|
|
||||||
assert report['rateText'] not in ('-', '0.0/s'), report
|
|
||||||
assert report['bars'] == 1 and report['historyLimit'] == 120 and report['newCardsHidden'], report
|
|
||||||
assert report['value'] and report['phpCapacity'] == '8 / 32', report
|
|
||||||
assert report['width'] <= width + 2, report
|
|
||||||
assert not [e for e in errors if any(x in e for x in ('ReferenceError', 'TypeError', 'SyntaxError'))], errors
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS: actual composed admin bundle starts without a JS TypeError')
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Static contracts for 0.33.0 WWW observability (not Rust compilation)."""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import tomllib
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
read = lambda file: (ROOT / file).read_text()
|
|
||||||
manifest = tomllib.loads(read('Cargo.toml'))
|
|
||||||
lock = tomllib.loads(read('Cargo.lock'))
|
|
||||||
version = manifest['package']['version']
|
|
||||||
assert tuple(map(int, version.split('.'))) >= (0, 33, 0)
|
|
||||||
assert [p['version'] for p in lock['package'] if p['name'] == 'rust-proxy'] == [version]
|
|
||||||
|
|
||||||
registry = json.loads(read('src/admin/web/dashboard-widgets.json'))
|
|
||||||
widgets = [w for w in registry if w['id'].startswith('www_')]
|
|
||||||
assert len(widgets) == 12
|
|
||||||
assert all(w['default_hidden'] is True for w in widgets)
|
|
||||||
assert len({w['id'] for w in registry}) == len(registry)
|
|
||||||
html = render_admin_index(ROOT)
|
|
||||||
english = json.loads(read('langs/en_US.json'))['translations']
|
|
||||||
polish = json.loads(read('langs/pl_pl.json'))['translations']
|
|
||||||
for widget in widgets:
|
|
||||||
assert f'data-dashboard-widget="{widget["id"]}" hidden' in html, widget['id']
|
|
||||||
assert widget['label_key'] in english, widget['label_key']
|
|
||||||
assert widget['label_key'] in polish, widget['label_key']
|
|
||||||
if widget['description_key']:
|
|
||||||
assert widget['description_key'] in english
|
|
||||||
assert widget['description_key'] in polish
|
|
||||||
|
|
||||||
# Safely assemble the admin app before core.js: the earlier release crashed on
|
|
||||||
# registering onChange callbacks before rustProxyOverviewSamples was defined.
|
|
||||||
router = read('src/admin/router.rs')
|
|
||||||
assert router.index('include_str!("web/app/overview-samples.js")') < router.index('include_str!("web/app/core.js")')
|
|
||||||
assert '"web/app/web-overview.js"' in router
|
|
||||||
assert '"web_server_metrics"' in read('src/admin/status_live.rs')
|
|
||||||
assert 'web_server_metrics_payload(&snapshot)' in read('src/admin/status.rs')
|
|
||||||
# record_web is only consumed by web_server::serve; expose no wider than the
|
|
||||||
# ResponseKind enum argument. This is a static visibility contract, not rustc.
|
|
||||||
metrics = read('src/web_server/metrics.rs')
|
|
||||||
assert 'pub(super) enum ResponseKind' in metrics
|
|
||||||
assert 'pub(super) fn record_web(' in metrics
|
|
||||||
assert 'pub(crate) fn record_web(' not in metrics
|
|
||||||
assert 'service.web_metrics.record_web(' in read('src/web_server/serve.rs')
|
|
||||||
|
|
||||||
families = read('src/prometheus/web_php.rs')
|
|
||||||
required = {
|
|
||||||
'rust_proxy_web_server_requests_total', 'rust_proxy_web_server_responses_total',
|
|
||||||
'rust_proxy_web_server_request_kind_total', 'rust_proxy_web_server_methods_total',
|
|
||||||
'rust_proxy_web_server_status_code_total', 'rust_proxy_web_server_response_header_duration_microseconds_total',
|
|
||||||
'rust_proxy_web_server_response_body_bytes_total', 'rust_proxy_web_server_requests_active',
|
|
||||||
'rust_proxy_web_server_compression_total', 'rust_proxy_php_requests_total',
|
|
||||||
'rust_proxy_php_responses_total', 'rust_proxy_php_response_header_duration_microseconds_total',
|
|
||||||
'rust_proxy_php_response_body_bytes_total', 'rust_proxy_php_concurrency_available',
|
|
||||||
'rust_proxy_php_concurrency_limit', 'rust_proxy_php_concurrency_in_use', 'rust_proxy_php_failures_total'
|
|
||||||
}
|
|
||||||
assert all(f'"{name}"' in families for name in required), required
|
|
||||||
assert 'PHP_FAILURE_NAMES' in families
|
|
||||||
assert re.search(r'\["GET", "POST", "PUT", "DELETE", "other"\]', families)
|
|
||||||
assert 'let wl = labels(&[("web_service", service.id.as_str())]);' in families
|
|
||||||
|
|
||||||
j = json.loads(read('grafana/rustproxy-overview.json'))
|
|
||||||
assert j['title'].startswith('RustProxy 0.33.')
|
|
||||||
ids = [p['id'] for p in j['panels']]
|
|
||||||
assert len(ids) == len(set(ids))
|
|
||||||
assert any(v.get('name') == 'web_service' and v.get('multi') and v.get('includeAll') for v in j['templating']['list'])
|
|
||||||
for title in ('Web Server 0.33.0', 'PHP-FPM / FastCGI', 'Per Web Service', 'WWW / PHP correlation'):
|
|
||||||
assert any(p['title'] == title and p['type'] == 'row' for p in j['panels'])
|
|
||||||
extra = [p for p in j['panels'] if p['id'] >= 89]
|
|
||||||
for i, p in enumerate(extra):
|
|
||||||
a = p['gridPos']
|
|
||||||
assert 0 <= a['x'] and a['w'] > 0 and a['x'] + a['w'] <= 24
|
|
||||||
for q in extra[i + 1:]:
|
|
||||||
b = q['gridPos']
|
|
||||||
overlap = a['x'] < b['x'] + b['w'] and b['x'] < a['x'] + a['w'] and a['y'] < b['y'] + b['h'] and b['y'] < a['y'] + a['h']
|
|
||||||
assert not overlap, (p['id'], q['id'])
|
|
||||||
|
|
||||||
production = [p for p in (ROOT / 'src').rglob('*.rs') if p.name != 'tests.rs' and 'tests' not in p.parts]
|
|
||||||
large = [str(p.relative_to(ROOT)) for p in production if len(p.read_text().splitlines()) > 500]
|
|
||||||
assert not large, large
|
|
||||||
print('PASS RustProxy 0.33.0 source version, opt-in widgets, localizations, status API,')
|
|
||||||
print(' exporter families, Grafana geometry/filtering and Rust module size (STATIC ONLY)')
|
|
||||||
@@ -1,153 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Simulated Web Server telemetry against the production admin JS/CSS bundle.
|
|
||||||
|
|
||||||
Browser-only test: does not assert live RustProxy, PHP-FPM, or network behavior.
|
|
||||||
Run: python3 scripts/tests/web_server_0330_ui_test.py
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
router = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', router)
|
|
||||||
assert match, 'admin_app_javascript definition missing'
|
|
||||||
portion = router[match.end():]
|
|
||||||
portion = portion[:re.search(r'\]\s*\.join\("\\n"\)', portion).start()]
|
|
||||||
fragments = []
|
|
||||||
for line in portion.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if 'include_str!("' in line:
|
|
||||||
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
|
|
||||||
fragments.append((ROOT / 'src/admin' / path).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
fragments.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
fragments.append(json.loads(line[:-1]))
|
|
||||||
BUNDLE = '\n'.join(fragments)
|
|
||||||
assert BUNDLE.index('window.rustProxyOverviewSamples =') < BUNDLE.index('window.rustProxyOverviewSamples.onChange(')
|
|
||||||
|
|
||||||
WEB = {
|
|
||||||
'requests': 100, 'status_2xx': 90, 'status_3xx': 2, 'status_4xx': 5, 'status_5xx': 3,
|
|
||||||
'response_headers_us': 2_500_000, 'response_bytes': 10_000, 'active_headers': 2,
|
|
||||||
'static_requests': 70, 'php_requests': 25, 'other_requests': 5,
|
|
||||||
'methods': [80, 10, 5, 2, 3], 'selected_status': [75, 5, 2, 0, 5, 1, 1, 1],
|
|
||||||
'failures': [0] * 9, 'compression': [5, 3, 2, 1],
|
|
||||||
}
|
|
||||||
PHP = {
|
|
||||||
'requests': 25, 'status_2xx': 22, 'status_3xx': 0, 'status_4xx': 1, 'status_5xx': 2,
|
|
||||||
'response_headers_us': 1_000_000, 'response_bytes': 2_000,
|
|
||||||
'failures': [1, 2, 3, 4, 5, 6, 7, 8, 9],
|
|
||||||
}
|
|
||||||
|
|
||||||
def observation(factor: int):
|
|
||||||
web = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
|
|
||||||
for key, value in WEB.items()}
|
|
||||||
php = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
|
|
||||||
for key, value in PHP.items()}
|
|
||||||
web['active_headers'] = 2
|
|
||||||
return {'revision': 10, 'web_server_metrics': {
|
|
||||||
'site-A': {'web': web, 'php': {'metrics': php, 'capacity_limit': 32, 'capacity_used': 18}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
with sync_playwright() as p:
|
|
||||||
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 800, 1440):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda err: errors.append(str(err)))
|
|
||||||
page.set_content(render_admin_index(ROOT))
|
|
||||||
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
|
|
||||||
page.add_script_tag(content=BUNDLE)
|
|
||||||
page.evaluate("""() => {
|
|
||||||
document.querySelector('#workspace').hidden = false;
|
|
||||||
document.querySelector('#panel-overview').hidden = false;
|
|
||||||
document.querySelectorAll('[data-dashboard-widget^="www_"]').forEach(n => n.hidden = false);
|
|
||||||
currentPanel = 'overview';
|
|
||||||
window.__clock = 1000;
|
|
||||||
Object.defineProperty(performance, 'now', { configurable: true, value: () => window.__clock });
|
|
||||||
}""")
|
|
||||||
first, second = observation(1), observation(2)
|
|
||||||
state = page.evaluate("""([one, two]) => {
|
|
||||||
const api = window.rustProxyWebOverview;
|
|
||||||
api.render(one);
|
|
||||||
window.__clock = 4000;
|
|
||||||
api.render(two);
|
|
||||||
return {
|
|
||||||
rps: document.getElementById('www_requests-all-value').textContent,
|
|
||||||
php: document.getElementById('www_requests-php-value').textContent,
|
|
||||||
latency: document.getElementById('www_latency-web-value').textContent,
|
|
||||||
capacity: document.getElementById('www_capacity-value').textContent,
|
|
||||||
types: document.getElementById('www_kind-value').textContent,
|
|
||||||
status: document.getElementById('www_status-value').textContent,
|
|
||||||
bytes: document.getElementById('www_traffic-value').textContent,
|
|
||||||
top: document.getElementById('www_top_services-value').textContent,
|
|
||||||
layoutWidth: document.documentElement.scrollWidth,
|
|
||||||
screenWidth: window.innerWidth,
|
|
||||||
children: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
|
|
||||||
sampleCount: document.getElementById('www_requests-all-line').getAttribute('d'),
|
|
||||||
limit: window.rustProxyOverviewSamples.getLimit(),
|
|
||||||
};
|
|
||||||
}""", [first, second])
|
|
||||||
assert state['children'] == 12, state
|
|
||||||
assert state['rps'] != '-' and state['php'] != '-', state
|
|
||||||
assert 'ms' in state['latency'] and '18' in state['capacity'], state
|
|
||||||
assert state['types'].endswith('%') and state['status'].endswith('%'), state
|
|
||||||
assert '/s' in state['top'], state
|
|
||||||
assert state['layoutWidth'] <= state['screenWidth'] + 2, (width, state)
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
# Exercise the always-visible per-service metrics with the same status,
|
|
||||||
# without any real HTTP/PHP request or extra polling.
|
|
||||||
details = page.evaluate("""(status) => {
|
|
||||||
envelope = { revision: 10, config: {
|
|
||||||
web_services: [{ id: 'site-A', name: 'Example', document_root: '/var/www/site-A',
|
|
||||||
index: ['index.html'], php: { enabled: true } }],
|
|
||||||
vhosts: [], php_fpm_profiles: []
|
|
||||||
}};
|
|
||||||
currentPanel = 'web-server';
|
|
||||||
window.rustProxyWebServer.render();
|
|
||||||
const detail = document.querySelector('#web-service-list .www-service-panel');
|
|
||||||
if (!detail) return { missing: true };
|
|
||||||
window.rustProxyWebServer.renderLive(status, false);
|
|
||||||
return { text: detail.textContent, rows: detail.querySelectorAll('[data-web-live-field]').length,
|
|
||||||
alwaysVisible: detail.tagName === 'SECTION' && !detail.querySelector('summary') };
|
|
||||||
}""", second)
|
|
||||||
assert not details.get('missing') and details['rows'] == 7 and details['alwaysVisible'], (width, details)
|
|
||||||
assert '18 / 32' in details['text'], (width, details)
|
|
||||||
# The first reading after an overdue browser interval must establish a
|
|
||||||
# fresh baseline without adding a fake 0/s dip to the service chart.
|
|
||||||
continuity = page.evaluate("""([first, second, paused, resumed]) => {
|
|
||||||
const api = window.rustProxyWebServer;
|
|
||||||
const chart = document.querySelector('#web-service-list [data-web-live-id="site-A"] polyline');
|
|
||||||
const rate = document.querySelector('#web-service-list [data-web-live-field="rate"]');
|
|
||||||
window.__clock = 5000; api.renderLive(first);
|
|
||||||
window.__clock = 8000; api.renderLive(second);
|
|
||||||
const before = chart.getAttribute('points');
|
|
||||||
window.__clock = 100000; api.renderLive(paused);
|
|
||||||
const during = chart.getAttribute('points'), unavailable = rate.textContent;
|
|
||||||
window.__clock = 103000; api.renderLive(resumed);
|
|
||||||
return { before, during, unavailable, after: chart.getAttribute('points'), resumedRate: rate.textContent };
|
|
||||||
}""", [observation(2), observation(3), observation(10), observation(12)])
|
|
||||||
assert continuity['before'] == continuity['during'], (width, continuity)
|
|
||||||
assert continuity['unavailable'] == '-', (width, continuity)
|
|
||||||
assert continuity['resumedRate'] != '-', (width, continuity)
|
|
||||||
assert continuity['after'] != continuity['before'], (width, continuity)
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
old = page.evaluate("""() => {
|
|
||||||
currentPanel = 'overview';
|
|
||||||
window.rustProxyWebOverview.render({ revision: 11, web_server_metrics: {} });
|
|
||||||
return {
|
|
||||||
rate: document.getElementById('www_requests-value').textContent,
|
|
||||||
status: document.getElementById('www_status-value').textContent,
|
|
||||||
top: document.getElementById('www_top_services-bars').childElementCount,
|
|
||||||
};
|
|
||||||
}""")
|
|
||||||
assert old == {'rate': '-', 'status': '-', 'top': 0}, (width, old)
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print(f'PASS simulated Web Server cards and always-visible service metrics at {width}px: {state["rps"]}, {state["php"]}, layout {state["layoutWidth"]}px')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS production JS bundle + all twelve visible WWW widgets (simulated data only)')
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""WWW/Vhost visual parity in production UI files, using simulated status.
|
|
||||||
|
|
||||||
Chromium exercises the real assembled admin JS/CSS and responsive layouts.
|
|
||||||
It does not establish Rust, SSE or PHP-FPM runtime acceptance.
|
|
||||||
"""
|
|
||||||
from pathlib import Path
|
|
||||||
from panel_html import render_admin_index
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from playwright.sync_api import sync_playwright
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
HTML = render_admin_index(ROOT)
|
|
||||||
CSS = (ROOT / 'src/admin/web/style.css').read_text()
|
|
||||||
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
|
|
||||||
|
|
||||||
|
|
||||||
def app_bundle():
|
|
||||||
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
|
|
||||||
assert found
|
|
||||||
code = ROUTER[found.end():]
|
|
||||||
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
|
|
||||||
fragments = []
|
|
||||||
for line in code.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
include = re.search(r'include_str!\("([^"]+)"\)', line)
|
|
||||||
if include:
|
|
||||||
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
|
|
||||||
elif line == 'admin_vhosts_javascript(),':
|
|
||||||
fragments.append('(() => {\n' + '\n'.join(
|
|
||||||
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
|
|
||||||
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
|
|
||||||
elif line.startswith('"') and line.endswith('",'):
|
|
||||||
fragments.append(json.loads(line[:-1]))
|
|
||||||
return '\n'.join(fragments)
|
|
||||||
|
|
||||||
|
|
||||||
def status(step):
|
|
||||||
return {'web_server_metrics': {
|
|
||||||
'site-A': {'web': {
|
|
||||||
'requests': step * 120, 'response_headers_us': step * 2_400_000,
|
|
||||||
'status_5xx': step * 4, 'response_bytes': step * 20_480,
|
|
||||||
'static_requests': step * 80, 'php_requests': step * 30,
|
|
||||||
'other_requests': step * 10,
|
|
||||||
}, 'php': {'capacity_used': 3, 'capacity_limit': 16}},
|
|
||||||
'site-B': {'web': {
|
|
||||||
'requests': step * 24, 'response_headers_us': step * 100_000,
|
|
||||||
'status_5xx': 0, 'response_bytes': step * 8_192,
|
|
||||||
'static_requests': step * 24, 'php_requests': 0,
|
|
||||||
'other_requests': 0,
|
|
||||||
}, 'php': None},
|
|
||||||
}}
|
|
||||||
|
|
||||||
|
|
||||||
with sync_playwright() as playwright:
|
|
||||||
browser = playwright.chromium.launch(
|
|
||||||
headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
|
|
||||||
for width in (320, 375, 640, 800, 980, 1440, 1920):
|
|
||||||
page = browser.new_page(viewport={'width': width, 'height': 900})
|
|
||||||
errors = []
|
|
||||||
page.on('pageerror', lambda e: errors.append(str(e)))
|
|
||||||
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
|
|
||||||
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
|
|
||||||
page.add_style_tag(content=CSS)
|
|
||||||
page.add_script_tag(content=app_bundle())
|
|
||||||
initial = page.evaluate('''(status) => {
|
|
||||||
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
|
|
||||||
document.getElementById('login').hidden = true;
|
|
||||||
document.getElementById('workspace').hidden = false;
|
|
||||||
document.getElementById('panel-web-server').hidden = false;
|
|
||||||
currentPanel = 'web-server';
|
|
||||||
envelope = {revision: 10, config: {web_services: [
|
|
||||||
{id: 'site-A', name: 'Example service', document_root: '/var/www/a', index: ['index.html'], php: {enabled: true}},
|
|
||||||
{id: 'site-B', name: 'Static service', document_root: '/var/www/b', index: ['index.html']}
|
|
||||||
], vhosts: [], php_fpm_profiles: []}};
|
|
||||||
window.__clock = 1000;
|
|
||||||
Object.defineProperty(performance, 'now', {configurable: true, value: () => window.__clock});
|
|
||||||
latestStatus = status;
|
|
||||||
window.rustProxyWebServer.render();
|
|
||||||
const list = document.querySelector('#web-service-list');
|
|
||||||
const cards = [...list.querySelectorAll(':scope > .vhost-card')];
|
|
||||||
const metricPanels = cards.map(card => card.querySelector('.www-service-panel'));
|
|
||||||
const grid = metricPanels[0].querySelector('.vhost-live-grid');
|
|
||||||
const chart = metricPanels[0].querySelector('.vhost-chart');
|
|
||||||
const svg = chart.querySelector('svg');
|
|
||||||
const svgBox = svg.getBoundingClientRect(), svgMatrix = svg.getScreenCTM();
|
|
||||||
const rect = node => node.getBoundingClientRect();
|
|
||||||
const expectedMetricLabels = ['webLiveRequests','webLiveRate','webLiveErrors',
|
|
||||||
'webLiveLatency','webLivePhpCapacity','webLiveBytes'];
|
|
||||||
return {
|
|
||||||
cards: cards.length, sectionTags: metricPanels.map(panel => panel.tagName),
|
|
||||||
collapsibles: list.querySelectorAll('details, summary').length,
|
|
||||||
liveFields: metricPanels.map(panel => panel.querySelectorAll('[data-web-live-field]').length),
|
|
||||||
labels: [...grid.children].map(tile => tile.querySelector('span')?.dataset.i18n),
|
|
||||||
tileClasses: [...grid.children].map(tile => tile.className),
|
|
||||||
values: [...grid.querySelectorAll('strong')].map(el => el.textContent),
|
|
||||||
phpMissing: metricPanels[1].querySelector('[data-web-live-field="php"]').textContent,
|
|
||||||
columns: getComputedStyle(grid).gridTemplateColumns.split(' ').length,
|
|
||||||
chartWidth: rect(chart).width, gridWidth: rect(grid).width,
|
|
||||||
cardWidth: rect(cards[0]).width, listWidth: rect(list).width,
|
|
||||||
svgWidth: svgBox.width, svgContentWidth: svg.viewBox.baseVal.width * Math.hypot(svgMatrix.a, svgMatrix.b),
|
|
||||||
preserveAspect: svg.getAttribute('preserveAspectRatio'),
|
|
||||||
stroke: getComputedStyle(svg.querySelector('polyline')).vectorEffect,
|
|
||||||
chartLabel: !!chart.querySelector('.vhost-chart-head .small-label'),
|
|
||||||
rendered: getComputedStyle(metricPanels[0]).display !== 'none' && rect(chart).height > 60,
|
|
||||||
fitsViewport: document.documentElement.scrollWidth <= innerWidth + 2,
|
|
||||||
expectedMetricLabels,
|
|
||||||
};
|
|
||||||
}''', status(1))
|
|
||||||
expected_cols = 2 if width <= 640 else 3 if width <= 980 else 6
|
|
||||||
assert initial['cards'] == 2, (width, initial)
|
|
||||||
assert initial['sectionTags'] == ['SECTION', 'SECTION'], (width, initial)
|
|
||||||
assert initial['collapsibles'] == 0, (width, initial)
|
|
||||||
assert initial['liveFields'] == [7, 7], (width, initial)
|
|
||||||
assert initial['labels'] == initial['expectedMetricLabels'], (width, initial)
|
|
||||||
assert initial['tileClasses'] == ['vhost-live-metric'] * 6, (width, initial)
|
|
||||||
assert initial['values'][0] != '-' and initial['values'][4] == '3 / 16', (width, initial)
|
|
||||||
assert initial['phpMissing'] != '-', (width, initial)
|
|
||||||
assert initial['columns'] == expected_cols, (width, initial)
|
|
||||||
assert initial['chartWidth'] > 0.95 * initial['gridWidth'], (width, initial)
|
|
||||||
assert initial['listWidth'] - initial['cardWidth'] < 3, (width, initial)
|
|
||||||
assert abs(initial['svgWidth'] - initial['svgContentWidth']) <= 2, (width, initial)
|
|
||||||
assert initial['preserveAspect'] == 'none' and initial['stroke'] == 'non-scaling-stroke', (width, initial)
|
|
||||||
assert initial['chartLabel'] and initial['rendered'] and initial['fitsViewport'], (width, initial)
|
|
||||||
|
|
||||||
sampled = page.evaluate('''([first, second, last]) => {
|
|
||||||
const api = window.rustProxyWebServer;
|
|
||||||
const panel = document.querySelector('[data-web-live-id="site-A"]');
|
|
||||||
const rate = panel.querySelector('[data-web-live-field="rate"]');
|
|
||||||
window.__clock = 4000; api.renderLive(first, true);
|
|
||||||
window.__clock = 7000; api.renderLive(second, true);
|
|
||||||
const before = {rate: rate.textContent, points: panel.querySelector('polyline').getAttribute('points')};
|
|
||||||
// Service-list redraw must show the current status without opening anything.
|
|
||||||
latestStatus = last;
|
|
||||||
api.render();
|
|
||||||
const updated = document.querySelector('[data-web-live-id="site-A"]');
|
|
||||||
const total = updated.querySelector('[data-web-live-field="requests"]').textContent;
|
|
||||||
const split = updated.querySelector('[data-web-live-field="split"]').textContent;
|
|
||||||
return {before, total, split,
|
|
||||||
pointsAfterRedraw: updated.querySelector('polyline').getAttribute('points'),
|
|
||||||
metricsRemainVisible: !!updated.querySelector('.vhost-live-metric'),
|
|
||||||
overflow: document.documentElement.scrollWidth - innerWidth};
|
|
||||||
}''', [status(2), status(3), status(4)])
|
|
||||||
assert sampled['before']['rate'] not in ('-', '0/s', '0.0/s'), (width, sampled)
|
|
||||||
assert sampled['before']['points'].count(' ') >= 1, (width, sampled)
|
|
||||||
assert sampled['total'] != initial['values'][0], (width, sampled)
|
|
||||||
assert sampled['split'] != '-' and sampled['metricsRemainVisible'], (width, sampled)
|
|
||||||
assert sampled['pointsAfterRedraw'] == sampled['before']['points'], (width, sampled)
|
|
||||||
assert sampled['overflow'] <= 2, (width, sampled)
|
|
||||||
assert not errors, (width, errors)
|
|
||||||
print(f'PASS 0.33.4 WWW live metrics visual parity at {width}px; {expected_cols} columns')
|
|
||||||
page.close()
|
|
||||||
browser.close()
|
|
||||||
print('PASS always-visible full-width WWW/Vhost parity with simulated updates (not Rust runtime)')
|
|
||||||
@@ -236,7 +236,6 @@ pub(super) static READER_DOCUMENTS: &[(&str, &str, &str)] = &[
|
|||||||
"/docs/SECURITY_AUDIT.md"
|
"/docs/SECURITY_AUDIT.md"
|
||||||
)),
|
)),
|
||||||
),
|
),
|
||||||
reader_doc!("docs/SECURITY_AUDIT_0.31.0.md", "audits"),
|
|
||||||
(
|
(
|
||||||
"docs/SETTINGS.md",
|
"docs/SETTINGS.md",
|
||||||
"guides",
|
"guides",
|
||||||
@@ -247,16 +246,6 @@ pub(super) static READER_DOCUMENTS: &[(&str, &str, &str)] = &[
|
|||||||
"guides",
|
"guides",
|
||||||
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/STREAMS.md")),
|
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/STREAMS.md")),
|
||||||
),
|
),
|
||||||
(
|
|
||||||
"docs/TEST_REPORT.md",
|
|
||||||
"guides",
|
|
||||||
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/TEST_REPORT.md")),
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"docs/TODO.md",
|
|
||||||
"guides",
|
|
||||||
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/TODO.md")),
|
|
||||||
),
|
|
||||||
(
|
(
|
||||||
"docs/TRANSFORM.md",
|
"docs/TRANSFORM.md",
|
||||||
"guides",
|
"guides",
|
||||||
|
|||||||
+1
-1
@@ -2370,7 +2370,7 @@ mod tests {
|
|||||||
.any(|document| document["path"] == "docs/PHP_FPM.md"));
|
.any(|document| document["path"] == "docs/PHP_FPM.md"));
|
||||||
assert!(documents
|
assert!(documents
|
||||||
.iter()
|
.iter()
|
||||||
.any(|document| document["path"] == "docs/SECURITY_AUDIT_0.31.0.md"));
|
.any(|document| document["path"] == "docs/SECURITY_AUDIT.md"));
|
||||||
assert!(documents
|
assert!(documents
|
||||||
.iter()
|
.iter()
|
||||||
.any(|document| document["path"] == "docs/TEST_REPORT.md"));
|
.any(|document| document["path"] == "docs/TEST_REPORT.md"));
|
||||||
|
|||||||
@@ -48,53 +48,61 @@ fn redact_regex(pattern: &'static str) -> &'static regex::Regex {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn is_secret_field(name: &str) -> bool {
|
fn is_secret_field(name: &str) -> bool {
|
||||||
matches!(
|
const FIELDS: &[&str] = &[
|
||||||
name.to_ascii_lowercase().as_str(),
|
"authorization",
|
||||||
"authorization"
|
"proxy_authorization",
|
||||||
| "proxy_authorization"
|
"api_token",
|
||||||
| "api_token"
|
"access_token",
|
||||||
| "access_token"
|
"application_secret",
|
||||||
| "application_secret"
|
"consumer_key",
|
||||||
| "consumer_key"
|
"key_secret",
|
||||||
| "key_secret"
|
"backup_password",
|
||||||
| "backup_password"
|
"password",
|
||||||
| "password"
|
"credential",
|
||||||
| "credential"
|
"credentials",
|
||||||
| "credentials"
|
"secret",
|
||||||
| "secret"
|
"token",
|
||||||
| "token"
|
"private_key",
|
||||||
| "private_key"
|
"master_key",
|
||||||
| "master_key"
|
];
|
||||||
)
|
// ASCII-insensitive matching has the same semantics as to_ascii_lowercase,
|
||||||
|
// without allocating a new key for every field in every JSON access log.
|
||||||
|
FIELDS.iter().any(|field| name.eq_ignore_ascii_case(field))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn redact_text_cow(input: &str) -> std::borrow::Cow<'_, str> {
|
||||||
|
let mut value = std::borrow::Cow::Borrowed(input);
|
||||||
|
// Keep the original order: a replacement can expose a pattern handled by
|
||||||
|
// the next rule. Ordinary, non-secret values stay borrowed (no six copies).
|
||||||
|
for (pattern, replacement) in [
|
||||||
|
("pem", "[REDACTED PRIVATE KEY]"),
|
||||||
|
("auth", "$1[REDACTED]"),
|
||||||
|
("query", "$1[REDACTED]"),
|
||||||
|
("escaped_json", "$1[REDACTED]$2"),
|
||||||
|
("json", "$1[REDACTED]$2"),
|
||||||
|
("kv", "$1[REDACTED]"),
|
||||||
|
] {
|
||||||
|
let regex = redact_regex(pattern);
|
||||||
|
if regex.is_match(value.as_ref()) {
|
||||||
|
value = std::borrow::Cow::Owned(
|
||||||
|
regex.replace_all(value.as_ref(), replacement).into_owned(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
value
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn redact_text(input: &str) -> String {
|
pub fn redact_text(input: &str) -> String {
|
||||||
let mut value = redact_regex("pem")
|
redact_text_cow(input).into_owned()
|
||||||
.replace_all(input, "[REDACTED PRIVATE KEY]")
|
|
||||||
.into_owned();
|
|
||||||
value = redact_regex("auth")
|
|
||||||
.replace_all(&value, "$1[REDACTED]")
|
|
||||||
.into_owned();
|
|
||||||
value = redact_regex("query")
|
|
||||||
.replace_all(&value, "$1[REDACTED]")
|
|
||||||
.into_owned();
|
|
||||||
// Debug-formatted fields can contain a JSON string with escaped quote
|
|
||||||
// delimiters. Redact that form before plain JSON, retaining quote escapes
|
|
||||||
// and consuming escaped quotes inside a secret instead of leaking its tail.
|
|
||||||
value = redact_regex("escaped_json")
|
|
||||||
.replace_all(&value, "$1[REDACTED]$2")
|
|
||||||
.into_owned();
|
|
||||||
value = redact_regex("json")
|
|
||||||
.replace_all(&value, "$1[REDACTED]$2")
|
|
||||||
.into_owned();
|
|
||||||
redact_regex("kv")
|
|
||||||
.replace_all(&value, "$1[REDACTED]")
|
|
||||||
.into_owned()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn redact_value(value: &mut Value) {
|
fn redact_value(value: &mut Value) {
|
||||||
match value {
|
match value {
|
||||||
Value::String(text) => *text = redact_text(text),
|
Value::String(text) => {
|
||||||
|
if let std::borrow::Cow::Owned(redacted) = redact_text_cow(text) {
|
||||||
|
*text = redacted;
|
||||||
|
}
|
||||||
|
}
|
||||||
Value::Array(items) => {
|
Value::Array(items) => {
|
||||||
for item in items {
|
for item in items {
|
||||||
redact_value(item);
|
redact_value(item);
|
||||||
|
|||||||
+16
-8
@@ -138,11 +138,21 @@ impl Logs {
|
|||||||
pub fn push(&self, entry: LogEntry) {
|
pub fn push(&self, entry: LogEntry) {
|
||||||
let has_viewers = self.live.receiver_count() != 0;
|
let has_viewers = self.live.receiver_count() != 0;
|
||||||
let has_sink = self.settings.load().enabled();
|
let has_sink = self.settings.load().enabled();
|
||||||
// The usual history-only configuration needs no deep copy at all.
|
// Reserve the bounded sink slot BEFORE cloning a log entry. Under
|
||||||
// When external consumers are present, clone exactly for history and
|
// saturation, the failed enqueue must not copy 30+ owned fields.
|
||||||
// keep the original entry available for streaming/sink delivery.
|
let sink_permit = if has_sink {
|
||||||
|
match self.sink.try_reserve() {
|
||||||
|
Ok(permit) => Some(permit),
|
||||||
|
Err(_) => {
|
||||||
|
self.sink_dropped.fetch_add(1, Ordering::Relaxed);
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
let mut external = Some(entry);
|
let mut external = Some(entry);
|
||||||
let history_entry = if has_viewers || has_sink {
|
let history_entry = if has_viewers || sink_permit.is_some() {
|
||||||
external.as_ref().expect("entry").clone()
|
external.as_ref().expect("entry").clone()
|
||||||
} else {
|
} else {
|
||||||
external.take().expect("entry")
|
external.take().expect("entry")
|
||||||
@@ -165,10 +175,8 @@ impl Logs {
|
|||||||
if has_viewers {
|
if has_viewers {
|
||||||
let _ = self.live.send(entry.clone());
|
let _ = self.live.send(entry.clone());
|
||||||
}
|
}
|
||||||
// Snapshot the sink choice once. This preserves sink drop
|
if let Some(permit) = sink_permit {
|
||||||
// accounting and avoids a second ArcSwap load per request.
|
permit.send(entry);
|
||||||
if has_sink && self.sink.try_send(entry).is_err() {
|
|
||||||
self.sink_dropped.fetch_add(1, Ordering::Relaxed);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+69
-3
@@ -41,6 +41,10 @@ fn format_entry(entry: &LogEntry, format: &JsonFormat) -> Value {
|
|||||||
Value::Object(object)
|
Value::Object(object)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Bound both write size and event count; never wait for the batch to fill.
|
||||||
|
const FILE_BATCH_EVENTS: u64 = 64;
|
||||||
|
const FILE_BATCH_BYTES: usize = 128 * 1024;
|
||||||
|
|
||||||
type LokiClient = Client<hyper_rustls::HttpsConnector<HttpConnector>, Body>;
|
type LokiClient = Client<hyper_rustls::HttpsConnector<HttpConnector>, Body>;
|
||||||
fn loki_client() -> Result<LokiClient> {
|
fn loki_client() -> Result<LokiClient> {
|
||||||
let mut roots = RootCertStore::empty();
|
let mut roots = RootCertStore::empty();
|
||||||
@@ -114,12 +118,74 @@ async fn sink_loop(
|
|||||||
let mut vhost_files: BTreeMap<PathBuf, File> = BTreeMap::new();
|
let mut vhost_files: BTreeMap<PathBuf, File> = BTreeMap::new();
|
||||||
let mut vhost_directory: Option<PathBuf> = None;
|
let mut vhost_directory: Option<PathBuf> = None;
|
||||||
let mut logstash: Option<LogstashState> = None;
|
let mut logstash: Option<LogstashState> = None;
|
||||||
|
let mut deferred_entry: Option<LogEntry> = None;
|
||||||
|
let mut file_batch = Vec::with_capacity(8 * 1024);
|
||||||
loop {
|
loop {
|
||||||
let entry = tokio::select! {
|
let entry = if let Some(entry) = deferred_entry.take() {
|
||||||
_ = shutdown.cancelled() => break,
|
entry
|
||||||
entry = rx.recv() => match entry { Some(entry) => entry, None => break },
|
} else {
|
||||||
|
tokio::select! {
|
||||||
|
_ = shutdown.cancelled() => break,
|
||||||
|
entry = rx.recv() => match entry { Some(entry) => entry, None => break },
|
||||||
|
}
|
||||||
};
|
};
|
||||||
let config = settings.load_full();
|
let config = settings.load_full();
|
||||||
|
// The common local-file-only sink has no per-event network or routing
|
||||||
|
// dependency. Coalesce consecutive JSONL records into one bounded write.
|
||||||
|
// Other sink combinations retain their original per-event semantics.
|
||||||
|
if config.file.enabled && !config.file.per_vhost
|
||||||
|
&& !config.logstash.enabled && !config.loki.enabled
|
||||||
|
{
|
||||||
|
vhost_files.clear();
|
||||||
|
vhost_directory = None;
|
||||||
|
let path = PathBuf::from(&config.file.path);
|
||||||
|
if file.as_ref().is_none_or(|state| state.path != path) {
|
||||||
|
file = match open_access_file(path.clone()).await {
|
||||||
|
Ok(opened) => Some(FileState { path: path.clone(), file: opened }),
|
||||||
|
Err(error) => {
|
||||||
|
tracing::warn!(%error, path=%path.display(), "access log file unavailable");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
file_batch.clear();
|
||||||
|
let mut count = 0u64;
|
||||||
|
let mut next = Some(entry);
|
||||||
|
while let Some(item) = next.take() {
|
||||||
|
let mut payload = format_entry(&item, &config.format);
|
||||||
|
redact_value(&mut payload);
|
||||||
|
let mut line = serde_json::to_vec(&payload).unwrap_or_default();
|
||||||
|
line.push(b'\n');
|
||||||
|
file_batch.extend_from_slice(&line);
|
||||||
|
count += 1;
|
||||||
|
if count >= FILE_BATCH_EVENTS || file_batch.len() >= FILE_BATCH_BYTES {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
match rx.try_recv() {
|
||||||
|
Ok(item) if Arc::ptr_eq(&settings.load_full(), &config) => {
|
||||||
|
next = Some(item);
|
||||||
|
}
|
||||||
|
Ok(item) => {
|
||||||
|
// A pending entry must be handled with the new snapshot;
|
||||||
|
// a batch never silently crosses a configuration change.
|
||||||
|
deferred_entry = Some(item);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
Err(_) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let written = match file.as_mut() {
|
||||||
|
Some(state) => state.file.write_all(&file_batch).await.is_ok(),
|
||||||
|
None => false,
|
||||||
|
};
|
||||||
|
if written {
|
||||||
|
sent.fetch_add(count, Ordering::Relaxed);
|
||||||
|
} else {
|
||||||
|
errors.fetch_add(count, Ordering::Relaxed);
|
||||||
|
file = None;
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
}
|
||||||
let mut payload = format_entry(&entry, &config.format);
|
let mut payload = format_entry(&entry, &config.format);
|
||||||
redact_value(&mut payload);
|
redact_value(&mut payload);
|
||||||
// File and Logstash consume the same JSONL record. Serialize once and
|
// File and Logstash consume the same JSONL record. Serialize once and
|
||||||
|
|||||||
@@ -458,4 +458,67 @@ mod tests {
|
|||||||
assert_eq!(logs.stats().queue_dropped, 1);
|
assert_eq!(logs.stats().queue_dropped, 1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn redaction_leaves_ordinary_strings_borrowed_and_preserves_secret_matching() {
|
||||||
|
use std::borrow::Cow;
|
||||||
|
assert!(matches!(redact_text_cow("GET /index.html"), Cow::Borrowed(_)));
|
||||||
|
assert!(matches!(redact_text_cow("hello € world"), Cow::Borrowed(_)));
|
||||||
|
assert!(matches!(redact_text_cow("?api_token=topsecret"), Cow::Owned(_)));
|
||||||
|
let mut value = json!({
|
||||||
|
"ToKeN": "secret-value",
|
||||||
|
"ordinary": "hello € world",
|
||||||
|
"nested": {"AUTHORIZATION": "Bearer private"},
|
||||||
|
"path": "/index?password=topsecret"
|
||||||
|
});
|
||||||
|
redact_value(&mut value);
|
||||||
|
assert_eq!(value["ToKeN"], "[REDACTED]");
|
||||||
|
assert_eq!(value["ordinary"], "hello € world");
|
||||||
|
assert_eq!(value["nested"]["AUTHORIZATION"], "[REDACTED]");
|
||||||
|
assert_eq!(value["path"], "/index?password=[REDACTED]");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn local_file_log_batch_keeps_every_jsonl_record_and_counters() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
let path = dir.path().join("access.jsonl");
|
||||||
|
let mut settings = Settings::default();
|
||||||
|
settings.file.enabled = true;
|
||||||
|
settings.file.path = path.to_string_lossy().into_owned();
|
||||||
|
settings.format.fields.insert("request_number".into(), "id".into());
|
||||||
|
settings.format.fields.insert("request_path".into(), "path".into());
|
||||||
|
settings.format.static_fields.insert("service".into(), "rustproxy".into());
|
||||||
|
let stop = CancellationToken::new();
|
||||||
|
let logs = Logs::start(settings, stop.clone());
|
||||||
|
const COUNT: usize = 180;
|
||||||
|
for id in 0..COUNT {
|
||||||
|
let mut item = entry();
|
||||||
|
item.id = id as u64;
|
||||||
|
item.path = format!("/test/{id}?token=secret");
|
||||||
|
logs.push(item);
|
||||||
|
}
|
||||||
|
let lines = tokio::time::timeout(Duration::from_secs(10), async {
|
||||||
|
loop {
|
||||||
|
if logs.stats().sent == COUNT as u64 {
|
||||||
|
if let Ok(text) = tokio::fs::read_to_string(&path).await {
|
||||||
|
if text.lines().count() == COUNT {
|
||||||
|
break text;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||||
|
}
|
||||||
|
}).await.expect("all accepted local file records must be written");
|
||||||
|
stop.cancel();
|
||||||
|
assert_eq!(logs.stats().queue_dropped, 0);
|
||||||
|
assert_eq!(logs.stats().errors, 0);
|
||||||
|
assert_eq!(logs.stats().sent, COUNT as u64);
|
||||||
|
for (id, line) in lines.lines().enumerate() {
|
||||||
|
let record: Value = serde_json::from_str(line).unwrap();
|
||||||
|
assert_eq!(record["request_number"], id as u64);
|
||||||
|
assert_eq!(record["request_path"], format!("/test/{id}?token=[REDACTED]"));
|
||||||
|
assert_eq!(record["service"], "rustproxy");
|
||||||
|
assert_eq!(record.as_object().unwrap().len(), 3);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
@@ -36,7 +36,12 @@ mod authority_tests {
|
|||||||
fn preserve_host_sets_http2_authority_without_changing_network_origin() {
|
fn preserve_host_sets_http2_authority_without_changing_network_origin() {
|
||||||
use crate::config::UpstreamProtocol;
|
use crate::config::UpstreamProtocol;
|
||||||
for (protocol, scheme, expected_authority, remove_host) in [
|
for (protocol, scheme, expected_authority, remove_host) in [
|
||||||
(UpstreamProtocol::Http1, "https", "backend.internal:443", false),
|
(
|
||||||
|
UpstreamProtocol::Http1,
|
||||||
|
"https",
|
||||||
|
"backend.internal:443",
|
||||||
|
false,
|
||||||
|
),
|
||||||
(UpstreamProtocol::Auto, "https", "unitraklub.pl", false),
|
(UpstreamProtocol::Auto, "https", "unitraklub.pl", false),
|
||||||
(UpstreamProtocol::Http2, "https", "unitraklub.pl", false),
|
(UpstreamProtocol::Http2, "https", "unitraklub.pl", false),
|
||||||
(UpstreamProtocol::H2, "https", "unitraklub.pl", true),
|
(UpstreamProtocol::H2, "https", "unitraklub.pl", true),
|
||||||
@@ -59,7 +64,11 @@ mod authority_tests {
|
|||||||
let mut headers = HeaderMap::new();
|
let mut headers = HeaderMap::new();
|
||||||
headers.insert(HOST, HeaderValue::from_static("unitraklub.pl"));
|
headers.insert(HOST, HeaderValue::from_static("unitraklub.pl"));
|
||||||
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
|
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
|
||||||
assert_eq!(target.authority().unwrap().as_str(), expected_authority, "{protocol:?}");
|
assert_eq!(
|
||||||
|
target.authority().unwrap().as_str(),
|
||||||
|
expected_authority,
|
||||||
|
"{protocol:?}"
|
||||||
|
);
|
||||||
assert_eq!(target.path(), "/block_refresh/block/10");
|
assert_eq!(target.path(), "/block_refresh/block/10");
|
||||||
assert_eq!(headers.contains_key(HOST), !remove_host);
|
assert_eq!(headers.contains_key(HOST), !remove_host);
|
||||||
assert_eq!(backend.origin, format!("{scheme}://10.87.20.2:443"));
|
assert_eq!(backend.origin, format!("{scheme}://10.87.20.2:443"));
|
||||||
@@ -70,8 +79,10 @@ mod authority_tests {
|
|||||||
fn optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes() {
|
fn optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes() {
|
||||||
assert_eq!(log_prefix("/small.html", 256), "/small.html");
|
assert_eq!(log_prefix("/small.html", 256), "/small.html");
|
||||||
assert_eq!(log_prefix("abcdefghijklmnopqrstuvwxyz", 5), "abcde");
|
assert_eq!(log_prefix("abcdefghijklmnopqrstuvwxyz", 5), "abcde");
|
||||||
assert_eq!(log_prefix("za\u{17c}\u{f3}\u{142}\u{107}\u{1f525}", 4),
|
assert_eq!(
|
||||||
"za\u{17c}\u{f3}");
|
log_prefix("za\u{17c}\u{f3}\u{142}\u{107}\u{1f525}", 4),
|
||||||
|
"za\u{17c}\u{f3}"
|
||||||
|
);
|
||||||
assert_eq!(log_prefix("", 0), "");
|
assert_eq!(log_prefix("", 0), "");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -83,8 +94,14 @@ mod authority_tests {
|
|||||||
let uri = upstream_uri(backend, "/keep?key=hello%2Fworld").unwrap();
|
let uri = upstream_uri(backend, "/keep?key=hello%2Fworld").unwrap();
|
||||||
let mut headers = HeaderMap::new();
|
let mut headers = HeaderMap::new();
|
||||||
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
|
let target = upstream_request_target(uri, &mut headers, backend, false, false).unwrap();
|
||||||
assert_eq!(target.path_and_query().unwrap().as_str(), "/keep?key=hello%2Fworld");
|
assert_eq!(
|
||||||
assert_eq!(target.authority().unwrap(), backend.request_origin_uri.authority().unwrap());
|
target.path_and_query().unwrap().as_str(),
|
||||||
|
"/keep?key=hello%2Fworld"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
target.authority().unwrap(),
|
||||||
|
backend.request_origin_uri.authority().unwrap()
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -109,8 +126,16 @@ mod authority_tests {
|
|||||||
let backend = &snapshot.pools[0].members[0].backend;
|
let backend = &snapshot.pools[0].members[0].backend;
|
||||||
let mut headers = HeaderMap::new();
|
let mut headers = HeaderMap::new();
|
||||||
headers.insert(HOST, HeaderValue::from_static("public.example"));
|
headers.insert(HOST, HeaderValue::from_static("public.example"));
|
||||||
let req = retry_request(&Method::GET, Version::HTTP_2, &headers, backend,
|
let req = retry_request(
|
||||||
&HeaderValue::from_static("public.example"), "/retry", true).unwrap();
|
&Method::GET,
|
||||||
|
Version::HTTP_2,
|
||||||
|
&headers,
|
||||||
|
backend,
|
||||||
|
&HeaderValue::from_static("public.example"),
|
||||||
|
"/retry",
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
assert_eq!(req.uri().authority().unwrap().as_str(), "public.example");
|
assert_eq!(req.uri().authority().unwrap().as_str(), "public.example");
|
||||||
assert!(req.headers().get(HOST).is_none());
|
assert!(req.headers().get(HOST).is_none());
|
||||||
}
|
}
|
||||||
@@ -361,7 +386,10 @@ mod authority_tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn backend_diagnostics_are_opt_in_and_strip_spoofed_values() {
|
fn backend_diagnostics_are_opt_in_and_strip_spoofed_values() {
|
||||||
let mut response = Response::new(Body::empty());
|
let mut response = Response::new(Body::empty());
|
||||||
response.headers_mut().insert("x-rustproxy-backend", HeaderValue::from_static("canonical-spoof"));
|
response.headers_mut().insert(
|
||||||
|
"x-rustproxy-backend",
|
||||||
|
HeaderValue::from_static("canonical-spoof"),
|
||||||
|
);
|
||||||
response
|
response
|
||||||
.headers_mut()
|
.headers_mut()
|
||||||
.insert("x-rust-proxy-backend", HeaderValue::from_static("spoofed"));
|
.insert("x-rust-proxy-backend", HeaderValue::from_static("spoofed"));
|
||||||
@@ -388,7 +416,10 @@ mod authority_tests {
|
|||||||
response
|
response
|
||||||
.extensions_mut()
|
.extensions_mut()
|
||||||
.insert(OriginTiming(Duration::from_millis(12)));
|
.insert(OriginTiming(Duration::from_millis(12)));
|
||||||
response.headers_mut().insert("x-rustproxy-backend", HeaderValue::from_static("canonical-spoof"));
|
response.headers_mut().insert(
|
||||||
|
"x-rustproxy-backend",
|
||||||
|
HeaderValue::from_static("canonical-spoof"),
|
||||||
|
);
|
||||||
apply_backend_diagnostic_values(&mut response, true, true);
|
apply_backend_diagnostic_values(&mut response, true, true);
|
||||||
assert!(response.headers().get("x-rustproxy-backend").is_none());
|
assert!(response.headers().get("x-rustproxy-backend").is_none());
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
@@ -623,16 +654,25 @@ mod graceful_h2_tests {
|
|||||||
config.vhosts[0].backends[0].tls_server_name = Some("backend.example".into());
|
config.vhosts[0].backends[0].tls_server_name = Some("backend.example".into());
|
||||||
let snapshot = crate::config::compile(config, 1, &validation_policy()).unwrap();
|
let snapshot = crate::config::compile(config, 1, &validation_policy()).unwrap();
|
||||||
let backend = &snapshot.pools[0].members[0].backend;
|
let backend = &snapshot.pools[0].members[0].backend;
|
||||||
assert_eq!(backend.request_origin_uri.to_string(), backend.request_origin);
|
assert_eq!(
|
||||||
|
backend.request_origin_uri.to_string(),
|
||||||
|
backend.request_origin
|
||||||
|
);
|
||||||
for suffix in [
|
for suffix in [
|
||||||
"/", "/static/main.css", "/path?x=1&y=2", "/p%2Fq?url=https%3A//example.org/a",
|
"/",
|
||||||
"/?q=%C5%82&empty=", "/double//slash", "/item#frag",
|
"/static/main.css",
|
||||||
|
"/path?x=1&y=2",
|
||||||
|
"/p%2Fq?url=https%3A//example.org/a",
|
||||||
|
"/?q=%C5%82&empty=",
|
||||||
|
"/double//slash",
|
||||||
|
"/item#frag",
|
||||||
] {
|
] {
|
||||||
let previous = format!("{}{suffix}", backend.request_origin).parse::<Uri>().ok();
|
let previous = format!("{}{suffix}", backend.request_origin)
|
||||||
|
.parse::<Uri>()
|
||||||
|
.ok();
|
||||||
assert_eq!(upstream_uri(backend, suffix), previous, "{suffix}");
|
assert_eq!(upstream_uri(backend, suffix), previous, "{suffix}");
|
||||||
}
|
}
|
||||||
assert!(upstream_uri(backend, "http://attacker.invalid/").is_none());
|
assert!(upstream_uri(backend, "http://attacker.invalid/").is_none());
|
||||||
assert_eq!(backend.origin, "https://127.0.0.1:9000");
|
assert_eq!(backend.origin, "https://127.0.0.1:9000");
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
Reference in new issue
Block a user