gru/RustProxy
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.34.0_8

Browse Source
This commit is contained in:
Mateusz Gruszczyński committed 2026-10-18 08:26:35 +02:00
1 parent b6a3e12225
commit 480c3ec769
92 files changed
+298 -10035
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/performance/README.md
-354
View File
@@ -1,354 +0,0 @@
# Live performance and regression harness
## DEV OPT3 / 1 KiB response: uncompiled source
`docs/PROXY_1KB_OPT3_DEV_0.34.0.md` describes 7 optimized hot-path sites
and how to benchmark their **newly rebuilt** executable. Baseline
`rust-proxy(2)` passed 89 real proxy/policy/cache checks. This runtime result
is not evidence that uncompiled OPT3 changes work, nor a measured OPT3 RPS gain.
These scripts exercise actual RustProxy executables through loopback sockets.
They do not simulate Rust request handling. The Python server in `peer.py` is a
controllable test **upstream**, used to verify streaming, non-replayable writes,
headers, cache policy and WebSocket frames. Performance measurements use nginx
as the shared upstream instead of Python.
## DEV 1 KiB histogram writes (source candidate)
`VhostMetrics::complete` now writes one exclusive histogram bin for each
completed exchange instead of up to 11 cumulative atomics. Both Prometheus
exporters reconstruct `le` and `+Inf` at scrape time. Unit tests cover boundary
semantics. This change has not been compiled or benchmarked: see
`docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`.
## DEV 1 KiB cache counter follow-up
See `docs/PROXY_1KB_COUNTERS_DEV_0.34.0.md`. Cache domain metrics no longer
lock the HOT store on BYPASS; analytics cloning is deferred until the queue
has capacity. The new **source** remains uncompiled in the editing environment.
For quick 1,040-byte loopback diagnostics with ApacheBench (not a production
acceptance benchmark), use `python3 scripts/performance/ab_1kb_dev.py --help`.
The authoritative long-run comparison remains `benchmark.py` with pinned
CPU sets, an explicitly equal upstream idle pool and multiple `wrk` repeats.
## DEV 1 KiB follow-up on 0.34.0 (newer source)
See `docs/PROXY_1KB_HOTPATH_DEV_0.34.0.md`. The previous executable's 4-vs-64
idle-connection tuning experiment is real loopback evidence, but the *edited*
source has not been compiled. Its new default is 64 for new vhosts, whereas
explicitly saved `4` remains `4`. Keep the idle limit equal between before/after
*code* benchmarks; measure configuration tuning in a separate labelled series.
The source release is not eligible for a claim of 100k RPS or 80% nginx yet.
## Reverse-proxy optimization revision 0.34.0
After compiling the refreshed candidate, measure the same unmodified 0.33.8
binary and nginx reference as before, without turning off logging, request
IDs, ACL, cache-policy enforcement or LB health. Profile request-size 1,040 B
first. Before RPS runs execute full proxy, LB, transport and concurrency
regressions (including single-member capacity, recovery, sticky cookies,
replays on safe methods, rejected non-replayable writes and H2 connection
shutdown). Treat `--max-idle` tuning and recycles as separately labelled
experiments rather than mixing them into the primary comparison. There are no
new measured RPS figures in the source candidate.
## Follow-up RPS candidate on 0.34.0
`docs/PROXY_RPS_0.34.0.md` tracks the code changes and their regressions.
Compile before/after binaries from the two 0.34.0 source revisions and use
the paired benchmark below; a live run against the older binary cannot
establish a measured RPS improvement for the uncompiled candidate. Keep
`max_idle_per_host`, recycling, HTTP1/H2 settings and all application security
and logging features equal between runs. Treat any configuration tuning as a
separate experiment. Profile the resulting binary with `perf` only after
correctness checks pass.
## Offline cache-retention model for 0.34.0
`python3 scripts/performance/simulate_cache_034.py` compares cache invalidation
policies under a deterministic synthetic sequence of vhost saves and uniform
cacheable GETs. The model contains **no RustProxy server**, no timing and no
network traffic; its percentages are **not measured RPS gains**. See
`docs/TEST_REPORT_0.34.0.md` for assumptions and the live acceptance gap.
## Requirements
- Linux with readable `/proc`, Python 3.10+, executable RustProxy binaries.
- nginx and wrk for performance comparisons. Both paths are explicit; no
system-wide installation is required.
- `taskset` from util-linux for the benchmark's explicit CPU affinity and for
optional affinity settings in the other scripts.
- Optional Python `brotli` and `zstandard` decoders increase compression
acceptance coverage. A missing optional decoder is reported as **skipped**.
- Permission to bind unprivileged loopback sockets and start local processes.
In a sandbox without sockets or `/proc`, run in the approved local execution
environment. Do not replace real runtime results with a peer self-test.
Each run creates a new private temporary data directory and initializes its own
administrator with a random password supplied only over stdin. Configurations
are applied through RustProxy's existing authenticated CLI/API and listener
changes receive a restart. No historical SQLite database is copied or edited.
TLS keys, databases and passwords are never written to the evidence directory.
Temporary state is outside the evidence tree and is removed on normal exit.
`fixture-cleanup.json` records the cleanup check. Interrupted runs can leave an
unrelated `/tmp/rustproxy-perf-*` directory; do not include it in a release.
Before timed benchmark, soak or syscall-profile work, `wait_quiescent()` waits
for the existing authenticated ACME runtime API to report `ready` or
`unavailable`. A fresh RustProxy data directory can start a Python/Certbot
environment bootstrap even when no certificates have an issuer. The gate
requires no configured issuers, jobs or renewal observations, the same live Rust
PID and start time, and no child processes for five continuous seconds. It polls
every 0.5 seconds and fails after 180 seconds instead of measuring installation
work. Selected status, process count and timing evidence is saved in
`<label>-quiescence.json`. No PATH, pip settings or application features are
changed. An `unavailable` result permits the HTTP fixture to proceed; these runs
do not test ACME issuance or certificate-provider availability.
## Regression tests
Run from the repository root, providing absolute binary and output paths:
```bash
python3 scripts/performance/regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/regression
```
`--groups static,proxy,policy,cache,transform` selects groups; all are enabled by
default. `--workers 4` sets the same `TOKIO_WORKER_THREADS` for each binary.
The report contains individual pass/fail/skip observations and a comparison
matrix. An existing failure in the supplied binary remains a failure in its
report; it is not silently converted to a successful baseline expectation.
Coverage includes:
- Static GET/HEAD and byte checksums at 1,040 B, 64 KiB and 1 MiB; empty files;
ETag/Last-Modified conditional requests; prefix/suffix/open/multipart ranges,
If-Range and 416; directory redirect/index/listing; decoded path rejection;
deny/inside-root symlinks; precompressed gzip; MIME/download headers;
`try_files`, redirects and the compiled `.htaccess` header subset.
- Proxy POST/PUT, 1 MiB upload checksums, chunked upload delivery **before the
last chunk is sent**, early response delivery, client disconnect, original
status/redirect preservation, failed-POST no replay, request/response header
scopes, hop-by-hop stripping, request-ID shape/uniqueness and WebSocket echo.
- Global host/vhost/profile/location ACL composition, upstream suppression on
denial, trusted versus spoofed forwarding, untrusted country hints, method
and rate policies, hot changes and rejection of invalid configuration.
- Private administration routes, login, session/CSRF and Origin enforcement,
Basic Auth, ACL-before-auth, private-origin rejection, retained exchange
logging fields and byte accounting.
- Cache global/vhost/route OFF and ON, MISS/HIT, Vary isolation, cookie,
Authorization/private/no-store bypass, PURGE, concurrent fills, validator
revalidation, static cached ranges/conditionals; transforms OFF/ON with cache
OFF/ON; compression negotiation, exclusions and scope/global codec ceilings.
The generic suite does not establish TLS/H2/H3, Tinyauth, real PHP-FPM, real DNS
rebinding, concurrent filesystem rename-race safety or long-soak acceptance.
Use the dedicated transport tests and existing Rust integration suites for
their covered cases. The sequential symlink-target test is explicitly not a
concurrent rename-race proof.
## TLS, HTTP/2, HTTP/3 and WebSocket interoperability
```bash
python3 scripts/performance/transport_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/transport
```
This suite requires `aioquic`, `httpx[http2]` and `websockets`; use
`--python-deps /absolute/path/pydeps` for a separate dependency directory.
It verifies the temporary server certificate, negotiated protocol, response
status and bytes for TLS H2 and QUIC H3, including HEAD, validators, ranges,
cache ON/OFF, 1 MiB bodies and concurrent streams. Dedicated WebSocket checks
include bidirectional echo and a normal close handshake. Client protocol
self-checks are reported separately from RustProxy acceptance cases.
This establishes covered interoperability, not TLS/H2/H3 throughput or
production-certificate compatibility.
## Two-origin load-balancer regression
```bash
python3 scripts/performance/lb_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--out /absolute/path/results/lb
```
This separate, small suite sends real requests through RustProxy to two local
TCP origins. It verifies equal round-robin allocation, GET failover after a 503
and after a stopped origin refuses connections, and the absence of POST replay.
A controlled response pauses after its first body fragment while configuration
changes drain or disable its member. New requests must use the other member,
the admitted response must finish intact, and its active lease must remain
counted until completion and then return to zero. The main HTTP case matrix is
unchanged. This suite does not cover sticky sessions, active health probes,
DNS rotation or TLS upstream failures.
## Finite concurrency correctness stress
```bash
python3 scripts/performance/concurrency_regression.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --out /absolute/path/results/concurrency
```
This separate functional check uses 1, 16 and 128 concurrent client tasks for
proxy and static responses at 1,040 B, 64 KiB and 1 MiB. Each finite batch sends
`max(32, 2 * clients)` requests and verifies every status, length and body SHA-256.
After the clients finish, backend active-exchange counters must settle to zero.
RSS and file descriptors are recorded before the batch and after its drain.
These are not peak resource measurements or throughput results. Client tasks
start behind a barrier; configured concurrency does not establish that the same
number of server exchanges were active simultaneously.
## Before/after/nginx benchmark
First verify starts, loopback routing and exact payload hashes without invoking
the load generator:
```bash
python3 scripts/performance/benchmark.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
--out /absolute/path/results/preflight --prepare-only
```
For a host with CPUs 0–8 available, a reproducible primary run is:
```bash
python3 scripts/performance/benchmark.py \
--binary before=/absolute/path/rust-proxy-before \
--binary after=/absolute/path/rust-proxy-after \
--nginx /absolute/path/nginx --wrk /absolute/path/wrk \
--out /absolute/path/results/primary \
--workers 4 --origin-workers 2 --client-threads 4 \
--server-cpus 0-3 --origin-cpus 4-5 --client-cpus 6-8 \
--concurrency 64 --sizes small,large,huge --modes proxy,static \
--duration 15 --warmup 3 --repeats 5
```
Choose explicit, nonempty and disjoint CPU sets allowed in the run environment;
adapt all three sets on a smaller host. Frontends use the same CPU set and worker
count, the shared origin uses another set, and the generator uses a third. The
harness checks each process/thread's actual affinity and refuses unavailable
resource counters. Four wrk threads share the three client CPUs in this example.
The structured result records thread count and effective connection count; at
64 connections, four threads keep exactly 64 connections, avoiding wrk's integer
division of connections between threads. Only one frontend is loaded at a time.
Stop builds, other tests and profilers during throughput measurement.
The primary run takes approximately 27 minutes: six scenarios, three variants,
five samples, and 18 seconds per sample including excluded warm-up. To explore
concurrency, repeat in a fresh output directory with
`--concurrency 1,16,64,128`. More scenarios intentionally take more time.
Short sanity checks require the explicit `--exploratory` option and cannot
satisfy the >=5 samples / >=15-second measurement gate.
Cache is explicitly disabled for the primary benchmark. Transforms retain their
disabled defaults. Identity encoding is requested. Logging history, analytics,
request IDs, origin policy, body accounting, health/lease plumbing and normal
application functionality remain active. nginx uses `sendfile on`, HTTP/1.1,
keep-alive and streaming proxy delivery. The generated exact configurations are
saved, so this intentional difference in nginx/application features is visible.
No cache-disabled claim is inferred from a cache-on speedup.
Optional keep-alive tuning uses `--max-idle label=64`. Keep this as a separately
labelled experiment. The harness does not turn off bounded pool recycling.
## Recorded evidence and interpretation
- `schedule.json`: complete deterministic crossover order, rotating and
reversing variants between repetitions. No best-run selection.
- `preflight.json`, `fixture/payloads.json`, binary/version/config metadata:
exact status, body SHA-256, payload sizes and inputs.
- Per-sample exact status/body checks before and after the measured interval,
plus a wire-bytes-per-request sanity check, detect wrong content and gross
transfer-size anomalies. These do not hash every response inside wrk.
- `raw/*.txt` and `raw/*.json`: unmodified wrk output and structured requests,
duration, bytes, RPS, p50/p95/p99, effective concurrency, and
connect/read/write/timeout/status errors.
- `raw/*-resources.json`: timestamped `/proc` samples for each server process
and its children, plus the separate origin process group.
- `raw/*-affinity.json`: observed process/thread CPU masks for the front end,
origin and generator.
- `runs.csv/json`, `summary.csv/json`: all samples, medians, RPS range and
coefficient of variation, latency medians, server CPU time per completed
request, peak/end RSS and file descriptors.
- `status/*.json`: numeric drop and pool-generation/rotation counters read
outside each measured interval. Warm-up output is kept separately.
- `completion.json` and `SHA256SUMS`: completion state and hashes of evidence.
Any generator/network/status error stops the run with `failure.json`; partial
data remains available and is not presented as completed acceptance.
Use `(after_median / before_median - 1) * 100` for throughput improvement and
`after_median / nginx_median * 100` for relative nginx throughput. Rebuilding
both revisions with one compiler, target and resolved lockfile is required for
source-change attribution. Compare the supplied executable separately: build
flags and dependency versions can change its performance.
These are same-host, closed-loop HTTP/1.1 diagnostics. They do not establish
production capacity, TLS/H2/H3 performance, an open-loop latency SLA, allocator
rates, CPU cycles or 24-hour resource stability. wrk latency follows wrk's own
histogram/correction behavior. RSS and descriptor peaks are sampled observations,
so shorter spikes may be missed. Summed nginx RSS can double-count shared pages.
Origin or generator CPU saturation can limit observed throughput; inspect their
resource data and move the generator/origin to separate machines for release
measurements. Run profilers separately so profiling overhead does not enter the
headline A/B throughput numbers.
## Mixed-traffic resource soak
Run this in its own phase, after correctness and throughput checks:
```bash
python3 scripts/performance/soak.py \
--binary /absolute/path/rust-proxy-after --label after \
--nginx /absolute/path/nginx --out /absolute/path/results/soak \
--duration 1800 --rate 100 --clients 8 \
--workers 4 --origin-workers 2 --server-cpus 0-3 --origin-cpus 4-5 \
--toggle-seconds 120 --sample-seconds 5 --abort-every 101
```
The bounded client executor alternates proxy/static and 1,040 B/64 KiB/1 MiB
payloads, verifies full bytes and occasional HEAD/range semantics, closes some
clients after an initial body prefix, and applies live cache OFF/ON changes.
Every workload request opens a new frontend HTTP/1.1 connection; upstream
keepalive retains its configured defaults. This exercises connection churn.
The primary wrk benchmark separately uses persistent frontend connections.
The nominal request rate is paced; scheduler delays and actual completion rate
are reported instead of hiding an under-driven client. Pending jobs are bounded
to twice the client count. The script restores base configuration, drains and
rechecks both listeners before shutdown. After each cache apply, two separate
GETs per source verify exact contents and require a HIT on the second GET when
enabled; globally disabled cache must expose no X-Cache header. These probe
requests are recorded separately from paced traffic. Backend active-exchange
counters must settle to zero after the final drain and cooldown.
Five-second resource/status records, body/status failures, observed cache-status
counts, cache transitions/probes, RSS/FD trend and completed traffic totals are
retained. Controlled client closes mean that the client closes after reading a
prefix. The server may already have sent the complete response, so those counts
are not presented as proven server-side cancellations.
Memory trend is descriptive: switching caching and allocator retention affect
RSS. This is a 30-minute resource gate, not a 24-hour leak proof. A short
`--exploratory --duration ...` sanity run stays explicitly outside that gate.
## 0.34.0 hypothesis-only sendfile/nginx gap simulator
```bash
python3 scripts/performance/simulate_nginx_gap_034.py --output /tmp/nginx-gap.json
```
This is **not** a benchmark of RustProxy 0.34.0. It reproduces measured
0.33.8/nginx medians with cache OFF and applies explicitly hypothetical
Amdahl service-time reductions (WWW and reverse proxy separately). A real
RustProxy binary is required to validate the code path, detect regressions
and measure latency/RPS against nginx under identical conditions.
scripts/performance/ab_1kb_dev.py
-108
View File
@@ -1,108 +0,0 @@
#!/usr/bin/env python3
"""Short, same-host ab diagnostic for 1,040-byte HTTP/1 proxy responses.
Requires an existing executable and nginx; keeps the same explicitly supplied
idle-pool limit across variants. Not a substitute for wrk/CPU-isolated acceptance.
"""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import re
import subprocess
import sys
import time
from common import Fixture, http, write_json
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--binary', action='append', required=True, metavar='LABEL=/abs/path')
parser.add_argument('--nginx', default='/usr/sbin/nginx', type=Path)
parser.add_argument('--ab', default='/usr/bin/ab', type=Path)
parser.add_argument('--out', required=True, type=Path)
parser.add_argument('--concurrency', default='16,64')
parser.add_argument('--duration', type=int, default=4)
parser.add_argument('--repeats', type=int, default=2)
parser.add_argument('--workers', type=int, default=4)
parser.add_argument('--max-idle', type=int, default=64)
args = parser.parse_args()
binaries = dict(item.split('=', 1) for item in args.binary)
if len(binaries) != len(args.binary) or not all(Path(x).is_absolute() for x in binaries.values()):
parser.error('binary labels must be distinct and paths must be absolute')
if args.duration < 2 or args.repeats < 1:
parser.error('duration >=2 seconds and repeats >=1 required')
concurrency = [int(v) for v in args.concurrency.split(',')]
args.out.mkdir(parents=True, exist_ok=True)
records = []
write_json(args.out / 'metadata.json', {
'simulation': False, 'binary_labels': list(binaries), 'nginx': str(args.nginx),
'max_idle_for_every_rust_variant': args.max_idle, 'duration_s': args.duration,
'repeats': args.repeats, 'concurrency': concurrency, 'workers': args.workers,
'limitations': 'same-host shared CPU and closed-loop ab; not build-level before/after of uncompiled source'
})
with Fixture(args.out, workers=args.workers) as fixture:
nginx = fixture.nginx(args.nginx, frontend=True)
instances = {name: fixture.rust(Path(binary), name, max_idle=args.max_idle)
for name, binary in binaries.items()}
for name, instance in instances.items():
instance.login()
print('Waiting for quiescence:', name, flush=True)
instance.wait_quiescent(timeout=150, quiet_seconds=3)
sources = {**{name: r.reverse_port for name, r in instances.items()}, 'nginx': nginx['reverse_port']}
payload = (fixture.site / 'small.html').read_bytes()
for name, port in sources.items():
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
if code != 200 or body != payload:
raise AssertionError(f'{name} precheck failed: {code}, {len(body)} bytes')
write_json(args.out / 'payload-check.json', {
'status': 200, 'bytes': len(payload), 'sha256': hashlib.sha256(payload).hexdigest(),
'all_variants_verified': True
})
labels = list(sources)
for c in concurrency:
for repeat in range(args.repeats):
order = labels[repeat % len(labels):] + labels[:repeat % len(labels)]
if repeat % 2:
order.reverse()
for name in order:
port = sources[name]
cmd = [str(args.ab), '-k', '-t', str(args.duration), '-n', '1000000', '-c', str(c),
'-H', 'Accept-Encoding: identity', f'http://127.0.0.1:{port}/small.html']
prefix = f'c{c}-rep{repeat+1}-{name}'
subprocess.run([str(args.ab), '-k', '-n', '256', '-c', str(c),
f'http://127.0.0.1:{port}/small.html'],
capture_output=True, timeout=25, check=True)
p = subprocess.run(cmd, capture_output=True, text=True, timeout=args.duration + 45)
raw = p.stdout + '\n' + p.stderr
(args.out / f'{prefix}.txt').write_text(raw)
if p.returncode:
raise RuntimeError(f'{prefix} ab failed: {p.returncode} {p.stderr}')
def field(label):
m = re.search(r'^' + re.escape(label) + r':\s*(\d+(?:\.\d+)?)', raw, re.M)
return float(m.group(1)) if m else None
# ApacheBench tracks both network failures and bad body lengths.
if not re.search(r'^Failed requests:\s+0\s*$', raw, re.M):
raise RuntimeError(f'{prefix} failed requests: {raw[-1600:]}')
if not re.search(r'^Non-2xx responses:\s*0\s*$', raw, re.M) and 'Non-2xx responses:' in raw:
raise RuntimeError(f'{prefix} received non-2xx')
code, _, body = http(port, '/small.html', headers={'Accept-Encoding': 'identity'})
if code != 200 or body != payload:
raise AssertionError(f'{prefix} postcheck failed')
rps = field('Requests per second')
if rps is None:
raise RuntimeError(f'{prefix} missing RPS')
sample = {'variant': name, 'concurrency': c, 'repeat': repeat+1,
'rps': rps, 'completed': field('Complete requests'), 'failed': 0,
'expected_sha256': hashlib.sha256(payload).hexdigest(), 'status': 200}
records.append(sample)
write_json(args.out / 'samples.json', records)
print(sample, flush=True)
write_json(args.out / 'complete.json', {'complete': True, 'samples': len(records)})
return 0
if __name__ == '__main__':
sys.exit(main())
scripts/performance/benchmark.py
-479
View File
@@ -1,479 +0,0 @@
#!/usr/bin/env python3
"""Interleaved before/after/nginx process benchmarks with immutable raw evidence.
This is a same-host diagnostic experiment, not a simulation or release sign-off.
Runs >=5 measured samples per variant/scenario by default, each >=15 seconds.
"""
from __future__ import annotations
import argparse
import csv
import hashlib
import itertools
import json
import os
from pathlib import Path
import platform
import resource
import statistics
import subprocess
import sys
import time
import traceback
from common import Fixture, http, seal_results, sha256, write_json
from resources import (Monitor, affinity_snapshot, cgroup_cpu_delta, cgroup_cpu_stat,
parse_cpu_set, validate_cpu_sets, wait_for_affinity)
from telemetry import traffic_analytics_delta, traffic_analytics_snapshot
SIZES = {"small": "small.html", "large": "large.bin", "huge": "huge.bin"}
DROP_COUNTER_FIELDS = {
"logs_dropped": "history_dropped_delta",
"logging.sinks.queue_dropped": "logging_sinks_queue_dropped_delta",
"logging.error_file.queue_dropped": "logging_error_file_queue_dropped_delta",
"cache.cold_queue_dropped": "cache_cold_queue_dropped_delta",
}
DROP_INTERVAL_NOTE = ("Delta brackets measured wrk plus one postflight request and possible in-flight completions; "
"percentage uses wrk completed requests as an approximate denominator. "
"History rejection does not imply rejection by live broadcast or external sinks.")
def usage_snapshot():
usage = resource.getrusage(resource.RUSAGE_CHILDREN)
return usage.ru_utime + usage.ru_stime
def effective_threads(requested, concurrency):
if requested < 1 or concurrency < 1:
raise ValueError("client threads and concurrency must be positive")
# wrk floor-divides the connection count between threads. Select a divisor
# so its advertised and effective connection counts remain identical.
return next(value for value in range(min(requested, concurrency), 0, -1)
if concurrency % value == 0)
def wrk_run(args, port, path, duration, concurrency, prefix, roots=None, expected_processes=None):
threads = effective_threads(args.client_threads, concurrency)
command = [str(args.wrk), "--latency", "-t", str(threads),
"-c", str(concurrency), "-d", f"{duration}s", "--timeout", "5s",
"-H", "Accept-Encoding: identity", "-s", str(Path(__file__).with_name("wrk_report.lua")),
f"http://127.0.0.1:{port}/{path}"]
if args.client_cpus:
command = ["taskset", "-c", args.client_cpus, *command]
affinities = {}
expected_processes = expected_processes or {}
if roots:
for label, pid in roots.items():
expected = parse_cpu_set(args.server_cpus if label == "server" else args.origin_cpus)
affinities[label] = affinity_snapshot(pid, expected, expected_processes.get(label))
monitor = Monitor(roots, args.resource_interval, expected_processes) if roots else None
if monitor:
monitor.start()
cpu_start = usage_snapshot()
started = time.monotonic()
process = None
stdout, stderr = "", ""
try:
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
affinities["generator"] = wait_for_affinity(process, parse_cpu_set(args.client_cpus))
stdout, stderr = process.communicate(timeout=duration + 30)
finally:
if process is not None and process.poll() is None:
process.kill()
stdout, stderr = process.communicate(timeout=5)
if monitor:
monitor.stop()
prefix.parent.mkdir(parents=True, exist_ok=True)
prefix.with_suffix(".txt").write_text(stdout + "\n" + stderr)
write_json(prefix.with_name(prefix.name + "-affinity.json"), affinities)
if monitor:
write_json(prefix.with_name(prefix.name + "-resources.json"), monitor.samples)
if monitor.errors:
write_json(prefix.with_name(prefix.name + "-resource-errors.json"), monitor.errors)
cpu_seconds = usage_snapshot() - cpu_start
wall_seconds = time.monotonic() - started
raw = stdout + "\n" + stderr
if process.returncode:
raise RuntimeError(f"wrk exited {process.returncode}: {raw[-2000:]}")
line = next((line for line in stdout.splitlines() if line.startswith("RUSTPROXY_BENCH_JSON ")), None)
if line is None:
raise RuntimeError("wrk produced no structured report")
result = json.loads(line.removeprefix("RUSTPROXY_BENCH_JSON "))
if result["requests"] <= 0 or result["duration_us"] <= 0:
write_json(prefix.with_suffix(".json"), result)
raise RuntimeError("wrk completed no requests or reported an invalid duration; raw evidence retained")
result.update({"command": command, "wall_seconds": wall_seconds,
"client_threads": threads, "requested_connections": concurrency,
"client_threads_requested": args.client_threads,
"effective_connections": threads * (concurrency // threads),
"generator_cpu_seconds": cpu_seconds,
"rps": result["requests"] * 1_000_000 / result["duration_us"],
"error_total": sum(value for key, value in result.items() if key.startswith("errors_"))})
if monitor:
result["resources"] = monitor.summary(result["requests"])
write_json(prefix.with_suffix(".json"), result)
return result
def payload_check(port, path, expected):
"""A full exact-body check outside the wrk measurement interval."""
try:
status, headers, body = http(port, "/" + path, headers={"Accept-Encoding": "identity"})
return {"status": status, "bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
"expected_bytes": len(expected), "expected_sha256": hashlib.sha256(expected).hexdigest(),
"content_length": headers.get("content-length"), "content_encoding": headers.get("content-encoding"),
"approximate_header_bytes": 19 + sum(len(key) + len(value) + 4 for key, value in headers.items()),
"passed": status == 200 and body == expected}
except Exception as error:
return {"passed": False, "error": str(error), "expected_bytes": len(expected),
"expected_sha256": hashlib.sha256(expected).hexdigest()}
def transfer_sanity(result, payload_bytes, header_bytes, concurrency):
"""Check gross truncation/size anomalies; this is not per-response hashing."""
requests, wire_bytes = result["requests"], result["bytes"]
minimum = requests * payload_bytes
# wrk counts headers and partially received final responses in its wire bytes.
header_allowance = max(1024, 2 * header_bytes)
maximum = (requests + concurrency) * (payload_bytes + header_allowance)
return {"passed": requests > 0 and minimum <= wire_bytes <= maximum,
"wire_bytes_per_request": wire_bytes / requests if requests else None,
"expected_payload_bytes_per_request": payload_bytes,
"minimum_wire_bytes": minimum, "maximum_wire_bytes": maximum,
"header_allowance_per_response": header_allowance,
"inflight_response_allowance": concurrency}
def counter_values(value, prefix=""):
"""Retain numeric drop and rotation counters without session/config data."""
found = {}
if isinstance(value, dict):
for key, child in value.items():
path = f"{prefix}.{key}" if prefix else key
if isinstance(child, (int, float)) and any(part in key for part in ["dropped", "rotations", "generation", "requests_in_generation"]):
found[path] = child
found.update(counter_values(child, path))
elif isinstance(value, list):
for index, child in enumerate(value):
found.update(counter_values(child, f"{prefix}[{index}]"))
return found
def validate_drop_counters(values):
for name in DROP_COUNTER_FIELDS:
if type(values.get(name)) is not int or values[name] < 0:
raise RuntimeError(f"missing or invalid cumulative status counter: {name}")
return values
def status_counter_snapshot(rust):
if rust is None:
return None
status, _, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"{rust.label} status capture failed: HTTP {status}")
return counter_values(json.loads(body))
def drop_counter_report(before, after, requests):
if requests <= 0:
raise ValueError("drop-rate denominator must contain completed requests")
if (before is None) != (after is None):
raise RuntimeError("both status snapshots are required for a drop delta")
if before is not None:
validate_drop_counters(before)
validate_drop_counters(after)
deltas = {name: after[name] - before[name] if before is not None else None
for name in DROP_COUNTER_FIELDS}
if any(value is not None and value < 0 for value in deltas.values()):
raise RuntimeError("cumulative drop counter decreased during measured interval")
fields = {field: deltas[name] for name, field in DROP_COUNTER_FIELDS.items()}
fields["history_drop_percent_of_completed_requests"] = (
100 * deltas["logs_dropped"] / requests if before is not None else None)
return {"applicable": before is not None, "counter_deltas": deltas, "row_fields": fields,
"completed_requests_denominator": requests, "denominator_is_approximate": True,
"interval_note": DROP_INTERVAL_NOTE}
def analytics_counter_report(before, after, requests):
if requests <= 0 or (before is None) != (after is None):
raise ValueError("analytics delta requires paired snapshots and completed requests")
delta = traffic_analytics_delta(before, after) if before is not None else {
"queue_dropped_delta": None, "recorded_requests_delta": None, "interval_seconds": None}
fields = {"analytics_queue_dropped_delta": delta["queue_dropped_delta"],
"analytics_recorded_requests_delta": delta["recorded_requests_delta"],
"analytics_queue_drop_percent_of_completed_requests": (
100 * delta["queue_dropped_delta"] / requests if before is not None else None)}
return {"applicable": before is not None, **delta, "row_fields": fields,
"completed_requests_denominator": requests, "denominator_is_approximate": True,
"interval_note": DROP_INTERVAL_NOTE,
"recorded_requests_note": "Asynchronous analytics records; not HTTP success or completion counts."}
def persist_rows(out, rows):
write_json(out / "runs.json", rows)
if not rows:
return
with (out / "runs.csv").open("w", newline="") as stream:
writer = csv.DictWriter(stream, fieldnames=list(rows[0]))
writer.writeheader()
writer.writerows(rows)
groups = {}
for row in rows:
groups.setdefault((row["mode"], row["size"], row["concurrency"], row["variant"]), []).append(row)
summary = []
for (mode, size, concurrency, variant), group in sorted(groups.items()):
rps = [row["rps"] for row in group]
summary.append({"mode": mode, "size": size, "concurrency": concurrency, "variant": variant,
"samples": len(group), "rps_median": statistics.median(rps),
"rps_min": min(rps), "rps_max": max(rps),
"rps_cv_percent": statistics.stdev(rps) / statistics.mean(rps) * 100 if len(rps) > 1 else None,
"p50_us_median": statistics.median(row["p50_us"] for row in group),
"p95_us_median": statistics.median(row["p95_us"] for row in group),
"p99_us_median": statistics.median(row["p99_us"] for row in group),
"server_cpu_us_per_request_median": statistics.median(row["server_cpu_us_per_request"] for row in group),
"server_rss_peak_bytes": max(row["server_rss_peak_bytes"] for row in group),
"server_fd_peak": max(row["server_fd_peak"] for row in group),
"requests": sum(row["requests"] for row in group),
"cgroup_throttled_samples": sum(row["cgroup_nr_throttled_delta"] > 0 for row in group),
"cgroup_nr_throttled_delta": sum(row["cgroup_nr_throttled_delta"] for row in group),
"cgroup_throttled_usec_delta": sum(row["cgroup_throttled_usec_delta"] for row in group),
"validation_failures": sum(not (row["payload_before_passed"] and row["payload_after_passed"]
and row["transfer_sanity_passed"]) for row in group),
"errors": sum(row["error_total"] for row in group)})
write_json(out / "summary.json", summary)
with (out / "summary.csv").open("w", newline="") as stream:
writer = csv.DictWriter(stream, fieldnames=list(summary[0]))
writer.writeheader()
writer.writerows(summary)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--max-idle", action="append", default=[], metavar="LABEL=COUNT")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--wrk", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=4)
parser.add_argument("--client-threads", type=int, default=4)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--client-cpus")
parser.add_argument("--duration", type=int, default=15)
parser.add_argument("--warmup", type=int, default=3)
parser.add_argument("--repeats", type=int, default=5)
parser.add_argument("--concurrency", default="64")
parser.add_argument("--sizes", default="small,large")
parser.add_argument("--modes", default="proxy,static")
parser.add_argument("--resource-interval", type=float, default=.5)
parser.add_argument("--exploratory", action="store_true", help="explicitly allow <5 repetitions or <15s samples")
parser.add_argument("--prepare-only", action="store_true", help="start processes and verify hashes; run no load generator")
args = parser.parse_args()
if (args.duration < 15 or args.repeats < 5) and not args.exploratory:
parser.error("acceptance requires >=15-second runs and >=5 repetitions; use --exploratory for labelled short diagnostics")
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be a new or empty directory so raw evidence is never overwritten")
modes, sizes = args.modes.split(","), args.sizes.split(",")
concurrencies = [int(value) for value in args.concurrency.split(",")]
if any(mode not in ("proxy", "static") for mode in modes) or any(size not in SIZES for size in sizes):
parser.error("supported modes: proxy,static; sizes: small,large,huge")
if min(concurrencies) < 1 or max(concurrencies) > 4096:
parser.error("concurrency must be 1..4096")
if args.client_threads < 1:
parser.error("--client-threads must be positive")
if args.workers < 1 or args.origin_workers < 1:
parser.error("--workers and --origin-workers must be positive")
try:
cpu_sets = validate_cpu_sets({"server": args.server_cpus, "origin": args.origin_cpus,
"client": args.client_cpus}, set(os.sched_getaffinity(0)))
except ValueError as error:
parser.error(str(error))
if args.resource_interval <= 0:
parser.error("--resource-interval must be positive")
args.nginx, args.wrk, args.out = args.nginx.resolve(), args.wrk.resolve(), args.out.resolve()
variants = dict(item.split("=", 1) for item in args.binary)
if len(variants) != len(args.binary) or "nginx" in variants:
parser.error("variant names must be unique and must not be nginx")
max_idle = dict(item.split("=", 1) for item in args.max_idle)
args.out.mkdir(parents=True, exist_ok=True)
environment = {"kind": "same-host real HTTP/1.1 process benchmark", "simulation": False,
"exploratory": args.exploratory, "prepare_only": args.prepare_only,
"acceptance_sampling": args.duration >= 15 and args.repeats >= 5 and not args.prepare_only,
"platform": platform.platform(), "python": sys.version,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"cpu_affinity_available": sorted(os.sched_getaffinity(0)),
"cpu_sets": {label: sorted(cpus) for label, cpus in cpu_sets.items()},
"resource_process_discovery": "PPid map from /proc/*/stat; root and configured nginx process counts must be readable",
"expected_nginx_processes": {"frontend": args.workers + 1, "origin": args.origin_workers + 1},
"generator_plan": [{"connections": concurrency, "requested_threads": args.client_threads,
"effective_threads": effective_threads(args.client_threads, concurrency)}
for concurrency in concurrencies],
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
"wrk_sha256": sha256(args.wrk), "nginx_sha256": sha256(args.nginx),
"limitations": ["Generator, upstream and frontend share one host and kernel; not production capacity",
"Closed-loop wrk latency includes its histogram correction; no open-loop arrival-rate SLA claim",
"RSS sums processes and may double-count shared pages in nginx",
"RSS and FD peaks are sampled, not absolute instantaneous peaks",
"Exact HTTP 200 and payload hashes are checked before/after each measured run, not for every response during load",
"wrk status errors count only status >399; aggregate bytes/request is a gross sanity check, not response integrity proof",
"cgroup CPU-throttling deltas cover all processes in the shared cgroup, not only the measured server",
DROP_INTERVAL_NOTE,
"No TLS/H2/H3 speed inference; no CPU cycles/allocation profiler claim",
"nginx has a smaller feature set; request logging/ACL/analytics work is not feature-matched"]}
for path in ["/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max", "/proc/version"]:
try:
environment[path] = Path(path).read_text().strip()
except OSError:
environment[path] = None
write_json(args.out / "environment.json", environment)
expected_samples = len(modes) * len(sizes) * len(concurrencies) * args.repeats * (len(variants) + 1)
write_json(args.out / "completion.json", {"completed": False, "samples": 0,
"expected_samples": expected_samples,
"simulation": False, "prepare_only": args.prepare_only})
rows, error = [], None
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
nginx = fixture.nginx(args.nginx)
servers = {"nginx": {"process": nginx["process"], "proxy": nginx["reverse_port"],
"static": nginx["static_port"], "rust": None}}
for label, binary in variants.items():
rust = fixture.rust(Path(binary), label, max_idle=int(max_idle[label]) if label in max_idle else None)
rust.login()
servers[label] = {"process": rust.process, "proxy": rust.reverse_port, "static": rust.static_port, "rust": rust}
quiescence = {label: server["rust"].wait_quiescent()
for label, server in servers.items() if server["rust"] is not None}
write_json(args.out / "quiescence-preflight.json", quiescence)
initial_affinity = {"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], args.origin_workers + 1)}
initial_affinity.update({label: affinity_snapshot(server["process"].pid, cpu_sets["server"],
args.workers + 1 if label == "nginx" else 1)
for label, server in servers.items()})
write_json(args.out / "affinity-preflight.json", initial_affinity)
# The fixture has only master + configured workers. Refuse a
# master-only snapshot even if HTTP payload checks would succeed.
prepare_monitor = Monitor({"server": nginx["process"].pid, "origin": nginx["origin"].pid},
args.resource_interval,
{"server": args.workers + 1, "origin": args.origin_workers + 1})
prepare_monitor.sample()
write_json(args.out / "resources-preflight.json", prepare_monitor.samples)
telemetry_preflight = {label: {"status": status_counter_snapshot(server["rust"]),
"traffic": traffic_analytics_snapshot(server["rust"]) if server["rust"] else None}
for label, server in servers.items()}
write_json(args.out / "telemetry-preflight.json", telemetry_preflight)
for snapshot in telemetry_preflight.values():
if snapshot["status"] is not None:
validate_drop_counters(snapshot["status"])
preflight = []
for label, server in servers.items():
for mode, size in itertools.product(modes, sizes):
payload = (fixture.site / SIZES[size]).read_bytes()
check = payload_check(server[mode], SIZES[size], payload)
preflight.append({"variant": label, "mode": mode, "size": size, **check})
write_json(args.out / "preflight.json", preflight)
if not check["passed"]:
raise AssertionError(f"payload preflight failed: {label} {mode} {size}: {check}")
write_json(args.out / "preflight.json", preflight)
labels = [*variants, "nginx"]
schedule = []
for mode, size, concurrency in itertools.product(modes, sizes, concurrencies):
for repetition in range(args.repeats):
offset = repetition % len(labels)
order = labels[offset:] + labels[:offset]
if repetition % 2:
order.reverse()
for variant in order:
schedule.append({"sequence": len(schedule) + 1, "mode": mode, "size": size,
"concurrency": concurrency, "repetition": repetition + 1, "variant": variant})
write_json(args.out / "schedule.json", schedule)
if not args.prepare_only:
for item in schedule:
server = servers[item["variant"]]
expected_processes = {"origin": args.origin_workers + 1,
"server": args.workers + 1 if item["variant"] == "nginx" else 1}
stem = f"{item['sequence']:04}-{item['mode']}-{item['size']}-c{item['concurrency']}-{item['variant']}"
if args.warmup:
warmup = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.warmup,
item["concurrency"], args.out / "warmup" / stem)
if warmup["error_total"]:
raise RuntimeError(f"warmup errors in {stem}; raw evidence retained")
payload = (fixture.site / SIZES[item["size"]]).read_bytes()
before = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
write_json(args.out / "checks" / (stem + "-before.json"), before)
if not before["passed"]:
raise RuntimeError(f"exact payload check failed before {stem}: {before}")
counters_before = status_counter_snapshot(server["rust"])
write_json(args.out / "status" / (stem + "-before.json"), counters_before)
if counters_before is not None:
validate_drop_counters(counters_before)
analytics_before = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
write_json(args.out / "status" / (stem + "-traffic-before.json"), analytics_before)
cpu_before = cgroup_cpu_stat()
try:
result = wrk_run(args, server[item["mode"]], SIZES[item["size"]], args.duration,
item["concurrency"], args.out / "raw" / stem,
{"server": server["process"].pid, "origin": nginx["origin"].pid},
expected_processes)
finally:
cpu_after = cgroup_cpu_stat()
cpu_delta = cgroup_cpu_delta(cpu_before, cpu_after)
throttling = {"before": cpu_before, "after": cpu_after, "delta": cpu_delta}
write_json(args.out / "checks" / (stem + "-cgroup-cpu.json"), throttling)
after = payload_check(server[item["mode"]], SIZES[item["size"]], payload)
write_json(args.out / "checks" / (stem + "-after.json"), after)
write_json(args.out / "checks" / (stem + "-affinity-after.json"), {
"server": affinity_snapshot(server["process"].pid, cpu_sets["server"], expected_processes.get("server")),
"origin": affinity_snapshot(nginx["origin"].pid, cpu_sets["origin"], expected_processes["origin"])})
counters_after = status_counter_snapshot(server["rust"])
write_json(args.out / "status" / (stem + ".json"), counters_after)
analytics_after = traffic_analytics_snapshot(server["rust"]) if server["rust"] else None
write_json(args.out / "status" / (stem + "-traffic-after.json"), analytics_after)
drops = drop_counter_report(counters_before, counters_after, result["requests"])
write_json(args.out / "status" / (stem + "-delta.json"), drops)
analytics = analytics_counter_report(analytics_before, analytics_after, result["requests"])
write_json(args.out / "status" / (stem + "-traffic-delta.json"), analytics)
sanity = transfer_sanity(result, len(payload), before["approximate_header_bytes"], item["concurrency"])
result.update({"payload_before": before, "payload_after": after, "transfer_sanity": sanity,
"cgroup_cpu": throttling, "observability": drops, "traffic_analytics": analytics})
write_json((args.out / "raw" / stem).with_suffix(".json"), result)
row = {**item, **{key: result[key] for key in ["requests", "bytes", "duration_us", "rps", "p50_us", "p95_us", "p99_us",
"client_threads_requested", "client_threads", "requested_connections", "effective_connections",
"error_total", "errors_connect", "errors_read", "errors_write", "errors_timeout", "errors_status", "generator_cpu_seconds"]}}
row.update({"payload_before_passed": before["passed"], "payload_after_passed": after["passed"],
"transfer_sanity_passed": sanity["passed"], "wire_bytes_per_request": sanity["wire_bytes_per_request"],
"cgroup_nr_throttled_delta": cpu_delta["nr_throttled"],
"cgroup_throttled_usec_delta": cpu_delta["throttled_usec"]})
row.update(drops["row_fields"])
row.update(analytics["row_fields"])
for group in ["server", "origin"]:
row.update({f"{group}_{key}": value for key, value in result["resources"][group].items()})
rows.append(row)
persist_rows(args.out, rows)
print(json.dumps({"sequence": item["sequence"], "total": len(schedule), "variant": item["variant"],
"scenario": f"{item['mode']}_{item['size']}_c{item['concurrency']}",
"rps": round(result["rps"], 2), "p95_ms": result["p95_us"] / 1000,
"errors": result["error_total"]}), flush=True)
if result["error_total"]:
raise RuntimeError(f"benchmark errors in {stem}; raw evidence retained")
if not after["passed"] or not sanity["passed"]:
raise RuntimeError(f"payload/byte-count validation failed in {stem}; raw evidence retained")
except (Exception, KeyboardInterrupt) as exc:
error = {"error": str(exc) or type(exc).__name__, "type": type(exc).__name__,
"interrupted": isinstance(exc, KeyboardInterrupt), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", error)
print(json.dumps(error), flush=True)
finally:
persist_rows(args.out, rows)
completed = error is None and (args.prepare_only or len(rows) == expected_samples)
write_json(args.out / "completion.json", {"completed": completed, "samples": len(rows),
"expected_samples": expected_samples,
"simulation": False, "prepare_only": args.prepare_only,
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
seal_results(args.out)
return (130 if error.get("interrupted") else 1) if error else (0 if completed else 1)
if __name__ == "__main__":
sys.exit(main())
scripts/performance/checks_cache.py
-180
View File
@@ -1,180 +0,0 @@
"""Cache and response-processing paths must survive disabled-path optimization."""
from concurrent.futures import ThreadPoolExecutor
import copy
import gzip
import io
import json
import time
import zlib
from common import http
def run(check, rust, peer):
port = rust.reverse_port
before = peer.count("GET", "/fresh")
results = [http(port, "/fresh") for _ in range(2)]
check("cache.global_off", all(s == 200 and h.get("x-cache") != "HIT" for s, h, b in results)
and peer.count("GET", "/fresh") - before == 2)
config = copy.deepcopy(rust.base)
config["cache"] = {"enabled": True, "expose_status": True, "expose_hits": True,
"expose_cache_status": True, "http_invalidation_allowed_cidrs": ["127.0.0.1/32"]}
config["web_services"][0]["cache_control"] = "public, max-age=60"
rust.apply(config, "cache-enabled")
first = http(port, "/fresh")
second = http(port, "/fresh")
check("cache.miss_then_hit", first[0] == 200 and first[1].get("x-cache") == "MISS"
and second[1].get("x-cache") == "HIT" and first[2] == second[2],
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
results = {}
for language in ["en", "pl", "en", "pl"]:
response = http(port, "/vary", headers={"Accept-Language": language})
previous = results.get(language)
if previous:
check("cache.vary_hit." + language, response[1].get("x-cache") == "HIT" and response[2] == previous[2],
cache=response[1].get("x-cache"))
results[language] = response
check("cache.vary_isolation", results["en"][2] != results["pl"][2])
for path in ["/private", "/no-store", "/cookie"]:
before = peer.count("GET", path)
first, second = http(port, path), http(port, path)
check("cache.nonpublic_bypass." + path, first[0] == second[0] == 200
and first[1].get("x-cache") != "HIT" and second[1].get("x-cache") != "HIT"
and peer.count("GET", path) - before == 2,
first=first[1].get("x-cache"), second=second[1].get("x-cache"))
for name, headers in [("authorization", {"Authorization": "Bearer fixture-value"}),
("cookie", {"Cookie": "application_session=fixture"})]:
before = peer.count("GET", "/fresh")
s, h, b = http(port, "/fresh", headers=headers)
check("cache.request_bypass." + name, s == 200 and h.get("x-cache") != "HIT"
and peer.count("GET", "/fresh") - before == 1, cache=h.get("x-cache"))
before = peer.count("GET", "/fresh")
s, h, b = http(port, "/fresh", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
check("cache.nonempty_get_bypass_and_preserve", s == 200 and h.get("x-cache") != "HIT"
and h.get("x-received-body-bytes") == "17" and peer.count("GET", "/fresh") - before == 1,
cache=h.get("x-cache"), origin_body_bytes=h.get("x-received-body-bytes"))
s, h, b = http(port, "/fresh", "PURGE")
after = http(port, "/fresh")
check("cache.http_purge", s in (200, 202, 204) and after[1].get("x-cache") == "MISS", status=s)
with ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(lambda _: http(port, "/concurrent"), range(8)))
check("cache.concurrent_fill", all(item[0] == 200 for item in results)
and len({item[2] for item in results}) == 1, origin_requests=peer.count("GET", "/concurrent"))
first = http(port, "/revalidate")
time.sleep(1.2)
second = http(port, "/revalidate")
check("cache.expired_validator_revalidation", first[0] == second[0] == 200 and first[2] == second[2]
and peer.count("GET", "/revalidate") >= 2, cache=second[1].get("x-cache"))
first, second = http(rust.static_port), http(rust.static_port)
check("cache.static_hit", first[0] == second[0] == 200 and second[1].get("x-cache") == "HIT"
and first[2] == second[2], cache=second[1].get("x-cache"))
empty_first, empty_second = http(rust.static_port, "/empty.txt"), http(rust.static_port, "/empty.txt")
check("cache.static_empty_hit", empty_first[0] == empty_second[0] == 200
and empty_second[1].get("x-cache") == "HIT" and empty_second[2] == b"",
first=empty_first[1].get("x-cache"), second=empty_second[1].get("x-cache"))
hs, hh, hb = http(rust.static_port, "/download.txt", "HEAD")
gs, gh, gb = http(rust.static_port, "/download.txt")
check("cache.static_head_cannot_fill_empty_get", hs == gs == 200 and hb == b""
and gb == (rust.fixture.site / "download.txt").read_bytes(), head_status=hs, get_status=gs)
s, h, b = http(rust.static_port, headers={"Range": "bytes=0-9"})
check("cache.static_range_survives", s == 206 and b == first[2][:10], status=s)
s, h, b = http(rust.static_port, headers={"If-None-Match": first[1]["etag"]})
check("cache.static_conditional_survives", s == 304 and b == b"", status=s)
config["vhosts"][0]["cache"] = {"enabled": False}
rust.apply(config, "cache-vhost-disabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.vhost_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
and first[2] != second[2])
config["vhosts"][0]["cache"] = {"enabled": True}
config["vhosts"][0]["routes"] = [{"id": "fresh-no-cache", "path_prefix": "/fresh", "cache": False}]
rust.apply(config, "cache-route-disabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.route_off", all(response[1].get("x-cache") != "HIT" for response in [first, second])
and first[2] != second[2])
config["cache"]["enabled"] = False
config["vhosts"][0]["routes"] = []
rust.apply(config, "cache-global-disabled-vhost-enabled")
first, second = http(port, "/fresh"), http(port, "/fresh")
check("cache.vhost_cannot_override_global_off", first[2] != second[2]
and second[1].get("x-cache") != "HIT")
def transforms(check, rust, peer):
config = copy.deepcopy(rust.base)
original = (rust.fixture.site / "app.json").read_bytes()
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/app.json")
check("transform.off." + source, s == 200 and b == original, status=s)
config["transform"] = {"enabled": True, "json": True, "html": True}
for vhost in config["vhosts"]:
vhost["transform"] = {"enabled": True, "json": True, "html": True}
rust.apply(config, "transform-enabled-cache-off")
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/app.json")
check("transform.on." + source, s == 200 and json.loads(b) == json.loads(original)
and len(b) < len(original), original_bytes=len(original), result_bytes=len(b), status=s)
s, h, b = http(port, "/large.bin")
check("transform.binary_unchanged." + source, s == 200 and b == (rust.fixture.site / "large.bin").read_bytes())
s, h, b = http(port, "/app.json", headers={"Range": "bytes=0-9"})
check("transform.range_exclusion." + source, s == 206 and b == original[:10], status=s)
config["cache"] = {"enabled": True, "expose_status": True}
rust.apply(config, "transform-enabled-cache-on")
first, second = http(rust.reverse_port, "/app.json"), http(rust.reverse_port, "/app.json")
check("transform.cache_combination", second[0] == 200 and second[1].get("x-cache") == "HIT"
and first[2] == second[2] and json.loads(second[2]) == json.loads(original),
cache=second[1].get("x-cache"))
rust.reset()
expected = (rust.fixture.site / "compress.txt").read_bytes()
for port, source in [(rust.reverse_port, "proxy"), (rust.static_port, "static")]:
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.gzip." + source, s == 200 and h.get("content-encoding") == "gzip"
and gzip.decompress(b) == expected and len(b) < len(expected), bytes=len(b))
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip;q=0, identity"})
check("compression.identity_qvalue." + source, s == 200 and "content-encoding" not in h and b == expected)
s, h, b = http(port, "/compress.txt", headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
check("compression.range_exclusion." + source, s == 206 and "content-encoding" not in h and b == expected[:10])
s, h, b = http(rust.reverse_port, "/no-transform", headers={"Accept-Encoding": "gzip"})
check("compression.no_transform_exclusion", s == 200 and "content-encoding" not in h)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["compression"] = {"enabled": False}
rust.apply(config, "compression-vhost-disabled")
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.vhost_off", s == 200 and "content-encoding" not in h and b == expected)
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
config["vhosts"][0]["compression"] = {"enabled": True, "gzip": True}
rust.apply(config, "compression-global-disabled")
s, h, b = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": "gzip"})
check("compression.global_codec_ceiling", s == 200 and "content-encoding" not in h and b == expected)
config = copy.deepcopy(rust.base)
config["compression"] = {"gzip": True, "br": True, "zstd": True, "deflate": True}
config["cache"] = {"enabled": True, "expose_status": True}
rust.apply(config, "compression-cache-enabled")
for codec in ["gzip", "deflate", "br", "zstd"]:
decoder = None
if codec == "gzip":
decoder = gzip.decompress
elif codec == "deflate":
decoder = zlib.decompress
elif codec == "br":
try:
import brotli
decoder = brotli.decompress
except ImportError:
pass
else:
try:
import zstandard
decoder = lambda data: zstandard.ZstdDecompressor().stream_reader(io.BytesIO(data)).read()
except ImportError:
pass
if decoder is None:
check("compression.codec." + codec, None, skip_reason="optional Python decoder is not installed")
continue
first = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
second = http(rust.reverse_port, "/compress.txt", headers={"Accept-Encoding": codec})
check("compression.codec." + codec, second[0] == 200 and second[1].get("content-encoding") == codec
and decoder(first[2]) == decoder(second[2]) == expected, cache=second[1].get("x-cache"))
scripts/performance/checks_policy.py
-184
View File
@@ -1,184 +0,0 @@
"""Live ACL, authentication, atomic reload and logging regressions."""
import base64
import copy
import json
import re
import time
from common import http, write_json
DENY = {"default": "deny", "rules": []}
ALLOW_LOCAL = {"default": "deny", "rules": [{"network": "127.0.0.1/32", "action": "allow"}]}
def run(check, rust, peer):
port = rust.reverse_port
config = copy.deepcopy(rust.base)
config["host_policies"] = [{"hosts": ["fixture.test"], "acl": DENY}]
rust.apply(config, "policy-global-host")
count = peer.count("GET", "/echo")
s, h, b = http(port, "/echo", headers={"Host": "fixture.test"})
check("acl.global_host_without_vhost_acl", s == 403 and peer.count("GET", "/echo") == count,
status=s, upstream_requests=peer.count("GET", "/echo") - count)
s, h, b = http(port, "/echo", headers={"Host": "localhost"})
check("acl.global_other_host_unchanged", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["acl"] = DENY
rust.apply(config, "policy-vhost-deny")
s, h, b = http(port)
check("acl.vhost_deny", s == 403, status=s)
s, h, b = http(rust.static_port)
check("acl.vhost_isolation", s == 200, status=s)
config["vhosts"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-vhost-allow")
s, h, b = http(port)
check("acl.hot_enable_allow", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["acl_profiles"] = [{"id": "restricted", "name": "Restricted", "acl": DENY}]
config["vhosts"][0]["acl_profile_id"] = "restricted"
config["vhosts"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-profile-deny")
s, h, b = http(port)
check("acl.profile_and_vhost_stack", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["routes"] = [{"id": "denied", "path_prefix": "/echo", "acl": DENY}]
rust.apply(config, "policy-location-deny")
count = peer.count("GET", "/echo")
s, h, b = http(port, "/echo")
check("acl.location_before_upstream", s == 403 and peer.count("GET", "/echo") == count, status=s)
s, h, b = http(port)
check("acl.location_other_path_unchanged", s == 200, status=s)
config["vhosts"][0]["acl"] = DENY
config["vhosts"][0]["routes"][0]["acl"] = ALLOW_LOCAL
rust.apply(config, "policy-location-allow-vhost-deny")
s, h, b = http(port, "/echo")
check("acl.location_cannot_override_vhost_deny", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["acl"] = {"default": "deny", "rules": [{"network": "203.0.113.22/32", "action": "allow"}]}
rust.apply(config, "policy-untrusted-forwarded")
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
check("acl.spoofed_forwarded_ip_denied", s == 403, status=s)
config["forwarding"] = {"trusted_proxies": ["127.0.0.1/32"]}
rust.apply(config, "policy-trusted-forwarded")
s, h, b = http(port, headers={"X-Forwarded-For": "203.0.113.22"})
check("acl.trusted_forwarded_ip_allowed", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["rate_limit"] = {"country_access": {"mode": "allow", "countries": ["PL"]}}
config["vhosts"][0]["cloudflare_client_ip"] = True
rust.apply(config, "policy-country")
s, h, b = http(port, headers={"CF-IPCountry": "PL", "CF-Connecting-IP": "203.0.113.22"})
check("acl.untrusted_country_header_cannot_allow", s == 403, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["method_policy"] = {"mode": "allow", "methods": ["GET", "HEAD"]}
rust.apply(config, "policy-method")
s, h, b = http(port, "/echo", "POST", body=b"denied method")
check("policy.method_allowlist", s in (403, 405), status=s)
s, h, b = http(port)
check("policy.allowed_method_survives", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["rate_limit"] = {"enabled": True, "requests": 2, "window_seconds": 60, "burst": 0}
rust.apply(config, "policy-rate-limit")
statuses = [http(port)[0] for _ in range(3)]
check("policy.rate_limit", statuses[:2] == [200, 200] and statuses[2] == 429, statuses=statuses)
rust.reset()
s, h, b = http(port)
check("policy.hot_disable_restores_request", s == 200, status=s)
# Every route check uses the real admin socket and no bearer/backdoor token.
for path in ["/api/config", "/api/status", "/api/logs", "/api/events", "/api/openapi.json", "/assets/app.js"]:
s, h, b = http(rust.admin_port, path)
check("admin.private." + path, s in (401, 403), status=s)
check("admin.login", rust.login() == 200)
s, h, b = rust.admin("/api/config")
envelope = json.loads(b)
check("admin.authenticated_config", s == 200 and "config" in envelope and "revision" in envelope, status=s)
s, h, b = rust.admin("/api/config", "PUT", {"expected_revision": envelope["revision"], "config": rust.base}, csrf=False)
check("admin.csrf_required", s == 403, status=s)
s, h, b = rust.admin("/api/config", origin="http://foreign.example")
check("admin.foreign_origin_denied", s == 403, status=s)
# Generate the disposable Basic Auth hash through the existing secured API.
s, h, b = rust.admin("/api/basic-auth/hash", "POST", {"username": "fixtureuser", "password": rust.password})
hashed = json.loads(b)
check("basic.hash_api", s == 200 and "password_hash" in hashed, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["basic_auth"] = {"enabled": True, "require_tls": False,
"users": [{"username": "fixtureuser", "password_hash": hashed["password_hash"]}]}
rust.apply(config, "policy-basic-auth")
s, h, b = http(port, "/echo")
check("basic.challenge", s == 401 and "Basic" in h.get("www-authenticate", ""), status=s)
credentials = base64.b64encode(("fixtureuser:" + rust.password).encode()).decode()
s, h, b = http(port, "/echo", headers={"Authorization": "Basic " + credentials})
check("basic.correct_password_and_strip", s == 200 and "authorization" not in json.loads(b)["headers"], status=s)
s, h, b = http(port, "/echo", headers={"Authorization": "Basic Zml4dHVyZXVzZXI6d3Jvbmc="})
check("basic.wrong_password_denied", s == 401, status=s)
config["vhosts"][0]["acl"] = DENY
rust.apply(config, "policy-acl-before-basic")
s, h, b = http(port, "/echo")
check("basic.acl_precedes_auth", s == 403, status=s)
rust.reset()
config = copy.deepcopy(rust.base)
config["vhosts"][0]["frontend"]["listener_ids"] = ["does-not-exist"]
rust.apply(config, "invalid-config-retains-snapshot", expected_success=False)
s, h, b = http(port)
check("reload.invalid_document_retains_previous_snapshot", s == 200, status=s)
s, h, b = http(rust.static_port)
check("reload.invalid_document_keeps_other_vhost", s == 200, status=s)
config = copy.deepcopy(rust.base)
config["vhosts"][0]["backends"][0]["allow_private"] = False
# Explicit private-address rejection is checked through apply, preserving the
# active configuration if compile-time validation rejects the backend.
rust.apply(config, "ssrf-private-origin-disallowed", expected_success=False)
s, h, b = http(port)
check("ssrf.rejected_origin_does_not_replace_snapshot", s == 200, status=s)
query = "/echo?observability=" + rust.label
s, h, b = http(port, query)
request_id = json.loads(b)["headers"]["x-request-id"]
log = None
for _ in range(20):
ls, lh, lb = rust.admin("/api/logs?vhost=reverse&limit=512")
logs = json.loads(lb)
log = next((entry for entry in logs.get("entries", []) if entry.get("request_id") == request_id), None)
if log:
break
time.sleep(.05)
check("observability.log_fields_and_body_bytes", log is not None and log.get("status") == 200
and log.get("response_bytes") == len(b) and log.get("request_bytes") == 0
and log.get("cache_effective_enabled") is False and log.get("transform_effective_enabled") is False
and bool(re.fullmatch("[0-9a-f]{64}", log.get("request_id", ""))),
recorded_status=log.get("status") if log else None,
recorded_response_bytes=log.get("response_bytes") if log else None, response_bytes=len(b))
if log:
write_json(rust.fixture.results / f"{rust.label}-sample-exchange.json", log)
config = copy.deepcopy(rust.base)
config["diagnostic_headers"] = {"request_id": True}
rust.apply(config, "observability-static-eos")
s, h, b = http(rust.static_port)
static_request_id = h.get("x-request-id")
static_log = None
for _ in range(20):
ls, lh, lb = rust.admin("/api/logs?vhost=static&limit=64")
static_log = next((entry for entry in json.loads(lb).get("entries", [])
if entry.get("request_id") == static_request_id), None)
if static_log:
break
time.sleep(.05)
check("observability.static_complete_after_content_length", s == 200 and static_log is not None
and static_log.get("outcome") == "complete" and static_log.get("response_bytes") == len(b),
outcome=static_log.get("outcome") if static_log else None,
recorded_response_bytes=static_log.get("response_bytes") if static_log else None,
response_bytes=len(b))
if static_log:
write_json(rust.fixture.results / f"{rust.label}-sample-static-exchange.json", static_log)
s, h, b = rust.admin("/api/status")
status = json.loads(b)
write_json(rust.fixture.results / f"{rust.label}-runtime-status.json", status)
check("observability.status_available", s == 200, status=s)
scripts/performance/checks_proxy.py
-135
View File
@@ -1,135 +0,0 @@
"""Proxy streaming, status, header, request-ID and WebSocket tests."""
import copy
import hashlib
from http.client import HTTPConnection, HTTPResponse
import json
import re
import socket
import time
from common import http
def run(check, rust, peer):
port = rust.reverse_port
for name in ["small.html", "large.bin", "huge.bin"]:
expected = (rust.fixture.site / name).read_bytes()
s, h, b = http(port, "/" + name)
check("proxy.get." + name, s == 200 and b == expected, status=s, bytes=len(b))
s, h, b = http(port, "/" + name, "HEAD")
check("proxy.head." + name, s == 200 and b == b"" and h.get("content-length") == str(len(expected)), status=s)
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-127"})
check("proxy.range", s == 206 and b == bytes(range(128)), status=s, bytes=len(b))
s, h, b = http(port)
s, h, b = http(port, headers={"If-None-Match": h["etag"]})
check("proxy.conditional304", s == 304 and b == b"", status=s)
for status in [201, 400, 404, 500, 503]:
s, h, b = http(port, f"/status/{status}")
check(f"proxy.origin_status.{status}", s == status and b == b"origin status payload\n"
and h.get("x-origin-fixture") == "yes", status=s)
s, h, b = http(port, "/redirect")
check("proxy.redirect", s == 302 and h.get("location") == "/small.html", status=s)
payload = bytes(range(256)) * 4096
ids = []
for method in ["POST", "PUT"]:
s, h, b = http(port, "/echo?query=preserved", method,
{"Content-Type": "application/octet-stream", "X-Request-ID": "untrusted-client-id"}, payload)
data = json.loads(b)
request_id = data["headers"].get("x-request-id", "")
ids.append(request_id)
check("proxy.upload." + method, s == 200 and data["method"] == method
and data["path"] == "/echo?query=preserved" and data["body_bytes"] == len(payload)
and data["body_sha256"] == hashlib.sha256(payload).hexdigest(), status=s, bytes=data["body_bytes"])
check("proxy.request_id." + method, bool(re.fullmatch("[0-9a-f]{64}", request_id)),
valid_64_hex=bool(re.fullmatch("[0-9a-f]{64}", request_id)))
check("proxy.request_id_unique", len(set(ids)) == 2)
s, h, b = http(port, "/echo", "GET", {"Content-Type": "application/octet-stream"}, b"nonempty-get-body")
data = json.loads(b)
check("proxy.nonempty_get_body_preserved", s == 200 and data["body_bytes"] == 17
and data["body_sha256"] == hashlib.sha256(b"nonempty-get-body").hexdigest(), status=s)
peer.started_upload.clear()
first, last = b"first streamed chunk" * 64, b"final streamed chunk" * 64
with socket.create_connection(("127.0.0.1", port), timeout=5) as sock:
sock.sendall((f"POST /stream-upload HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\n"
"Transfer-Encoding: chunked\r\nContent-Type: application/octet-stream\r\n\r\n").encode())
sock.sendall(f"{len(first):x}\r\n".encode() + first + b"\r\n")
forwarded_before_end = peer.started_upload.wait(2)
sock.sendall(f"{len(last):x}\r\n".encode() + last + b"\r\n0\r\n\r\n")
response = HTTPResponse(sock)
response.begin()
data = json.loads(response.read())
check("proxy.chunked_streaming_upload", forwarded_before_end and response.status == 200
and data["body_sha256"] == hashlib.sha256(first + last).hexdigest(),
first_chunk_arrived_before_end=forwarded_before_end, status=response.status)
peer.release_stream.clear()
connection = HTTPConnection("127.0.0.1", port, timeout=3)
connection.request("GET", "/stream-download")
started = time.monotonic()
response = connection.getresponse()
first = response.read(5)
first_seconds = time.monotonic() - started
peer.release_stream.set()
last = response.read()
connection.close()
check("proxy.streaming_response", response.status == 200 and first == b"first" and last == b"last"
and first_seconds < 2, first_chunk_seconds=first_seconds)
peer.release_stream.clear()
connection = HTTPConnection("127.0.0.1", port, timeout=3)
connection.request("GET", "/stream-download")
response = connection.getresponse()
response.read(5)
connection.close()
peer.release_stream.set()
s, h, b = http(port)
check("proxy.client_disconnect_survives", s == 200, status=s)
before = peer.count("POST", "/no-replay")
s, h, b = http(port, "/no-replay", "POST", {"Content-Type": "application/octet-stream"}, b"write-once")
after = peer.count("POST", "/no-replay")
check("proxy.failed_post_not_replayed", s in (502, 503, 504) and after - before == 1,
status=s, upstream_writes=after - before)
config = copy.deepcopy(rust.base)
config["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Global", "value": "global"}],
"response": [{"op": "set", "name": "X-Fixture-Global", "value": "response"}]}
config["vhosts"][0]["headers"] = {"request": [{"op": "set", "name": "X-Fixture-Vhost", "value": "vhost"}]}
config["vhosts"][0]["routes"] = [{"id": "echo", "path_prefix": "/echo", "headers": {
"request": [{"op": "set", "name": "X-Fixture-Route", "value": "route"}],
"response": [{"op": "set", "name": "X-Fixture-Route", "value": "response"}]}}]
rust.apply(config, "proxy-headers")
s, h, b = http(port, "/echo", headers={"Connection": "keep-alive, x-hop-fixture", "X-Hop-Fixture": "must-be-stripped"})
observed = json.loads(b)["headers"]
check("proxy.header_scopes", s == 200 and observed.get("x-fixture-global") == "global"
and observed.get("x-fixture-vhost") == "vhost" and observed.get("x-fixture-route") == "route"
and h.get("x-fixture-global") == "response" and h.get("x-fixture-route") == "response", status=s)
check("proxy.hop_by_hop_removed", "x-hop-fixture" not in observed)
s, h, b = http(port, "/echo", headers={"X-Forwarded-For": "203.0.113.22", "X-Real-IP": "203.0.113.22"})
observed = json.loads(b)["headers"]
check("proxy.untrusted_forwarded_ip", observed.get("x-real-ip") == "127.0.0.1"
and "203.0.113.22" not in observed.get("x-forwarded-for", ""),
observed_x_real_ip=observed.get("x-real-ip"), observed_x_forwarded_for=observed.get("x-forwarded-for"))
rust.reset()
with socket.create_connection(("127.0.0.1", port), timeout=4) as sock:
sock.sendall((f"GET /ws HTTP/1.1\r\nHost: 127.0.0.1:{port}\r\nUpgrade: websocket\r\n"
"Connection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
"Sec-WebSocket-Version: 13\r\n\r\n").encode())
buffer = b""
while b"\r\n\r\n" not in buffer:
piece = sock.recv(4096)
if not piece:
break
buffer += piece
upgraded = buffer.startswith(b"HTTP/1.1 101")
echoed = b""
if upgraded:
message, mask = b"hello", b"\x11\x22\x33\x44"
sock.sendall(bytes([0x81, 0x80 | len(message)]) + mask +
bytes(c ^ mask[i % 4] for i, c in enumerate(message)))
frame = sock.recv(2)
if len(frame) == 2:
echoed = sock.recv(frame[1] & 127)
check("proxy.websocket_echo", upgraded and echoed == b"hello", upgraded=upgraded, bytes=len(echoed))
scripts/performance/checks_static.py
-132
View File
@@ -1,132 +0,0 @@
"""Live static-file regression cases against RustProxy's network listener."""
import copy
from email.parser import BytesParser
from email.policy import default
import gzip
from common import http
def run(check, rust, peer):
port, fixture = rust.static_port, rust.fixture
for name in ["small.html", "large.bin", "huge.bin", "empty.txt"]:
expected = (fixture.site / name).read_bytes()
status, headers, body = http(port, "/" + name)
check(f"static.get.{name}", status == 200 and body == expected,
status=status, bytes=len(body))
head_status, head_headers, head_body = http(port, "/" + name, "HEAD")
check(f"static.head.{name}", head_status == 200 and head_body == b""
and head_headers.get("content-length") == str(len(expected)),
status=head_status, content_length=head_headers.get("content-length"))
status, headers, body = http(port)
tag = headers.get("etag")
modified = headers.get("last-modified")
check("static.validators_present", bool(tag and modified), etag=tag, last_modified=modified)
for label, condition in [("etag", {"If-None-Match": tag}),
("modified", {"If-Modified-Since": modified})]:
s, h, b = http(port, headers=condition)
check("static.conditional304." + label, s == 304 and b == b"", status=s, bytes=len(b))
s, h, b = http(port, headers={"If-None-Match": '"missing"'})
check("static.conditional_miss", s == 200 and b == body, status=s)
data = (fixture.site / "large.bin").read_bytes()
cases = [("prefix", "bytes=0-127", data[:128], "bytes 0-127/65536"),
("suffix", "bytes=-37", data[-37:], "bytes 65499-65535/65536"),
("open", "bytes=65500-", data[65500:], "bytes 65500-65535/65536")]
for name, range_header, expected, content_range in cases:
s, h, b = http(port, "/large.bin", headers={"Range": range_header})
check("static.range." + name, s == 206 and b == expected
and h.get("content-range") == content_range,
status=s, bytes=len(b), content_range=h.get("content-range"))
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=999999-"})
check("static.range.unsatisfiable", s == 416 and h.get("content-range") == "bytes */65536",
status=s, content_range=h.get("content-range"))
s, h, b = http(port, "/large.bin", headers={"Range": "bytes=0-15,100-115"})
mime = BytesParser(policy=default).parsebytes(
b"Content-Type: " + h.get("content-type", "").encode() + b"\r\n\r\n" + b)
parts = list(mime.iter_parts())
check("static.range.multipart", s == 206 and len(parts) == 2
and parts[0].get_payload(decode=True) == data[:16]
and parts[1].get_payload(decode=True) == data[100:116], status=s, parts=len(parts))
for name, value, expected_status in [("match", tag, 206), ("mismatch", '"other"', 200)]:
s, h, b = http(port, headers={"Range": "bytes=0-15", "If-Range": value})
expected = body[:16] if expected_status == 206 else body
check("static.if_range." + name, s == expected_status and b == expected,
status=s, bytes=len(b))
s, h, b = http(port, "/directory")
check("static.directory_slash", s in (301, 308) and h.get("location", "").endswith("/directory/"),
status=s, location=h.get("location"))
s, h, b = http(port, "/directory/")
check("static.directory_index", s == 200 and b == b"directory index\n", status=s)
s, h, b = http(port, "/listing/")
check("static.directory_listing_denied", s == 403, status=s)
for path in ["/escape-link.txt", "/inside-link.txt"]:
s, h, b = http(port, path)
check("static.symlink_deny." + path, s == 403 and b"MUST-NOT" not in b, status=s)
for path in ["/%2e%2e/outside-secret.txt", "/%2fetc/passwd", "/%252e%252e/outside-secret.txt",
"/%00", "/..%5coutside-secret.txt"]:
s, h, b = http(port, path)
check("static.invalid_path." + path, s in (400, 403, 404) and b"MUST-NOT" not in b,
status=s)
config = copy.deepcopy(rust.base)
service = config["web_services"][0]
service["symlink_policy"] = "inside_root"
service["directory_listing"] = True
rust.apply(config, "static-inside-root")
s, h, b = http(port, "/inside-link.txt")
check("static.symlink_inside_root_allowed", s == 200 and b == b"download fixture\n", status=s)
s, h, b = http(port, "/escape-link.txt")
check("static.symlink_inside_root_escape_denied", s == 403 and b"MUST-NOT" not in b, status=s)
s, h, b = http(port, "/listing/")
check("static.directory_listing_enabled", s == 200 and b"visible.txt" in b, status=s)
# Alternate a symlink target between valid and outside-root files. The file
# read must stay confined even when a pathname changes between requests.
link = fixture.site / "changing-link.txt"
race_statuses = []
escaped = False
for index in range(12):
if link.is_symlink():
link.unlink()
link.symlink_to(fixture.outside if index % 2 else fixture.site / "download.txt")
s, h, b = http(port, "/changing-link.txt")
race_statuses.append(s)
escaped = escaped or b"MUST-NOT" in b
check("static.changing_symlink_confinement", not escaped and all(s in (200, 403) for s in race_statuses),
statuses=race_statuses, note="Sequential target replacement; not a concurrent rename-race proof")
config = copy.deepcopy(rust.base)
config["web_services"][0]["precompressed"] = {"enabled": True, "gzip": True, "brotli": False}
rust.apply(config, "static-precompressed")
s, h, b = http(port, headers={"Accept-Encoding": "gzip"})
sidecar = (fixture.site / "small.html.gz").read_bytes()
check("static.precompressed_gzip", s == 200 and h.get("content-encoding") == "gzip"
and b == sidecar and gzip.decompress(b) == body, status=s, bytes=len(b))
s, h, b = http(port, headers={"Accept-Encoding": "gzip", "Range": "bytes=0-9"})
check("static.precompressed_range", s == 206 and b == sidecar[:10], status=s, bytes=len(b))
config = copy.deepcopy(rust.base)
config["web_services"][0]["try_files"] = ["$uri", "/small.html"]
config["web_services"][0]["rewrites"] = [{"pattern": "^/old$", "replacement": "/small.html", "status": 302}]
config["web_services"][0]["download_extensions"] = ["txt"]
config["web_services"][0]["mime_types"] = {"bin": "application/x-fixture"}
rust.apply(config, "static-routing")
s, h, b = http(port, "/spa/deep/link")
check("static.try_files_spa", s == 200 and b == body, status=s)
s, h, b = http(port, "/old")
check("static.rewrite_redirect", s == 302 and h.get("location") == "/small.html", status=s)
s, h, b = http(port, "/download.txt")
check("static.download_header", s == 200 and "attachment" in h.get("content-disposition", ""), status=s)
s, h, b = http(port, "/large.bin")
check("static.mime_override", s == 200 and h.get("content-type") == "application/x-fixture", status=s)
config = copy.deepcopy(rust.base)
(fixture.site / ".htaccess").write_text("Header set X-Fixture-Htaccess yes\nDirectoryIndex small.html\n")
config["web_services"][0]["htaccess"] = {"enabled": True, "strict": True}
rust.apply(config, "static-htaccess")
s, h, b = http(port)
check("static.htaccess_header", s == 200 and h.get("x-fixture-htaccess") == "yes", status=s)
s, h, b = http(port, "/.htaccess")
check("static.htaccess_not_served", s in (403, 404) and b"Header set" not in b, status=s)
scripts/performance/common.py
-448
View File
@@ -1,448 +0,0 @@
#!/usr/bin/env python3
"""Disposable, loopback-only fixtures for testing actual RustProxy processes.
Runtime databases and credentials live in TemporaryDirectory, never in results.
No pre-existing SQLite database, fixed port, or historical /mnt/data path is used.
"""
from __future__ import annotations
import copy
import gzip
import hashlib
from http.client import HTTPConnection, HTTPException
import json
import os
from pathlib import Path
import pwd
import secrets
import shutil
import signal
import socket
import subprocess
import tempfile
import time
def sha256(path: Path) -> str:
result = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
result.update(block)
return result.hexdigest()
def write_json(path: Path, value) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n")
def seal_results(directory: Path) -> None:
"""Checksums cover every regular evidence file, except the checksum manifest."""
paths = sorted(p for p in directory.rglob("*") if p.is_file()
and p.name != "SHA256SUMS")
(directory / "SHA256SUMS").write_text("".join(
f"{sha256(path)} {path.relative_to(directory)}\n" for path in paths))
def http(port: int, path="/small.html", method="GET", headers=None, body=None,
timeout=10, source_address=None):
connection = HTTPConnection("127.0.0.1", port, timeout=timeout,
source_address=source_address)
try:
connection.request(method, path, body, headers or {})
response = connection.getresponse()
payload = response.read()
return response.status, {k.lower(): v for k, v in response.getheaders()}, payload
finally:
connection.close()
def free_port() -> int:
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
def wait_ready(port: int, process: subprocess.Popen, timeout=30):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if process.poll() is not None:
raise RuntimeError(f"process {process.pid} exited with {process.returncode}")
try:
http(port, "/", timeout=.5)
return
except (OSError, HTTPException):
time.sleep(.05)
raise RuntimeError(f"loopback listener {port} did not start")
class Fixture:
def __init__(self, results: Path, workers=4, temp_parent: Path | None = None,
server_cpus=None, origin_cpus=None, origin_workers=None):
self.results = results.resolve()
self.results.mkdir(parents=True, exist_ok=True)
# Keep live state outside the evidence tree. Even an interrupted run or
# workspace synchronization must never put databases/keys in results.
self.temp = tempfile.TemporaryDirectory(prefix="rustproxy-perf-", dir=temp_parent)
self.root = Path(self.temp.name)
self.root.chmod(0o755) # nginx workers must traverse the public fixture root
self.site = self.root / "site"
self.site.mkdir(mode=0o755)
self.workers = workers
self.server_cpus = server_cpus
self.origin_cpus = origin_cpus
self.origin_workers = origin_workers or workers
self.children = []
self.handles = []
self.ports = set()
self.origin_port = self.port()
self.make_site()
def port(self):
while True:
port = free_port()
if port not in self.ports:
self.ports.add(port)
return port
def make_site(self):
# Exact, reproducible payload sizes match the original report's cases.
small = b"<!doctype html><html><body>RustProxy benchmark "
small += b"x" * (1040 - len(small) - len(b"</body></html>"))
small += b"</body></html>"
files = {"small.html": small,
"large.bin": bytes(range(256)) * 256,
"huge.bin": bytes(range(256)) * 4096,
"empty.txt": b"",
"app.json": (b'{\n "message": "stable fixture",\n "values": [1, 2, 3]\n}\n'),
"app.css": b"/* fixture comment */\nbody { color: red; padding: 0px; }\n",
"compress.txt": (b"compressible RustProxy text fixture. " * 4096),
"download.txt": b"download fixture\n"}
for name, body in files.items():
(self.site / name).write_bytes(body)
(self.site / "small.html.gz").write_bytes(gzip.compress(small, mtime=0))
(self.site / "directory").mkdir()
(self.site / "directory" / "index.html").write_bytes(b"directory index\n")
(self.site / "listing").mkdir()
(self.site / "listing" / "visible.txt").write_bytes(b"listing fixture\n")
(self.site / "inside-link.txt").symlink_to(self.site / "download.txt")
self.outside = self.root / "outside-secret.txt"
self.outside.write_bytes(b"MUST-NOT-BE-SERVED-OUTSIDE-ROOT\n")
(self.site / "escape-link.txt").symlink_to(self.outside)
write_json(self.results / "payloads.json", {
name: {"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest()}
for name, body in files.items()})
def spawn(self, argv, name, env=None, affinity=None):
log = (self.results / f"{name}.log").open("w")
self.handles.append(log)
if affinity:
argv = ["taskset", "-c", affinity, *argv]
child = subprocess.Popen([str(x) for x in argv], stdout=log,
stderr=subprocess.STDOUT, env=env, start_new_session=True)
self.children.append(child)
return child
@staticmethod
def stop(process):
if process.poll() is None:
os.killpg(process.pid, signal.SIGTERM)
try:
process.wait(timeout=12)
except subprocess.TimeoutExpired:
os.killpg(process.pid, signal.SIGKILL)
process.wait(timeout=3)
def close(self):
for process in reversed(self.children):
self.stop(process)
for handle in self.handles:
handle.close()
self.temp.cleanup()
write_json(self.results / "fixture-cleanup.json", {
"temporary_state_removed": not self.root.exists(),
"state_was_outside_results": not self.root.is_relative_to(self.results)})
if self.root.exists():
raise RuntimeError("temporary runtime state was not removed; do not package the fixture directory")
def __enter__(self):
return self
def __exit__(self, *_):
self.close()
def base_config(self, reverse_port, static_port):
endpoints = []
for label, port in [("reverse", reverse_port), ("static", static_port)]:
endpoints.append({"id": label, "name": label, "scope": "internal",
"enabled": True, "binding": {
"address": f"127.0.0.1:{port}", "tls": False,
"http2": False, "http3": None}})
return {
"cache": {"enabled": False},
"geoip_updates": {"enabled": False},
"listeners": {"endpoints": endpoints},
"web_services": [{"id": "fixture-static", "name": "Fixture static",
"document_root": str(self.site), "index": ["index.html", "small.html"],
"directory_listing": False}],
"vhosts": [{"id": "reverse", "frontend": {
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
"listener_ids": ["reverse"]},
"backends": [{"id": "origin", "url": f"http://127.0.0.1:{self.origin_port}",
"allow_private": True, "preserve_host": False}],
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []},
{"id": "static", "frontend": {
"hosts": ["127.0.0.1", "localhost", "fixture.test"],
"listener_ids": ["static"]},
"source": {"type": "web_server", "web_service_id": "fixture-static"},
"routes": []}]
}
def rust(self, binary: Path, label: str, max_idle=None):
return RustInstance(self, binary.resolve(), label, max_idle=max_idle)
def nginx(self, executable: Path, frontend=True):
"""Use a separate nginx master for the shared origin and comparator."""
executable = executable.resolve()
result = subprocess.run([str(executable), "-V"], capture_output=True, text=True, timeout=10)
(self.results / "nginx-version.txt").write_text(result.stdout + result.stderr)
origin_conf = self.nginx_config("nginx-origin", f"""
server {{ listen 127.0.0.1:{self.origin_port}; server_name _;
root "{self.site}"; location / {{ add_header Cache-Control "public, max-age=60"; }}
}}
""", workers=self.origin_workers)
origin = self.spawn([executable, "-p", self.root, "-c", origin_conf], "nginx-origin",
affinity=self.origin_cpus)
wait_ready(self.origin_port, origin)
result = {"origin": origin, "origin_port": self.origin_port}
if frontend:
reverse_port, static_port = self.port(), self.port()
conf = self.nginx_config("nginx-frontend", f"""
upstream fixture_origin {{ server 127.0.0.1:{self.origin_port}; keepalive 64; }}
server {{ listen 127.0.0.1:{reverse_port}; server_name _;
location / {{ proxy_pass http://fixture_origin; proxy_http_version 1.1;
proxy_set_header Connection ""; proxy_buffering off; }}
}}
server {{ listen 127.0.0.1:{static_port}; server_name _; root "{self.site}"; }}
""")
process = self.spawn([executable, "-p", self.root, "-c", conf], "nginx-frontend",
affinity=self.server_cpus)
wait_ready(reverse_port, process)
wait_ready(static_port, process)
result.update({"process": process, "reverse_port": reverse_port,
"static_port": static_port})
return result
def nginx_config(self, name, servers, workers=None):
path = self.root / f"{name}.conf"
text = f"""daemon off;
master_process on;
user {pwd.getpwuid(os.geteuid()).pw_name};
worker_processes {workers or self.workers};
pid {self.root}/{name}.pid;
error_log stderr warn;
events {{ worker_connections 8192; }}
http {{
access_log off;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
keepalive_requests 1000000;
default_type application/octet-stream;
types {{ text/html html; text/css css; application/json json; text/plain txt; }}
client_body_temp_path {self.root}/{name}-body;
proxy_temp_path {self.root}/{name}-proxy;
{servers}
}}
"""
path.write_text(text)
(self.results / path.name).write_text(text)
return path
class RustInstance:
def __init__(self, fixture, binary, label, max_idle=None):
self.fixture, self.binary, self.label = fixture, binary, label
if not os.access(binary, os.X_OK):
target = fixture.root / f"{label}-rust-proxy"
shutil.copyfile(binary, target)
target.chmod(0o700)
self.binary = target
self.admin_port, self.reverse_port, self.static_port = (fixture.port() for _ in range(3))
self.data = fixture.root / f"{label}-data"
self.base = fixture.base_config(self.reverse_port, self.static_port)
if max_idle is not None:
self.base["vhosts"][0]["load_balancing"]["keepalive"] = {"max_idle_per_host": max_idle}
self.password = "Fixture!" + secrets.token_hex(20)
self.username = "performance"
self.env = os.environ.copy()
self.env["TOKIO_WORKER_THREADS"] = str(fixture.workers)
self.env["RUST_PROXY_PROXY_LISTEN"] = f"127.0.0.1:{self.reverse_port}"
self.env["RUST_PROXY_UPDATE_CHECK_INTERVAL_SECS"] = "0"
self.env.pop("RUST_PROXY_TLS_LISTEN", None)
self.applies = []
self.cli("admin", "init", "--username", self.username, "--password-stdin",
input=self.password + "\n")
self.start("bootstrap")
self.apply(self.base, "base")
fixture.stop(self.process)
self.start("runtime")
wait_ready(self.static_port, self.process)
version = self.cli("--version").stdout.strip()
write_json(fixture.results / f"{label}-binary.json", {
"label": label, "path": str(binary), "sha256": sha256(binary),
"version": version, "worker_threads": fixture.workers,
"pid": self.process.pid, "protocol": "HTTP/1.1 plaintext loopback",
"cpu_affinity": fixture.server_cpus,
"max_idle_per_host_override": max_idle})
def cli(self, *args, input=None, check=True):
result = subprocess.run([str(self.binary), "--data-dir", str(self.data),
"--admin-listen", f"127.0.0.1:{self.admin_port}", *args],
input=input, text=True, capture_output=True,
env=self.env, timeout=40)
if check and result.returncode:
raise RuntimeError(f"{self.label} CLI {' '.join(args[:2])}: "
f"{result.stdout[-2000:]} {result.stderr[-3000:]}")
return result
def start(self, phase):
self.process = self.fixture.spawn([
self.binary, "--data-dir", self.data, "--admin-listen", f"127.0.0.1:{self.admin_port}",
"--update-check-interval-secs", "0", "serve", "--proxy-listen",
f"127.0.0.1:{self.reverse_port}", "--max-connections", "4096", "--max-inflight", "4096"],
f"{self.label}-{phase}", env=self.env, affinity=self.fixture.server_cpus)
wait_ready(self.admin_port, self.process)
wait_ready(self.reverse_port, self.process)
def apply(self, config, name, expected_success=True):
config_path = self.fixture.root / f"{self.label}-{name}.json"
write_json(config_path, config)
result = self.cli("config", "apply", str(config_path), check=False)
# Configurations with ephemeral test passwords/hashes are intentionally not copied.
safe = copy.deepcopy(config)
for vhost in safe.get("vhosts", []):
if "basic_auth" in vhost:
vhost["basic_auth"]["users"] = "REDACTED fixture users"
write_json(self.fixture.results / "configs" / f"{self.label}-{name}.json", safe)
event = {"name": name, "sha256": sha256(config_path), "returncode": result.returncode,
"expected_success": expected_success, "stdout": result.stdout[-2000:],
"stderr": result.stderr[-3000:]}
self.applies.append(event)
write_json(self.fixture.results / f"{self.label}-config-applies.json", self.applies)
if (result.returncode == 0) != expected_success:
raise RuntimeError(f"config {name}: {result.stdout} {result.stderr}")
return result
def reset(self):
return self.apply(self.base, "restored-base")
def login(self):
status, headers, body = http(self.admin_port, "/api/auth/login", "POST",
{"Origin": f"http://127.0.0.1:{self.admin_port}", "Content-Type": "application/json",
"X-RustProxy-Request": "1"},
json.dumps({"username": self.username, "password": self.password}))
if status != 200:
raise RuntimeError(f"fixture login failed: HTTP {status}: {body[:200]!r}")
self.cookie = headers["set-cookie"].split(";", 1)[0]
self.csrf = json.loads(body)["csrf_token"]
return status
def wait_quiescent(self, timeout=180, quiet_seconds=5, poll_seconds=.5):
"""Wait for the real, one-time ACME bootstrap before any timed work.
This does not disable ACME or change the process environment. The
private fixture has no issuers/jobs; a terminal unavailable state is
valid for HTTP tests, but does not establish ACME issuance support.
"""
from resources import descendants, process_stats
if timeout <= 0 or quiet_seconds <= 0 or poll_seconds <= 0 or timeout <= quiet_seconds:
raise ValueError("quiescence timeout must exceed the positive quiet period")
started = time.monotonic()
deadline = started + timeout
output = self.fixture.results / f"{self.label}-quiescence.json"
report = {"completed": False, "label": self.label, "pid": self.process.pid,
"timeout_seconds": timeout, "required_quiet_seconds": quiet_seconds,
"poll_seconds": poll_seconds, "observations": [],
"acme_issuance_tested": False}
def read_admin(path):
remaining = deadline - time.monotonic()
if remaining <= 0:
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
if self.process.poll() is not None:
raise RuntimeError(f"{self.label} exited during quiescence")
status, _, body = self.admin(path, timeout=min(1, remaining))
if status != 200:
raise RuntimeError(f"{self.label} quiescence {path}: HTTP {status}")
return json.loads(body)
try:
root = process_stats(self.process.pid)
if root is None:
raise RuntimeError(f"{self.label} has no live root process")
identity = (root["pid"], root["start_ticks"])
report["start_ticks"] = root["start_ticks"]
envelope = read_admin("/api/config")
if not isinstance(envelope, dict) or not isinstance(envelope.get("config"), dict):
raise RuntimeError("quiescence requires the actual configuration envelope")
certificates = envelope["config"].get("tls", {}).get("certificates", [])
if not isinstance(certificates, list) or any(not isinstance(cert, dict) for cert in certificates):
raise RuntimeError("unexpected certificate configuration during quiescence")
report["configured_issuers"] = sum(cert.get("issuer") is not None for cert in certificates)
if report["configured_issuers"]:
raise RuntimeError("performance fixture must have no configured ACME issuers")
quiet_started, previous_terminal = None, None
while True:
payload = read_admin("/api/acme")
if not isinstance(payload, dict) or not {"runtime", "job", "renewals"} <= payload.keys():
raise RuntimeError("quiescence requires explicit ACME runtime/job/renewal state")
if payload["job"] is not None or payload["renewals"] != {}:
raise RuntimeError("performance fixture has an unexpected ACME job or renewal")
runtime = payload["runtime"]
if not isinstance(runtime, dict) or runtime.get("status") not in {
"checking", "initializing", "ready", "unavailable"}:
raise RuntimeError("unexpected ACME runtime state during quiescence")
pids = descendants(self.process.pid)
root = process_stats(self.process.pid)
if root is None or (root["pid"], root["start_ticks"]) != identity:
raise RuntimeError(f"{self.label} root process identity changed during quiescence")
now = time.monotonic()
if now >= deadline:
raise TimeoutError(f"{self.label} did not become quiescent within {timeout}s")
terminal = tuple(runtime.get(key) for key in ("status", "mode", "message"))
quiet = runtime["status"] in {"ready", "unavailable"} and pids == [self.process.pid]
if not quiet:
quiet_started, previous_terminal = None, None
elif quiet_started is None or terminal != previous_terminal:
quiet_started, previous_terminal = now, terminal
elapsed_quiet = now - quiet_started if quiet_started is not None else 0
report["observations"].append({
"elapsed_seconds": now - started, "process_count": len(pids),
"acme_status": runtime["status"], "acme_mode": runtime.get("mode"),
"acme_message": runtime.get("message"), "quiet_seconds": elapsed_quiet})
if elapsed_quiet >= quiet_seconds:
report.update({"completed": True, "elapsed_seconds": now - started,
"terminal_status": runtime["status"], "process_count": len(pids)})
write_json(output, report)
return report
write_json(output, report)
time.sleep(min(poll_seconds, max(0, deadline - time.monotonic())))
except (Exception, KeyboardInterrupt) as error:
report.update({"elapsed_seconds": time.monotonic() - started,
"error_type": type(error).__name__, "error": str(error)})
write_json(output, report)
raise
def admin(self, path, method="GET", body=None, csrf=True, cookie=True, origin=None,
timeout=10):
headers = {"Origin": origin or f"http://127.0.0.1:{self.admin_port}", "X-RustProxy-Request": "1"}
if cookie:
headers["Cookie"] = self.cookie
if csrf:
headers["X-CSRF-Token"] = self.csrf
if body is not None:
headers["Content-Type"] = "application/json"
body = json.dumps(body)
return http(self.admin_port, path, method, headers, body, timeout=timeout)
scripts/performance/concurrency_regression.py
-138
View File
@@ -1,138 +0,0 @@
#!/usr/bin/env python3
"""Finite live HTTP concurrency correctness stress; no throughput/latency claim."""
from __future__ import annotations
import argparse
from concurrent.futures import ThreadPoolExecutor
import hashlib
import json
from pathlib import Path
import threading
import time
import traceback
from common import Fixture, http, seal_results, sha256, write_json
from resources import process_stats
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
def active_backends(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
return {"error": f"HTTP {status}"}
values = []
def walk(value):
if isinstance(value, dict):
if "id" in value and "active" in value and "status" in value and "requests" in value:
values.append({key: value[key] for key in ("id", "active", "status", "requests")})
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return values
def batch(rust, port, name, expected, concurrency):
count = max(32, concurrency * 2)
expected_sha = hashlib.sha256(expected).hexdigest()
ready = threading.Barrier(concurrency, timeout=15)
lock = threading.Lock()
running = peak = 0
def fetch(index):
nonlocal running, peak
with lock:
running += 1
peak = max(peak, running)
try:
if index < concurrency:
ready.wait()
status, headers, body = http(port, "/" + name, timeout=15)
digest = hashlib.sha256(body).hexdigest()
return {"index": index, "status": status, "bytes": len(body), "sha256": digest,
"passed": status == 200 and len(body) == len(expected) and digest == expected_sha}
except Exception as error:
return {"index": index, "passed": False, "error": str(error)}
finally:
with lock:
running -= 1
before = process_stats(rust.process.pid)
started = time.monotonic()
with ThreadPoolExecutor(max_workers=concurrency) as executor:
responses = list(executor.map(fetch, range(count)))
elapsed = time.monotonic() - started
# Completion means every client consumed or failed its full response. Give
# asynchronous server bookkeeping a bounded interval to release its leases.
for _ in range(40):
active = active_backends(rust)
if isinstance(active, list) and active and all(row["active"] == 0 for row in active):
break
time.sleep(.05)
after = process_stats(rust.process.pid)
return {"configured_concurrent_clients": concurrency, "peak_running_client_tasks": peak,
"client_tasks_include_barrier": True, "requests": count, "elapsed_seconds": elapsed,
"expected_bytes": len(expected), "expected_sha256": expected_sha,
"response_failures": sum(not response["passed"] for response in responses),
"passed": all(response["passed"] for response in responses)
and isinstance(active, list) and bool(active) and all(row["active"] == 0 for row in active),
"responses": responses, "active_backends_after_drain": active,
"server_before": before, "server_after_drain": after}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be new or empty")
reports = {}
for specification in args.binary:
label, binary = specification.split("=", 1)
cases, error = [], None
try:
with Fixture(args.out / label, workers=4, origin_workers=2) as fixture:
fixture.nginx(args.nginx, frontend=False)
rust = fixture.rust(Path(binary), label)
rust.login()
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
for name in PAYLOADS:
expected = (fixture.site / name).read_bytes()
for concurrency in [1, 16, 128]:
result = batch(rust, port, name, expected, concurrency)
case = {"case": f"concurrency.{source}.{name}.c{concurrency}", **result}
cases.append(case)
write_json(args.out / label / "cases" / f"{source}-{name}-c{concurrency}.json", case)
print(f"{label} {'PASS' if case['passed'] else 'FAIL'} {case['case']}", flush=True)
if rust.process.poll() is not None:
raise RuntimeError("RustProxy exited during concurrency regression")
except Exception as exc:
error = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / label / "failure.json", error)
report = {"kind": "live HTTP/1.1 finite concurrency functional stress", "simulation": False,
"binary_sha256": sha256(Path(binary)), "throughput_measurement": False,
"passed_cases": sum(case["passed"] for case in cases),
"failed_cases": sum(not case["passed"] for case in cases),
"requests": sum(case["requests"] for case in cases),
"response_failures": sum(case["response_failures"] for case in cases),
"completed": error is None and len(cases) == 18,
"cases": [{key: case[key] for key in ("case", "passed", "requests", "response_failures")} for case in cases],
"limitations": ["configured client concurrency is not proof of simultaneous active server exchanges",
"Python client is not a throughput or latency benchmark",
"FD/RSS are before/after snapshots, not peaks or a long-term leak test"]}
reports[label] = report
write_json(args.out / label / "report.json", report)
write_json(args.out / "comparison.json", reports)
seal_results(args.out)
return 0 if all(report["completed"] and not report["failed_cases"] for report in reports.values()) else 1
if __name__ == "__main__":
raise SystemExit(main())
scripts/performance/lb_regression.py
-223
View File
@@ -1,223 +0,0 @@
#!/usr/bin/env python3
"""Small live two-origin failover and active-exchange regression, not a benchmark."""
from __future__ import annotations
import argparse
from collections import Counter
import copy
import hashlib
from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
from pathlib import Path
import threading
import time
import traceback
from common import Fixture, http, seal_results, write_json
class Origin:
def __init__(self, port, name):
self.name, self.port = name, port
self.fail_get = False
self.counts, self.post_hashes = Counter(), []
self.release, self.lock = threading.Event(), threading.Lock()
origin = self
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def log_message(self, *_):
pass
def do_GET(self):
self.respond()
def do_POST(self):
self.respond()
def respond(self):
body = self.rfile.read(int(self.headers.get("Content-Length", "0")))
with origin.lock:
origin.counts[(self.command, self.path)] += 1
if self.command == "POST":
origin.post_hashes.append(hashlib.sha256(body).hexdigest())
hold = self.path == "/hold"
status = 503 if self.command == "POST" or origin.fail_get else 200
payload = b"firstlast" if hold else origin.name.encode()
try:
self.send_response(status)
self.send_header("Content-Length", str(len(payload)))
self.send_header("Content-Type", "application/octet-stream")
self.send_header("Cache-Control", "no-store")
# Fresh connections make refused-listener tests independent
# of already accepted upstream keepalive connections.
self.send_header("Connection", "close")
self.end_headers()
if hold:
self.wfile.write(payload[:5])
self.wfile.flush()
origin.release.wait(15)
self.wfile.write(payload[5:])
else:
self.wfile.write(payload)
self.wfile.flush()
except (BrokenPipeError, ConnectionResetError):
pass
self.close_connection = True
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
self.closed = False
def close(self):
self.release.set()
if not self.closed:
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=3)
self.closed = True
def backend_views(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"status unavailable: {status}")
found = {}
def walk(value):
if isinstance(value, dict):
if value.get("id") in ("a", "b") and "active" in value and "status" in value:
found[value["id"]] = {key: value[key] for key in value
if key in ("id", "status", "active", "requests", "errors", "bytes_out")
or key.startswith("retry_") or key.startswith("connect_errors")}
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return found
def run(check, fixture, binary, label):
a, b = Origin(fixture.origin_port, "a"), Origin(fixture.port(), "b")
try:
rust = fixture.rust(binary, label)
rust.login()
config = copy.deepcopy(rust.base)
vhost = config["vhosts"][0]
vhost["backends"] = [{"id": origin.name, "url": f"http://127.0.0.1:{origin.port}",
"allow_private": True, "preserve_host": False} for origin in [a, b]]
vhost["load_balancing"] = {"algorithm": "round_robin", "health_check": {"mode": "off"},
"slow_start_seconds": 0, "passive_health": {"enabled": False},
"retry": {"max_retries": 1, "retry_statuses": [503]}}
vhost["routes"] = [{"id": "hold-a", "path_prefix": "/hold", "backend_id": "a"}]
rust.apply(config, "lb-two-origins")
replies = [http(rust.reverse_port, "/rr") for _ in range(8)]
counts = Counter(body.decode() for status, _, body in replies if status == 200)
check("lb.round_robin_both_origins", counts == {"a": 4, "b": 4}, counts=dict(counts))
a.fail_get = True
replies = [http(rust.reverse_port, "/retry-status") for _ in range(6)]
check("lb.get_503_failover", all(status == 200 and body == b"b" for status, _, body in replies)
and a.counts[("GET", "/retry-status")] > 0,
failed_origin_attempts=a.counts[("GET", "/retry-status")])
views = backend_views(rust)
check("lb.retry_observations_recorded", sum(view.get("retry_attempts_total", 0) for view in views.values()) > 0,
backends=views)
a.fail_get = False
payload = b"one POST must be attempted once"
replies = [http(rust.reverse_port, "/post-no-replay", "POST", body=payload) for _ in range(4)]
attempts = sum(origin.counts[("POST", "/post-no-replay")] for origin in [a, b])
check("lb.post_503_never_replayed", all(status == 503 for status, _, _ in replies) and attempts == 4,
client_requests=4, origin_attempts=attempts)
check("lb.post_body_preserved", len(a.post_hashes + b.post_hashes) == 4 and
all(digest == hashlib.sha256(payload).hexdigest() for digest in a.post_hashes + b.post_hashes))
for mode in ["drain", "disable"]:
vhost["backends"][0].update({"enabled": True, "drain": False})
rust.apply(config, f"lb-{mode}-before")
a.release.clear()
connection = HTTPConnection("127.0.0.1", rust.reverse_port, timeout=10)
try:
connection.request("GET", "/hold")
response = connection.getresponse()
first = response.read(5)
before = backend_views(rust)
check(f"lb.{mode}.lease_spans_response_body", response.status == 200 and first == b"first"
and before.get("a", {}).get("active") == 1, backends=before)
vhost["backends"][0].update({"drain": mode == "drain", "enabled": mode != "disable"})
rust.apply(config, f"lb-{mode}-active")
replies = [http(rust.reverse_port, "/new") for _ in range(4)]
check(f"lb.{mode}.new_requests_use_healthy_member",
all(status == 200 and body == b"b" for status, _, body in replies))
during = backend_views(rust)
check(f"lb.{mode}.active_lease_survives_hot_edit", during.get("a", {}).get("active") == 1,
backends=during)
a.release.set()
last = response.read()
check(f"lb.{mode}.admitted_body_finishes", first + last == b"firstlast")
for _ in range(20):
after = backend_views(rust)
if after.get("a", {}).get("active") == 0:
break
time.sleep(.05)
check(f"lb.{mode}.lease_released_after_body", after.get("a", {}).get("active") == 0,
backends=after)
finally:
a.release.set()
connection.close()
vhost["backends"][0].update({"enabled": True, "drain": False})
rust.apply(config, "lb-connect-refused")
a.close()
replies = [http(rust.reverse_port, "/retry-connect") for _ in range(6)]
views = backend_views(rust)
check("lb.connect_refused_failover", all(status == 200 and body == b"b" for status, _, body in replies)
and views.get("a", {}).get("connect_errors_total", 0) > 0, backends=views)
rust.reset()
check("lb.static_vhost_survives_reload", http(rust.static_port)[0] == 200)
finally:
a.close()
b.close()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be new or empty")
reports = {}
for specification in args.binary:
label, binary = specification.split("=", 1)
cases = []
def check(case, passed, **details):
cases.append({"case": case, "passed": passed, **details})
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(args.out / label / "checks.json", cases)
try:
with Fixture(args.out / label) as fixture:
run(check, fixture, Path(binary), label)
except Exception as exc:
check("lb.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"passed": sum(case["passed"] for case in cases),
"failed": sum(not case["passed"] for case in cases), "cases": cases,
"simulation": False, "kind": "live RustProxy two-origin HTTP/1.1 regression",
"not_covered": ["sticky affinity", "active health probes", "long-running connection soak",
"TLS origin failures", "DNS rotation"]}
reports[label] = report
write_json(args.out / label / "report.json", report)
write_json(args.out / "comparison.json", reports)
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
raise SystemExit(main())
scripts/performance/peer.py
-223
View File
@@ -1,223 +0,0 @@
#!/usr/bin/env python3
"""A protocol peer, not a RustProxy emulator; used only as the test upstream.
Performance comparisons use nginx as the upstream, never this Python fixture.
"""
from __future__ import annotations
import base64
import collections
import hashlib
import json
import socket
import struct
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import urlsplit, parse_qs
class Peer:
def __init__(self, port, site):
self.site = site
self.counts = collections.Counter()
self.received = {}
self.lock = threading.Lock()
self.started_upload = threading.Event()
self.release_stream = threading.Event()
peer = self
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
def log_message(self, *_):
pass
def emit(self, status, body=b"", headers=None):
self.send_response(status)
self.send_header("Content-Length", str(len(body)))
if "Content-Type" not in (headers or {}):
self.send_header("Content-Type", "application/octet-stream")
for key, value in (headers or {}).items():
self.send_header(key, value)
self.end_headers()
if self.command != "HEAD" and body:
self.wfile.write(body)
def upload(self):
chunks = []
if self.headers.get("Transfer-Encoding", "").lower() == "chunked":
while True:
length = int(self.rfile.readline().split(b";", 1)[0], 16)
if length == 0:
while self.rfile.readline().strip():
pass
break
payload = self.rfile.read(length)
self.rfile.read(2)
chunks.append(payload)
peer.started_upload.set()
else:
remaining = int(self.headers.get("Content-Length", 0))
while remaining:
payload = self.rfile.read(min(remaining, 1024))
if not payload:
break
chunks.append(payload)
remaining -= len(payload)
peer.started_upload.set()
return b"".join(chunks)
def do_POST(self):
self.handle_method()
def do_PUT(self):
self.handle_method()
def do_HEAD(self):
self.handle_method()
def do_GET(self):
self.handle_method()
def handle_method(self):
try:
self.route()
except (BrokenPipeError, ConnectionResetError, socket.timeout):
self.close_connection = True
def route(self):
parsed = urlsplit(self.path)
path, query = parsed.path, parse_qs(parsed.query)
with peer.lock:
peer.counts[(self.command, path)] += 1
count = peer.counts[(self.command, path)]
body = self.upload() if self.command in ("POST", "PUT") or self.headers.get("Content-Length") or self.headers.get("Transfer-Encoding") else b""
if path == "/no-replay":
self.close_connection = True
self.connection.shutdown(socket.SHUT_RDWR)
self.connection.close()
return
if path in ("/echo", "/stream-upload"):
data = {"method": self.command, "path": self.path,
"headers": {key.lower(): value for key, value in self.headers.items()},
"body_bytes": len(body), "body_sha256": hashlib.sha256(body).hexdigest()}
with peer.lock:
peer.received[path] = data
self.emit(200, json.dumps(data).encode(), {"Content-Type": "application/json",
"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
return
if path == "/stream-download":
self.send_response(200)
self.send_header("Transfer-Encoding", "chunked")
self.send_header("Content-Type", "application/octet-stream")
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(b"5\r\nfirst\r\n")
self.wfile.flush()
peer.release_stream.wait(5)
self.wfile.write(b"4\r\nlast\r\n0\r\n\r\n")
self.wfile.flush()
return
if path == "/ws":
key = self.headers.get("Sec-WebSocket-Key", "")
accept = base64.b64encode(hashlib.sha1(
(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").encode()).digest()).decode()
self.send_response(101)
self.send_header("Upgrade", "websocket")
self.send_header("Connection", "Upgrade")
self.send_header("Sec-WebSocket-Accept", accept)
self.end_headers()
self.wfile.flush()
frame = self.rfile.read(2)
if len(frame) == 2:
length = frame[1] & 127
if length == 126:
length = struct.unpack("!H", self.rfile.read(2))[0]
mask = self.rfile.read(4) if frame[1] & 128 else b"\0\0\0\0"
data = self.rfile.read(length)
data = bytes(byte ^ mask[i % 4] for i, byte in enumerate(data))
self.wfile.write(bytes([0x81, len(data)]) + data)
self.wfile.flush()
self.close_connection = True
return
if path.startswith("/status/"):
self.emit(int(path.rsplit("/", 1)[1]), b"origin status payload\n",
{"Cache-Control": "no-store", "X-Origin-Fixture": "yes"})
return
if path == "/redirect":
self.emit(302, b"redirect body\n", {"Location": "/small.html", "Cache-Control": "no-store"})
return
if path in ("/private", "/cookie", "/no-store", "/vary", "/fresh", "/revalidate", "/concurrent"):
cache_control = "public, max-age=60"
headers = {"Content-Type": "text/plain"}
if body:
headers["X-Received-Body-Bytes"] = str(len(body))
if path == "/private":
cache_control = "private, max-age=60"
if path == "/no-store":
cache_control = "no-store"
if path == "/cookie":
headers["Set-Cookie"] = "application-session=fixture; Path=/"
if path == "/vary":
headers["Vary"] = "Accept-Language"
if path == "/concurrent":
time.sleep(.08)
if path == "/revalidate":
cache_control = "public, max-age=1, stale-if-error=10"
headers["ETag"] = '"fixture-v1"'
if self.headers.get("If-None-Match") == headers["ETag"]:
self.emit(304, b"", {**headers, "Cache-Control": cache_control})
return
data = f"{path}:count={count}:lang={self.headers.get('Accept-Language', '')}".encode()
if path == "/revalidate":
data = b"stable revalidated object"
self.emit(200, data, {**headers, "Cache-Control": cache_control})
return
if path == "/no-transform":
self.emit(200, b"unchanged compression fixture " * 1024,
{"Content-Type": "text/plain", "Cache-Control": "no-transform, public, max-age=60"})
return
file = peer.site / path.lstrip("/")
if not file.is_file() or ".." in file.parts:
self.emit(404, b"fixture not found")
return
payload = file.read_bytes()
tag = '"' + hashlib.sha256(payload).hexdigest() + '"'
headers = {"Cache-Control": "public, max-age=60", "ETag": tag,
"Accept-Ranges": "bytes", "X-Origin-Fixture": "yes"}
headers["Content-Type"] = {".html": "text/html", ".txt": "text/plain",
".css": "text/css", ".json": "application/json"}.get(file.suffix, "application/octet-stream")
if self.headers.get("If-None-Match") == tag:
self.emit(304, b"", headers)
return
range_value = self.headers.get("Range")
if range_value and self.headers.get("If-Range", tag) == tag:
try:
first, last = range_value.removeprefix("bytes=").split("-", 1)
start = int(first) if first else max(0, len(payload) - int(last))
end = min(int(last), len(payload) - 1) if first and last else len(payload) - 1
if start >= len(payload) or end < start:
self.emit(416, b"", {**headers, "Content-Range": f"bytes */{len(payload)}"})
else:
self.emit(206, payload[start:end + 1], {**headers,
"Content-Range": f"bytes {start}-{end}/{len(payload)}"})
return
except ValueError:
pass
self.emit(200, payload, headers)
self.server = ThreadingHTTPServer(("127.0.0.1", port), Handler)
self.server.daemon_threads = True
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
def count(self, method, path):
with self.lock:
return self.counts[(method, path)]
def close(self):
self.release_stream.set()
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=2)
scripts/performance/profile_syscalls.py
-284
View File
@@ -1,284 +0,0 @@
#!/usr/bin/env python3
"""Profile selected server syscalls separately from throughput measurements.
Runs each RustProxy binary sequentially against a disposable loopback fixture.
The measured operation is one persistent HTTP/1.1 connection requesting the
64-KiB static file with cache disabled and Accept-Encoding: identity.
Every response is validated; syscall counts include background server work.
strace elapsed/CPU times must not be interpreted as uninstrumented performance.
"""
from __future__ import annotations
import argparse
import hashlib
from http.client import HTTPConnection
import json
import os
from pathlib import Path
import platform
import re
import shutil
import signal
import subprocess
import sys
import time
import traceback
from common import Fixture, seal_results, sha256, write_json
from resources import descendants
SYSCALLS = ("read", "pread64", "write", "writev", "openat", "close", "fcntl", "lseek", "futex")
REQUEST_PATH = "/large.bin"
def thread_tracers(pid: int) -> dict[int, int]:
result = {}
for task in Path(f"/proc/{pid}/task").iterdir():
try:
status = (task / "status").read_text()
except FileNotFoundError:
continue # A thread may exit between the directory read and status.
match = re.search(r"^TracerPid:\s+(\d+)$", status, re.MULTILINE)
if match:
result[int(task.name)] = int(match.group(1))
return result
def wait_attached(tracer, server, timeout=10):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if tracer.poll() is not None:
raise RuntimeError("strace exited before attachment; see strace diagnostic log (ptrace may be unavailable)")
if server.poll() is not None:
raise RuntimeError("server exited during strace attachment")
traced = thread_tracers(server.pid)
if traced and all(traced.values()):
return sorted(traced)
time.sleep(.01)
raise RuntimeError("strace did not attach to every server thread within 10 seconds")
def detach(tracer, server):
# Do not signal the process group: only the independently started tracer.
if tracer.poll() is None:
tracer.send_signal(signal.SIGINT)
try:
tracer.wait(timeout=10)
except subprocess.TimeoutExpired:
tracer.terminate()
try:
tracer.wait(timeout=3)
except subprocess.TimeoutExpired:
tracer.kill()
tracer.wait(timeout=3)
raise RuntimeError("strace failed to finish on SIGINT; profile is incomplete")
if server.poll() is not None:
raise RuntimeError("server exited while the tracer was detached")
traced = thread_tracers(server.pid)
if not traced or any(traced.values()):
raise RuntimeError("server still has traced threads after strace exited")
return {"signal": "SIGINT to strace PID only", "strace_returncode": tracer.returncode,
"server_alive": True, "remaining_traced_threads": 0}
def parse_summary(path: Path, requests: int):
rows = {name: {"calls": 0, "errors": 0, "calls_per_request": 0.0}
for name in SYSCALLS}
total = None
for line in path.read_text().splitlines():
fields = line.split()
if len(fields) not in (5, 6) or fields[-1] not in (*SYSCALLS, "total"):
continue
percent, seconds, usecs, calls = map(float, fields[:4])
errors = int(fields[4]) if len(fields) == 6 else 0
row = {"calls": int(calls), "errors": errors,
"calls_per_request": int(calls) / requests,
"strace_system_seconds": seconds, "strace_percent_time": percent,
"strace_us_per_call": usecs}
if fields[-1] == "total":
total = row
else:
rows[fields[-1]] = row
if total is None:
raise RuntimeError(f"no complete strace summary found in {path}")
if sum(row["calls"] for row in rows.values()) != total["calls"]:
raise RuntimeError("parsed syscall counts do not match the raw strace total")
return {"selected_syscalls": rows, "selected_total": total}
def request_checked(connection, expected_socket, expected_sha, phase, sequence, evidence):
if connection.sock is not expected_socket:
raise RuntimeError("keepalive connection was replaced before the next request")
connection.request("GET", REQUEST_PATH,
headers={"Accept-Encoding": "identity", "Connection": "keep-alive"})
response = connection.getresponse()
payload = response.read()
digest = hashlib.sha256(payload).hexdigest()
headers = {key.lower(): value for key, value in response.getheaders()}
valid = (response.status == 200 and len(payload) == 65536 and digest == expected_sha
and headers.get("content-length") == "65536"
and headers.get("content-encoding", "identity") == "identity"
and connection.sock is expected_socket and not response.will_close)
result = {"phase": phase, "sequence": sequence, "status": response.status,
"bytes": len(payload), "sha256": digest,
"content_length": headers.get("content-length"),
"content_encoding": headers.get("content-encoding"),
"connection_reused": connection.sock is expected_socket,
"will_close": response.will_close, "passed": valid}
evidence.write(json.dumps(result, sort_keys=True) + "\n")
evidence.flush()
if not valid:
raise RuntimeError(f"response verification failed in {phase} request {sequence}: {result}")
def profile_variant(fixture, label, binary, args):
rust = fixture.rust(binary, label)
rust.login()
rust.wait_quiescent()
raw = args.out / "raw"
raw.mkdir(exist_ok=True)
summary_path = raw / f"{label}.strace.txt"
diagnostic_path = raw / f"{label}.strace.log"
responses_path = raw / f"{label}.responses.jsonl"
expected_sha = sha256(fixture.site / "large.bin")
quiescence_path = fixture.results / f"{label}-quiescence.json"
config_path = fixture.results / "configs" / f"{label}-base.json"
config_sha = sha256(config_path)
if config_sha != rust.applies[-1]["sha256"]:
raise RuntimeError("saved fixture configuration differs from the applied configuration")
command = [str(args.strace), "-f", "-c", "-e", "trace=" + ",".join(SYSCALLS),
"-p", str(rust.process.pid), "-o", str(summary_path)]
connection = HTTPConnection("127.0.0.1", rust.static_port, timeout=args.timeout)
tracer = None
result = {"variant": label, "binary": str(binary), "binary_sha256": sha256(binary),
"config": str(config_path.relative_to(args.out)), "config_sha256": config_sha,
"quiescence": str(quiescence_path.relative_to(args.out)),
"quiescence_sha256": sha256(quiescence_path),
"pid": rust.process.pid, "request_path": REQUEST_PATH, "payload_bytes": 65536,
"payload_sha256": expected_sha, "cache_enabled": False,
"request_encoding": "identity", "concurrency": 1,
"warmup_requests": args.warmup, "profile_requests": args.requests,
"strace_command": command, "raw_summary": str(summary_path.relative_to(args.out)),
"raw_diagnostics": str(diagnostic_path.relative_to(args.out)),
"responses": str(responses_path.relative_to(args.out)), "passed": False}
try:
with responses_path.open("w") as responses, diagnostic_path.open("w") as diagnostics:
connection.connect()
expected_socket = connection.sock
result["client_local_address"] = expected_socket.getsockname()
for index in range(args.warmup):
request_checked(connection, expected_socket, expected_sha, "warmup", index + 1, responses)
result["processes_before_attach"] = descendants(rust.process.pid)
if result["processes_before_attach"] != [rust.process.pid]:
raise RuntimeError("server spawned a child after quiescence; refusing contaminated syscall trace")
environment = os.environ.copy()
environment["LC_ALL"] = "C"
tracer = subprocess.Popen(command, stdout=diagnostics, stderr=diagnostics,
env=environment, start_new_session=True)
result["initial_traced_threads"] = wait_attached(tracer, rust.process)
started = time.monotonic()
try:
for index in range(args.requests):
request_checked(connection, expected_socket, expected_sha, "profile", index + 1, responses)
if tracer.poll() is not None:
raise RuntimeError("strace exited before all profile requests completed")
finally:
result["instrumented_request_loop_seconds"] = time.monotonic() - started
result["detach"] = detach(tracer, rust.process)
tracer = None
# An actual HTTP response after detaching proves the server stayed usable.
request_checked(connection, expected_socket, expected_sha, "after_detach", 1, responses)
result["processes_after_detach"] = descendants(rust.process.pid)
if result["processes_after_detach"] != [rust.process.pid]:
raise RuntimeError("server has a child after profiling; syscall trace needs review")
result.update(parse_summary(summary_path, args.requests))
result["verified_responses"] = args.requests + args.warmup + 1
result["passed"] = True
return result
finally:
if tracer is not None:
detach(tracer, rust.process)
connection.close()
fixture.stop(rust.process)
write_json(args.out / f"{label}.json", result)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--nginx", type=Path, required=True, help="nginx executable for the shared fixture origin")
parser.add_argument("--strace", type=Path, default=shutil.which("strace"))
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--requests", type=int, default=500)
parser.add_argument("--warmup", type=int, default=20)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=4)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--timeout", type=float, default=20)
args = parser.parse_args()
if args.requests < 1 or args.warmup < 0 or args.workers < 1 or args.origin_workers < 1 or args.timeout <= 0:
parser.error("requests/workers/timeout must be positive, warmup must be nonnegative")
if args.strace is None:
parser.error("strace is required; pass --strace /absolute/path")
if args.out.exists() and (not args.out.is_dir() or any(args.out.iterdir())):
parser.error("--out must be a new or empty directory; existing evidence is never overwritten")
variants = {}
for value in args.binary:
label, separator, filename = value.partition("=")
if not separator or not re.fullmatch(r"[A-Za-z][A-Za-z0-9_-]{0,48}", label) or label in variants:
parser.error("each --binary must use a unique filesystem-safe LABEL=/ABS/PATH")
binary = Path(filename).resolve()
if not binary.is_file():
parser.error(f"binary does not exist: {binary}")
variants[label] = binary
args.out, args.nginx, args.strace = args.out.resolve(), args.nginx.resolve(), args.strace.resolve()
for executable in (args.nginx, args.strace):
if not executable.is_file() or not os.access(executable, os.X_OK):
parser.error(f"executable is missing or is not executable: {executable}")
args.out.mkdir(parents=True, exist_ok=True)
version = subprocess.run([str(args.strace), "-V"], capture_output=True, text=True, timeout=10)
environment = {"kind": "selected syscall counts with ptrace instrumentation", "simulation": False,
"platform": platform.platform(), "python": sys.version, "strace_version": version.stdout + version.stderr,
"strace_sha256": sha256(args.strace), "nginx_sha256": sha256(args.nginx),
"variant_order": list(variants), "selected_syscalls": SYSCALLS,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"settings": {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()},
"limitations": ["strace changes scheduling and latency; do not use instrumented times as RPS evidence",
"Counts cover selected syscalls in every traced server thread, including background work",
"Normalized counts divide the whole trace by verified profile requests; they do not isolate per-request causality",
"This is a warm-file-cache, c1, HTTP/1.1 plaintext static-file scenario; no general capacity inference",
"Syscall counts do not directly count Tokio task dispatches or memory allocations"]}
for path in ("/proc/sys/kernel/yama/ptrace_scope", "/sys/fs/cgroup/cpu.max", "/sys/fs/cgroup/memory.max"):
try:
environment[path] = Path(path).read_text().strip()
except OSError:
environment[path] = None
write_json(args.out / "environment.json", environment)
rows, error = [], None
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
fixture.nginx(args.nginx, frontend=False)
for label, binary in variants.items():
result = profile_variant(fixture, label, binary, args)
rows.append(result)
write_json(args.out / "profiles.json", rows)
print(json.dumps({"variant": label, "requests": args.requests, "passed": result["passed"],
"selected_syscall_calls": result["selected_total"]["calls"]}), flush=True)
except Exception as exc:
error = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", error)
print(json.dumps(error), flush=True)
finally:
write_json(args.out / "profiles.json", rows)
write_json(args.out / "completion.json", {"completed": error is None, "profiles": len(rows),
"simulation": False, "throughput_measurement": False,
"finished_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())})
seal_results(args.out)
return 1 if error else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/regression.py
-107
View File
@@ -1,107 +0,0 @@
#!/usr/bin/env python3
"""Run black-box acceptance tests against one or more real RustProxy binaries.
Example:
python3 scripts/performance/regression.py --binary before=/abs/before \
--binary after=/abs/after --out /abs/results/regression
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import platform
import sys
import time
import traceback
from common import Fixture, seal_results, write_json
from peer import Peer
import checks_cache
import checks_policy
import checks_proxy
import checks_static
GROUPS = {"static": checks_static.run, "proxy": checks_proxy.run,
"policy": checks_policy.run, "cache": checks_cache.run,
"transform": checks_cache.transforms}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", required=True, type=Path)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--groups", default=",".join(GROUPS))
args = parser.parse_args()
groups = args.groups.split(",")
if any(group not in GROUPS for group in groups):
parser.error(f"groups must be selected from {','.join(GROUPS)}")
args.out = args.out.resolve()
args.out.mkdir(parents=True, exist_ok=True)
metadata = {"kind": "live RustProxy black-box HTTP acceptance", "simulation": False,
"python": sys.version, "platform": platform.platform(),
"cpu_affinity": sorted(os.sched_getaffinity(0)) if hasattr(os, "sched_getaffinity") else None,
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"groups": groups,
"not_covered": ["TLS/ALPN", "HTTP/2", "HTTP/3", "PHP-FPM",
"Tinyauth verifier", "real DNS rebinding", "24-hour soak",
"concurrent filesystem rename-race proof"]}
write_json(args.out / "environment.json", metadata)
reports = {}
for binary_arg in args.binary:
label, separator, name = binary_arg.partition("=")
if not separator or not label.replace("-", "").replace("_", "").isalnum():
parser.error("--binary must be LABEL=/ABS/PATH with a simple unique label")
if label in reports:
parser.error("binary labels must be unique")
cases = []
started = time.monotonic()
output = args.out / label
def check(case, passed, **details):
row = {"case": case, "passed": passed, **details}
cases.append(row)
print(f"{label} {'SKIP' if passed is None else 'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(output / "checks.json", cases)
try:
with Fixture(output, workers=args.workers) as fixture:
peer = Peer(fixture.origin_port, fixture.site)
try:
rust = fixture.rust(Path(name), label)
check("runtime.started", True)
for group in groups:
try:
rust.reset()
GROUPS[group](check, rust, peer)
except Exception as exc:
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
finally:
rust.reset()
check("runtime.still_alive", rust.process.poll() is None)
finally:
peer.close()
except Exception as exc:
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"label": label, "kind": metadata["kind"], "simulation": False,
"seconds": time.monotonic() - started, "passed": sum(c["passed"] is True for c in cases),
"failed": sum(c["passed"] is False for c in cases),
"skipped": sum(c["passed"] is None for c in cases), "cases": cases}
reports[label] = report
write_json(output / "report.json", report)
print(f"{label}: {report['passed']} passed, {report['failed']} failed, {report['skipped']} skipped", flush=True)
comparison = {"variants": {name: {key: report[key] for key in ["passed", "failed", "skipped", "seconds"]}
for name, report in reports.items()},
"cases": [{"case": case, **{name: next((entry["passed"] for entry in report["cases"]
if entry["case"] == case), "not_run") for name, report in reports.items()}}
for case in sorted({entry["case"] for report in reports.values() for entry in report["cases"]})]}
write_json(args.out / "comparison.json", comparison)
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/resources.py
-228
View File
@@ -1,228 +0,0 @@
"""Linux /proc sampling outside the server process; no synthetic resource data."""
from __future__ import annotations
import os
from pathlib import Path
import threading
import time
def cgroup_cpu_stat(path=Path("/sys/fs/cgroup/cpu.stat")):
values = {key: int(value) for key, value in (line.split() for line in path.read_text().splitlines())}
if not {"nr_throttled", "throttled_usec"} <= values.keys():
raise RuntimeError(f"{path} lacks cgroup v2 throttling counters")
return {"path": str(path), "observed_monotonic": time.monotonic(), "values": values}
def cgroup_cpu_delta(before, after):
delta = {key: after["values"][key] - value for key, value in before["values"].items()
if key in after["values"]}
if any(value < 0 for value in delta.values()):
raise RuntimeError("cgroup CPU counters reset during the measured run")
return delta
def parse_cpu_set(value):
if not value:
raise ValueError("an explicit, nonempty CPU set is required")
result = set()
for part in value.split(","):
bounds = part.strip().split("-")
if len(bounds) not in (1, 2) or any(not item.isascii() or not item.isdigit() for item in bounds):
raise ValueError(f"invalid CPU set {value!r}; use comma-separated numbers or ranges")
start, end = int(bounds[0]), int(bounds[-1])
if start > end or end - start > 65536:
raise ValueError(f"invalid CPU range {part!r}")
result.update(range(start, end + 1))
return result
def validate_cpu_sets(values, allowed):
groups = {label: parse_cpu_set(value) for label, value in values.items()}
for label, cpus in groups.items():
if not cpus <= allowed:
raise ValueError(f"{label} CPUs {sorted(cpus - allowed)} are outside the allowed affinity")
for other, other_cpus in groups.items():
if label < other and cpus & other_cpus:
raise ValueError(f"{label} and {other} CPU sets overlap: {sorted(cpus & other_cpus)}")
return groups
def affinity_snapshot(root, expected, expected_processes=None):
"""Verify every currently visible thread in the root process and children."""
tasks = []
for pid in descendants(root):
try:
threads = list(Path(f"/proc/{pid}/task").iterdir())
except FileNotFoundError:
continue
for thread in threads:
try:
actual = set(os.sched_getaffinity(int(thread.name)))
except ProcessLookupError:
continue
if actual != expected:
raise RuntimeError(f"affinity mismatch for PID {pid} TID {thread.name}: "
f"expected {sorted(expected)}, got {sorted(actual)}")
tasks.append({"pid": pid, "tid": int(thread.name), "cpus": sorted(actual)})
if not tasks or not any(task["pid"] == root for task in tasks):
raise RuntimeError(f"cannot inspect affinity of live process {root}")
processes = len({task["pid"] for task in tasks})
if expected_processes is not None and processes != expected_processes:
raise RuntimeError(f"process count mismatch for PID {root}: "
f"expected {expected_processes}, inspected {processes}")
return {"root_pid": root, "processes": processes,
"threads": len(tasks), "tasks": tasks}
def wait_for_affinity(process, expected, timeout=2):
"""Allow taskset to execute before inspecting the generator's inherited mask."""
deadline = time.monotonic() + timeout
while True:
if process.poll() is not None:
raise RuntimeError(f"generator exited before affinity verification: {process.returncode}")
try:
return affinity_snapshot(process.pid, expected)
except RuntimeError:
if time.monotonic() >= deadline:
raise
time.sleep(.01)
def descendants(pid, proc_root=Path("/proc")):
"""Find the process tree through stat PPids, including nginx workers.
/proc/PID/task/PID/children is optional on some Linux kernels and may also
omit children created by another thread. Missing that file is not evidence
that a live process is a leaf. A single PPid-map scan avoids either case.
"""
children, seen = {}, set()
for entry in proc_root.iterdir():
if not entry.name.isdecimal():
continue
current = int(entry.name)
try:
fields = (entry / "stat").read_text().rsplit(")", 1)[1].split()
parent = int(fields[1])
except (FileNotFoundError, ProcessLookupError):
# A non-root process may finish while the snapshot is assembled.
continue
except (OSError, ValueError, IndexError) as error:
raise RuntimeError(f"cannot discover process tree from PID {current}: {error}") from error
if fields[0] in {"Z", "X", "x"}:
# A dead/zombie worker cannot satisfy the configured live count.
continue
seen.add(current)
children.setdefault(parent, []).append(current)
if pid not in seen:
raise RuntimeError(f"no readable live root process for PID {pid}")
result, pending = [], [pid]
while pending:
current = pending.pop()
if current in result:
continue
result.append(current)
pending.extend(sorted(children.get(current, []), reverse=True))
return result
def process_stats(pid):
try:
data = Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()
if data[0] in {"Z", "X", "x"}:
return None
ticks = os.sysconf("SC_CLK_TCK")
pages = os.sysconf("SC_PAGE_SIZE")
return {"pid": pid, "start_ticks": int(data[19]),
"cpu_seconds": (int(data[11]) + int(data[12])) / ticks,
"rss_bytes": int(data[21]) * pages,
"fd": len(list(Path(f"/proc/{pid}/fd").iterdir())),
"threads": int(data[17])}
except (FileNotFoundError, ProcessLookupError):
return None
except (OSError, ValueError, IndexError) as error:
raise RuntimeError(f"cannot read resource counters for PID {pid}: {error}") from error
class Monitor:
def __init__(self, roots, interval=.5, expected_processes=None):
self.roots = roots
self.expected_processes = expected_processes or {}
self.process_identities = {}
self.interval = interval
if interval <= 0:
raise ValueError("resource interval must be positive")
self.samples = []
self.errors = []
self.stop_event = threading.Event()
self.started = time.monotonic()
self.thread = threading.Thread(target=self.run, daemon=True)
def sample(self):
groups = {}
for label, root in self.roots.items():
processes = [stats for pid in descendants(root) if (stats := process_stats(pid)) is not None]
if not any(process["pid"] == root for process in processes):
raise RuntimeError(f"no readable live root process for {label} (PID {root})")
expected = self.expected_processes.get(label)
if expected is not None and len(processes) != expected:
raise RuntimeError(f"resource process count mismatch for {label} (PID {root}): "
f"expected {expected}, read {len(processes)}")
identities = {(process["pid"], process["start_ticks"]) for process in processes}
if expected is not None:
original = self.process_identities.setdefault(label, identities)
if identities != original:
raise RuntimeError(f"process identities changed while measuring {label}: "
f"expected {sorted(original)}, read {sorted(identities)}")
groups[label] = {"processes": processes,
"rss_bytes": sum(p["rss_bytes"] for p in processes),
"fd": sum(p["fd"] for p in processes),
"threads": sum(p["threads"] for p in processes)}
self.samples.append({"seconds": time.monotonic() - self.started, "groups": groups})
def run(self):
while not self.stop_event.wait(self.interval):
try:
self.sample()
except Exception as error:
self.errors.append(str(error))
self.stop_event.set()
def start(self):
self.sample()
self.thread.start()
def stop(self):
self.stop_event.set()
self.thread.join(timeout=3)
if self.thread.is_alive():
self.errors.append("resource sampler did not stop")
try:
self.sample()
except Exception as error:
self.errors.append(str(error))
def summary(self, requests):
if self.errors:
raise RuntimeError("resource sampling failed: " + "; ".join(self.errors))
result = {}
for label in self.roots:
observations = [sample["groups"][label] for sample in self.samples]
totals = {}
for group in observations:
for process in group["processes"]:
key = (process["pid"], process["start_ticks"])
low, high = totals.get(key, (process["cpu_seconds"], process["cpu_seconds"]))
totals[key] = (min(low, process["cpu_seconds"]), max(high, process["cpu_seconds"]))
if not totals:
raise RuntimeError(f"no readable process samples for {label}; refusing zero resource metrics")
seconds = sum(high - low for low, high in totals.values())
result[label] = {"cpu_seconds": seconds,
"cpu_us_per_request": seconds * 1_000_000 / requests if requests else None,
"rss_peak_bytes": max(group["rss_bytes"] for group in observations),
"rss_start_bytes": observations[0]["rss_bytes"],
"rss_end_bytes": observations[-1]["rss_bytes"],
"fd_peak": max(group["fd"] for group in observations),
"fd_start": observations[0]["fd"], "fd_end": observations[-1]["fd"],
"processes_seen": len(totals)}
return result
scripts/performance/simulate_cache_034.py
-112
View File
@@ -1,112 +0,0 @@
#!/usr/bin/env python3
"""Deterministic analytical/synthetic cache-retention scenario, not a RustProxy load test.
Model: 20 vhosts with 100 HOT + 100 COLD uniformly requested objects each.
A new configuration is saved 1,000 times. Eighty percent of saves edit only
keepalive/LB operational controls, and 20% edit content policy on one vhost.
Every save is followed by 500 uniform GETs; every object is public/cacheable and
has infinite TTL. A miss immediately inserts the object in its original tier.
Capacity is assumed adequate and there is no origin latency or concurrent IO.
The baseline flushes both tiers globally on every save, as in 0.33.8.
The candidate changes only the affected vhost after content edits.
"""
import argparse
import json
import random
def simulate(vhosts=20, objects_per_tier=100, saves=1000, requests_per_save=500, seed=3400):
if min(vhosts, objects_per_tier, saves, requests_per_save) < 1:
raise ValueError("all counts must be positive")
rng = random.Random(seed)
objects = vhosts * objects_per_tier * 2
old = set(range(objects))
new = set(range(objects))
old_hits = new_hits = old_misses = new_misses = 0
old_evicted = new_evicted = 0
edits_operational = edits_content = 0
for step in range(saves):
changed_vhost = rng.randrange(vhosts)
content_edit = step % 5 == 0
if content_edit:
edits_content += 1
else:
edits_operational += 1
old_evicted += len(old)
old.clear()
if content_edit:
low = changed_vhost * objects_per_tier * 2
high = low + objects_per_tier * 2
removed = set(range(low, high)) & new
new_evicted += len(removed)
new.difference_update(removed)
for _ in range(requests_per_save):
object_id = rng.randrange(objects)
if object_id in old:
old_hits += 1
else:
old_misses += 1
old.add(object_id)
if object_id in new:
new_hits += 1
else:
new_misses += 1
new.add(object_id)
total = saves * requests_per_save
assert old_hits + old_misses == total
assert new_hits + new_misses == total
assert new_evicted <= old_evicted
assert new_misses <= old_misses
return {
"scenario": {
"vhosts": vhosts,
"hot_objects_per_vhost": objects_per_tier,
"cold_objects_per_vhost": objects_per_tier,
"saves": saves,
"operational_saves": edits_operational,
"single_vhost_content_saves": edits_content,
"gets_per_save": requests_per_save,
"total_gets": total,
"seed": seed,
"assumptions": "All objects public/cacheable, infinite TTL, instantly refilled, uniform GETs; no bandwidth/CPU/origin timings",
},
"legacy_global_purge": {
"invalidated_objects": old_evicted,
"misses": old_misses,
"hits": old_hits,
"hit_ratio_percent": round(100 * old_hits / total, 3),
},
"scoped_cache": {
"invalidated_objects": new_evicted,
"misses": new_misses,
"hits": new_hits,
"hit_ratio_percent": round(100 * new_hits / total, 3),
},
"improvement": {
"fewer_invalidations_percent": round(100 * (1 - new_evicted / old_evicted), 3) if old_evicted else 0,
"fewer_cache_misses_percent": round(100 * (1 - new_misses / old_misses), 3) if old_misses else 0,
"hit_ratio_gain_percentage_points": round(100 * (new_hits - old_hits) / total, 3),
"observed_rps_gain_percent": None,
},
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--vhosts", type=int, default=20)
parser.add_argument("--objects-per-tier", type=int, default=100)
parser.add_argument("--saves", type=int, default=1000)
parser.add_argument("--gets-per-save", type=int, default=500)
parser.add_argument("--seed", type=int, default=3400)
parser.add_argument("--output", help="optional JSON results file")
args = parser.parse_args()
result = simulate(args.vhosts, args.objects_per_tier, args.saves, args.gets_per_save, args.seed)
encoded = json.dumps(result, indent=2, sort_keys=True)
if args.output:
from pathlib import Path
Path(args.output).write_text(encoded + "\n")
print(encoded)
if __name__ == "__main__":
main()
scripts/performance/simulate_nginx_gap_034.py
-98
View File
@@ -1,98 +0,0 @@
#!/usr/bin/env python3
"""Amdahl sensitivity study only: 0.33.8 measured baseline, 0.34.0 not measured.
No synthetic RPS is described as benchmarked RustProxy throughput. Amdahl uses
RPS_new = RPS_old / (1 - eligible_cost_fraction * fraction_eliminated).
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
# Verified medians reproduced from the supplied 0.33.8 evidence. Cache OFF.
BASELINE = [
("reverse_proxy", 1040, 29334.12, 170610.16, False),
("reverse_proxy", 65536, 20735.07, 35393.29, False),
("reverse_proxy", 1048576, 4461.19, 2730.23, False),
("www", 1040, 24906.20, 351605.13, False),
("www", 65536, 20856.67, 203709.91, True),
("www", 1048576, 2833.71, 12611.49, True),
]
# WWW scenarios: share of per-request service time due to file copy/read;
# assume sendfile removes HALF of that part, not the full service time.
WWW_SHARES = {"low": 0.20, "middle": 0.40, "high": 0.60}
COPY_COST_REMOVED = 0.50
# Proxy independent optimization: hypothetical reduction in *total* request
# service time, modeling removal of redundant response-frame mutex, nothing else.
PROXY_TIME_SAVED = {"low": 0.01, "middle": 0.03, "high": 0.05}
def run() -> dict:
items = []
for kind, payload_bytes, rust_rps, nginx_rps, eligible in BASELINE:
baseline_ratio = rust_rps / nginx_rps
row = {
"kind": kind,
"bytes": payload_bytes,
"baseline_version": "0.33.8",
"baseline_rust_rps": rust_rps,
"baseline_nginx_rps": nginx_rps,
"baseline_of_nginx_pct": round(100 * baseline_ratio, 2),
"factor_to_match_nginx": round(nginx_rps / rust_rps, 2),
"sendfile_eligible_only_if_plain_linux_http1_1_no_cache_no_compression": eligible,
"scenarios": {},
}
for level in ("low", "middle", "high"):
if kind == "www" and eligible:
saved_fraction = WWW_SHARES[level] * COPY_COST_REMOVED
elif kind == "reverse_proxy":
saved_fraction = PROXY_TIME_SAVED[level]
else:
saved_fraction = 0.0
simulated_rps = rust_rps / (1.0 - saved_fraction)
row["scenarios"][level] = {
"modeled_total_time_saved_pct": round(100 * saved_fraction, 2),
"simulated_rps": round(simulated_rps),
"simulated_rps_gain_pct": round(100 * (simulated_rps / rust_rps - 1), 2),
"simulated_of_nginx_pct": round(100 * simulated_rps / nginx_rps, 2),
"simulated_remaining_gap_to_nginx_pct_points": round(
100 * (1 - simulated_rps / nginx_rps), 2
),
}
items.append(row)
return {
"kind": "HYPOTHETICAL_AMDAHL_SENSITIVITY_NOT_A_BENCHMARK",
"comparison": "Existing 0.33.8 measurements (5 runs, cache off) vs measured nginx",
"not_measured": "0.34.0 server performance and sendfile syscall/runtime correctness",
"conditions": [
"Identical CPU and network assumptions to supplied 0.33.8 baseline",
"WWW zero-copy is limited to Linux, cleartext HTTP/1.1, GET >=64KiB, no cache, range, transform, compression or TLS",
"WWW low/middle/high: file-copy contributes 20%/40%/60% of total request time; modeled sendfile eliminates 50% of that component",
"Reverse proxy low/middle/high: removing response-frame mutex saves a hypothetical 1%/3%/5% of total service time",
"These fractions are chosen sensitivity parameters; not measured or inferred from profiling",
"No change is assumed for WWW 1040B or out-of-scope protocol paths",
"No network saturation, TLS record processing, queueing, origin latency, filesystem cache misses, CPU contention or concurrency effects modeled",
"No additional speedup due to cache-retention changes is assumed",
],
"formula": "RPS_sim = RPS_0.33.8 / (1 - fraction_of_total_service_time_eliminated)",
"entries": items,
}
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, help="write JSON to this file")
args = parser.parse_args()
payload = json.dumps(run(), indent=2, ensure_ascii=False) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(payload)
print(f"wrote {args.output}")
else:
print(payload)
if __name__ == "__main__":
main()
scripts/performance/soak.py
-325
View File
@@ -1,325 +0,0 @@
#!/usr/bin/env python3
"""Fixed-rate mixed traffic, controlled client closes and cache hot-toggle soak.
The default is a 30-minute live run. Short invocations must be explicitly marked
--exploratory and cannot satisfy the release soak gate. No run is auto-scheduled.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, wait, FIRST_COMPLETED
import copy
import hashlib
from http.client import HTTPConnection
import json
import os
from pathlib import Path
import platform
import statistics
import sys
import time
import traceback
from benchmark import counter_values
from common import Fixture, http, seal_results, sha256, write_json
from resources import Monitor
from telemetry import traffic_analytics_snapshot, traffic_analytics_delta
PAYLOADS = ["small.html", "large.bin", "huge.bin"]
def fetch(index, rust, expected, abort_every):
name = PAYLOADS[index % len(PAYLOADS)]
source = "proxy" if (index // len(PAYLOADS)) % 2 else "static"
port = rust.reverse_port if source == "proxy" else rust.static_port
started = time.monotonic()
case = {"index": index, "source": source, "file": name, "method": "GET", "intentional_abort": False}
try:
if abort_every and index % abort_every == 0:
case.update({"file": "huge.bin", "intentional_abort": True})
connection = HTTPConnection("127.0.0.1", port, timeout=10)
try:
connection.request("GET", "/huge.bin", headers={"Accept-Encoding": "identity"})
response = connection.getresponse()
body = response.read(128)
case.update({"status": response.status, "bytes": len(body),
"cache": response.getheader("X-Cache"),
"passed": response.status == 200 and body == expected["huge.bin"][:128]})
finally:
connection.close()
else:
headers = {"Accept-Encoding": "identity"}
method = "HEAD" if index % 29 == 0 else "GET"
if method == "GET" and index % 37 == 0:
headers["Range"] = "bytes=0-127"
expected_status, expected_body = 206, expected[name][:128]
else:
expected_status, expected_body = 200, b"" if method == "HEAD" else expected[name]
status, response_headers, body = http(port, "/" + name, method, headers)
case.update({"method": method, "status": status, "bytes": len(body),
"cache": response_headers.get("x-cache"),
"passed": status == expected_status and body == expected_body})
if not case["passed"]:
case.update({"expected_status": expected_status, "expected_bytes": len(expected_body),
"body_sha256": hashlib.sha256(body).hexdigest()})
except Exception as exc:
case.update({"passed": False, "error": str(exc)})
case["seconds"] = time.monotonic() - started
return case
def probe_cache_mode(rust, expected, enabled, phase, out, observations):
"""Record two exact GETs per source, separate from the paced workload."""
phase_observations = []
for source, port in [("proxy", rust.reverse_port), ("static", rust.static_port)]:
replies = []
for attempt in [1, 2]:
try:
status, headers, body = http(port, "/small.html?soak_cache_probe=1",
headers={"Accept-Encoding": "identity"})
replies.append({"attempt": attempt, "status": status, "cache": headers.get("x-cache"),
"bytes": len(body), "sha256": hashlib.sha256(body).hexdigest(),
"body_passed": status == 200 and body == expected["small.html"]})
except Exception as error:
replies.append({"attempt": attempt, "body_passed": False, "error": str(error)})
mode_passed = replies[1].get("cache") == "HIT" if enabled else all(
reply.get("cache") is None for reply in replies)
phase_observations.append({"phase": phase, "source": source, "cache_enabled": enabled,
"responses": replies,
"passed": all(reply["body_passed"] for reply in replies) and mode_passed})
observations.extend(phase_observations)
write_json(out / "cache-probes.json", observations)
if not all(observation["passed"] for observation in phase_observations):
raise RuntimeError(f"cache mode/body probe failed after {phase}; see cache-probes.json")
def backend_drain_snapshot(rust):
status, _, body = rust.admin("/api/status")
if status != 200:
return {"passed": False, "error": f"admin status HTTP {status}"}
backends = []
def walk(value):
if isinstance(value, dict):
if "id" in value and "active" in value and "status" in value and "requests" in value:
backends.append({key: value[key] for key in ("id", "active", "status", "requests")})
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(json.loads(body))
return {"passed": bool(backends) and all(backend["active"] == 0 for backend in backends),
"backends": backends}
def memory_trend(samples, label="server"):
observations = [(sample["seconds"], sample["groups"][label]["rss_bytes"], sample["groups"][label]["fd"])
for sample in samples if sample["seconds"] >= 60]
if len(observations) < 3:
return {"available": False, "reason": "less than three samples after the first minute"}
tail_seconds = observations[-1][0]
early = [row[1] for row in observations if row[0] < observations[0][0] + 60]
late = [row[1] for row in observations if row[0] >= tail_seconds - 60]
x_mean = statistics.mean(row[0] for row in observations)
y_mean = statistics.mean(row[1] for row in observations)
denominator = sum((row[0] - x_mean) ** 2 for row in observations)
slope = sum((row[0] - x_mean) * (row[1] - y_mean) for row in observations) / denominator if denominator else 0
return {"available": True, "rss_first_minute_after_warmup_median": statistics.median(early),
"rss_last_minute_median": statistics.median(late),
"rss_growth_bytes": statistics.median(late) - statistics.median(early),
"rss_linear_bytes_per_hour": slope * 3600, "fd_first": observations[0][2],
"fd_last": observations[-1][2], "fd_peak": max(row[2] for row in observations),
"interpretation": "Descriptive only: cache toggles and allocator retention can change RSS without a leak"}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", type=Path, required=True)
parser.add_argument("--label", default="after")
parser.add_argument("--nginx", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--duration", type=int, default=1800)
parser.add_argument("--rate", type=float, default=100, help="offered requests per second, not a saturation benchmark")
parser.add_argument("--clients", type=int, default=8)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--origin-workers", type=int, default=2)
parser.add_argument("--server-cpus")
parser.add_argument("--origin-cpus")
parser.add_argument("--toggle-seconds", type=int, default=120)
parser.add_argument("--sample-seconds", type=int, default=5)
parser.add_argument("--abort-every", type=int, default=101)
parser.add_argument("--cooldown-seconds", type=int, default=15)
parser.add_argument("--exploratory", action="store_true")
args = parser.parse_args()
if args.duration < 1800 and not args.exploratory:
parser.error("the soak gate requires >=1800 seconds; label short diagnostics with --exploratory")
if args.rate <= 0 or not 1 <= args.clients <= 128 or args.toggle_seconds < 1 or args.sample_seconds < 1:
parser.error("rate, sample/toggle intervals and clients must be positive; clients <=128")
args.out = args.out.resolve()
if args.out.exists() and any(args.out.iterdir()):
parser.error("--out must be a new or empty evidence directory")
args.out.mkdir(parents=True, exist_ok=True)
settings = {key: str(value) if isinstance(value, Path) else value for key, value in vars(args).items()}
write_json(args.out / "environment.json", {
"kind": "actual RustProxy fixed-rate mixed-traffic resource soak", "simulation": False,
"settings": settings, "platform": platform.platform(), "binary_sha256": sha256(args.binary),
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"frontend_connection_policy": "A new HTTP/1.1 TCP connection for each workload request and probe",
"controlled_close_interpretation": "Client closes after reading 128 bytes; the server may already have sent the full body. No server-cancellation count is inferred.",
"traffic_analytics_observation_policy": "Global counters are read before traffic, periodically with status diagnostics, and after final drain. Deltas include cache-mode probes and cooldown, and are separate from HTTP errors.",
"not_covered": ["24-hour soak", "TLS/H2/H3 soak", "multi-backend failures", "PHP-FPM",
"real application write load", "origin DNS/TLS rotation", "production traffic",
"long-lived frontend keepalive connections"]})
totals = {"submitted": 0, "completed": 0, "passed": 0, "failed": 0, "controlled_client_closes": 0,
"response_bytes": 0, "scheduler_late_events": 0, "peak_pending": 0}
failure_examples, progress, toggles, counter_samples = [], [], [], []
traffic_samples, traffic_delta = [], None
cache_probes, cache_status_counts = [], {"proxy": Counter(), "static": Counter()}
drain_snapshot, configuration_restored = None, False
samples, resources = [], {}
fatal, runtime_seconds = None, 0
try:
with Fixture(args.out / "fixture", workers=args.workers, server_cpus=args.server_cpus,
origin_cpus=args.origin_cpus, origin_workers=args.origin_workers) as fixture:
nginx = fixture.nginx(args.nginx, frontend=False)
rust = fixture.rust(args.binary, args.label)
rust.login()
rust.wait_quiescent()
expected = {name: (fixture.site / name).read_bytes() for name in PAYLOADS}
config = copy.deepcopy(rust.base)
config["cache"].update({"expose_status": True})
config["web_services"][0]["cache_control"] = "public, max-age=60"
rust.apply(config, "soak-initial")
probe_cache_mode(rust, expected, False, "initial-off", args.out, cache_probes)
traffic_before = traffic_analytics_snapshot(rust)
traffic_samples.append({"phase": "before", "seconds": 0, "snapshot": traffic_before})
write_json(args.out / "traffic-analytics.json", traffic_samples)
monitor = Monitor({"server": rust.process.pid, "origin": nginx["origin"].pid}, args.sample_seconds)
monitor.start()
start = time.monotonic()
deadline = start + args.duration
next_toggle, next_snapshot, next_submit = start + args.toggle_seconds, start, start
pending = set()
def consume(done):
for future in done:
result = future.result()
totals["completed"] += 1
totals["passed" if result["passed"] else "failed"] += 1
totals["controlled_client_closes"] += int(result["intentional_abort"])
totals["response_bytes"] += result.get("bytes", 0)
cache_status = (result.get("cache") or "absent") if "status" in result else "no_response"
cache_status_counts[result["source"]][cache_status] += 1
if not result["passed"] and len(failure_examples) < 100:
failure_examples.append(result)
try:
with ThreadPoolExecutor(max_workers=args.clients) as pool:
while time.monotonic() < deadline:
now = time.monotonic()
done = {future for future in pending if future.done()}
consume(done)
pending -= done
if now >= next_toggle:
config["cache"]["enabled"] = not config["cache"]["enabled"]
rust.apply(config, f"soak-toggle-{len(toggles) + 1}")
toggles.append({"seconds": time.monotonic() - start, "cache_enabled": config["cache"]["enabled"]})
write_json(args.out / "cache-toggles.json", toggles)
probe_cache_mode(rust, expected, config["cache"]["enabled"],
f"toggle-{len(toggles)}", args.out, cache_probes)
next_toggle += args.toggle_seconds
if now >= next_snapshot:
status, headers, body = rust.admin("/api/status")
if status != 200:
raise RuntimeError(f"admin status became unavailable: HTTP {status}")
counter_samples.append({"seconds": time.monotonic() - start,
"cache_enabled": config["cache"]["enabled"],
"counters": counter_values(json.loads(body))})
traffic_now = traffic_analytics_snapshot(rust)
traffic_delta = traffic_analytics_delta(traffic_before, traffic_now)
traffic_samples.append({"phase": "during", "seconds": time.monotonic() - start,
"snapshot": traffic_now, "delta_from_start": traffic_delta})
row = {"seconds": time.monotonic() - start, **totals,
"cache_enabled": config["cache"]["enabled"], "pending": len(pending),
"observed_cache_status_counts": {source: dict(counts)
for source, counts in cache_status_counts.items()}}
progress.append(row)
write_json(args.out / "progress.json", progress)
write_json(args.out / "status-counters.json", counter_samples)
write_json(args.out / "traffic-analytics.json", traffic_samples)
write_json(args.out / "resources.json", list(monitor.samples))
print(json.dumps(row), flush=True)
next_snapshot = time.monotonic() + args.sample_seconds
now = time.monotonic()
if len(pending) >= args.clients * 2:
done, pending = wait(pending, timeout=.1, return_when=FIRST_COMPLETED)
consume(done)
continue
if now < next_submit:
time.sleep(min(.02, next_submit - now))
continue
if now > next_submit + .25:
totals["scheduler_late_events"] += 1
next_submit = now
totals["submitted"] += 1
pending.add(pool.submit(fetch, totals["submitted"], rust, expected, args.abort_every))
totals["peak_pending"] = max(totals["peak_pending"], len(pending))
next_submit += 1 / args.rate
done, pending = wait(pending, timeout=30)
consume(done)
if pending:
raise RuntimeError("requests failed to settle during bounded soak drain")
runtime_seconds = time.monotonic() - start
rust.reset()
configuration_restored = True
cooldown_until = time.monotonic() + args.cooldown_seconds
while time.monotonic() < cooldown_until:
time.sleep(max(0, min(.5, cooldown_until - time.monotonic())))
probe_cache_mode(rust, expected, False, "restored-off", args.out, cache_probes)
for _ in range(40):
drain_snapshot = backend_drain_snapshot(rust)
if drain_snapshot["passed"]:
break
time.sleep(.05)
write_json(args.out / "backend-drain.json", drain_snapshot)
if not drain_snapshot["passed"]:
raise RuntimeError("backend active exchanges did not drain to zero")
traffic_after = traffic_analytics_snapshot(rust)
traffic_delta = traffic_analytics_delta(traffic_before, traffic_after)
traffic_samples.append({"phase": "after_drain", "seconds": time.monotonic() - start,
"snapshot": traffic_after, "delta_from_start": traffic_delta})
finally:
monitor.stop()
samples = monitor.samples
resources = monitor.summary(totals["completed"])
write_json(args.out / "resources.json", samples)
write_json(args.out / "status-counters.json", counter_samples)
write_json(args.out / "traffic-analytics.json", traffic_samples)
write_json(args.out / "failure-examples.json", failure_examples)
except Exception as exc:
fatal = {"error": str(exc), "traceback": traceback.format_exc()}
write_json(args.out / "failure.json", fatal)
report = {"completed": fatal is None, "simulation": False, "exploratory": args.exploratory,
"duration_seconds": runtime_seconds, "soak_duration_gate": runtime_seconds >= 1800,
"totals": totals, "resources": resources, "memory_trend": memory_trend(samples) if samples else None,
"cache_toggles": len(toggles), "errors": failure_examples,
"traffic_analytics": {"observations": len(traffic_samples), "delta": traffic_delta,
"includes_cache_mode_probes_and_cooldown": True},
"cache_mode_probes": {"source_checks": len(cache_probes),
"requests": sum(len(probe["responses"]) for probe in cache_probes),
"passed": bool(cache_probes) and all(probe["passed"] for probe in cache_probes)},
"observed_cache_status_counts": cache_status_counts,
"configuration_restored": configuration_restored, "backend_drain": drain_snapshot,
"actual_completed_rps": totals["completed"] / runtime_seconds if runtime_seconds else None,
"passed": fatal is None and totals["failed"] == 0 and runtime_seconds >= args.duration}
write_json(args.out / "report.json", report)
seal_results(args.out)
print(json.dumps(report), flush=True)
return 0 if report["passed"] else 1
if __name__ == "__main__":
sys.exit(main())
scripts/performance/telemetry.py
-47
View File
@@ -1,47 +0,0 @@
"""Selected counters from existing authenticated APIs, without traffic records."""
from __future__ import annotations
import json
import time
def traffic_analytics_snapshot(rust, timeout=2):
"""Read global analytics queue loss; never retain paths, IPs or user agents.
queue_dropped is process-global even when the API's vhost query is used.
recorded_requests is asynchronous analytics work, not an HTTP success count.
Call outside a timed load interval so this control-plane read is not charged
to the request path being measured.
"""
status, _, body = rust.admin("/api/traffic", timeout=timeout)
if status != 200:
raise RuntimeError(f"traffic analytics counter capture failed: HTTP {status}")
payload = json.loads(body)
if not isinstance(payload, dict) or payload.get("scope") != "*" or payload.get("since") != "process_start":
raise RuntimeError("traffic analytics requires the global process-start scope")
analytics = payload.get("analytics")
if not isinstance(analytics, dict) or analytics.get("scope") != "*":
raise RuntimeError("traffic analytics response lacks the global queue view")
data = analytics.get("data")
if not isinstance(data, dict):
raise RuntimeError("traffic analytics response lacks the recorded counters")
values = {"queue_capacity": analytics.get("queue_capacity"),
"queue_dropped": analytics.get("queue_dropped"),
"recorded_requests": data.get("requests")}
if any(type(value) is not int or value < 0 for value in values.values()) or not values["queue_capacity"]:
raise RuntimeError("traffic analytics queue counters are absent or malformed")
return {"endpoint": "/api/traffic", "scope": "*", "since": "process_start",
"pid": rust.process.pid, "observed_monotonic": time.monotonic(), **values}
def traffic_analytics_delta(before, after):
"""Keep telemetry loss separate from failed HTTP requests and log rotation."""
for key in ["pid", "endpoint", "scope", "since", "queue_capacity"]:
if before[key] != after[key]:
raise RuntimeError(f"traffic analytics identity changed during capture: {key}")
result = {"queue_dropped_delta": after["queue_dropped"] - before["queue_dropped"],
"recorded_requests_delta": after["recorded_requests"] - before["recorded_requests"],
"interval_seconds": after["observed_monotonic"] - before["observed_monotonic"]}
if any(value < 0 for value in result.values()):
raise RuntimeError("traffic analytics counters or clock reset during capture")
return result
scripts/performance/transport_clients.py
-191
View File
@@ -1,191 +0,0 @@
"""Strict H2/TLS and H3/QUIC clients for local RustProxy acceptance tests.
The generated fixture certificate is explicitly trusted. Neither transport
silently falls back to HTTP/1.1 or disables certificate verification.
"""
from __future__ import annotations
import asyncio
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from pathlib import Path
import ssl
import httpx
from aioquic.asyncio import QuicConnectionProtocol, connect
from aioquic.h3.connection import H3Connection, MessageError
from aioquic.h3.events import DataReceived, HeadersReceived
from aioquic.quic.configuration import QuicConfiguration
from aioquic.quic.events import ConnectionTerminated, ProtocolNegotiated, StreamReset
@dataclass
class Response:
status: int
headers: dict[str, str]
body: bytes
protocol: str
alpn: str | None
stream_id: int | None = None
class H2Client:
def __init__(self, client: httpx.AsyncClient, port: int):
self.client, self.port = client, port
async def request(self, method, path, headers=None, body=None):
async def chunks():
for offset in range(0, len(body), 16384):
yield body[offset:offset + 16384]
await asyncio.sleep(0)
response = await self.client.request(
method, f"https://127.0.0.1:{self.port}{path}",
headers=headers or {}, content=chunks() if body is not None else None,
)
network = response.extensions.get("network_stream")
tls = network.get_extra_info("ssl_object") if network else None
alpn = tls.selected_alpn_protocol() if tls else None
if response.http_version != "HTTP/2" or alpn != "h2":
raise AssertionError(f"H2 required, got {response.http_version}, ALPN={alpn!r}")
return Response(response.status_code, dict(response.headers), response.content,
"HTTP/2", alpn, response.extensions.get("stream_id"))
@dataclass
class Pending:
future: asyncio.Future
status: int = 0
headers: dict[str, str] = field(default_factory=dict)
body: bytearray = field(default_factory=bytearray)
class HeadAwareH3Connection(H3Connection):
"""Supply request-method context missing from aioquic 1.7.0's length check.
RFC 9110 section 8.6 permits HEAD's Content-Length to describe a GET body.
Still reject any actual HEAD DATA; keep aioquic's exact length check for all
other requests. This changes only the test client, never the target server.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.head_streams = set()
def _check_content_length(self, stream):
if stream.stream_id in self.head_streams:
if stream.content_length:
raise MessageError("HEAD response contained DATA")
return
super()._check_content_length(stream)
def validate_head_adapter():
"""Keep adapter checks separate from the real-server acceptance totals."""
from types import SimpleNamespace
connection = HeadAwareH3Connection.__new__(HeadAwareH3Connection)
connection.head_streams = {0}
for stream_id, declared, received, valid in [(0, 1040, 0, True), (0, 1040, 1, False),
(4, 1040, 1040, True), (4, 1040, 0, False)]:
stream = SimpleNamespace(stream_id=stream_id, expected_content_length=declared,
content_length=received)
try:
connection._check_content_length(stream)
accepted = True
except MessageError:
accepted = False
assert accepted == valid, "H3 fixture HEAD adapter changed ordinary framing validation"
return {"cases": 4, "passed": 4, "kind": "test-client self-test, not RustProxy coverage"}
class H3Client(QuicConnectionProtocol):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.http = HeadAwareH3Connection(self._quic)
self.pending: dict[int, Pending] = {}
self.alpn = None
self.authority = None
def quic_event_received(self, event):
if isinstance(event, ProtocolNegotiated):
self.alpn = event.alpn_protocol
if isinstance(event, ConnectionTerminated):
for item in self.pending.values():
if not item.future.done():
item.future.set_exception(ConnectionError(
f"QUIC closed code={event.error_code}: {event.reason_phrase}"))
if isinstance(event, StreamReset):
item = self.pending.get(event.stream_id)
if item is not None and not item.future.done():
item.future.set_exception(ConnectionError(
f"H3 stream reset code={event.error_code}"))
for http_event in self.http.handle_event(event):
if not isinstance(http_event, (HeadersReceived, DataReceived)):
continue
item = self.pending.get(http_event.stream_id)
if item is None or item.future.done():
continue
if isinstance(http_event, HeadersReceived):
for name, value in http_event.headers:
if name == b":status":
item.status = int(value)
elif not name.startswith(b":"):
item.headers[name.decode("ascii").lower()] = value.decode("latin-1")
else:
item.body.extend(http_event.data)
if len(item.body) > 4 * 1024 * 1024:
item.future.set_exception(ValueError("fixture response exceeded 4 MiB bound"))
continue
if http_event.stream_ended:
item.future.set_result(Response(item.status, item.headers, bytes(item.body),
"HTTP/3", self.alpn, http_event.stream_id))
async def request(self, method, path, headers=None, body=None):
if self.alpn != "h3":
raise AssertionError(f"H3 required, got ALPN={self.alpn!r}")
stream_id = self._quic.get_next_available_stream_id()
item = Pending(asyncio.get_running_loop().create_future())
self.pending[stream_id] = item
if method == "HEAD":
self.http.head_streams.add(stream_id)
fields = [(b":method", method.encode("ascii")), (b":scheme", b"https"),
(b":authority", self.authority.encode("ascii")),
(b":path", path.encode("ascii"))]
fields += [(name.lower().encode("ascii"), value.encode("latin-1"))
for name, value in (headers or {}).items()]
try:
self.http.send_headers(stream_id, fields, end_stream=body is None)
self.transmit()
if body is not None:
if not body:
self.http.send_data(stream_id, b"", end_stream=True)
self.transmit()
for offset in range(0, len(body), 16384):
end = min(offset + 16384, len(body))
self.http.send_data(stream_id, body[offset:end], end_stream=end == len(body))
self.transmit()
await asyncio.sleep(0)
return await asyncio.wait_for(item.future, timeout=10)
finally:
self.pending.pop(stream_id, None)
self.http.head_streams.discard(stream_id)
@asynccontextmanager
async def client(protocol: str, port: int, certificate: Path):
if protocol == "h2":
context = ssl.create_default_context(cafile=str(certificate))
async with httpx.AsyncClient(http2=True, verify=context, trust_env=False,
timeout=10, limits=httpx.Limits(max_connections=1)) as session:
yield H2Client(session, port)
elif protocol == "h3":
config = QuicConfiguration(is_client=True, alpn_protocols=["h3"],
server_name="localhost", idle_timeout=10)
config.verify_mode = ssl.CERT_REQUIRED
config.load_verify_locations(cafile=str(certificate))
async with connect("127.0.0.1", port, configuration=config,
create_protocol=H3Client) as session:
session.authority = f"127.0.0.1:{port}"
yield session
else:
raise ValueError(f"unsupported fixture protocol {protocol!r}")
scripts/performance/transport_regression.py
-281
View File
@@ -1,281 +0,0 @@
#!/usr/bin/env python3
"""Real, bounded RustProxy H2/TLS, H3/QUIC and H1 WebSocket acceptance.
Install Python dependencies aioquic, httpx[http2] and websockets. Example:
python3 scripts/performance/transport_regression.py --binary before=/abs/binary \
--binary after=/abs/binary --out /abs/results --python-deps /abs/optional/pydeps
Only fresh temporary databases and loopback listeners are used. Certificates
are generated by RustProxy and verified by clients; no protocol fallback is used.
This is an interoperability/functional suite, not a throughput benchmark.
"""
from __future__ import annotations
import argparse
import asyncio
import copy
import hashlib
import importlib.metadata
import json
from pathlib import Path
import platform
import socket
import sys
import time
import traceback
from common import Fixture, seal_results, sha256, write_json
from peer import Peer
def response_details(response):
return {"status": response.status, "protocol": response.protocol,
"alpn": response.alpn, "stream_id": response.stream_id,
"bytes": len(response.body), "sha256": hashlib.sha256(response.body).hexdigest(),
"cache": response.headers.get("x-cache"),
"cache_reason": response.headers.get("x-cache-reason")}
def tls_port(fixture):
while True:
port = fixture.port()
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as udp:
try:
udp.bind(("127.0.0.1", port))
return port
except OSError:
continue
def transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin):
config = copy.deepcopy(rust.base)
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
identifier = f"{source}-tls"
config["listeners"]["endpoints"].append({
"id": identifier, "name": identifier, "scope": "internal", "enabled": True,
"binding": {"address": f"127.0.0.1:{port}", "tls": True, "http2": True,
"http3": {"max_concurrent_connections": 16,
"max_concurrent_bidistream": 16,
"max_concurrent_unistream": 16, "max_idle_timeout": 10}}})
vhost = next(v for v in config["vhosts"] if v["id"] == source)
vhost["frontend"]["listener_ids"].append(identifier)
vhost["http2_enabled"] = True
vhost["http3_enabled"] = True
config["listeners"]["endpoints"].append({
"id": "websocket", "name": "WebSocket", "scope": "internal", "enabled": True,
"binding": {"address": f"127.0.0.1:{websocket_port}", "tls": False,
"http2": False, "http3": None}})
config["vhosts"].append({
"id": "websocket", "frontend": {"hosts": ["127.0.0.1", "localhost"],
"listener_ids": ["websocket"]},
"websockets": True,
"backends": [{"id": "echo", "url": f"http://127.0.0.1:{websocket_origin}",
"allow_private": True}],
"load_balancing": {"health_check": {"mode": "off"}}, "routes": []})
config["cache"].update({"enabled": False, "expose_status": True, "expose_hits": True,
"expose_cache_status": True, "expose_only_hits": False})
config["compression"] = {"gzip": False, "br": False, "zstd": False, "deflate": False}
config["web_services"][0]["cache_control"] = "public, max-age=60"
return config
async def http_cases(check, session, protocol, source, cache, fixture):
prefix = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
expected = (fixture.site / "small.html").read_bytes()
path = f"/transport-{protocol}.html"
first = await session.request("GET", path)
check(prefix + ".get", first.status == 200 and first.body == expected,
**response_details(first))
second = await session.request("GET", path)
check(prefix + ".cache_delivery", second.status == 200 and second.body == expected
and ((first.headers.get("x-cache") == "MISS" and second.headers.get("x-cache") == "HIT")
if cache else second.headers.get("x-cache") not in ("MISS", "HIT")),
first_cache=first.headers.get("x-cache"), **response_details(second))
head = await session.request("HEAD", path)
check(prefix + ".head", head.status == 200 and head.body == b""
and head.headers.get("content-length") == str(len(expected)), **response_details(head))
etag = second.headers.get("etag")
check(prefix + ".validator", bool(etag), etag=etag)
ranged = await session.request("GET", path, {"range": "bytes=11-40", "if-range": etag or ""})
check(prefix + ".range", ranged.status == 206 and ranged.body == expected[11:41]
and ranged.headers.get("content-range") == f"bytes 11-40/{len(expected)}",
**response_details(ranged))
mismatch = await session.request("GET", path, {"range": "bytes=11-40", "if-range": '"different"'})
check(prefix + ".if_range_mismatch", mismatch.status == 200 and mismatch.body == expected,
**response_details(mismatch))
conditional = await session.request("GET", path, {"if-none-match": etag or ""})
check(prefix + ".conditional_304", conditional.status == 304 and conditional.body == b"",
**response_details(conditional))
invalid = await session.request("GET", path, {"range": "bytes=999999-"})
check(prefix + ".range_416", invalid.status == 416 and invalid.body == b""
and invalid.headers.get("content-range") == f"bytes */{len(expected)}",
**response_details(invalid))
for name in ["large.bin", "huge.bin"]:
expected_file = (fixture.site / name).read_bytes()
response = await session.request("GET", "/" + name)
check(prefix + ".get." + name, response.status == 200 and response.body == expected_file,
**response_details(response))
parallel = await asyncio.gather(*(session.request("GET", "/large.bin") for _ in range(4)))
expected_large = (fixture.site / "large.bin").read_bytes()
check(prefix + ".multiplex_four_streams", all(r.status == 200 and r.body == expected_large for r in parallel)
and len({r.stream_id for r in parallel}) == 4,
stream_ids=[r.stream_id for r in parallel], response_bytes=[len(r.body) for r in parallel])
if source == "reverse":
for payload in [b"", (bytes(range(256)) * 512) + b"streamed-end"]:
response = await session.request("POST", "/echo?transport=preserved",
{"content-type": "application/octet-stream"}, payload)
data = json.loads(response.body) if response.status == 200 else {}
check(prefix + f".post_stream_{len(payload)}", response.status == 200
and data.get("method") == "POST" and data.get("body_bytes") == len(payload)
and data.get("body_sha256") == hashlib.sha256(payload).hexdigest()
and data.get("path") == "/echo?transport=preserved",
upload_bytes=len(payload), **response_details(response))
else:
response = await session.request("POST", path, body=b"static-method-gate")
check(prefix + ".post_static_method_gate", response.status == 405, **response_details(response))
async def websocket_cases(check, port, cache):
import websockets
prefix = f"h1.websocket.cache_{'on' if cache else 'off'}"
async with websockets.connect(f"ws://127.0.0.1:{port}/echo", open_timeout=5,
close_timeout=3, ping_interval=None, compression=None) as session:
text = "RustProxy echo: Zażółć gęślą jaźń"
await session.send(text)
returned = await asyncio.wait_for(session.recv(), timeout=5)
check(prefix + ".text_echo", returned == text)
binary = bytes(range(256)) * 256
await session.send(binary)
returned = await asyncio.wait_for(session.recv(), timeout=5)
check(prefix + ".binary_echo", returned == binary, bytes=len(returned))
pong = await session.ping(b"rustproxy-transport")
await asyncio.wait_for(pong, timeout=5)
check(prefix + ".ping_pong", True)
await session.close(code=1000, reason="fixture complete")
check(prefix + ".normal_close", session.close_code == 1000, close_code=session.close_code)
async def run_variant(binary, label, output, workers):
import websockets
from transport_clients import client
cases = []
started = time.monotonic()
def check(case, passed, **details):
cases.append({"case": case, "passed": bool(passed), **details})
print(f"{label} {'PASS' if passed else 'FAIL'} {case}", flush=True)
write_json(output / "checks.json", cases)
try:
with Fixture(output, workers=workers) as fixture:
plain = (fixture.site / "small.html").read_bytes()
for name in ["transport-h2.html", "transport-h3.html", "transport-shared.html"]:
(fixture.site / name).write_bytes(plain)
peer = Peer(fixture.origin_port, fixture.site)
websocket_origin, websocket_port = fixture.port(), fixture.port()
async def echo(session, _path=None):
async for message in session:
await session.send(message)
try:
async with websockets.serve(echo, "127.0.0.1", websocket_origin,
ping_interval=None, compression=None, close_timeout=3):
rust = fixture.rust(binary, label)
reverse_tls, static_tls = tls_port(fixture), tls_port(fixture)
config = transport_config(rust, reverse_tls, static_tls, websocket_port, websocket_origin)
rust.apply(config, "transport-listeners")
fixture.stop(rust.process)
rust.start("transport")
certificate = rust.data / "certs" / "default.pem"
write_json(output / "transport.json", {
"certificate_sha256": sha256(certificate), "certificate_verification": True,
"h2_alpn_required": "h2", "h3_alpn_required": "h3",
"reverse_tls_tcp_udp": reverse_tls, "static_tls_tcp_udp": static_tls,
"websocket_h1_tcp": websocket_port, "fallback_allowed": False})
for cache in [False, True]:
if cache:
config["cache"]["enabled"] = True
rust.apply(config, "transport-cache-on")
for protocol in ["h2", "h3"]:
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
group = f"{protocol}.{source}.cache_{'on' if cache else 'off'}"
try:
async with client(protocol, port, certificate) as session:
await http_cases(check, session, protocol, source, cache, fixture)
except Exception as exc:
check(group + ".runner", False, error=str(exc), traceback=traceback.format_exc())
try:
await websocket_cases(check, websocket_port, cache)
except Exception as exc:
check(f"h1.websocket.cache_{'on' if cache else 'off'}.runner", False,
error=str(exc), traceback=traceback.format_exc())
for source, port in [("reverse", reverse_tls), ("static", static_tls)]:
try:
async with client("h2", port, certificate) as h2:
warm = await h2.request("GET", "/transport-shared.html")
async with client("h3", port, certificate) as h3:
hit = await h3.request("GET", "/transport-shared.html")
check(f"cache_cross_protocol.{source}.h2_to_h3",
warm.status == 200 and hit.status == 200 and hit.body == plain
and hit.headers.get("x-cache") == "HIT",
first_cache=warm.headers.get("x-cache"), **response_details(hit))
except Exception as exc:
check(f"cache_cross_protocol.{source}.runner", False,
error=str(exc), traceback=traceback.format_exc())
check("runtime.still_alive", rust.process.poll() is None)
finally:
peer.close()
except Exception as exc:
check("runtime.runner", False, error=str(exc), traceback=traceback.format_exc())
report = {"label": label, "simulation": False, "seconds": time.monotonic() - started,
"passed": sum(case["passed"] for case in cases),
"failed": sum(not case["passed"] for case in cases), "cases": cases}
write_json(output / "report.json", report)
print(f"{label}: {report['passed']} passed, {report['failed']} failed", flush=True)
return report
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", action="append", required=True, metavar="LABEL=/ABS/PATH")
parser.add_argument("--out", type=Path, required=True)
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--python-deps", type=Path)
args = parser.parse_args()
if args.python_deps:
sys.path.insert(0, str(args.python_deps.resolve()))
variants = {}
for value in args.binary:
label, separator, path = value.partition("=")
if not separator or not label.replace("-", "").replace("_", "").isalnum() or label in variants:
parser.error("--binary must be unique SIMPLE_LABEL=/ABS/PATH")
variants[label] = Path(path).resolve()
args.out = args.out.resolve()
args.out.mkdir(parents=True, exist_ok=True)
from transport_clients import validate_head_adapter
write_json(args.out / "environment.json", {
"kind": "real RustProxy transport acceptance", "simulation": False,
"python": sys.version, "platform": platform.platform(),
"dependencies": {name: importlib.metadata.version(name) for name in ["aioquic", "httpx", "h2", "websockets"]},
"aioquic_head_adapter": validate_head_adapter(),
"started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"not_covered": ["production certificates", "TLS upstreams", "H3 upstreams", "H2/H3 WebSocket extended CONNECT",
"network loss or migration", "24-hour soak", "real-browser navigation"]})
reports = {}
for label, binary in variants.items():
reports[label] = asyncio.run(run_variant(binary, label, args.out / label, args.workers))
write_json(args.out / "comparison.json", {
"variants": {label: {key: report[key] for key in ["passed", "failed", "seconds"]}
for label, report in reports.items()},
"cases": [{"case": case, **{label: next((row["passed"] for row in report["cases"] if row["case"] == case), "not_run")
for label, report in reports.items()}}
for case in sorted({row["case"] for report in reports.values() for row in report["cases"]})]})
seal_results(args.out)
return 1 if any(report["failed"] for report in reports.values()) else 0
if __name__ == "__main__":
sys.exit(main())
scripts/performance/wrk_report.lua
-14
View File
@@ -1,14 +0,0 @@
-- No response hook: benchmarks must not be limited by Lua body processing.
-- Known payload/status/headers are verified separately before measurement.
done = function(summary, latency, requests)
io.write("RUSTPROXY_BENCH_JSON {")
io.write(string.format('"requests":%d,"duration_us":%d,"bytes":%d,',
summary.requests, summary.duration, summary.bytes))
io.write(string.format('"p50_us":%.3f,"p95_us":%.3f,"p99_us":%.3f,',
latency:percentile(50), latency:percentile(95), latency:percentile(99)))
io.write(string.format('"latency_mean_us":%.3f,"latency_max_us":%.3f,', latency.mean, latency.max))
io.write(string.format('"errors_connect":%d,"errors_read":%d,"errors_write":%d,"errors_timeout":%d,"errors_status":%d',
summary.errors.connect, summary.errors.read, summary.errors.write,
summary.errors.timeout, summary.errors.status))
io.write("}\n")
end
scripts/preflight.py
+1 -1
View File
@@ -74,7 +74,7 @@ def check(root: Path, repair_known: bool = False) -> list[str]:
flat = module.parent.with_suffix(".rs")
if flat.is_file():
raise ValueError(f"Conflicting Rust modules: {flat.relative_to(root)} and {module.relative_to(root)}")
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/TODO.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
for path in ["src/auth/mod.rs", "src/auth/store.rs", "src/backup/snapshot.rs", "src/upstream.rs", "src/tls.rs", "src/config/internal_routing.rs", "src/proxy/websocket.rs", "src/proxy/error_fallback.rs", "src/cache/mod.rs", "src/cache/model.rs", "src/cache/lifecycle.rs", "src/cache/cold.rs", "src/cache/dashboard.rs", "src/cache/tickets.rs", "src/cache/lookup.rs", "src/cache/policy.rs", "src/cache/fill.rs", "src/cache/tests.rs", "src/settings.rs", "src/compression.rs", "src/diagnostics.rs", "src/traffic.rs", "src/system_stats/mod.rs", "src/system_stats/linux.rs", "src/vhost.rs", "src/vhost/error_fallback.rs", "src/web_server/mod.rs", "src/web_server/config.rs", "src/web_server/filesystem.rs", "src/web_server/security.rs", "src/web_server/response.rs", "src/web_server/request.rs", "src/web_server/routing.rs", "src/web_server/fastcgi.rs", "src/web_server/htaccess.rs", "src/web_server/serve.rs", "src/web_server/tests.rs", "src/balance.rs", "src/security.rs", "src/admin/web/vhosts/core.js", "src/admin/web/vhosts/rates.js", "src/admin/web/vhosts/health.js", "src/admin/web/vhosts/editor.js", "src/admin/web/vhosts/render.js", "src/admin/web/web-server.js", "src/admin/web/streams.js", "src/admin/web/host404.js", "src/admin/web/app/core.js", "src/admin/web/app/session-resume.js", "src/admin/web/app/preferences.js", "src/admin/web/app/config-history.js", "src/admin/web/app/runtime.js", "src/admin/web/app/settings.js", "src/admin/web/management.js", "src/admin/web/drafts.js", "src/pages.rs", "tools/dev/mock_server.py", "tools/dev/mock_auth.py", "src/forwarding.rs", "src/tinyauth.rs", "src/http3.rs", "src/stream.rs", "src/admin/web/integrations.js", "src/admin/web/traffic-panel.js", "src/prometheus.rs", "src/admin/web/settings.js", "src/admin/web/docker.js", "src/docker.rs", "docker-agent/main.go", "docker-agent/model.go", "docker-agent/docker.go", "docker-agent/network.go", "docker-agent/certificate.go", "docker-agent/server.go", "docker-agent/main_test.go", "docker-agent/rustproxy-docker-agent.service", "docker-agent/install.sh", "docs/DOCKER_DISCOVERY.md", "docs/WEB_SERVER.md", "docs/PHP_FPM.md", "docs/README.md", "docs/CHANGELOG_TECHNICAL_ARCHIVE.md", "docs/ROADMAP_HISTORY.md", "scripts/runtime/acme_provider.py", "src/basic_auth.rs", "scripts/deploy/deploy.py", "scripts/deploy/deploy_core.py", "scripts/deploy/deploy_runtime.py", "scripts/migration/migrate_npm.py", "scripts/migration/migrate_npm_core.py", "scripts/migration/migrate_npm_api.py", "docs/NPM_MIGRATION.md", "docs/STREAMS.md", "install.sh", "install-debian13.sh", "update.sh", "build.rs", "scripts/build/swagger_assets.py", "src/admin/documentation.rs", "src/admin/openapi.json", "langs/en_US.json", "langs/pl_pl.json", "Dockerfile", ".dockerignore", "docs/CONTAINER.md", "docs/PROMETHEUS.md", "docs/NOT_FOUND_HOSTS.md", "grafana/rustproxy-overview.json"]:
if not (root / path).is_file():
raise ValueError(f"Incomplete RustProxy 0.31.9 source tree: missing {path}")
# The Reader uses compile-time include_str! for its bundled Markdown. A
scripts/tests/cache_age_policy_034_model.py
-62
View File
@@ -1,62 +0,0 @@
#!/usr/bin/env python3
"""Deterministic oracle for HTTP Age / local freshness-cap algebra.
No binary execution, no claim of measured throughput. Run with Python 3 only.
"""
import json
import random
from pathlib import Path
def old_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
age = max(header_age, apparent_age) + response_delay
return max(0, min(max_age, local_limit) - age)
def new_freshness(max_age, header_age, apparent_age, response_delay, local_limit):
# RFC 9111 corrected_initial_age = max(apparent_age, age_value + response_delay)
age = max(apparent_age, header_age + response_delay)
lifetime = min(max_age, age + local_limit)
return max(0, lifetime - age)
def run():
cases = [
('user_cdn_example', 2_592_000, 1_553_483, 0, 0, 86_400),
('user_cdn_example_7d', 2_592_000, 1_553_483, 0, 0, 604_800),
('origin_nearly_stale', 2_592_000, 2_591_990, 0, 0, 86_400),
('origin_stale', 2_592_000, 2_592_010, 0, 0, 86_400),
('normal_small', 3_600, 0, 0, 0, 86_400),
('date_is_older', 3_600, 10, 600, 2, 86_400),
('rfc_delay_case', 3_600, 3, 10, 2, 86_400),
]
report = {'reference': 'RFC 9111 4.2.3 (Age) and 4.2.1 (freshness lifetime)',
'kind': 'algebra simulation; not RustProxy RPS benchmark',
'scenarios': [], 'randomized_checks': 0, 'false_miss_cases_old': 0}
for name, max_age, age, apparent, delay, local in cases:
old = old_freshness(max_age, age, apparent, delay, local)
new = new_freshness(max_age, age, apparent, delay, local)
report['scenarios'].append({'scenario': name, 'origin_max_age_seconds': max_age,
'received_age_seconds': age, 'apparent_age_seconds': apparent,
'response_delay_seconds': delay, 'local_max_ttl_seconds': local,
'old_freshness_seconds': old, 'new_freshness_seconds': new})
rng = random.Random(0x3400)
for _ in range(100_000):
origin = rng.randrange(0, 1 << 24)
header_age = rng.randrange(0, 1 << 24)
apparent = rng.randrange(0, 1 << 24)
delay = rng.randrange(0, 300)
local = rng.randrange(0, 604_801)
corrected_age = max(apparent, header_age + delay)
new = new_freshness(origin, header_age, apparent, delay, local)
expected = min(max(0, origin - corrected_age), local)
assert new == expected, (origin, header_age, apparent, delay, local, new, expected)
assert new <= local and new <= max(0, origin - corrected_age)
old = old_freshness(origin, header_age, apparent, delay, local)
report['false_miss_cases_old'] += old == 0 < new
report['randomized_checks'] += 1
return report
if __name__ == '__main__':
report = run()
print(json.dumps(report, ensure_ascii=False, indent=2))
scripts/tests/cache_age_transform_034_live.py
-212
View File
@@ -1,212 +0,0 @@
#!/usr/bin/env python3
"""Loopback integration regression: transformed Age, HOT/COLD, diagnostics, isolation.
Run after compiling:
python3 scripts/tests/cache_age_transform_034_live.py --binary target/release/rust-proxy
To reproduce the original regression with the user-supplied 0.34.0 binary:
python3 scripts/tests/cache_age_transform_034_live.py --binary /path/to/old/rust-proxy --expect-old-age-bug
No internet and no user data; test config/data/passwords are in a disposable temporary dir.
"""
import argparse
import collections
import copy
import json
import sys
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'performance'))
from common import Fixture, http, sha256, write_json
BODY = b'/* test cache */\nbody { color: red; padding: 0px; }\n'
# Fixed sample from Brotli encoder, embedded to avoid Python dependency on brotli.
BROTLI_BODY = b"\x1b3\x00\xe8\x1d\x07\xee\xbe\r\xa2\xbc3'\x8cP\xe9\xad\xa0\xf2\xb2\xa7\xe4\xfa\x12\x84\xac\x0c\x1cr\xc0\xfe\xc0>\x0fR*a\x9c\x9e2\xe8\x08\xcb\x1cP\x881\xac\xdd\x88s5(\x9d\x01"
COUNTS = collections.Counter()
class Origin(BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_GET(self):
path = self.path.split('?', 1)[0]
if path not in {'/aged.css', '/normal.css', '/private.css', '/no-store.css', '/other.css', '/br.css', '/aged-br.css'}:
self.send_error(404)
return
COUNTS[path] += 1
self.send_response(200)
if path in ('/aged.css', '/aged-br.css'):
policy = 'public, max-age=2592000'
elif path == '/private.css':
policy = 'private, max-age=2592000'
elif path == '/no-store.css':
policy = 'no-store'
else:
policy = 'public, max-age=3600'
self.send_header('Cache-Control', policy)
self.send_header('Content-Type', 'text/css')
payload = BROTLI_BODY if path in ('/br.css', '/aged-br.css') else BODY
self.send_header('Content-Length', str(len(payload)))
if path in ('/br.css', '/aged-br.css'):
self.send_header('Content-Encoding', 'br')
self.send_header('ETag', '"origin-css"')
self.send_header('Last-Modified', 'Thu, 07 Jan 2016 20:04:59 GMT')
self.send_header('Vary', 'Accept-Encoding')
if path in ('/aged.css', '/aged-br.css'):
self.send_header('Age', '1553483')
self.end_headers()
self.wfile.write(payload)
def request(port, path, encoding='identity'):
status, headers, body = http(port, path, headers={'Accept-Encoding': encoding})
return {'status': status, 'x_cache': headers.get('x-cache'),
'x_cache_hits': headers.get('x-cache-hits'), 'x_cache_resident_age': headers.get('x-cache-resident-age'),
'cache_status': headers.get('cache-status'), 'age': headers.get('age'),
'body': body.decode('utf-8', 'replace'), 'content_length': headers.get('content-length'),
'etag': headers.get('etag'), 'encoding': headers.get('content-encoding'), 'origin_count': COUNTS[path.split('?', 1)[0]]}
def dash(rust):
status, _, body = rust.admin('/api/cache?limit=200')
assert status == 200, ('cache dashboard', status, body[:200])
data = json.loads(body)['dashboard']
return {'objects': [{k: row.get(k) for k in ('vhost','path','tier','transformed',
'hits','age_seconds','resident_age_seconds','remaining_ttl_seconds')} for row in data['objects']],
'domains': data['domains']}
def find(d, path):
matches = [item for item in d['objects'] if item['path'] == path and item['vhost'] == 'reverse']
assert len(matches) == 1, (path, matches)
return matches[0]
def run(binary: Path, output: Path, expect_old_age_bug: bool):
outcome = {'binary_sha256': sha256(binary), 'expect_old_age_bug': expect_old_age_bug,
'tests': [], 'passed': False}
output.mkdir(parents=True, exist_ok=True)
with Fixture(output, workers=2) as fixture:
origin = ThreadingHTTPServer(('127.0.0.1', fixture.origin_port), Origin)
thread = threading.Thread(target=origin.serve_forever, daemon=True)
thread.start()
try:
rust = fixture.rust(binary, 'cache-age-034')
rust.login()
cfg = copy.deepcopy(rust.base)
cfg['cache'] = {'enabled': True, 'max_ttl_seconds': 86_400,
'expose_status': True, 'expose_hits': True,
'expose_cache_status': True, 'expose_resident_age': True}
cfg['transform'] = {'enabled': True, 'css': True}
for vhost in cfg['vhosts']:
vhost['transform'] = {'enabled': True, 'css': True}
rust.apply(cfg, 'fresh-cache-transform')
for path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css', '/private.css', '/no-store.css'):
before = COUNTS[path]
encoding = 'br' if path in ('/br.css', '/aged-br.css') else 'identity'
first, second = request(rust.reverse_port, path, encoding), request(rust.reverse_port, path, encoding)
row = {'path': path, 'first': first, 'second': second,
'origin_delta': COUNTS[path] - before}
outcome['tests'].append(row)
if path in ('/aged.css', '/aged-br.css') and expect_old_age_bug:
assert first['x_cache'] == second['x_cache'] == 'MISS', row
assert row['origin_delta'] == 2, row
elif path in ('/normal.css', '/aged.css', '/br.css', '/aged-br.css'):
assert first['x_cache'] == 'MISS' and second['x_cache'] == 'HIT', row
assert row['origin_delta'] == 1, row
assert second['cache_status'] and 'transformed' in second['cache_status'], row
assert first['etag'] == second['etag'] and first['body'] == second['body'], row
else:
assert first['x_cache'] == second['x_cache'] == 'MISS', row
print(path, first['x_cache'], second['x_cache'], 'origin+', row['origin_delta'], flush=True)
snapshot = dash(rust)
outcome['tests'].append({'dashboard': snapshot})
assert find(snapshot, '/normal.css')['transformed'], snapshot
aged = find(snapshot, '/aged.css')
if expect_old_age_bug:
assert aged['remaining_ttl_seconds'] == 0, aged
else:
assert aged['transformed'] and aged['remaining_ttl_seconds'] >= 86_000, aged
assert aged['age_seconds'] >= 1_553_483, aged
assert aged['resident_age_seconds'] < 10, aged
aged_br = find(snapshot, '/aged-br.css')
assert aged_br['transformed'] and aged_br['remaining_ttl_seconds'] >= 86_000, aged_br
# Query-string bypass is an explicit administrator rule. Even a
# transformed CSS with public max-age stays BYPASS while it is ON,
# then the *same* URL must become MISS -> HIT after disabling it.
# This is not a transform regression: it is a request policy.
cfg['cache']['bypass_query'] = True
rust.apply(cfg, 'query-bypass-enabled')
query_path = '/normal.css?v=transformed-public'
before = COUNTS['/normal.css']
q_bypass = [request(rust.reverse_port, query_path) for _ in range(2)]
assert [row['x_cache'] for row in q_bypass] == ['BYPASS', 'BYPASS'], q_bypass
assert COUNTS['/normal.css'] - before == 2, q_bypass
cfg['cache']['bypass_query'] = False
rust.apply(cfg, 'query-bypass-disabled')
q_cached = [request(rust.reverse_port, query_path) for _ in range(2)]
assert [row['x_cache'] for row in q_cached] == ['MISS', 'HIT'], q_cached
assert q_cached[1]['cache_status'] and 'transformed' in q_cached[1]['cache_status']
assert COUNTS['/normal.css'] - before == 3, q_cached
outcome['tests'].append({'query_bypass_policy': {'bypass': q_bypass, 'cached': q_cached}})
# Operational LB tuning must not invalidate a cached representation.
before = COUNTS['/normal.css']
cfg['vhosts'][0]['load_balancing']['keepalive'] = {'max_idle_per_host': 16}
rust.apply(cfg, 'only-backend-keepalive-changed')
normal = request(rust.reverse_port, '/normal.css')
assert normal['x_cache'] == 'HIT' and COUNTS['/normal.css'] == before
outcome['tests'].append({'keepalive_update_without_purge': normal})
# Diagnostics can be intentionally hidden on MISS and remain visible on HIT.
cfg['cache']['expose_only_hits'] = True
rust.apply(cfg, 'only-show-hit-headers')
before = COUNTS['/other.css']
first, second = request(rust.reverse_port, '/other.css'), request(rust.reverse_port, '/other.css')
assert first['x_cache'] is None and second['x_cache'] == 'HIT'
assert COUNTS['/other.css'] - before == 1
outcome['tests'].append({'only_hits': [first, second]})
if not expect_old_age_bug:
# Cold path on a 1-entry HOT tier: demotion and on-disk reuse.
cfg['cache'].update({'max_entries': 1, 'cold_enabled': True, 'cold_on_lru': True,
'cold_path': 'cache/age-regression-cold'})
cfg['cache'].pop('expose_only_hits', None)
rust.apply(cfg, 'cold-enabled-one-hot-entry')
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'MISS'
assert request(rust.reverse_port, '/aged.css')['x_cache'] == 'HIT'
assert request(rust.reverse_port, '/other.css')['x_cache'] == 'MISS'
deadline = time.monotonic() + 4
while True:
snapshot = dash(rust)
matches = [x for x in snapshot['objects'] if x['path'] == '/aged.css' and x['tier'] == 'cold']
if matches:
assert matches[0]['remaining_ttl_seconds'] >= 86_000, matches
break
if time.monotonic() > deadline:
raise AssertionError('aged transformed CSS never demoted to COLD: ' + str(snapshot))
time.sleep(.05)
cold_hit = request(rust.reverse_port, '/aged.css')
assert cold_hit['x_cache'] == 'HIT' and 'transformed' in cold_hit['cache_status'], cold_hit
outcome['tests'].append({'cold': {'row': matches[0], 'hit': cold_hit}})
outcome['passed'] = True
finally:
origin.shutdown()
origin.server_close()
write_json(output / 'cache-age-transform-result.json', outcome)
return outcome
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--binary', type=Path, required=True)
parser.add_argument('--output', type=Path, default=Path('/tmp/rustproxy-cache-age-034-live'))
parser.add_argument('--expect-old-age-bug', action='store_true')
args = parser.parse_args()
try:
report = run(args.binary.resolve(), args.output.resolve(), args.expect_old_age_bug)
except Exception as error:
print('FAIL', type(error).__name__, str(error)[:1000], file=sys.stderr)
raise
print('PASS', len(report['tests']), 'scenarios; see', args.output / 'cache-age-transform-result.json')
scripts/tests/cache_hot_034_static_test.py
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env python3
"""Source-only hot-cache safety guards; NOT a Rust compile, concurrency test or RPS result."""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
read = lambda path: (ROOT / path).read_text()
hot = read("src/cache/lookup/hot.rs")
entry = read("src/cache/model/cache.rs")
model = read("src/cache/model/maintenance.rs")
inv = read("src/cache/invalidation/invalidate.rs")
dash = read("src/cache/dashboard.rs")
assert 'headers: Arc<HeaderMap>' in entry
assert 'Arc::clone(&entry.headers)' in hot
assert 'drop(store);' in hot
assert hot.index('drop(store);\n\n let mut response') < hot.index('headers.as_ref().clone()')
assert 'self.domain_stats.saved_body_hit(&ticket.scope, &ticket.domain, body_bytes)' in hot
assert 'store.lru.remove(&old_sequence)' in hot and 'store.lru.insert(sequence, storage_key)' in hot
assert '.sweep()' not in hot
assert 'fn sweep(' not in entry
stats = read('src/cache/stats.rs')
assert 'fn saved_body_hit(' in stats
assert 'const DOMAIN_STATS_LIMIT: usize = 4096' in stats
assert 'ArcSwap<PublishedDomains>' in stats
assert 'fn record_reason(&self, reason:' in stats
assert 'self.store.lock()' not in stats
assert 'if self.ban_rule_count.load(Ordering::Acquire) == 0' in inv
assert 'bans.iter().any(|rule| rule.matches_hot' in inv
assert 'fn matching_hot_key(' in model and 'entry.vary.is_empty()' in model
assert 'self.domain_stats.clear()' in dash
assert 'self.domain_stats.record(scope, domain, outcome, reason);' in dash
assert 'include!("tests/hot_hit_034_tests.rs")' in read('src/cache/tests.rs')
for path in (ROOT / 'src').rglob('*.rs'):
if '/tests/' not in path.as_posix() and path.name != 'tests.rs':
assert len(path.read_text().splitlines()) <= 500, path
print('PASS HOT path static invariants (no Rust compilation or throughput claim)')
scripts/tests/cache_scope_034_static_test.py
-40
View File
@@ -1,40 +0,0 @@
#!/usr/bin/env python3
"""0.34.0 source-wiring checks only; do not mistake these for Rust tests."""
from pathlib import Path
import tomllib
root = Path(__file__).resolve().parents[2]
read = lambda path: (root / path).read_text()
assert tomllib.loads(read("Cargo.toml"))["package"]["version"] == "0.34.0"
assert 'version = "0.34.0"' in read("Cargo.lock")
app = read("src/runtime/updates.rs")
assert "CacheInvalidation::Scopes(scopes)" in app
assert "this.cache.purge_scope(&scope)" in app
assert "this.cache.set_control_revision(envelope.revision)" in app
assert app.index("this.cache.set_control_revision(envelope.revision)") < app.index("match invalidation")
source = read("src/runtime/types.rs")
assert "view.load_balancing = Default::default();" in source
assert "member.max_concurrent_requests = 0;" in source
assert "fn vhost_cache_revision" in source
assert "&web_services" in source
assert "service.sendfile = crate::web_server::SendfileMode::Off;" in source
assert "&config.php_fpm_profiles" in source
for filename in ("src/proxy/forward/route_cache.rs", "src/proxy/forward/web_source.rs"):
assert "ticket_scoped_for_config(" in read(filename)
assert "snapshot.envelope.revision" in read(filename)
assert "expected_control_revision" in read("src/cache/tickets.rs")
assert "file.scope_revision != expected_scope_revision" in read("src/cache/cold/startup.rs")
assert "scope_revision: draft.scope_revision" in read("src/cache/cold/worker.rs")
body = read("src/proxy/body.rs")
assert "response_backend: Option<Arc<crate::balance::Lease>>" in body
assert "lease.bytes(count, false);" in body
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
assert "lease.bytes(count, true);" in body
assert "cold_scope_fingerprint_preserves_unmodified_vhost" in read("src/cache/tests/cold_tier_tests.rs")
assert "stale_snapshot_cannot_issue_cache_ticket" in read("src/cache/tests/scope_tests.rs")
for file in (root / 'src').rglob('*.rs'):
if '/tests/' in file.as_posix() or file.name == 'tests.rs':
continue
if len(file.read_text().splitlines()) > 500:
raise AssertionError(f"production Rust source exceeds 500 lines: {file}")
print('PASS source wiring for scoped invalidation, cold fingerprints, stale-snapshot barrier, response accounting and <=500-line modules (STATIC ONLY)')
scripts/tests/dashboard_chart_headers_0335_browser_test.py
-97
View File
@@ -1,97 +0,0 @@
#!/usr/bin/env python3
"""Overview traffic/connection heading cleanup against Chromium and the bundled JS.
Runs a simulated authenticated DOM rather than a live RustProxy service.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
ROUTER = (ROOT / 'src/admin/router.rs').read_text(encoding='utf-8')
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text(encoding='utf-8'))
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text(encoding='utf-8')
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
bundle = app_bundle()
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=bundle)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
currentPanel = 'overview';
samples.splice(0); samples.push(1.5, 85.5, 12.3);
redrawOverviewHistoryCharts();
}''')
for density in ('normal', 'compact', 'ultra'):
for span in (3, 6):
data = page.evaluate('''({density,span}) => {
document.documentElement.dataset.dashboardDensity = density;
const visible = ['traffic_chart', 'connection_chart'];
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !visible.includes(id));
applyDashboardLayout([...visible,...hidden], hidden,
{traffic_chart:span,connection_chart:span});
const traffic = document.querySelector('[data-dashboard-widget="traffic_chart"]');
const connection = document.querySelector('[data-dashboard-widget="connection_chart"]');
const heading = traffic.querySelector('.card-heading');
const peak = traffic.querySelector('#chart-peak');
const peakRect = peak.getBoundingClientRect();
const cardRect = traffic.getBoundingClientRect();
const headingRect = heading.getBoundingClientRect();
return {
title: heading.querySelector('h2')?.textContent.trim(),
peak: peak.textContent,
peakInHeading: peak.parentElement === heading,
peakNotInFooter: !traffic.querySelector('.chart-footer #chart-peak'),
noTrafficEyebrow: !traffic.querySelector('.small-label'),
noTrafficLiveBadge: !traffic.querySelector('.pill.live'),
noConnectionLiveBadge: !connection.querySelector('.pill.live'),
connectionTitleIntact: !!connection.querySelector('.card-heading h3.overview-chart-heading[data-i18n="connectionActivity"]'),
peakInsideCard: peakRect.left >= cardRect.left - 1 && peakRect.right <= cardRect.right + 1,
peakInsideHeading: peakRect.top >= headingRect.top - 1 && peakRect.bottom <= headingRect.bottom + 1,
browserOverflow: document.documentElement.scrollWidth - innerWidth,
};
}''', {'density':density,'span':span})
assert data['title'] == 'Requests per second', (width, density, span, data)
assert data['peak'] == 'max 85.5 req/s', (width, density, span, data)
for key in ('peakInHeading', 'peakNotInFooter', 'noTrafficEyebrow',
'noTrafficLiveBadge', 'noConnectionLiveBadge',
'connectionTitleIntact', 'peakInsideCard', 'peakInsideHeading'):
assert data[key], (width, density, span, key, data)
assert data['browserOverflow'] <= 2, (width, density, span, data)
assert not errors, (width, errors)
print(f'PASS traffic/connection chart headers at {width}px (all densities, spans 3 and 6)')
page.close()
browser.close()
print('PASS Overview chart heading cleanup')
scripts/tests/dashboard_chart_typography_0337_browser_test.py
-168
View File
@@ -1,168 +0,0 @@
#!/usr/bin/env python3
"""All chart titles and series labels in simulated Chromium (no live Rust server).
Covers Overview, diagnostics, Web Server, Traffic, Cache/Transform, Vhosts,
Streams, and DOM-created chart/legend variants. Preserve localized DOM text.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
CHECK = r'''(settings) => {
const { density, compactBody, theme } = settings;
document.documentElement.dataset.dashboardDensity = density;
document.documentElement.dataset.theme = theme;
document.body.dataset.density = compactBody ? 'compact' : 'comfortable';
const titleSelectors = [
'#panel-overview .overview-wide-chart .card-heading h2',
'#panel-overview .activity-card .card-heading h2',
'#panel-overview :is(.system-card, .connection-card) .overview-chart-heading',
'#panel-overview :is(.stat-chart, .stat-donut) > span:first-child',
'.mini-live-chart > .mini-live-chart-head > span:first-child',
'.traffic-chart-card > h3.subheading',
'.function-live-chart-heading > strong',
'.vhost-chart-head > span:first-child',
];
const monoSelectors = [
'.overview-wide-summary', '.overview-wide-legend',
'.overview-donut-legend', '.www-method-legend', '.stat-chart-legend',
'.mini-chart-legend', '.connection-legend',
'.traffic-protocol-legend', '.traffic-chart-legend',
'.function-live-chart-legend', '.stream-chart-legend',
'.chart-footer', '.vhost-chart-head > :not(:first-child)',
'.mini-live-chart-head > b', '.mini-live-chart-head > .mini-live-chart-actions',
'.network-interface-rate',
'.function-live-chart-summary > b', '.function-live-chart-heading small',
'.function-live-chart-samples',
'#panel-overview .stat-chart > small',
'#panel-overview .overview-wide-chart > .fine',
'#panel-overview .www-top-row', '.traffic-bar-row',
'#panel-overview .system-card .card-heading h2',
'#panel-overview .system-card .card-heading .meta',
'#panel-overview .system-card .memory-line',
'#panel-overview .connection-card .card-heading h2',
];
const legendSelectors = [
'.overview-wide-legend', '.overview-donut-legend', '.www-ring-legend',
'.www-method-legend', '.stat-chart-legend', '.mini-chart-legend',
'.connection-legend', '.traffic-protocol-legend', '.traffic-chart-legend',
'.function-live-chart-legend', '.stream-chart-legend', '.stream-traffic-legend'
];
const legendNodes = [...document.querySelectorAll(legendSelectors.join(','))];
const requestLegend = document.querySelector('[data-dashboard-widget="traffic_chart"] .traffic-protocol-legend');
const referenceLegendSize = getComputedStyle(requestLegend).fontSize;
const titleNodes = Array.from(document.querySelectorAll(titleSelectors.join(',')));
const monoNodes = Array.from(document.querySelectorAll(monoSelectors.join(',')));
const base = document.createElement('div');
base.style.fontFamily = 'var(--mono)';
document.body.append(base);
const mono = getComputedStyle(base).fontFamily;
base.style.fontFamily = 'var(--sans)';
const sans = getComputedStyle(base).fontFamily;
base.remove();
const titleErrors = titleNodes.flatMap(node => {
const css = getComputedStyle(node);
const inOverview = !!node.closest('#panel-overview');
const isSmallCaption = inOverview && !!node.closest('.mini-live-chart-head');
const wantedFont = isSmallCaption ? mono : sans;
const wantedSize = isSmallCaption ? '10px' : '12px';
const wantedCase = inOverview ? 'none' : 'uppercase';
const problems = [];
if (css.fontFamily !== wantedFont) problems.push(`font=${css.fontFamily}`);
if (css.fontSize !== wantedSize) problems.push(`size=${css.fontSize}`);
if (css.textTransform !== wantedCase) problems.push(`case=${css.textTransform}`);
return problems.map(problem => `${node.outerHTML.slice(0, 100)}: ${problem}`);
});
const monoErrors = monoNodes.flatMap(node => {
const all = [node, ...node.querySelectorAll('*')].filter(el => el.tagName.toLowerCase() !== 'svg');
return all.filter(el => getComputedStyle(el).fontFamily !== mono)
.map(el => `${el.outerHTML.slice(0, 100)}: ${getComputedStyle(el).fontFamily}`);
});
const legendErrors = legendNodes.flatMap(root => {
const children = [root, ...root.querySelectorAll('span, b, strong, small, li, em')];
return children.flatMap(node => {
const style = getComputedStyle(node);
const issues = [];
if (style.fontSize !== referenceLegendSize) issues.push(`size=${style.fontSize}`);
if (style.fontFamily !== mono) issues.push(`font=${style.fontFamily}`);
return issues.map(issue => `${node.outerHTML.slice(0, 95)} ${issue}`);
});
});
// Keep localized Overview titles in sentence case; other sections retain
// their independent uppercase chart-heading convention.
const compression = document.querySelector('[data-dashboard-widget="wide_compression_activity"]');
const compressionHeading = compression.querySelector('.card-heading h2');
const compressionLegend = compression.querySelector('.overview-wide-legend');
const originalTranslation = compressionHeading.textContent.trim() === 'Aktywność kompresji';
const legendFonts = [...compressionLegend.querySelectorAll('span, b')]
.map(el => getComputedStyle(el).fontFamily);
const unrelatedHeading = document.querySelector('#panel-traffic .card-heading h2');
const numericReading = document.querySelector('#system-cpu');
return {
counts: {titles: titleNodes.length, mono: monoNodes.length, legends: legendNodes.length},
titleErrors, monoErrors, legendErrors, referenceLegendSize, mono, sans,
originalTranslation, compressionTitleSize: getComputedStyle(compressionHeading).fontSize,
compressionTitleCase: getComputedStyle(compressionHeading).textTransform,
compressionLegendIsMono: legendFonts.every(font => font === mono),
unrelatedTitleSize: getComputedStyle(unrelatedHeading).fontSize,
numericReadingIsMono: getComputedStyle(numericReading).fontFamily === mono,
};
}'''
with sync_playwright() as api:
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 960})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
// Representative dynamically rendered Vhost/Web Service/Stream charts.
for (const panel of ['panel-vhosts','panel-web-server','panel-streams']) {
document.getElementById(panel).insertAdjacentHTML('beforeend', `
<div class="vhost-chart"><div class="vhost-chart-head">
<span class="small-label">Connection history</span>
<span class="fine">42 req/s</span></div>
<svg viewBox="0 0 300 54"><path d=""></path></svg>
<div class="stream-chart-legend"><span>RX 2 MB/s</span><span>TX 1 MB/s</span></div>
</div>`);
}
const comp = document.querySelector('[data-dashboard-widget="wide_compression_activity"] .card-heading h2');
comp.textContent = 'Aktywność kompresji';
// The traffic renderer inserts these series dynamically.
document.querySelectorAll('.traffic-chart-legend').forEach(root => {
root.innerHTML = `<li><span class="traffic-chart-label">GET requests</span><b>12</b></li>`;
});
const sample = document.createElement('div');
sample.innerHTML = `<div class="www-top-row"><span>php-worker</span><b>11/s</b></div>
<div class="traffic-bar-row"><span class="traffic-bar-label">GET</span><b class="traffic-bar-value">22</b></div>
<div class="network-interface-rate">100 Mb/s</div>`;
document.getElementById('panel-overview').append(sample);
}''')
for density in ('normal','compact','ultra'):
for compact_body in (False, True):
for theme in ('dark', 'light'):
result = page.evaluate(CHECK, {'density': density, 'compactBody': compact_body, 'theme': theme})
description = (width, density, compact_body, theme)
assert result['counts']['titles'] >= 43, (description, result['counts'])
assert result['counts']['mono'] >= 90, (description, result['counts'])
assert not result['titleErrors'], (description, result['titleErrors'][:10])
assert not result['monoErrors'], (description, result['monoErrors'][:10])
assert result['counts']['legends'] >= 39, (description, result['counts'])
assert result['referenceLegendSize'] == '9px', description
assert not result['legendErrors'], (description, result['legendErrors'][:10])
assert result['originalTranslation'], description
assert result['compressionTitleSize'] == '12px', description
assert result['compressionTitleCase'] == 'none', description
assert result['compressionLegendIsMono'], description
assert result['numericReadingIsMono'], description
assert result['unrelatedTitleSize'] != '12px', (description, result['unrelatedTitleSize'])
print(f'PASS all chart families at {width}px (3 dashboard densities, 2 interface densities, 2 themes)')
page.close()
browser.close()
scripts/tests/dashboard_chart_width_0333_browser_test.py
-110
View File
@@ -1,110 +0,0 @@
#!/usr/bin/env python3
"""Real-browser regression for Overview chart content when card width changes.
Runs the production HTML/CSS/assembled admin JS against a simulated DOM and
fake network-interface samples. This does not exercise Rust HTTP/SSE.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
scripts = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
scripts.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
scripts.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
scripts.append(json.loads(line[:-1]))
return '\n'.join(scripts)
def inspect(page):
return page.evaluate('''() => {
const selectors = {
'websocket_protocol':'[data-dashboard-widget="websocket_protocol"] svg',
'wide_transform_activity':'[data-dashboard-widget="wide_transform_activity"] svg',
'www_requests':'[data-dashboard-widget="www_requests"] svg',
'traffic_chart':'[data-dashboard-widget="traffic_chart"] svg',
'system_chart':'[data-dashboard-widget="system_chart"] svg',
'cache_diagnostics':'[data-dashboard-widget="cache_diagnostics"] .mini-live-chart svg',
'network_diagnostics':'[data-dashboard-widget="network_diagnostics"] .mini-live-chart svg',
'network_interface':'[data-dashboard-widget="network_diagnostics"] .network-interface-row svg',
};
return Object.fromEntries(Object.entries(selectors).map(([name, selector]) => {
const svg = document.querySelector(selector);
if (!svg) throw new Error('Missing SVG: ' + name);
const box = svg.getBoundingClientRect();
const matrix = svg.getScreenCTM();
const viewBox = svg.viewBox.baseVal;
const contentWidth = Math.hypot(matrix.a, matrix.b) * viewBox.width;
const series = svg.querySelector('.overview-wide-series');
return [name, {width:box.width, contentWidth,
preservation:svg.getAttribute('preserveAspectRatio'),
lineStroke: series ? getComputedStyle(series).vectorEffect : null}];
}));
}''')
with sync_playwright() as browser_api:
browser = browser_api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for screen_width in (320, 375, 800, 1440, 1920):
page = browser.new_page(viewport={'width': screen_width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
currentPanel = 'overview';
renderNetworkInterfaceCharts([{name:'eth0',rx_bytes_per_second:12500,tx_bytes_per_second:5200}], true);
}''')
ids = ['websocket_protocol', 'wide_transform_activity', 'www_requests', 'traffic_chart',
'system_chart', 'cache_diagnostics', 'network_diagnostics']
results = []
for span in (3, 12):
page.evaluate('''({ids,span}) => {
const hidden = ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id));
const sizes = Object.fromEntries(ids.map(id => [id, span]));
applyDashboardLayout([...ids, ...hidden], hidden, sizes);
}''', {'ids': ids, 'span': span})
measured = inspect(page)
results.append(measured)
for chart, value in measured.items():
assert value['width'] > 50, (screen_width, span, chart, value)
assert abs(value['width'] - value['contentWidth']) <= 2, (
'chart not filling its own SVG viewport', screen_width, span, chart, value)
assert value['preservation'] == 'none', (screen_width, chart, value)
if value['lineStroke'] is not None:
assert value['lineStroke'] == 'non-scaling-stroke', (chart, value)
if screen_width >= 1440:
for name in results[0]:
assert results[1][name]['width'] > results[0][name]['width'] * 1.4, (
'chart does not grow when widening card', screen_width, name, results)
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, screen_width
assert not errors, (screen_width, errors)
print('PASS SVG chart uses full width at spans 3 and 12:', screen_width)
page.close()
browser.close()
print('PASS resizable Overview chart content regression')
scripts/tests/dashboard_cpu_autoscale_0335_browser_test.py
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env python3
"""Legacy CPU plot regression: low and high real values on a fixed 0-100% axis.
Autoscaling was intentionally replaced by 0.33.5 fixed scaling. This test
retains the original history/reset and label checks using the current scale.
Chromium is simulated, not a live RustProxy server.
"""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
def inspect(page):
return page.evaluate('''() => {
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
const backendSvg = get('backend_traffic').querySelector('svg');
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
const bounds = cpu.getBoundingClientRect();
const matrix = cpuSvg.getScreenCTM();
return {
cpuHeight: cpuSvg.getBoundingClientRect().height,
protocolHeight: protocolSvg.getBoundingClientRect().height,
backendHeight: backendSvg.getBoundingClientRect().height,
svgWidth: cpuSvg.getBoundingClientRect().width,
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
overflow: document.documentElement.scrollWidth - innerWidth,
};
}''')
def render_cpu(page, revision, system, process):
return page.evaluate('''({revision,system,process}) => {
window.rustProxyWideOverview.render({revision,live_system:{
cpu_percent:system, process_cpu_percent:process
}});
const read = name => {
const prefix = `overview-wide-wide_cpu-${name}`;
return {
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
label: document.getElementById(`${prefix}-value`).textContent,
};
};
return {system:read('system'),process:read('process')};
}''', {'revision': revision, 'system': system, 'process': process})
def ys(path):
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': viewport, 'height': 950})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const grid = document.getElementById('dashboard-overview-grid');
grid.classList.add('is-aligned');
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic'].includes(card.dataset.dashboardWidget);
}
window.currentPanel = 'overview';
window.$ = id => document.getElementById(id);
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
window.fmtBytes = value => `${value} B`;
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
}''')
page.add_script_tag(content=SCALE_JS)
page.add_script_tag(content=WIDE_JS)
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
result = inspect(page)
assert result['protocolHeight'] >= 124, (viewport, density, result)
assert abs(result['cpuHeight'] - result['protocolHeight']) <= 1, (
'CPU must use standard chart height', viewport, density, result)
assert result['backendHeight'] > 0, (viewport, density, result)
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
# Measured 0.5-0.8% stay numerically visible without axis auto-expansion.
first = render_cpu(page, 1, 0.5, 0.2)
second = render_cpu(page, 1, 0.8, 0.6)
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
assert ys(second['system']['path']) == [136, 136], (viewport, second)
assert ys(second['process']['path']) == [137, 136], (viewport, second)
# Multi-core values display accurately but clip at the fixed 100% ceiling.
third = render_cpu(page, 1, 3.4, 2.2)
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
high = render_cpu(page, 1, 250, 123)
assert high['system']['label'] == '250.0%'
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
# Revision changes reset the history. Zero stays at the plot baseline.
idle = render_cpu(page, 2, 0, 0)
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
missing = render_cpu(page, 3, None, None)
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
assert not errors, (viewport, errors)
print(f'PASS legacy CPU labels/history on fixed Y-scale at {viewport}px (all densities)')
page.close()
browser.close()
scripts/tests/dashboard_cpu_fixed_scale_0335_browser_test.py
-149
View File
@@ -1,149 +0,0 @@
#!/usr/bin/env python3
"""0.33.5 fixed 0-100% CPU plot and height regression with production HTML/CSS/JS.
Uses Chromium with simulated authenticated DOM and status samples, not a live
RustProxy server, SSE stream or real CPU telemetry.
"""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALE_JS = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
WIDE_JS = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
def inspect(page):
return page.evaluate('''() => {
const get = id => document.querySelector(`#dashboard-overview-grid [data-dashboard-widget="${id}"]`);
const cpu = get('wide_cpu'), cpuSvg = cpu.querySelector('svg');
const protocolSvg = get('wide_protocol_rates').querySelector('svg');
const connection = get('connection_chart');
const backendSvg = get('backend_traffic').querySelector('svg');
const legend = cpu.querySelector('.overview-wide-legend').getBoundingClientRect();
const bounds = cpu.getBoundingClientRect();
const matrix = cpuSvg.getScreenCTM();
return {
cpuHeight: cpuSvg.getBoundingClientRect().height,
cpuCardHeight: bounds.height,
connectionCardHeight: connection.getBoundingClientRect().height,
protocolHeight: protocolSvg.getBoundingClientRect().height,
backendHeight: backendSvg.getBoundingClientRect().height,
svgWidth: cpuSvg.getBoundingClientRect().width,
drawnWidth: Math.hypot(matrix.a, matrix.b) * cpuSvg.viewBox.baseVal.width,
ratio: cpuSvg.getAttribute('preserveAspectRatio'),
cpuSeries: cpuSvg.querySelectorAll('.overview-wide-series').length,
lineEffect: getComputedStyle(cpuSvg.querySelector('.overview-wide-series')).vectorEffect,
gridEffect: getComputedStyle(cpuSvg.querySelector('.chart-grid')).vectorEffect,
gridPath: cpuSvg.querySelector('.chart-grid').getAttribute('d'),
legendInCard: legend.top >= bounds.top && legend.bottom <= bounds.bottom + 1,
overflow: document.documentElement.scrollWidth - innerWidth,
};
}''')
def render_cpu(page, revision, system, process):
return page.evaluate('''({revision,system,process}) => {
window.rustProxyWideOverview.render({revision,live_system:{
cpu_percent:system, process_cpu_percent:process
}});
const read = name => {
const prefix = `overview-wide-wide_cpu-${name}`;
return {
path: document.getElementById(`${prefix}-line`).getAttribute('d'),
label: document.getElementById(`${prefix}-value`).textContent,
};
};
return {system:read('system'),process:read('process')};
}''', {'revision': revision, 'system': system, 'process': process})
def ys(path):
return [int(v) for v in re.findall(r'[ML]\d+\s+(\d+)', path)]
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': viewport, 'height': 950})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const grid = document.getElementById('dashboard-overview-grid');
grid.classList.add('is-aligned');
for (const card of grid.querySelectorAll(':scope > [data-dashboard-widget]')) {
card.hidden = !['wide_cpu', 'wide_protocol_rates', 'backend_traffic', 'connection_chart'].includes(card.dataset.dashboardWidget);
}
window.currentPanel = 'overview';
window.$ = id => document.getElementById(id);
window.fmt = (value, digits = 0) => Number(value).toFixed(digits);
window.fmtBytes = value => `${value} B`;
window.rustProxyOverviewSamples = {getLimit: () => 60, onChange: () => {}};
}''')
page.add_script_tag(content=SCALE_JS)
page.add_script_tag(content=WIDE_JS)
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
result = inspect(page)
assert result['protocolHeight'] >= 124, (viewport, density, result)
assert abs(result['cpuHeight'] - 125) <= 1, (
'CPU height must remain fixed at 125px', viewport, density, result)
assert result['backendHeight'] > 0, (viewport, density, result)
assert result['ratio'] == 'none' and result['cpuSeries'] == 2, result
assert result['lineEffect'] == result['gridEffect'] == 'non-scaling-stroke', result
assert result['gridPath'] == 'M0 47H600 M0 77H600 M0 107H600', result
assert result['svgWidth'] > 100 and abs(result['svgWidth'] - result['drawnWidth']) < 2, result
assert result['legendInCard'] and result['overflow'] <= 2, (viewport, density, result)
# Low measured CPU values stay near zero instead of filling the plot.
first = render_cpu(page, 1, 0.5, 0.2)
second = render_cpu(page, 1, 0.8, 0.6)
assert first['system']['label'] == '0.5%' and second['process']['label'] == '0.6%'
assert ys(second['system']['path']) == [136, 136], (viewport, second)
assert ys(second['process']['path']) == [137, 136], (viewport, second)
# The Y axis is always 0-100% regardless of recorded history.
third = render_cpu(page, 1, 3.4, 2.2)
assert ys(third['system']['path'])[-1] == 133, (viewport, third)
assert ys(third['process']['path'])[-1] == 134, (viewport, third)
high = render_cpu(page, 1, 250, 123)
assert high['system']['label'] == '250.0%' and high['process']['label'] == '123.0%'
assert ys(high['system']['path'])[-1] == 17, (viewport, high)
assert ys(high['process']['path'])[-1] == 17, (viewport, high)
mid = render_cpu(page, 1, 50, 25)
assert ys(mid['system']['path'])[-1] == 77, (viewport, mid)
assert ys(mid['process']['path'])[-1] == 107, (viewport, mid)
# Revision changes reset history; missing readings do not plot a zero.
idle = render_cpu(page, 2, 0, 0)
assert ys(idle['system']['path']) == [137] and ys(idle['process']['path']) == [137]
full = render_cpu(page, 2, 100, 100)
assert ys(full['system']['path'])[-1] == 17 and ys(full['process']['path'])[-1] == 17
missing = render_cpu(page, 3, None, None)
assert missing['system']['path'] == '' and missing['process']['label'] == '-'
# Samples may change paths, but not the plot/card or neighboring card sizes.
for density in ('compact', 'normal', 'ultra'):
page.evaluate('(density) => document.documentElement.dataset.dashboardDensity = density', density)
unchanged = inspect(page)
assert abs(unchanged['cpuHeight'] - 125) <= 1, (viewport, density, unchanged)
for value in (0.1, 99.9, 250, 0.0):
before = inspect(page)
render_cpu(page, 4, value, value)
after = inspect(page)
for key in ('cpuHeight', 'cpuCardHeight', 'connectionCardHeight'):
assert abs(before[key] - after[key]) < 1, (viewport, density, value, key, before, after)
assert not errors, (viewport, errors)
print(f'PASS CPU fixed 0-100% Y-scale and 125px plot height at {viewport}px (all densities)')
page.close()
browser.close()
scripts/tests/dashboard_layout_0331_browser_test.py
-199
View File
@@ -1,199 +0,0 @@
#!/usr/bin/env python3
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
The authenticated app bundle runs against simulated status/preferences; this is
not a Rust server integration test.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
BUNDLE = app_bundle()
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
def inspect(page, ids):
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
}))''', ids)
def assert_visible(page, visible, hidden):
result = inspect(page, visible + hidden)
for widget in visible:
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
for widget in hidden:
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in [320, 375, 800, 1440]:
context = browser.new_context(viewport={'width': width, 'height': 950})
page = context.new_page()
errors = []
page.on('pageerror', lambda e: errors.append(str(e)))
load(page)
result = page.evaluate('''() => {
const prefs = ADMIN_APPEARANCE_DEFAULTS;
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = JSON.parse(JSON.stringify(prefs));
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
appConfirm = async () => true;
return {
order: ADMIN_DASHBOARD_WIDGETS,
hidden: ADMIN_DEFAULT_HIDDEN_WIDGETS,
density: document.documentElement.dataset.dashboardDensity,
samples:window.rustProxyOverviewSamples.getLimit(),
serviceCount:document.querySelectorAll('#dashboard-overview-grid [data-dashboard-widget]').length,
};
}''')
assert result['order'] == PRESET['dashboard_widgets'], (width, result['order'])
assert result['hidden'] == PRESET['dashboard_hidden_widgets'], (width, result['hidden'])
assert result['samples'] == 600 and result['density'] == 'compact' and result['serviceCount'] == 78, result
assert_visible(page, ['backend_traffic', 'extra_response_latency', 'wide_transform_activity'],
['backend_health', 'stream_sessions', 'www_status', 'bandwidth'])
# Reproduce the previously flaky operation: opt in hidden backend +
# Stream cards and opt out a visible backend card, WITHOUT saving yet.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
outcome = page.evaluate('''() => {
const add = id => {
const label = dashboardWidgetLabel(id);
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
.find(node => node.firstElementChild.textContent === label);
if (!item) throw new Error('no hidden card to restore: ' + id);
item.querySelector('button').click();
};
add('backend_health');
add('stream_sessions');
const card = document.querySelector('[data-dashboard-widget="backend_traffic"]');
card.querySelector('.overview-inline-hide').click();
return window.rustProxyInlineLayout.currentDraft();
}''')
assert 'backend_traffic' in outcome['hidden'] and 'backend_health' not in outcome['hidden']
assert 'stream_sessions' not in outcome['hidden']
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
for _ in range(5):
page.evaluate('''() => {
const status = {revision:4,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
}''')
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
# Unrelated settings update must not repaint the saved layout over the draft.
page.evaluate('''() => applyAdminPreferences({...window.__savedDashboard, ui_scale:105})''')
assert_visible(page, ['backend_health', 'stream_sessions'], ['backend_traffic'])
assert page.evaluate('window.rustProxyInlineLayout.isEditing()')
page.locator('#overview-inline-cancel').click()
assert not page.evaluate('window.rustProxyInlineLayout.isEditing()')
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions'])
# Save under live traffic, ensure the in-flight preference application
# uses the current draft before the editor exits.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
page.evaluate('''() => {
const label = dashboardWidgetLabel('backend_health');
const item = [...document.querySelectorAll('#overview-inline-hidden-list .overview-inline-hidden-item')]
.find(node => node.firstElementChild.textContent === label);
item.querySelector('button').click();
document.querySelector('[data-dashboard-widget="backend_traffic"] .overview-inline-hide').click();
}''')
page.locator('#overview-inline-save').click()
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
assert_visible(page, ['backend_health'], ['backend_traffic'])
saved = page.evaluate('''() => window.__savedDashboard.dashboard_hidden_widgets''')
assert 'backend_traffic' in saved and 'backend_health' not in saved
page.evaluate('''() => {
const status={backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
}''')
assert_visible(page, ['backend_health'], ['backend_traffic'])
# Restore factory layout through the new named-presets modal.
page.locator('#overview-layout-open').evaluate('(node) => node.hidden = false')
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert_visible(page, ['backend_traffic'], ['backend_health', 'stream_sessions', 'www_status'])
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
assert page.evaluate('window.__savedDashboard.dashboard_hidden_widgets') == PRESET['dashboard_hidden_widgets']
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2
assert not errors, (width, errors)
context.close()
print('PASS inline layout defaults, add/hide, live ticks, other preferences, cancel/save', width)
# An already-persisted sample choice must override the factory 600.
context = browser.new_context()
page = context.new_page()
# This sandbox blocks all network origins; stub browser storage before
# loading the bundle rather than navigating to a real host.
page.evaluate("""() => Object.defineProperty(window, 'localStorage', {
configurable: true,
value: {getItem: key => key === 'rustproxy.overview.liveChartSamples' ? '120' : null,
setItem: () => undefined},
})""")
load(page)
assert page.evaluate('window.rustProxyOverviewSamples.getLimit()') == 120
context.close()
browser.close()
print('PASS RustProxy 0.33.1 simulated browser layout regression')
scripts/tests/dashboard_metric_visuals_ui_test.mjs
-134
View File
@@ -1,134 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Deterministic Overview widget regression; simulated DOM/status, no running API.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const html = renderAdminIndex(root);
const registry = JSON.parse(fs.readFileSync(path.join(root, 'src/admin/web/dashboard-widgets.json'), 'utf8'));
const lang = JSON.parse(fs.readFileSync(path.join(root, 'langs/en_US.json'), 'utf8')).translations;
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
const widgets = new Map(), ids = new Map();
class Element {
constructor() { this.hidden = false; this.attributes = new Map(); this.style = {}; this.textContent = '-'; }
setAttribute(key, value) { this.attributes.set(key, String(value)); }
removeAttribute(key) { this.attributes.delete(key); }
getAttribute(key) { return this.attributes.get(key) ?? null; }
}
for (const [, id] of html.matchAll(/\bid="([^"]+)"/g)) {
assert.ok(!ids.has(id), `duplicate DOM id: ${id}`);
ids.set(id, new Element());
}
for (const [, id] of html.matchAll(/data-dashboard-widget="([^"]+)"/g)) widgets.set(id, new Element());
assert.equal(registry.length, widgets.size, 'registry and DOM widget count must match');
for (const entry of registry) assert.ok(widgets.has(entry.id), `registry references missing DOM card ${entry.id}`);
let now = 1000;
const document = {
hidden: false,
addEventListener() {},
querySelector(query) { return widgets.get(query.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) ?? null; },
};
const window = { addEventListener() {}, rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
const context = vm.createContext({
window, document, performance: { now: () => now },
currentPanel: 'overview', $: id => ids.get(id) ?? null,
fmt: (v, digits = 0) => Number(v).toFixed(digits),
fmtBytes: v => `${Number(v)} B`,
t: key => lang[key] || key,
console,
});
for (const script of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-extra-metrics.js', 'app/overview-wide-metrics.js']) {
vm.runInContext(code(script), context, { filename: script });
}
const extra = window.rustProxyExtraOverview, wide = window.rustProxyWideOverview;
const status = {
revision: 9,
system: { open_fds: 60, max_fds: 300 },
cache: { hits: 100, misses: 50, resident_bytes: 150, hot_limit_bytes: 300,
cold_queue_bytes: 25, cold_queue_limit_bytes: 100, cold_bytes: 400, cold_limit_bytes: 1000 },
metrics: { requests: 300, http1_requests: 200, http2_requests: 80, http3_requests: 20,
compression_gzip: 10, compression_br: 10, compression_zstd: 10, compression_deflate: 1 },
transform_metrics: { applied: 10, cache_hits: 10, skipped: 10, errors: 1 },
};
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_memory-meter-fill').style.width, '50%');
assert.equal(ids.get('overview-extra-extra_cold_queue_memory-meter-fill').style.width, '25%');
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '40%');
widgets.get('extra_cache_memory').hidden = false;
widgets.get('extra_cache_hit_rate').hidden = false;
widgets.get('extra_h2_share').hidden = false;
widgets.get('extra_h1_share').hidden = false;
widgets.get('extra_h3_share').hidden = false;
widgets.get('extra_process_fds').hidden = false;
extra.render(status);
assert.equal(ids.get('overview-extra-extra_process_fds-meter-fill').style.width, '20%');
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-'); // first counter sample has no rate
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '-');
now = 4000;
status.cache.hits += 30;
status.cache.misses += 10;
status.metrics.requests += 40;
status.metrics.http1_requests += 10;
status.metrics.http2_requests += 20;
status.metrics.http3_requests += 10;
status.metrics.compression_gzip += 3;
status.metrics.compression_br += 6;
status.metrics.compression_zstd += 9;
status.metrics.compression_deflate += 12;
status.transform_metrics.applied += 15;
status.transform_metrics.cache_hits += 3;
status.transform_metrics.skipped += 6;
status.transform_metrics.errors += 1;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '75.0%');
assert.equal(ids.get('extra-cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
assert.equal(ids.get('extra-cache-ratio-miss-arc').getAttribute('stroke-dashoffset'), '-75');
assert.equal(ids.get('extra-cache-ratio-hit-count').textContent, '30');
assert.equal(ids.get('extra-cache-ratio-miss-count').textContent, '10');
assert.equal(ids.get('overview-extra-extra_h2_share-meter-fill').style.width, '50%');
assert.equal(ids.get('overview-wide-wide_transform_activity-summary').textContent, '8.3/s');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '10.0/s');
for (const name of ['applied', 'cache_hits', 'skipped', 'errors'])
assert.ok(ids.get(`overview-wide-wide_transform_activity-${name}-line`).getAttribute('d')?.length);
for (const name of ['gzip', 'br', 'zstd', 'deflate'])
assert.ok(ids.get(`overview-wide-wide_compression_activity-${name}-line`).getAttribute('d')?.length);
// No hidden-tab spike or fake zero when a long interval is skipped.
now = 100000;
status.cache.hits += 300;
status.cache.misses += 200;
status.metrics.compression_gzip += 600;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '-');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
now = 103000;
status.cache.hits += 5;
status.cache.misses += 5;
status.metrics.compression_gzip += 3;
extra.render(status);
wide.render(status);
assert.equal(ids.get('overview-extra-extra_cache_hit_rate').textContent, '50.0%');
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '1.0/s');
// Bounded metric with no configured budget remains unknown, not a made-up percentage.
status.cache.cold_limit_bytes = 0;
extra.render(status);
assert.equal(ids.get('overview-extra-extra_cold_storage').textContent, '400 B');
assert.equal(ids.get('overview-extra-extra_cold_storage-meter').getAttribute('aria-valuenow'), null);
assert.equal(ids.get('overview-extra-extra_cold_storage-meter-fill').style.width, '0%');
extra.donut('cache-ratio', 75, 25);
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '75 25');
assert.equal(ids.get('cache-ratio-chart').getAttribute('aria-label'), 'HIT 75, MISS 25');
extra.donut('cache-ratio', 0, 0);
assert.equal(ids.get('cache-ratio-hit-arc').getAttribute('stroke-dasharray'), '0 100');
assert.ok(code('app/runtime.js').includes('rustProxyExtraOverview?.donut?.("cache-ratio"'),
'main HIT/MISS tile must share tested donut rendering');
console.log('PASS: bounded meters, COLD tiers, HIT/MISS rings, transform and compression rates, pause/no-limit');
scripts/tests/dashboard_modal_0333_browser_test.py
-176
View File
@@ -1,176 +0,0 @@
#!/usr/bin/env python3
"""Regression: named Overview presets, responsive modal and browser actions.
Runs the real panel DOM/CSS/bundled JavaScript with mocked preferences/API.
This is browser layout testing, not Rust server acceptance.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
PACKS = {
name: {'id': name, **json.loads((ROOT / 'langs' / (name + '.json')).read_text())}
for name in ('en_US', 'pl_pl')
}
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
BUNDLE = app_bundle()
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyDashboardLayout')
page.evaluate('''packs => {
languagePacks = new Map(Object.entries(packs));
defaultLanguage = 'en_US';
language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
}''', PACKS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
document.getElementById('overview-layout-open').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.querySelectorAll('#overview-presets-dialog [data-i18n]').forEach(node => node.textContent = t(node.dataset.i18n));
}''')
# Regression for the replacement named-presets modal, with a simulated SQLite API.
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 800, 1440):
context = browser.new_context(viewport={'width': width, 'height': 880})
page = context.new_page()
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
load(page)
page.evaluate('''() => {
const saved = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.__savedDashboard = saved;
window.__presets = new Map();
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
api = async (path, options={}) => {
const method = options.method || 'GET';
if (path === '/api/dashboard-presets' && method === 'GET')
return [...window.__presets.entries()].map(([name, data]) => ({ name, ...structuredClone(data) }));
if (path.startsWith('/api/dashboard-presets/')) {
const name = decodeURIComponent(path.split('/').slice(3).join('/'));
if (method === 'PUT') { window.__presets.set(name, JSON.parse(options.body)); return {}; }
if (method === 'DELETE') { window.__presets.delete(name); return {}; }
if (method === 'PATCH') {
const changed = JSON.parse(options.body).name;
if (window.__presets.has(changed)) throw new Error('already exists');
window.__presets.set(changed, window.__presets.get(name));
window.__presets.delete(name);
return {};
}
}
throw new Error('unexpected API: ' + method + ' ' + path);
};
appConfirm = async () => true;
applyAdminPreferences(window.__savedDashboard);
document.getElementById('overview-layout-open').hidden = false;
}''')
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
assert page.locator('#overview-presets-list .overview-presets-row').count() == 0
widths = page.evaluate('''() => {
const d = document.getElementById('overview-presets-dialog');
const r = d.getBoundingClientRect();
return {left:r.left,right:r.right,overflow:d.scrollWidth - d.clientWidth};
}''')
assert widths['left'] >= -1 and widths['right'] <= width+1 and widths['overflow'] <= 2, (width, widths)
# Persist the factory arrangement as a user preset.
page.locator('#overview-presets-name').fill('Daily')
page.locator('#overview-presets-save').click()
page.wait_for_function("window.__presets.has('Daily')")
assert page.locator('#overview-presets-list .overview-presets-row').count() == 1
assert page.evaluate("window.__presets.get('Daily').overview_samples") == 600
# Create another view, with customized widths, density and history.
page.evaluate('''async () => {
await window.rustProxyPreferences.update({
dashboard_hidden_widgets:['requests'], dashboard_widget_sizes:{requests:3},
dashboard_density:'ultra',
});
window.rustProxyOverviewSamples.setLimit(120, true);
}''')
page.locator('#overview-presets-name').fill('Detailed')
page.locator('#overview-presets-save').click()
page.wait_for_function("window.__presets.has('Detailed')")
assert page.evaluate('''() => {
const p=window.__presets.get('Detailed');
return [p.dashboard_widget_sizes.requests,p.dashboard_density,p.overview_samples];
}''') == [3,'ultra',120]
# Named load must restore full layout and history.
page.locator('.overview-presets-row').filter(has_text='Daily').locator('[data-preset-action="dashboardPresetsLoad"]').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate('''() => [window.__savedDashboard.dashboard_density,
window.__savedDashboard.dashboard_widget_sizes,
window.rustProxyOverviewSamples.getLimit()]''') == ['compact',None,600]
# Rename and delete remain administrator-scoped operations.
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-name').fill('Nightly')
page.locator('.overview-presets-row').filter(has_text='Detailed').locator('[data-preset-action="dashboardPresetsRename"]').click()
page.wait_for_function("window.__presets.has('Nightly')")
assert not page.evaluate("window.__presets.has('Detailed')")
page.locator('.overview-presets-row').filter(has_text='Nightly').locator('[data-preset-action="dashboardPresetsDelete"]').click()
page.wait_for_function("!window.__presets.has('Nightly')")
assert page.evaluate("window.__presets.has('Daily')")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate("window.__savedDashboard.dashboard_widgets") == json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())['dashboard_widgets']
assert not errors, (width, errors)
print('PASS named Dashboard presets save/load/rename/delete/factory and responsive modal:', width)
context.close()
browser.close()
print('PASS Overview preset modal (simulated SQLite API; no running Rust server)')
scripts/tests/dashboard_overview_labels_0337_browser_test.py
-102
View File
@@ -1,102 +0,0 @@
#!/usr/bin/env python3
"""All Overview widget labels and small values in simulated Chromium.
This is a DOM/CSS regression test, not a live RustProxy integration test.
"""
from pathlib import Path
import json
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
PACKS = {name: json.loads((ROOT / f'langs/{name}.json').read_text(encoding='utf-8'))['translations']
for name in ('en_US', 'pl_pl')}
CHECK = r'''({density, lang, translations}) => {
document.documentElement.dataset.dashboardDensity = density;
document.documentElement.lang = lang === 'pl_pl' ? 'pl-PL' : 'en-US';
const grid = document.getElementById('dashboard-overview-grid');
grid.querySelectorAll('[data-i18n]').forEach(node => {
if (Object.hasOwn(translations, node.dataset.i18n)) node.textContent = translations[node.dataset.i18n];
});
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
const font = value => getComputedStyle(value).fontFamily;
const probe = document.createElement('div');
grid.append(probe);
probe.style.fontFamily = 'var(--sans)'; const sans = font(probe);
probe.style.fontFamily = 'var(--mono)'; const mono = font(probe);
probe.remove();
const titleErrors = [];
const titles = [];
const labelErrors = [];
const numericErrors = [];
for (const card of cards) {
let title;
if (card.classList.contains('stat')) title = card.querySelector(':scope > span:first-child');
else if (card.classList.contains('system-card') || card.classList.contains('connection-card'))
title = card.querySelector('.overview-chart-heading');
else if (card.querySelector('.mini-heading > h3')) title = card.querySelector('.mini-heading > h3');
else title = card.querySelector('.card-heading h2');
if (!title) { titleErrors.push(`${card.dataset.dashboardWidget}: missing title`); continue; }
const style = getComputedStyle(title);
titles.push(title);
if (style.fontFamily !== sans || style.fontSize !== '12px' || style.textTransform !== 'none')
titleErrors.push(`${card.dataset.dashboardWidget}: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
if (title.textContent.trim() === title.textContent.trim().toLocaleUpperCase('pl') &&
/[a-ząćęłńóśżź]{4}/i.test(title.textContent) &&
!/^(?:HIT|MISS|HTTP|CPU|RAM|WWW|RX|TX)/.test(title.textContent.trim()))
titleErrors.push(`${card.dataset.dashboardWidget}: all-uppercase title ${title.textContent.trim()}`);
for (const node of card.querySelectorAll(':scope > small, .mini-metrics dt, .mini-metrics dd, .mini-live-chart-head, .memory-line, .chart-footer, .meta, .small-label')) {
if (font(node) !== mono) labelErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
}
const readings = card.classList.contains('stat')
? [...card.querySelectorAll(':scope > strong'), ...card.querySelectorAll(':scope > small')]
: [...card.querySelectorAll('.mini-metrics dd')];
for (const node of readings) if (font(node) !== mono)
numericErrors.push(`${card.dataset.dashboardWidget}: ${node.outerHTML.slice(0, 85)}`);
}
const nested = [...grid.querySelectorAll('.mini-live-chart > .mini-live-chart-head > span:first-child')];
for (const node of nested) {
const style = getComputedStyle(node);
if (style.fontFamily !== mono || style.fontSize !== '10px' || style.textTransform !== 'none')
labelErrors.push(`mini-live-chart: ${style.fontFamily} ${style.fontSize} ${style.textTransform}`);
}
const systemNumeric = getComputedStyle(document.querySelector('#system-cpu'));
return {
cardCount: cards.length, titleCount: titles.length, secondaryCount: nested.length,
titleErrors, labelErrors, numericErrors,
systemNumericMono: systemNumeric.fontFamily === mono,
cpuLabel: document.querySelector('[data-dashboard-widget="system_chart"] .overview-chart-heading').textContent,
connectionsLabel: document.querySelector('[data-dashboard-widget="connection_chart"] .overview-chart-heading').textContent,
memoryLabel: document.querySelector('[data-dashboard-widget="memory"] > span:first-child').textContent,
backendLabel: document.querySelector('[data-dashboard-widget="backend_health"] > span:first-child').textContent,
diagnosticsTitle: document.querySelector('[data-dashboard-widget="process_diagnostics"] .mini-heading h3').textContent,
};
}'''
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 900})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
for density in ('normal', 'compact', 'ultra'):
for lang in ('en_US', 'pl_pl'):
result = page.evaluate(CHECK, {'density': density, 'lang': lang, 'translations': PACKS[lang]})
descriptor = (width, density, lang)
assert result['cardCount'] == result['titleCount'] >= 75, (descriptor, result)
assert result['secondaryCount'] >= 6, descriptor
assert not result['titleErrors'], (descriptor, result['titleErrors'][:10])
assert not result['labelErrors'], (descriptor, result['labelErrors'][:10])
assert not result['numericErrors'], (descriptor, result['numericErrors'][:10])
assert result['systemNumericMono'], descriptor
expected = ('System load', 'Connection activity') if lang == 'en_US' else ('Obciążenie systemu', 'Aktywność połączeń')
assert (result['cpuLabel'], result['connectionsLabel']) == expected, (descriptor, result)
assert result['memoryLabel'] and result['backendLabel'] and result['diagnosticsTitle'], descriptor
print(f'PASS all Overview titles and small readings at {width}px (3 densities, 2 languages)')
page.close()
browser.close()
scripts/tests/dashboard_presets_sql_test.py
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env python3
"""Offline SQLite regression for additive admin preset migration and 16-slot upsert."""
from pathlib import Path
import sqlite3
ROOT = Path(__file__).resolve().parents[2]
all_scripts = sorted((ROOT / 'migrations').glob('*.sql'))
assert [p.name for p in all_scripts][-1] == '0010_dashboard_presets.sql'
con = sqlite3.connect(':memory:')
con.execute('PRAGMA foreign_keys = ON')
for _ in range(2):
for script in all_scripts:
con.executescript(script.read_text())
con.execute("INSERT INTO administrator(id,username,password_hash,generation,created_at) VALUES(1,'owner','hash',1,0)")
con.execute("INSERT INTO admin_preferences(administrator_id,document) VALUES(1,'{}')")
upsert = '''
INSERT INTO dashboard_presets(administrator_id, name, document, updated_at)
SELECT a.id, ?, ?, ? FROM administrator a WHERE a.id = 1 AND a.username = ?
AND ((SELECT count(*) FROM dashboard_presets WHERE administrator_id = a.id) < 16
OR EXISTS(SELECT 1 FROM dashboard_presets WHERE administrator_id = a.id AND name = ?))
ON CONFLICT(administrator_id, name) DO UPDATE SET document = excluded.document, updated_at = excluded.updated_at
'''
for i in range(16):
name = f'Preset {i}'
assert con.execute(upsert, (name,'{}',i,'owner',name)).rowcount == 1
assert con.execute(upsert, ('Overflow','{}',16,'owner','Overflow')).rowcount == 0
assert con.execute(upsert, ('Preset 0','{"changed":true}',17,'owner','Preset 0')).rowcount == 1
assert con.execute(upsert, ('Other account','{}',18,'intruder','Other account')).rowcount == 0
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 16
assert con.execute('SELECT document FROM dashboard_presets WHERE name="Preset 0"').fetchone()[0] == '{"changed":true}'
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Preset 1','Preset 0')).rowcount == 0
assert con.execute('UPDATE OR IGNORE dashboard_presets SET name=? WHERE name=?', ('Renamed','Preset 0')).rowcount == 1
con.execute('DELETE FROM administrator WHERE id=1')
assert con.execute('SELECT count(*) FROM dashboard_presets').fetchone()[0] == 0
assert con.execute('SELECT count(*) FROM admin_preferences').fetchone()[0] == 0
print('PASS SQLite migrations repeated, named-preset CRUD, cap, account ownership and cascade')
scripts/tests/dashboard_resizing_0333_browser_test.py
-200
View File
@@ -1,200 +0,0 @@
#!/usr/bin/env python3
"""0.33.1 factory/default + unsaved layout regression against the full bundled JS.
The authenticated app bundle runs against simulated status/preferences; this is
not a Rust server integration test.
"""
import json
import re
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
PRESET = json.loads((ROOT / 'src/admin/web/dashboard-defaults.json').read_text())
REGISTRY = json.loads((ROOT / 'src/admin/web/dashboard-widgets.json').read_text())
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert match
body = ROUTER[match.end():]
body = body[:re.search(r'\]\s*\.join\("\\n"\)', body).start()]
result = []
for line in body.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
result.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
assert len(PRESET['dashboard_widgets']) == len(REGISTRY)
assert set(PRESET['dashboard_widgets']) == {w['id'] for w in REGISTRY}
assert len(set(PRESET['dashboard_hidden_widgets'])) == len(PRESET['dashboard_hidden_widgets'])
assert set(PRESET['dashboard_hidden_widgets']).issubset(set(PRESET['dashboard_widgets']))
assert PRESET['dashboard_density'] == 'compact' and PRESET['overview_samples'] == 600
assert sum(w['id'].startswith('www_') and w['id'] in PRESET['dashboard_hidden_widgets'] for w in REGISTRY) == 12
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
BUNDLE = app_bundle()
assert BUNDLE.index('const ADMIN_DASHBOARD_DEFAULTS =') < BUNDLE.index('window.rustProxyOverviewSamples =')
def load(page):
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=BUNDLE)
page.wait_for_function('window.rustProxyInlineLayout && window.rustProxyPreferences && window.rustProxyBackendDashboard')
def inspect(page, ids):
return page.evaluate('''ids => Object.fromEntries(ids.map(id => {
const node = document.querySelector(`[data-dashboard-widget="${id}"]`);
return [id, {hidden:node.hidden, display:getComputedStyle(node).display}];
}))''', ids)
def assert_visible(page, visible, hidden):
result = inspect(page, visible + hidden)
for widget in visible:
assert result[widget]['hidden'] is False and result[widget]['display'] != 'none', (widget, result)
for widget in hidden:
assert result[widget]['hidden'] is True and result[widget]['display'] == 'none', (widget, result)
# Test actual production CSS/JS in a browser with simulated preferences/status.
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in [320, 375, 800, 1440, 1920]:
context = browser.new_context(viewport={'width': width, 'height': 1000})
page = context.new_page()
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
load(page)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('overview-layout-unlock').hidden = false;
document.getElementById('overview-layout-open').hidden = false;
currentPanel = 'overview';
window.__savedDashboard = structuredClone(ADMIN_APPEARANCE_DEFAULTS);
window.rustProxyPreferences.load = async () => window.__savedDashboard;
window.rustProxyPreferences.current = () => window.__savedDashboard;
window.rustProxyPreferences.update = async patch => {
window.__savedDashboard = {...window.__savedDashboard, ...patch};
applyAdminPreferences(window.__savedDashboard);
return window.__savedDashboard;
};
applyAdminPreferences(window.__savedDashboard);
api = async path => { if (path === '/api/dashboard-presets') return []; throw new Error('unexpected API ' + path); };
appConfirm = async () => true;
}''')
# Unchanged previous accounts preserve historic grouped rows.
assert page.evaluate('''() => ({
legacy: document.getElementById('dashboard-overview-grid').dataset.overviewLayoutMode,
widths:window.__savedDashboard.dashboard_widget_sizes,
})''') == {'legacy':'legacy','widths':None}
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
# Widen a number+sparkline card from 2 to 3 and then 4 columns.
selector = '[data-dashboard-widget="http_protocol"]'
old_width = page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width')
page.locator(selector + ' .overview-inline-widen').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '3'
page.locator(selector + ' .overview-inline-widen').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
assert page.locator(selector).evaluate('(node) => node.getBoundingClientRect().width') >= old_width * 1.7
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().sizes.http_protocol') == 4
# Minimize a large multi-series chart to the new 3-column option.
for _ in range(2):
page.locator('[data-dashboard-widget="wide_transform_activity"] .overview-inline-narrow').click()
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
# Real UI controls allow crossing old groups: send mini-chart to the front
# with the upward arrow, then across the original stat/chart boundary.
# A direct drag onto the first card also uses the actual event handlers.
page.evaluate('''() => {
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
const transfer = new DataTransfer();
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true, cancelable:true, dataTransfer:transfer}));
to.dispatchEvent(new DragEvent('drop', {bubbles:true, cancelable:true, dataTransfer:transfer, clientX:0}));
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
}''')
assert page.evaluate('window.rustProxyInlineLayout.currentDraft().order[0]') == 'wide_transform_activity'
# Background status and unrelated preference changes cannot reset live sizes.
page.evaluate('''() => {
const status={revision:5,backend_pools:[],streams:{active:[],desired:[],metrics:{}}};
window.rustProxyBackendDashboard.render(status);
window.rustProxyStreamDashboard.render(status);
applyAdminPreferences({...window.__savedDashboard, chart_style:'soft'});
}''')
assert page.locator(selector).get_attribute('data-dashboard-span') == '4'
# Cancel should remove both size and layout changes.
page.locator('#overview-inline-cancel').click()
assert page.locator(selector).get_attribute('data-dashboard-span') == '2'
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '6'
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
# The actual unlocked dashboard remains the only layout editor.
page.locator('#overview-layout-unlock').click()
page.wait_for_function('window.rustProxyInlineLayout.isEditing()')
tile = '[data-dashboard-widget="wide_transform_activity"]'
page.locator(tile + ' .overview-inline-narrow').click()
assert page.locator(tile).get_attribute('data-dashboard-span') == '4'
page.locator(tile + ' .overview-inline-narrow').click()
assert page.locator(tile).get_attribute('data-dashboard-span') == '3'
page.evaluate('''() => {
const from = document.querySelector('[data-dashboard-widget="wide_transform_activity"] .overview-inline-grip');
const to = document.querySelector('[data-dashboard-widget="requests"] .overview-inline-grip');
const transfer = new DataTransfer();
from.dispatchEvent(new DragEvent('dragstart', {bubbles:true,cancelable:true,dataTransfer:transfer}));
to.dispatchEvent(new DragEvent('drop', {bubbles:true,cancelable:true,dataTransfer:transfer,clientX:0}));
from.dispatchEvent(new DragEvent('dragend', {bubbles:true}));
}''')
page.locator('#overview-inline-save').click()
page.wait_for_function('!window.rustProxyInlineLayout.isEditing()')
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes.wide_transform_activity') == 3
assert page.evaluate('window.__savedDashboard.dashboard_widgets[0]') == 'wide_transform_activity'
# Reload account settings: new card width persists.
page.evaluate('applyAdminPreferences(window.__savedDashboard)')
assert page.locator('[data-dashboard-widget="wide_transform_activity"]').get_attribute('data-dashboard-span') == '3'
assert page.evaluate('window.__savedDashboard.dashboard_density') == 'compact'
# Four charts in one row, set through the same layout function used by UI.
data = page.evaluate('''() => {
const ids=['wide_transform_activity','wide_compression_activity','http_protocol','websocket_protocol'];
const order=[...ids,...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
const hidden=[...ADMIN_DASHBOARD_WIDGETS.filter(id => !ids.includes(id))];
const sizes=Object.fromEntries(ids.map(id => [id,3]));
applyDashboardLayout(order,hidden,sizes);
const grid=document.getElementById('dashboard-overview-grid');
return { gridWidth:grid.getBoundingClientRect().width,
template:getComputedStyle(grid).gridTemplateColumns.split(' ').length,
boxes:ids.map(id=>{const b=document.querySelector(`[data-dashboard-widget="${id}"]`).getBoundingClientRect();return {id,x:Math.round(b.x),y:Math.round(b.y),w:Math.round(b.width),scroll:b.width};})
};
}''')
if data['template'] == 12:
assert len(set(b['y'] for b in data['boxes'])) == 1, (width, data)
assert all(b['w'] > 150 for b in data['boxes']), (width,data)
assert page.evaluate('document.documentElement.scrollWidth - window.innerWidth') <= 2, (width, data)
page.locator('#overview-layout-open').click()
page.wait_for_function("document.getElementById('overview-presets-dialog').open")
page.locator('#overview-presets-factory').click()
page.wait_for_function("!document.getElementById('overview-presets-dialog').open")
assert page.evaluate('window.__savedDashboard.dashboard_widget_sizes') is None
assert page.evaluate('window.__savedDashboard.dashboard_widgets') == PRESET['dashboard_widgets']
assert page.evaluate('document.getElementById("dashboard-overview-grid").dataset.overviewLayoutMode') == 'legacy'
assert not errors, (width, errors)
print('PASS free-order, resize, live refresh, cancel/save, four chart grid', width, data['template'])
context.close()
browser.close()
print('PASS 0.33.3 resizable Overview browser regression')
scripts/tests/dashboard_resume_rate_charts_ui_test.mjs
-115
View File
@@ -1,115 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Simulated DOM/status regression: no holes or artificial zeros after tab resume.
// This intentionally does not exercise the running Rust server or real browser.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const code = file => fs.readFileSync(path.join(root, 'src/admin/web', file), 'utf8');
const html = renderAdminIndex(root);
class Element {
constructor() { this.attributes = new Map(); this.style = {}; this.textContent = ''; this.children = []; this.hidden = false; }
getAttribute(key) { return this.attributes.get(key) ?? null; }
setAttribute(key, value) { this.attributes.set(key, String(value)); }
removeAttribute(key) { this.attributes.delete(key); }
append(...children) { this.children.push(...children); }
replaceChildren(...children) { this.children = children; }
}
const ids = new Map([...html.matchAll(/\bid="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
const widgets = new Map([...html.matchAll(/data-dashboard-widget="([^"]+)"/g)].map(([, id]) => [id, new Element()]));
const listeners = new Map();
const document = {
hidden: false,
addEventListener: (type, handler) => listeners.set(type, handler),
querySelector: selector => widgets.get(selector.match(/^\[data-dashboard-widget="([^"]+)"\]$/)?.[1]) || null,
createElement: () => new Element(),
};
const samples = {
getLimit: () => 60,
trim: values => { if (values.length > 60) values.splice(0, values.length - 60); },
onChange: () => {},
};
const window = {
addEventListener() {}, rustProxyOverviewSamples: samples,
rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) },
};
let now = 1000;
const context = vm.createContext({
window, document, performance: { now: () => now }, currentPanel: 'overview', latestStatus: null,
$: id => ids.get(id) || null, fmt: (value, n = 0) => Number(value).toFixed(n),
fmtBytes: value => `${value} B`, t: value => value,
});
for (const file of ['app/rate-sampling.js', 'app/chart-scales.js', 'app/overview-wide-metrics.js', 'app/web-overview.js'])
vm.runInContext(code(file), context, { filename: file });
const web = {
requests: 200, status_2xx: 190, status_3xx: 2, status_4xx: 5, status_5xx: 3,
static_requests: 100, php_requests: 80, other_requests: 20,
methods: [160, 20, 10, 5, 5], selected_status: [0, 0, 0, 0, 0, 0, 0, 0],
response_headers_us: 300000, response_bytes: 1200000, active_headers: 2,
};
const php = {
metrics: { requests: 80, response_headers_us: 100000, response_bytes: 320000, failures: Array(9).fill(0) },
capacity_limit: 6, capacity_used: 1,
};
const status = {
revision: 4,
metrics: { requests: 200, http1_requests: 120, http2_requests: 60, http3_requests: 20,
compression_gzip: 10, compression_br: 20, compression_zstd: 5, compression_deflate: 8 },
cache: { hits: 50, misses: 30 },
transform_metrics: { applied: 10, cache_hits: 5, skipped: 1, errors: 0 },
web_server_metrics: { demo: { web, php } },
};
function advance(delta) {
for (const key of Object.keys(status.metrics)) status.metrics[key] += delta;
for (const key of ['hits', 'misses']) status.cache[key] += delta;
for (const key of Object.keys(status.transform_metrics)) status.transform_metrics[key] += delta;
for (const key of ['requests', 'status_2xx', 'static_requests', 'php_requests', 'response_headers_us', 'response_bytes']) web[key] += delta;
for (const key of ['requests', 'response_headers_us', 'response_bytes']) php.metrics[key] += delta;
}
function render() { window.rustProxyWideOverview.render(status); window.rustProxyWebOverview.render(status); }
const line = id => ids.get(id).getAttribute('d') || '';
const compression = 'overview-wide-wide_compression_activity-gzip-line';
const website = 'www_requests-all-line';
const phpLine = 'www_php_requests-line';
const starts = path => (path.match(/M/g) || []).length;
render(); // no first-interval rate: establish baseline only
now = 4000; advance(12); render();
const beforeWide = line(compression), beforeWeb = line(website);
const beforePhp = ids.get(phpLine).getAttribute('points');
assert.equal(starts(beforeWide), 1);
assert.equal(starts(beforeWeb), 1);
assert.ok(beforePhp);
// Background tab -> a new generation, regardless of counter increments.
document.hidden = true; listeners.get('visibilitychange')();
document.hidden = false;
now = 100000; advance(600); render();
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '-');
assert.equal(ids.get('www_requests-value').textContent, '-');
assert.equal(line(compression), beforeWide, 'wide rate chart must not receive a null point');
assert.equal(line(website), beforeWeb, 'WWW rate chart must not receive a null point');
assert.equal(ids.get(phpLine).getAttribute('points'), beforePhp, 'small PHP chart must preserve valid history');
// A fresh interval then connects to previous valid samples without a gap.
now = 103000; advance(9); render();
assert.equal(starts(line(compression)), 1, 'wide chart must have one connected path');
assert.equal(starts(line(website)), 1, 'WWW chart must have one connected path');
assert.ok(line(compression).includes('L'));
assert.ok(line(website).includes('L'));
// Real measured zeros must continue to be plotted, not treated as pauses.
now = 106000; render();
assert.equal(ids.get('overview-wide-wide_compression_activity-summary').textContent, '0.0/s');
assert.equal(ids.get('www_requests-value').textContent, '0.0/s');
assert.equal(starts(line(compression)), 1);
assert.equal(starts(line(website)), 1);
assert.match(code('web-server.js'), /if \(record && rate !== null\) \{ previous\.push\(rate\)/,
'Web Server chart must also skip unavailable rates instead of plotting a zero dip');
console.log('PASS: resumed wide/WWW/PHP charts stay connected; first invalid rate is unavailable; measured zero persists');
scripts/tests/dashboard_stable_scales_0336_browser_test.py
-129
View File
@@ -1,129 +0,0 @@
#!/usr/bin/env python3
"""0.33.6 plot size/scale checks in simulated Chromium, not a running RustProxy."""
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
SCALES = (ROOT / 'src/admin/web/app/chart-scales.js').read_text(encoding='utf-8')
RATES = (ROOT / 'src/admin/web/app/rate-sampling.js').read_text(encoding='utf-8')
WIDE = (ROOT / 'src/admin/web/app/overview-wide-metrics.js').read_text(encoding='utf-8')
WIDE_IDS = ('wide_protocol_rates', 'wide_cpu', 'wide_memory', 'wide_cache_activity',
'wide_connections', 'wide_backend_health', 'wide_latency',
'wide_transform_activity', 'wide_compression_activity',
'www_requests', 'www_latency', 'www_failures', 'www_traffic', 'www_status_trend')
with sync_playwright() as api:
browser = api.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 920})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading','app-startup-failed');
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
document.getElementById('dashboard-overview-grid').classList.add('is-aligned');
for (const card of document.querySelectorAll('#dashboard-overview-grid>[data-dashboard-widget]'))
card.hidden = !['wide_protocol_rates','wide_cpu','wide_memory','wide_cache_activity',
'wide_connections','wide_backend_health','wide_latency',
'wide_transform_activity','wide_compression_activity',
'www_requests','www_latency','www_failures','www_traffic','www_status_trend'].includes(card.dataset.dashboardWidget);
window.currentPanel='overview';
window.$ = id => document.getElementById(id);
window.fmt = (n, digits=0) => Number(n).toFixed(digits);
window.fmtBytes = n => `${n} B`;
window.rustProxyOverviewSamples = {getLimit:()=>60,onChange:()=>{}};
window.rustProxyRefresh = {current:()=>({overview_seconds:3})};
window.__clock = 1000;
Object.defineProperty(performance,'now',{configurable:true,value:()=>window.__clock});
}''')
for script in (RATES, SCALES, WIDE):
page.add_script_tag(content=script)
for density in ('normal','compact','ultra'):
measured = page.evaluate('''density => {
document.documentElement.dataset.dashboardDensity=density;
const results={};
for (const card of document.querySelectorAll('#dashboard-overview-grid > .overview-wide-chart')) {
const svg=card.querySelector('svg.overview-wide-svg');
if (!svg) continue;
const r=svg.getBoundingClientRect();
results[card.dataset.dashboardWidget]={height:r.height, width:r.width,
grid:svg.querySelector('.chart-grid').getAttribute('d'),
contentWidth:Math.hypot(svg.getScreenCTM().a,svg.getScreenCTM().b)*600};
}
document.getElementById('panel-transform').hidden=false;
document.getElementById('panel-cache').hidden=false;
for(const id of ['transform-live-chart','cache-live-chart'])document.getElementById(id).open=true;
results.transformDetails=document.getElementById('transform-live-activity-chart').getBoundingClientRect().height;
results.cacheDetails=document.getElementById('cache-live-activity-chart').getBoundingClientRect().height;
return results;
}''', density)
assert abs(measured['transformDetails'] - 180) < 1, (width,density,measured)
assert abs(measured['cacheDetails'] - 180) < 1, (width,density,measured)
for name in WIDE_IDS:
data=measured[name]
assert abs(data['height']-125)<1, (width,density,name,data)
assert data['width']>50 and abs(data['width']-data['contentWidth'])<2, (width,density,name,data)
assert data['grid']=='M0 47H600 M0 77H600 M0 107H600', (width,density,name,data)
result = page.evaluate('''() => {
const c=window.rustProxyChartScale;
const first=c.peak('probe:rate','rate',[[.2,.7]]);
const count=c.peak('probe:count','count',[[0,1]]);
const latency=c.peak('probe:ms','ms',[[.1,1.5]]);
const memory=c.peak('probe:bytes','bytes',[[1000]]);
const high=c.peak('probe:rate','rate',[[101]]);
const held=c.peak('probe:rate','rate',[[.3]]);
c.reset('probe:');
const reset=c.peak('probe:rate','rate',[[.3]]);
const percent=c.peak('probe:cpu','percent',[[.1,500]]);
const blank=c.peak('probe:blank','rate',[[null,NaN,Infinity]]);
const before={revision:10,live_system:{cpu_percent:0.7,process_cpu_percent:0.2},
metrics:{compression_gzip:50,compression_br:100,compression_zstd:10,compression_deflate:5},
transform_metrics:{applied:2,cache_hits:10,skipped:1,errors:0}};
window.rustProxyWideOverview.render(before);
window.__clock=4000;
before.metrics.compression_gzip+=3;before.metrics.compression_br+=6;
before.metrics.compression_zstd+=0;before.metrics.compression_deflate+=0;
before.transform_metrics.applied+=6;before.transform_metrics.cache_hits+=3;
window.rustProxyWideOverview.render(before);
const get = (id,key) => document.getElementById(`overview-wide-${id}-${key}-line`).getAttribute('d');
const compressionLow=get('wide_compression_activity','gzip');
const transformLow=get('wide_transform_activity','applied');
const cpu=get('wide_cpu','system');
const lowLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
window.__clock=7000;
before.metrics.compression_gzip+=603; // now 201/s => ceiling expands to 500/s
before.transform_metrics.applied+=603;
window.rustProxyWideOverview.render(before);
const highLabel=document.getElementById('overview-wide-wide_compression_activity-gzip-value').textContent;
const compressionHigh=get('wide_compression_activity','gzip');
window.__clock=10000;
before.metrics.compression_gzip+=3;
before.transform_metrics.applied+=3;
window.rustProxyWideOverview.render(before);
const afterSpike=get('wide_compression_activity','gzip');
return {first,count,latency,memory,high,held,reset,percent,blank,
lowLabel,highLabel,compressionLow,transformLow,cpu,compressionHigh,afterSpike};
}''')
assert result['first']==100 and result['count']==10 and result['latency']==100, (width,result)
assert result['memory']>=64*1024*1024, (width,result)
assert (result['high'],result['held'],result['reset'],result['percent'],result['blank']) == (200,200,100,100,100), (width,result)
assert result['lowLabel']=='1.0/s' and result['highLabel']=='201.0/s', (width,result)
def yvalues(line): return [int(n) for n in re.findall(r'[ML]\d+ (\d+)',line)]
assert yvalues(result['compressionLow'])[-1]>=135, (width,result)
assert yvalues(result['transformLow'])[-1]>=130, (width,result)
assert yvalues(result['cpu'])==[136,136], (width,result)
assert yvalues(result['compressionHigh'])[-1]<100, (width,result)
assert yvalues(result['afterSpike'])[-1]>=135, (width,result)
assert not errors, (width,errors)
print('PASS all 14 overview-wide plot sizes/grids + full Cache/Transform charts + stable scales:', width)
page.close()
browser.close()
scripts/tests/dashboard_title_color_0337_browser_test.py
-92
View File
@@ -1,92 +0,0 @@
#!/usr/bin/env python3
"""Simulated Chromium regression for unified subdued metric/chart titles.
Tests computed CSS in the assembled panel DOM; not a live Rust server.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
CHECK = r'''({theme, density}) => {
document.documentElement.dataset.theme = theme;
document.documentElement.dataset.dashboardDensity = density;
const grid = document.getElementById('dashboard-overview-grid');
const probe = document.createElement('span');
probe.style.color = 'var(--muted)';
document.body.append(probe);
const expected = getComputedStyle(probe).color;
probe.style.color = 'var(--text)';
const primary = getComputedStyle(probe).color;
probe.remove();
const color = node => getComputedStyle(node).color;
const cards = [...grid.children].filter(node => node.hasAttribute('data-dashboard-widget'));
const titles = cards.map(card => {
let node;
if (card.classList.contains('stat')) node = card.querySelector(':scope > span:first-child');
else if (card.matches('.system-card,.connection-card')) node = card.querySelector('.overview-chart-heading');
else node = card.querySelector('.mini-heading > h3') || card.querySelector('.card-heading h2');
return [card.dataset.dashboardWidget, node];
});
const otherSelectors = [
'#panel-cache .stats-grid .stat > span:first-child',
'#panel-transform .stats-grid .stat > span:first-child',
'#panel-traffic .stat > span:first-child',
'#panel-cache > .card .card-heading h2',
'#panel-transform > .card .card-heading h2',
'.traffic-chart-card > h3.subheading',
'.function-live-chart-heading > strong',
'.mini-live-chart-head > span:first-child',
'.vhost-chart-head > span:first-child',
'.vhost-live-metric > span',
];
// Include dynamically rendered Vhost/Streams chart and Vhost metric titles.
const container = document.getElementById('panel-vhosts');
const fixture = document.createElement('div');
fixture.innerHTML = `<div class="vhost-chart"><div class="vhost-chart-head">
<span class="small-label">Connections history</span><span class="fine">42/s</span>
</div></div><div class="vhost-live-metric"><span>Requests</span><strong>42</strong></div>`;
container.append(fixture);
const otherTitles = otherSelectors.flatMap(selector => [...document.querySelectorAll(selector)]
.map(node => [selector, node]));
const errors = [...titles, ...otherTitles].flatMap(([name, node]) => {
if (!node) return [`${name} missing title`];
if (color(node) !== expected) return [`${name}: ${color(node)} != ${expected} (${node.textContent.trim().slice(0, 35)})`];
return [];
});
const numeric = [
document.querySelector('#system-cpu'),
document.querySelector('#panel-overview .system-card .card-heading h2'),
document.querySelector('#panel-overview .connection-card .card-heading h2'),
document.querySelector('[data-dashboard-widget="requests"] > strong'),
fixture.querySelector('.vhost-live-metric strong'),
].map(node => ({node: node && node.outerHTML.slice(0, 80), color: node && color(node)}));
fixture.remove();
return {expected, primary, count: titles.length, otherCount: otherTitles.length,
errors, numeric};
}'''
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 768, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
for theme in ('dark', 'light', 'ultra_dark'):
for density in ('normal', 'compact', 'ultra'):
result = page.evaluate(CHECK, {'theme': theme, 'density': density})
descriptor = width, theme, density
assert result['count'] == 78, (descriptor, result['count'])
assert result['otherCount'] >= 35, (descriptor, result['otherCount'])
assert result['primary'] != result['expected'], (descriptor, result)
assert not result['errors'], (descriptor, result['errors'][:25])
assert all(item['color'] == result['primary'] for item in result['numeric']), (descriptor, result['numeric'])
print(f'PASS muted titles, primary numeric readings at {width}px (3 themes, 3 densities)')
page.close()
browser.close()
print('PASS subdued metric/chart title CSS (simulated Chromium, no Rust runtime)')
scripts/tests/overview_samples_0330_ui_test.mjs
-116
View File
@@ -1,116 +0,0 @@
import { renderAdminIndex } from './panel_html.mjs';
// Simulated browser/status contract: this does not exercise the Rust server.
import assert from 'node:assert/strict';
import fs from 'node:fs';
import path from 'node:path';
import vm from 'node:vm';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const read = file => fs.readFileSync(path.join(root, file), 'utf8');
const html = renderAdminIndex(root);
const controllerSource = read('src/admin/web/app/overview-samples.js');
const store = new Map();
assert.match(html, /id="overview-layout-unlock"/);
// History controls share the connected-core eyebrow row (not a separate
// dashboard row), and navigation shows them only on Overview.
const metaStart = html.indexOf('class="page-intro-meta-row"');
const metaEnd = html.indexOf('class="page-intro-title-row"');
const statusPos = html.indexOf('id="connection-label"');
const selectPos = html.indexOf('id="overview-live-samples"');
assert.ok(metaStart < statusPos && statusPos < selectPos && selectPos < metaEnd);
assert.ok(selectPos < html.indexOf('id="panel-overview"'));
assert.match(html, /id="overview-samples-toolbar" hidden role="group"/);
assert.match(read('src/admin/web/app/core.js'), /\$\("overview-samples-toolbar"\)\.hidden = name !== "overview"/);
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="overview-layout-unlock"'));
assert.ok(html.indexOf('id="overview-live-samples"') < html.indexOf('id="dashboard-overview-grid"'));
for (const sample of [60, 120, 300, 600]) assert.match(html, new RegExp(`<option value="${sample}">`));
class Element {
constructor() { this.listeners = {}; this.value = ''; this.dataset = {}; this.disabled = false; this.attributes = new Map(); this.textContent = ''; this.hidden = false; }
addEventListener(name, callback) { this.listeners[name] = callback; }
fire(name) { this.listeners[name](); }
setAttribute(name, value) { this.attributes.set(name, String(value)); }
getAttribute(name) { return this.attributes.get(name); }
}
const nodes = new Map();
for (const id of ['overview-live-samples', 'overview-samples-save', 'overview-samples-state']) nodes.set(id, new Element());
const widgets = new Map([['wide_compression_activity', new Element()]]);
const document = {
readyState: 'complete', getElementById: id => nodes.get(id) || null,
querySelector: query => widgets.get(query.match(/data-dashboard-widget="([^"]+)"/)?.[1]) || null,
};
const localStorage = {
getItem: key => store.get(key) ?? null,
setItem: (key, value) => { store.set(key, value); },
};
let now = 1000;
const window = { rustProxyRefresh: { current: () => ({ overview_seconds: 3 }) } };
const context = vm.createContext({ document, localStorage, window, queueMicrotask: callback => callback(),
t: key => key, performance: { now: () => now },
currentPanel: 'overview', latestStatus: null, $: id => { if (!nodes.has(id)) nodes.set(id, new Element()); return nodes.get(id); },
fmt: n => String(n), fmtBytes: n => `${n} B`,
});
vm.runInContext(controllerSource, context);
const samples = window.rustProxyOverviewSamples;
const select = document.getElementById('overview-live-samples');
const save = document.getElementById('overview-samples-save');
const state = document.getElementById('overview-samples-state');
assert.equal(samples.getLimit(), 60);
assert.equal(save.disabled, true);
select.value = '600'; select.fire('change');
assert.equal(samples.getLimit(), 600);
assert.equal(state.dataset.i18n, 'overviewSamplesTemporary');
assert.equal(store.size, 0, 'temporary samples must not touch localStorage');
assert.equal(save.disabled, false);
const range = Array.from({ length: 700 }, (_, i) => i);
let calls = 0;
samples.onChange(() => { samples.trim(range); calls++; });
select.value = '120'; select.fire('change');
assert.equal(calls, 1);
assert.equal(range.length, 120);
assert.equal(range[0], 580);
assert.equal(samples.setLimit(999), false);
assert.equal(samples.getLimit(), 120);
save.fire('click');
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '120');
assert.equal(state.dataset.i18n, 'overviewSamplesSaved');
assert.equal(save.disabled, true);
// Fresh load reads only the saved choice, not a temporary override.
select.value = '300'; select.fire('change');
assert.equal(samples.getLimit(), 300);
const freshDocument = { readyState: 'loading', addEventListener() {} };
const fresh = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
vm.runInContext(controllerSource, fresh);
assert.equal(fresh.window.rustProxyOverviewSamples.getLimit(), 120);
// Permanently saved choice survives a fresh load.
save.fire('click');
const again = vm.createContext({ document: freshDocument, localStorage, window: {}, t: key => key });
vm.runInContext(controllerSource, again);
assert.equal(again.window.rustProxyOverviewSamples.getLimit(), 300);
// Wide Overview charts must respect the same limit and redraw immediately when
// the limit shrinks, without recording an extra point or hitting the network.
vm.runInContext(read('src/admin/web/app/rate-sampling.js'), context);
vm.runInContext(read('src/admin/web/app/chart-scales.js'), context);
vm.runInContext(read('src/admin/web/app/overview-wide-metrics.js'), context);
select.value = '600'; select.fire('change');
const status = {
revision: 1, metrics: { compression_gzip: 0, compression_br: 0, compression_zstd: 0, compression_deflate: 0 },
};
context.latestStatus = status;
for (let i = 0; i <= 650; i++) {
now += 3000;
for (const key of ['compression_gzip', 'compression_br', 'compression_zstd', 'compression_deflate']) status.metrics[key] += i % 3 + 1;
window.rustProxyWideOverview.render(status);
}
const line = () => nodes.get('overview-wide-wide_compression_activity-gzip-line').getAttribute('d');
assert.equal((line().match(/ L/g) || []).length, 599, 'maximum 600 samples must be retained');
select.value = '120'; select.fire('change');
assert.equal((line().match(/ L/g) || []).length, 119, 'changing selection must trim and redraw without a new sample');
assert.equal(store.get('rustproxy.overview.liveChartSamples'), '300', 'temporary choice must preserve the saved limit');
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('fetch('));
assert.ok(!read('src/admin/web/app/overview-samples.js').includes('setInterval('));
console.log('PASS: Overview samples 60/120/300/600, temporary vs persistent, immediate trim/redraw, no extra requests');
scripts/tests/panel_html.mjs
-22
View File
@@ -1,22 +0,0 @@
import fs from 'node:fs';
import path from 'node:path';
// Same four compile-time substitutions as src/admin/public.rs.
export function renderAdminIndex(root) {
const web = path.join(root, 'src/admin/web');
let html = fs.readFileSync(path.join(web, 'index.html'), 'utf8');
const metrics = fs.readFileSync(path.join(web, 'partials/overview-metrics.html'), 'utf8')
.split('<!-- RP_METRICS_SPLIT -->');
const charts = fs.readFileSync(path.join(web, 'partials/overview-charts.html'), 'utf8')
.split('<!-- RP_CHARTS_SPLIT -->');
if (metrics.length !== 2 || charts.length !== 2) throw new Error('Invalid Overview partials');
for (const [marker, text] of [
['METRICS_MAIN', metrics[0]], ['CHARTS_MAIN', charts[0]],
['METRICS_WEB', metrics[1]], ['CHARTS_WEB', charts[1]],
]) {
const token = `<!-- RP_${marker} -->`;
if (!html.includes(token)) throw new Error(`Missing ${token}`);
html = html.replace(token, text);
}
return html;
}
scripts/tests/panel_html.py
-18
View File
@@ -1,18 +0,0 @@
"""Return the actual assembled embedded Overview HTML to simulated-browser tests."""
from pathlib import Path
def render_admin_index(root: Path) -> str:
web = root / 'src/admin/web'
html = (web / 'index.html').read_text(encoding='utf-8')
metrics = (web / 'partials/overview-metrics.html').read_text(encoding='utf-8').split('<!-- RP_METRICS_SPLIT -->')
charts = (web / 'partials/overview-charts.html').read_text(encoding='utf-8').split('<!-- RP_CHARTS_SPLIT -->')
assert len(metrics) == len(charts) == 2
for key, fragment in (
('METRICS_MAIN', metrics[0]), ('CHARTS_MAIN', charts[0]),
('METRICS_WEB', metrics[1]), ('CHARTS_WEB', charts[1]),
):
marker = '<!-- RP_' + key + ' -->'
assert html.count(marker) == 1
html = html.replace(marker, fragment)
return html
scripts/tests/progressbar_0332_browser_test.py
-107
View File
@@ -1,107 +0,0 @@
#!/usr/bin/env python3
"""Progressbar Appearance regression using the production CSS in Chromium.
Browser-only visual/CSS verification. Does not execute RustProxy or authenticate.
"""
from __future__ import annotations
from io import BytesIO
from pathlib import Path
from panel_html import render_admin_index
import re
from PIL import Image
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
APP_HTML = render_admin_index(ROOT)
assert re.search(r'<div class="overview-meter system-memory-meter" id="memory-meter"', APP_HTML)
assert 'id="memory-meter-fill"' in APP_HTML
CORNERS = {'default': '3px', 'square': '0px', 'subtle': '2px', 'rounded': '4px', 'extra': '999px'}
STYLES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
HTML = '''<!doctype html><html><body>
<section id="panel-cache"><div class="stats-grid cache-stats">
<article class="stat"><progress id="cp-hot-entries-meter" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-memory" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-cold-entries-meter" value="55" max="100"></progress></article>
<article class="stat"><progress id="cp-cold-disk" value="55" max="100"></progress></article>
</div></section>
<main id="panel-overview">
<div class="stat stat-meter"><div class="overview-meter" id="overview-progress"><span class="overview-meter-fill" id="overview-fill" style="width:55%"></span></div></div>
<article class="card system-card"><div class="overview-meter system-memory-meter" id="memory-meter" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="55"><span class="overview-meter-fill" id="memory-meter-fill" style="width:55%"></span></div></article>
<div class="www-segmented" id="www-segmented"><span style="width:55%;background:var(--chart-accent)"></span></div>
<div class="www-top-meter" id="www-top"><span id="www-top-fill" style="width:55%"></span></div>
</main>
<div class="traffic-bar-track" id="traffic-track"><span class="traffic-bar-fill" id="traffic-fill" style="width:55%"></span></div>
<div id="chart-swatch" style="background:var(--chart-accent);width:4px;height:4px"></div>
</body></html>'''
def color_at_progress(page, progress_id: str) -> tuple[int, int, int]:
rect = page.locator(f'#{progress_id}').bounding_box()
assert rect and rect['width'] > 100
screenshot = Image.open(BytesIO(page.screenshot())).convert('RGB')
x = round(rect['x'] + rect['width'] * .22)
y = round(rect['y'] + rect['height'] / 2)
return screenshot.getpixel((x, y))
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML)
page.add_style_tag(content=CSS)
for corner, radius in CORNERS.items():
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
actual = page.evaluate('''() => {
const ids = ['cp-hot-entries-meter','cp-memory','cp-cold-entries-meter','cp-cold-disk',
'overview-progress','overview-fill','memory-meter','memory-meter-fill','www-segmented','www-top','www-top-fill',
'traffic-track','traffic-fill'];
return Object.fromEntries(ids.map(id => [id, getComputedStyle(document.getElementById(id)).borderRadius]));
}''')
assert all(value == radius for value in actual.values()), (corner, radius, actual)
actual_pseudo = page.evaluate('''() => {
const p = document.getElementById('cp-memory');
return ['::-webkit-progress-bar', '::-webkit-progress-value'].map(pseudo => getComputedStyle(p, pseudo).borderRadius);
}''')
assert actual_pseudo == [radius, radius], (corner, actual_pseudo)
for theme in ('dark', 'light'):
page.evaluate('theme => document.documentElement.dataset.theme = theme', theme)
for chart_style in STYLES:
page.evaluate('style => document.documentElement.dataset.chartStyle = style', chart_style)
observed = page.evaluate('''() => {
const color = id => getComputedStyle(document.getElementById(id)).backgroundColor;
const progress = document.getElementById('cp-memory');
return {
chart:color('chart-swatch'), native:getComputedStyle(progress).accentColor,
overview:color('overview-fill'), systemRam:color('memory-meter-fill'), www:color('www-top-fill'), traffic:color('traffic-fill'),
track:getComputedStyle(progress).backgroundColor,
overviewTrack:color('overview-progress'), systemTrack:color('memory-meter'), wwwTrack:color('www-top'),
};
}''')
assert len({observed[key] for key in ('chart', 'native', 'overview', 'systemRam', 'www', 'traffic')}) == 1, (theme, chart_style, observed)
assert len({observed[key] for key in ('track', 'overviewTrack', 'systemTrack', 'wwwTrack')}) == 1, (theme, chart_style, observed)
# Native progress on Chromium is painted by the WebKit pseudo-element;
# a pixel check confirms the color, not just the accent-color property.
if chart_style in ('default', 'neon', 'monochrome'):
color = color_at_progress(page, 'cp-memory')
chart_rgb = tuple(page.evaluate("""() => {
const canvas = document.createElement('canvas');
canvas.width = canvas.height = 1;
const ctx = canvas.getContext('2d');
ctx.fillStyle = getComputedStyle(document.getElementById('chart-swatch')).backgroundColor;
ctx.fillRect(0, 0, 1, 1);
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
}"""))
assert color == chart_rgb, (theme, chart_style, color, chart_rgb)
assert not errors, errors
page.close()
browser.close()
print('PASS: native Cache, System RAM, Overview, WWW and Traffic bars respond to 5 corners and 8 palettes (dark/light, 375/1440px).')
scripts/tests/proxy_1kb_opt3_static_test.py
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env python3
"""Source-only regression guards for the OPT3 patch (NOT a Rust compile)."""
from pathlib import Path
root = Path(__file__).resolve().parents[2]
def read(name):
return (root / name).read_text(encoding='utf-8')
assert 'version = "0.34.0"' in read('Cargo.toml')
acl = read('src/acl.rs')
assert "pub fn routing_path_view(raw: &str)" in acl
assert "return routing_path(raw).map(std::borrow::Cow::Owned)" in acl
assert 'raw.split' in acl and 'raw.len() <= 8192' in acl
forward = read('src/proxy/forward_entry.rs')
assert 'let routing_uri = request.uri().clone();' in forward
assert 'crate::acl::routing_path_view(routing_uri.path())' in forward
assert 'if context.client_ip != peer.0.ip()' in forward
route = read('src/proxy/cache_helpers.rs')
assert 'crate::acl::routing_path_view(request.uri().path())' in route
for name in ('src/proxy/forward/cache_lookup.rs', 'src/proxy/forward/success_finish.rs', 'src/proxy/forward/web_source.rs'):
assert 'path.clone()' not in read(name), name
assert 'if !trusted_peer' in read('src/forwarding.rs')
assert 'let mut ips = chain(incoming).unwrap_or_default();' in read('src/forwarding.rs')
headers = read('src/proxy/headers.rs')
assert 'let no_nominations' in headers
assert 'let nominated: Vec<HeaderName>' in headers
assert 'headers.remove(name)' in headers
assert 'String::from_utf8(encoded)' in read('src/proxy/request_id.rs')
assert 'if reason == "cache_disabled"' in read('src/cache/stats.rs')
assert not read('src/cache/stats.rs').startswith('//!') # previous E0753 build issue
for name in ('src/acl/tests.rs', 'src/proxy/headers/tests.rs', 'src/forwarding/tests.rs'):
assert (root / name).exists()
for source in (root / 'src').rglob('*.rs'):
if '/tests/' in source.as_posix() or source.name == 'tests.rs':
continue
assert len(source.read_text(encoding='utf-8').splitlines()) <= 500, source
print('PASS OPT3 source guards (STATIC ONLY; not cargo check, live HTTP or throughput)')
scripts/tests/proxy_1kb_opt4_static_test.py
-52
View File
@@ -1,52 +0,0 @@
#!/usr/bin/env python3
"""OPT4 source invariants: not compilation or performance proof."""
from pathlib import Path
r = Path(__file__).resolve().parents[2]
def source(name):
return (r / name).read_text(encoding="utf-8")
assert 'version = "0.34.0"' in source('Cargo.toml')
route = source('src/proxy/forward/route_cache.rs')
assert "Cow<'_, str>" in route
assert 'routing_uri.path_and_query()' in route
assert 'Ok(None) => public_suffix.clone()' in route
assert 'std::borrow::Cow::Owned(encoded)' in route
for name in ('src/proxy/forward/cache_lookup.rs','src/proxy/forward/success_finish.rs'):
assert 'public_suffix: public_suffix.clone()' not in source(name)
assert 'public_suffix: public_suffix.to_string()' in source(name)
helpers = source('src/proxy/upstream_helpers.rs')
assert 'fn wrap_observed_request' in helpers
assert 'if body.is_end_stream() { body }' in helpers
assert 'let request = request.map(|body| wrap_observed_request' in helpers
assert 'fn log_prefix' in helpers
assert 'input.is_ascii()' in helpers and 'input.chars().take(max_chars).collect()' in helpers
assert 'path: log_prefix(request.uri().path(), 256)' in helpers
assert 'pub request_origin_uri: http::Uri' in source('src/config/model.rs')
target = source('src/proxy/types.rs')
assert 'fn upstream_request_target(\n uri: Uri' in target
assert 'return Some(uri);' in target
assert 'let mut parts = uri.into_parts();' in target
assert 'upstream_request_target(uri, request.headers_mut()' in target
assert 'upstream_request_target(\n uri, request.headers_mut()' in source('src/proxy/forward/upstream.rs')
forward = source('src/forwarding.rs')
assert 'peer_node.as_deref().unwrap_or(&peer_text)' in forward
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forward
headers = source('src/proxy/headers.rs')
assert 'let no_nominations' in headers
assert 'eq_ignore_ascii_case(b"close")' in headers
assert 'let nominated: Vec<HeaderName>' in headers
for name, test in [
('src/proxy/body/tests.rs','optimized_empty_request_keeps_exchange_until_response'),
('src/proxy/body/tests.rs','optimized_nonempty_request_still_meters_data_and_holds_capacity'),
('src/proxy/tests/authority_tests.rs','optimized_log_prefix_preserves_unicode_char_count_and_ascii_bytes'),
('src/proxy/tests/authority_tests.rs','moved_uri_keeps_h1_origin_and_query'),
('src/proxy/headers/tests.rs','absent_and_close_connection_preserve_non_hop_headers'),
]:
assert test in source(name), (name, test)
for p in (r / 'src').rglob('*.rs'):
if '/tests/' in p.as_posix() or p.name == 'tests.rs':
continue
assert len(p.read_text(encoding='utf-8').splitlines()) <= 500, p
assert not source('src/cache/stats.rs').startswith('//!')
print('PASS OPT4 source guard (STATIC ONLY; no cargo / no OPT4 benchmark)')
scripts/tests/proxy_1kb_opt5_static_test.py
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env python3
"""OPT5 invariants, static-only. Does not compile or benchmark Rust."""
from pathlib import Path
root = Path(__file__).resolve().parents[2]
def read(path):
return (root / path).read_text(encoding='utf-8')
assert 'version = "0.34.0"' in read('Cargo.toml')
identifier = read('src/proxy/request_id.rs')
assert 'rand::random()' in identifier
assert 'let mut token: [u8; 32]' in identifier
assert 'sequence.to_be_bytes()' in identifier
assert '*byte ^= sequence_byte;' in identifier
assert 'request_id_hex(&token)' in identifier
assert 'let mut encoded = vec![0u8; digest.len() * 2]' in identifier
assert 'Sha256::new()' not in identifier
forwarding = read('src/forwarding.rs')
assert 'self.context_with_original_url(incoming, peer, local, tls, authority, uri, true)' in forwarding
assert forwarding.count('if include_original_url {') == 2
assert 'let mut ips = chain(incoming).unwrap_or_default();' in forwarding
assert 'crate::config::request_host(authority_text)?;' in forwarding
assert 'frontend.is_some_and(|frontend| frontend.tinyauth.is_some())' in read('src/proxy/forward_entry.rs')
assert '&context.original_url' in read('src/tinyauth.rs')
lease = read('src/balance/lease_runtime.rs')
assert 'let failure_detail = is_5xx.then(|| format!("status={status}"));' in lease
assert 'let detail = failure_detail.as_deref().unwrap_or(reason);' in lease
assert 'observe_passive_with_detail(' in lease
assert 'lease.response(' in read('src/proxy/forward/upstream.rs') or 'lease.response(' in read('src/proxy/forward/success_prepare.rs')
analytics = read('src/traffic.rs')
for literal in ['mpsc::channel::<Sample>(QUEUE_CAPACITY)', 'recv_many(&mut batch, 64)',
'for sample in batch.drain(..)', 'get_mut(sample.vhost.as_str())',
'state.vhosts.len() < MAX_VHOSTS', 'try_reserve()',
'self.dropped.fetch_add(1, Ordering::Relaxed)',
'candidate.eq_ignore_ascii_case(needle.as_bytes())',
'2 => self.statuses.add("2xx", 1)', 'map(Counters::view)']:
assert literal in analytics, literal
assert 'let lower = value.to_ascii_lowercase();' not in analytics
helpers = read('src/proxy/upstream_helpers.rs')
assert 'preferred_encoding(&original_headers, global_compression).is_some()' in helpers
assert 'compression.resolve(' not in helpers
assert 'compression::encode(' in helpers
assert 'wrap_observed_request(body, &exchange' in helpers
for rel, case in [
('src/traffic/tests.rs', 'batched_worker_keeps_exact_counts_and_bounded_vhosts'),
('src/traffic/tests.rs', 'status_top_list_preserves_common_and_extension_family_names'),
('src/forwarding/tests.rs', 'original_url_is_omitted_only_when_unneeded_and_forwarding_is_identical'),
('src/compression/tests.rs', 'unaccepted_codecs_cannot_be_reenabled_by_scoped_overrides'),
('src/balance/tests.rs', 'successful_lease_response_keeps_passive_observability_and_failure_detail'),
('src/proxy/request_id/tests.rs', 'independent_sequential_ids_keep_full_length_and_unique_values'),
]:
assert case in read(rel), (rel, case)
for path in (root / 'src').rglob('*.rs'):
if path.name == 'tests.rs' or '/tests/' in path.as_posix():
continue
assert len(path.read_text(encoding='utf-8').splitlines()) <= 500, path
assert not read('src/cache/stats.rs').startswith('//!')
print('PASS OPT5 static source checks; Rust build and runtime acceptance still required')
scripts/tests/proxy_hotpath_034_static_test.py
-47
View File
@@ -1,47 +0,0 @@
#!/usr/bin/env python3
"""Source-only guardrails for the uncompiled 0.34.0 hot-path candidate."""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def src(path):
return (ROOT / path).read_text()
snapshot = src("src/config/snapshot.rs")
assert "fn host_patterns(host: &str) -> impl Iterator" in snapshot
assert "pub fn select_on_arc(" in snapshot
assert "self.select_on_arc(host, path, local, tls).map(Arc::as_ref)" in snapshot
assert "Vec<Arc<CompiledRoute>>" in src("src/config/model.rs")
assert "route: Option<Arc<crate::config::CompiledRoute>>" in src("src/proxy/types.rs")
assert "request.extensions_mut().insert(SelectedRouteContext" in src("src/proxy/upstream_helpers.rs")
assert "selection.route.as_deref()" in src("src/proxy/forward_entry.rs")
assert "let fallback_request = selected" in src("src/proxy/forward_entry.rs")
assert "filter(|route| route.error_fallback.is_some())" in src("src/proxy/forward_entry.rs")
assert "let Some(next_request) = replay_headers" in src("src/proxy/forward/upstream.rs") or "replay_headers.as_ref().and_then" in src("src/proxy/forward/upstream.rs")
assert "let replay_headers = ((pool_retry_possible || h2_replay_possible)" in src("src/proxy/forward/upstream.rs")
pool = src("src/balance/pool_core.rs")
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
assert "let mut scheduling = lock(&self.scheduling);" in pool
assert "fn candidates_excluding" in pool
assert "health: member.health_settings.clone()" in pool
assert "health: selected.health_settings.clone()" in pool
assert "health_settings: Arc<HealthCheck>" in src("src/balance/health.rs")
assert "healthy_cached: AtomicBool" in src("src/balance/model.rs")
assert "self.healthy_cached.store(h.healthy, Ordering::Release)" in src("src/balance/health.rs")
exchange = src("src/proxy/body.rs")
for field in ("source", "cache_effective_enabled", "transform_effective_enabled"):
assert f"pub {field}: AtomicU8" in exchange
assert f"pub {field}: Mutex" not in exchange
assert "exchange.set_source(" in src("src/proxy/forward/route_cache.rs")
assert "exchange.set_cache_enabled(" in src("src/proxy/forward/web_source.rs")
assert "self.live.receiver_count() != 0" in src("src/logging/format.rs")
assert "fn route_handles_reuse_snapshot_ownership_without_cloning_compiled_routes" in src("src/config/tests.rs")
assert "fn single_backend_fast_path_observes_capacity_and_health_without_affinity_cookie" in src("src/balance/tests.rs")
assert "fn atomic_exchange_tags_keep_original_observability_fields" in src("src/proxy/body/tests.rs")
for path in (ROOT / "src").rglob("*.rs"):
if "/tests/" in path.as_posix() or path.name == "tests.rs" or path.is_relative_to(ROOT / "tests"):
continue
assert len(path.read_text().splitlines()) <= 500, path
assert 'version = "0.34.0"' in src("Cargo.toml")
assert "max_idle_per_host: 64" in src("src/balance/model.rs")
assert "pool_recycle_requests: 25_000" in src("src/balance/model.rs")
print("PASS 0.34.0 proxy hot-path wiring, observability, bounded tuned defaults and source-size limits (STATIC ONLY)")
scripts/tests/rps_hotpath_034_static_test.py
-129
View File
@@ -1,129 +0,0 @@
#!/usr/bin/env python3
"""Source invariants for 0.34.0 RPS optimizations (NOT a Rust build or RPS test).
These guards reject accidental removal of important branch/safety conditions.
They must be paired with ``cargo test`` and live black-box HTTP acceptance.
"""
from pathlib import Path
import json
ROOT = Path(__file__).resolve().parents[2]
def source(path):
return (ROOT / path).read_text(encoding="utf-8")
# Random request identifiers still use the OS-seeded thread-local CSPRNG,
# preserve a 256-bit unpredictable 64-hex token and mix the per-process sequence.
# OPT5 removes the extra SHA-256 operation, not the entropy or correlation ID.
identifier = source("src/proxy/request_id.rs")
assert "rand::random()" in identifier
assert "sequence.to_be_bytes()" in identifier
assert "*byte ^= sequence_byte;" in identifier
assert "request_id_hex(&token)" in identifier
assert "SysRng.try_fill_bytes" not in identifier
# Compiled/validated target authority survives request URI construction.
config = source("src/config/backend.rs")
model = source("src/config/model.rs")
request = source("src/proxy/types.rs")
assert "let request_origin_uri: http::Uri = request_origin.parse()?;" in config
assert "request_origin_uri," in config
assert "pub request_origin_uri: http::Uri," in model
assert "let path: http::uri::PathAndQuery = suffix.parse().ok()?;" in request
assert "let mut parts = backend.request_origin_uri.clone().into_parts();" in request
assert "parts.path_and_query = Some(path);" in request
assert "Uri::from_parts(parts).ok()" in request
# No new unbounded caches or disabled LB safety gates.
pool = source("src/balance/pool_core.rs")
assert "if self.members.len() == 1 && !self.cookie_affinity()" in pool
assert "try_acquire_active(member.definition.max_concurrent_requests)" in pool
assert "m.state.healthy()" in pool
assert "scheduling.get_mut(member.definition.id.as_str())" in pool
assert "scheduling.entry(member.definition.id.clone())" in pool
health = source("src/balance/lease_runtime.rs")
assert "async fn probe(member: Arc<Member>, settings: Arc<HealthCheck>)" in health
assert "checks.spawn(probe(Arc::clone(member), health.clone()))" in health
assert "tokio::time::timeout" in health
# Configurable keep-alive policy, exact recycle-request threshold and age backup.
upstream = source("src/upstream/client.rs")
assert "self.current.load_full()" in upstream
assert "request_due || age_due" in upstream
assert "let now = monotonic_millis();" in upstream
assert "let age_ms = now.saturating_sub(self.generation_started_ms.load(Ordering::Relaxed));" in upstream
assert "fn maintain_rotation_at(" in upstream
assert "self.factory.keepalive.pool_recycle_requests" in upstream
assert "self.factory.keepalive.pool_recycle_seconds" in upstream
assert "pool_max_idle_per_host(if keepalive.enabled" in upstream
assert "retry_canceled_requests(false)" in upstream
assert "observe_protocol" in upstream
# Request/response streamed payloads retain accounting and timeout/cancellation.
body = source("src/proxy/body.rs")
assert "request_backend: Option<Arc<crate::balance::Lease>>" in body
assert "this.request_backend = exchange.backend" in body
assert "lease.bytes(count, true)" in body
assert "lease.bytes(count, false)" in body
assert "exchange.request_bytes.fetch_add(count" in body
assert "exchange.response_bytes.fetch_add(count" in body
assert "get_or_insert_with(|| Box::pin(sleep_until(this.deadline)))" in body
assert "this.finish();" in body
# Persist history and preserve live streaming/sink drop accounting.
logs = source("src/logging/format.rs")
assert "let has_viewers = self.live.receiver_count() != 0;" in logs
assert "let has_sink = self.settings.load().enabled();" in logs
assert "external.take().expect(\"entry\")" in logs
assert "self.live.send(entry.clone())" in logs
assert "self.sink.try_send(entry)" in logs
assert "self.sink_dropped.fetch_add(1" in logs
# Never weaken routing, identity, HTTP version and cache eligibility while tuning.
route = source("src/proxy/forward/route_cache.rs")
assert "admin_session_passthrough" in route
assert "cache_plan.request_policy" in route
assert "exchange.set_cache_enabled" in route
assert "exchange.set_transform_enabled" in route
assert "upstream_uri(&backend, &origin_suffix)" in route
assert "headers::strip_hop_by_hop" in route
assert "pool.select_wait(context.client_ip" in route or ".select_wait(context.client_ip" in route
assert "pool.select_member_wait(id)" in route
assert "max_idle_per_host: 64" in source("src/balance/model.rs")
assert "pool_recycle_requests: 25_000" in source("src/balance/model.rs")
assert "(1..=64).contains(&k.max_idle_per_host)" in source("src/balance/model.rs")
assert "keepalive.max_idle_per_host ?? 64" in source("src/admin/web/vhosts/editor.js")
openapi = json.loads(source("src/admin/openapi.json"))
api_keepalive = openapi["components"]["schemas"]
assert any(isinstance(schema, dict) and "keepalive" in schema.get("properties", {})
and schema["properties"]["keepalive"].get("properties", {})
.get("max_idle_per_host", {}).get("default") == 64
for schema in api_keepalive.values())
assert "| `max_idle_per_host` | 64 | 1..64 |" in source("docs/VHOSTS.md")
assert '"keepalive": { "max_idle_per_host": 4 }' in source("src/balance/tests.rs")
assert "self.route.get_mut()" in source("src/proxy/body.rs")
assert "let client_ip = self" in source("src/proxy/body.rs")
assert "if !headers.contains_key(http::header::COOKIE)" in source("src/proxy/headers.rs")
assert "if !headers.contains_key(http::header::SET_COOKIE)" in source("src/proxy/headers.rs")
assert 'version = "0.34.0"' in source("Cargo.toml")
# Full exchange duration histogram still exports cumulative Prometheus series,
# but only one exclusive bucket is updated per request; per-vhost and Docker
# exporters must reconstruct bucket counts from exactly the same interface.
histogram = source("src/metrics/vhost.rs")
exporter = source("src/prometheus/docker_misc.rs")
assert "let bucket = DURATION_BUCKETS_US.partition_point" in histogram
assert "self.buckets[bucket].fetch_add(1, Ordering::Relaxed);" in histogram
assert "pub fn duration_histogram(&self)" in histogram
assert exporter.count("let (buckets, count) = m.duration_histogram();") == 2
assert "rust_proxy_vhost_exchange_duration_seconds_bucket" in exporter
assert "rust_proxy_docker_vhost_exchange_duration_seconds_bucket" in exporter
assert "exclusive_bins_reconstruct_exact_prometheus_cumulative_boundaries" in source("src/metrics/vhost/tests.rs")
for file in (ROOT / "src").rglob("*.rs"):
if "/tests/" in file.as_posix() or file.name == "tests.rs":
continue
assert len(file.read_text(encoding="utf-8").splitlines()) <= 500, file
print("PASS RPS hot-path source invariants (STATIC ONLY; new binary not compiled)")
scripts/tests/sendfile_034_panel_browser_test.py
-82
View File
@@ -1,82 +0,0 @@
#!/usr/bin/env python3
"""Offline Chromium test: sendfile toggle serializes in real admin bundle.
It uses simulated config and a mocked saveSection; never claims native
sendfile execution, Rust compilation or live server acceptance.
"""
from pathlib import Path
import json
import re
from playwright.sync_api import sync_playwright
from panel_html import render_admin_index
ROOT = Path(__file__).resolve().parents[2]
ROUTER = (ROOT / "src/admin/router.rs").read_text()
start = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert start
array = ROUTER[start.end():]
array = array[:re.search(r'\]\s*\.join\("\\n"\)', array).start()]
chunks = []
for line in array.splitlines():
line = line.strip()
match = re.search(r'include_str!\("([^"]+)"\)', line)
if match:
chunks.append((ROOT / "src/admin" / match.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
chunks.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (f + '.js')).read_text()
for f in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
chunks.append(json.loads(line[:-1]))
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 950})
page.set_default_timeout(4500)
errors = []
page.on('pageerror', lambda err: errors.append(str(err)))
page.set_content(render_admin_index(ROOT))
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
page.add_script_tag(content='\n'.join(chunks))
page.evaluate('''() => {
envelope = {revision: 10, config: {web_services: [{
id: 'site-A', name: 'Static Site', document_root: '/srv/static',
index: ['index.html'], php: {enabled: false}
}], vhosts: [{id: 'vhost-A', keepalive: {max_connections: 8}}], php_fpm_profiles: []}};
currentPanel = 'web-server';
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-web-server').hidden = false;
window.__saved = [];
saveSection = async (section, value) => {
window.__saved.push({section, value: structuredClone(value)});
if (section === 'web-services') envelope.config.web_services = structuredClone(value);
return true;
};
window.rustProxyWebServer.render();
}''')
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
assert page.locator('#web-service-sendfile').input_value() == 'off'
page.evaluate("document.getElementById('web-service-sendfile').value = 'auto'")
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
page.wait_for_function('window.__saved.length === 1')
result = page.evaluate('''() => ({
saved: window.__saved[0],
cacheStillUnrelated: envelope.config.vhosts[0].keepalive.max_connections === 8,
closed: !document.getElementById('web-service-dialog').open
})''')
assert result['saved']['section'] == 'web-services', result
assert result['saved']['value'][0]['sendfile'] == 'auto', result
assert result['saved']['value'][0]['document_root'] == '/srv/static', result
assert result['cacheStillUnrelated'] and result['closed'], result
page.evaluate("document.querySelector('#web-service-list .entity-actions button').click()")
assert page.locator('#web-service-sendfile').input_value() == 'auto'
page.evaluate("document.getElementById('web-service-sendfile').value = 'off'")
page.evaluate('''() => { const form = document.getElementById('web-service-form'); if (!form.checkValidity()) throw new Error('invalid fields: ' + [...form.elements].filter(e=>!e.checkValidity()).map(e=>e.id).join(', ')); form.requestSubmit(); }''')
page.wait_for_function('window.__saved.length === 2')
assert page.evaluate('envelope.config.web_services[0].sendfile') == 'off'
assert not errors, errors
print(f'PASS simulated panel sendfile off -> auto -> off config save at {width}px')
browser.close()
scripts/tests/sendfile_034_static_test.py
-44
View File
@@ -1,44 +0,0 @@
#!/usr/bin/env python3
"""RustProxy 0.34.0 sendfile source-wiring assertions ONLY, not native tests."""
from pathlib import Path
import json
import re
root = Path(__file__).resolve().parents[2]
get = lambda path: (root / path).read_text()
model = get('src/web_server/config/model.rs')
assert 'pub enum SendfileMode' in model and 'sendfile: SendfileMode::Off' in model
assert 'pub sendfile: SendfileMode' in model
assert 'sendfile: self.sendfile' in get('src/web_server/config/compile.rs')
assert 'pub sendfile: SendfileMode' in get('src/web_server/config/validation.rs')
assert 'service.sendfile = crate::web_server::SendfileMode::Off;' in get('src/runtime/types.rs')
source = get('src/proxy/forward/web_source.rs')
for guard in ('!enabled && identity_request', 'request_method == Method::GET',
'response.status() == StatusCode::OK', 'content-encoding',
'transfer-encoding', 'content-range', 'candidate.length',
'http::Version::HTTP_11'):
assert guard in source, guard
assert 'content_encoding.is_none()' in get('src/web_server/response/file.rs')
assert 'single_range.is_none()' in get('src/web_server/response/file.rs')
assert 'opened.file.try_clone()' in get('src/web_server/response/file.rs')
bridge = get('src/server/sendfile.rs')
for item in ('libc::sendfile(', 'MAX_PENDING: usize = 8', 'MIN_SIZE: u64 = 64 * 1024',
'registry.slots.is_empty()', 'native_sendfile_transfers_exact_file',
'sendfile placeholder did not match', 'UnexpectedEof'):
assert item in bridge, item
assert 'pub(crate) mod sendfile' in get('src/server.rs')
assert 'SendfileIo::new(stream)' in get('src/server/tls_runtime.rs')
assert 'sendfile_slots' in get('src/server/http1_connection.rs')
js = get('src/admin/web/web-server.js')
html = get('src/admin/web/index.html')
assert '"web-service-sendfile"' in js and 'sendfile: "off"' in js
assert 'sendfile: $("web-service-sendfile").value' in js
assert html.count('id="web-service-sendfile"') == 1
assert 'value="auto"' in html
for locale in ('en_US', 'pl_pl'):
keys = json.loads(get(f'langs/{locale}.json'))['translations']
assert all(k in keys for k in ('webServiceSendfile', 'webServiceSendfileHelp', 'webServiceSendfileAuto'))
api = json.loads(get('src/admin/openapi.json'))
assert 'sendfile' in json.dumps(api)
print('PASS 0.34.0 sendfile source guards, default OFF, Linux syscall, UI, translations, API and scoped cache (STATIC ONLY)')
scripts/tests/sidebar_nav_0333_browser_test.py
-116
View File
@@ -1,116 +0,0 @@
#!/usr/bin/env python3
"""Verify sidebar colors use only the UI theme; chart palettes affect charts only.
Uses actual production HTML/CSS in headless Chromium, not a live RustProxy
server or authenticated preference/session roundtrip.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text(encoding='utf-8')
THEMES = ('dark', 'ultra_dark', 'light', 'cream')
PALETTES = ('default', 'soft', 'technical', 'high_contrast', 'neon', 'monochrome', 'warm', 'cool')
CORNERS = {'default': 0, 'square': 0, 'subtle': 4, 'rounded': 8, 'extra': 12}
STYLE_INFO = r'''() => {
const color = value => {
const c = document.createElement('canvas'); c.width = c.height = 1;
const ctx = c.getContext('2d'); ctx.fillStyle = value; ctx.fillRect(0, 0, 1, 1);
return [...ctx.getImageData(0, 0, 1, 1).data].slice(0, 3);
};
const root = getComputedStyle(document.documentElement);
const active = document.querySelector('.sidebar .main-navigation .nav.active');
const idle = document.querySelector('.sidebar .main-navigation .nav:not(.active)');
const sa = getComputedStyle(active), si = getComputedStyle(idle);
return {
chartAccent:color(root.getPropertyValue('--chart-accent').trim()),
themeSelected:color(root.getPropertyValue('--nav-active-bg').trim()),
themeHovered:color(root.getPropertyValue('--nav-hover-bg').trim()),
foreground:color(sa.color),
background:color(sa.backgroundColor),
borderLeft:color(sa.borderLeftColor),
borderRight:color(sa.borderRightColor),
radius:parseFloat(sa.borderTopLeftRadius),
idleBackground:color(si.backgroundColor),
mobileBackground:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).backgroundColor),
mobileColor:color(getComputedStyle(document.querySelector('.mobile-menu-toggle')).color),
navWidth:document.querySelector('.main-navigation').scrollWidth,
navClientWidth:document.querySelector('.main-navigation').clientWidth,
activeWidth:active.scrollWidth,
activeClientWidth:active.clientWidth,
};
}'''
def contrast(fg, bg):
def luminance(rgb):
channels = [x / 255 for x in rgb]
channels = [x / 12.92 if x <= 0.04045 else ((x + 0.055) / 1.055) ** 2.4 for x in channels]
return .2126 * channels[0] + .7152 * channels[1] + .0722 * channels[2]
lo, hi = sorted((luminance(fg), luminance(bg)))
return (hi + .05) / (lo + .05)
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for viewport in (375, 1200):
page = browser.new_page(viewport={'width': viewport, 'height': 820})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', '').replace('/assets/app___RUSTPROXY_ASSET_ID__.js', ''))
page.add_style_tag(content=CSS)
page.evaluate('''() => {
document.documentElement.classList.remove('app-loading');
document.getElementById('app-startup').remove();
document.querySelector('.sidebar').classList.add('mobile-nav-open');
document.querySelector('.mobile-menu-toggle')?.setAttribute('aria-expanded', 'true');
}''')
theme_colors = []
for theme in THEMES:
baseline = None
chart_accents = set()
for palette in PALETTES:
page.evaluate('''([theme, palette]) => {
document.documentElement.dataset.theme = theme;
document.documentElement.dataset.chartStyle = palette;
}''', [theme, palette])
for corner, expected_radius in CORNERS.items():
page.evaluate('corner => document.documentElement.dataset.cornerStyle = corner', corner)
style = page.evaluate(STYLE_INFO)
assert style['radius'] == expected_radius, (viewport, theme, palette, corner, style['radius'])
assert style['borderLeft'] == style['foreground'], (viewport, theme, palette, corner, style)
assert style['background'] == style['themeSelected'], (viewport, theme, palette, corner, style)
if viewport == 375:
assert style['mobileBackground'] == style['background'], (viewport, theme, palette, corner, style)
assert style['mobileColor'] == style['foreground'], (viewport, theme, palette, corner, style)
assert contrast(style['foreground'], style['background']) >= 4.5, (viewport, theme, palette, corner, style)
assert style['navWidth'] <= style['navClientWidth'] + 2, (viewport, theme, palette, corner, style)
assert style['activeWidth'] <= style['activeClientWidth'] + 2, (viewport, theme, palette, corner, style)
chart_accents.add(tuple(style['chartAccent']))
# The entire selected state is unaffected by chart-style changes.
nav_colors = (tuple(style['foreground']), tuple(style['background']), tuple(style['borderLeft']))
if baseline is None:
baseline = nav_colors
else:
assert baseline == nav_colors, (viewport, theme, palette, baseline, nav_colors)
page.locator('.nav[data-panel="vhosts"]').hover()
hovered = page.evaluate(STYLE_INFO)
assert hovered['idleBackground'] == hovered['themeHovered'], (viewport, theme, palette, hovered)
assert hovered['idleBackground'] != style['idleBackground'], (viewport, theme, palette)
page.mouse.move(0, 0)
if viewport == 1200 and theme == 'ultra_dark' and palette == 'cool':
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_desktop.png'))
if viewport == 375 and theme == 'dark' and palette == 'soft':
page.screenshot(path=str(ROOT.parent / 'sidebar_preview_mobile.png'))
assert len(chart_accents) > 1, (viewport, theme, chart_accents)
theme_colors.append(baseline)
assert len(set(theme_colors)) == len(THEMES), (viewport, theme_colors)
page.evaluate("document.documentElement.dir = 'rtl'")
rtl = page.evaluate(STYLE_INFO)
assert rtl['borderRight'] == rtl['foreground'], (viewport, rtl)
assert not errors, errors
print('PASS themed navigation independence: viewport', viewport, 'themes', len(THEMES), 'palettes', len(PALETTES), 'corners', len(CORNERS))
page.close()
browser.close()
print('PASS: sidebar changes with UI theme and corner setting, never with chart style')
scripts/tests/system_ram_meter_0332_browser_test.py
-72
View File
@@ -1,72 +0,0 @@
#!/usr/bin/env python3
"""System Load RAM meter updates using the actual combined panel JavaScript.
This is a simulated authenticated DOM/status test, not a Rust runtime integration test.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
router = (ROOT / 'src/admin/router.rs').read_text()
def bundle_for(section):
match = re.search(r'fn ' + section + r'\([^)]*\)[^{]*\{', router)
assert match, section
text = router[match.end():]
text = text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
result = []
for line in text.splitlines():
line = line.strip()
if 'include_str!("' in line:
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
result.append((ROOT / 'src/admin' / path).read_text())
elif line == 'admin_vhosts_javascript(),':
result.append('(() => {\n' + ''.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() + '\n'
for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
html = render_admin_index(ROOT)
css = (ROOT / 'src/admin/web/style.css').read_text()
js = bundle_for('admin_app_javascript')
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (375, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '').replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=css)
page.add_script_tag(content=js)
page.wait_for_timeout(100)
report = page.evaluate('''() => {
currentPanel = "overview";
document.getElementById('workspace').hidden = false;
document.getElementById('panel-overview').hidden = false;
const meter = document.getElementById('memory-meter');
const fill = document.getElementById('memory-meter-fill');
const snapshot = (used, total) => ({system: {source:'measured',cpu_percent:2.1,
memory_used_bytes:used, memory_total_bytes:total},
live_system:{cpu_percent:2.1}, cache:{}, metrics:{}, listeners:{}});
const values = [];
for (const [used, total] of [[250,1000],[800,1000],[0,1000],[null,null]]) {
renderDiagnostics(snapshot(used,total), false, true);
values.push({fill:fill.style.width, now:meter.getAttribute('aria-valuenow'),
hidden:meter.hidden, text:meter.getAttribute('aria-valuetext')});
}
return { values, role:meter.getAttribute('role'), cssHeight:getComputedStyle(meter).height,
scrollbar:document.documentElement.scrollWidth>window.innerWidth+2 };
}''')
assert report['role'] == 'progressbar' and report['cssHeight'] == '5px', report
assert report['values'][0]['fill'] == '25%' and report['values'][0]['now'] == '25', report
assert report['values'][1]['fill'] == '80%' and report['values'][1]['now'] == '80', report
assert report['values'][2]['fill'] == '0%' and report['values'][2]['now'] == '0', report
assert report['values'][3]['fill'] == '0%' and report['values'][3]['now'] is None and report['values'][3]['hidden'], report
assert not report['scrollbar'] and not errors, (width, errors, report)
page.close()
browser.close()
print('PASS: System RAM meter live percentage and unknown/zero handling on 375/1440px in bundled panel.')
scripts/tests/test_stream_tcp.py
-137
View File
@@ -1,137 +0,0 @@
#!/usr/bin/env python3
"""TCP Stream black-box test: use against a running RustProxy and mock backends.
The 'selftest' mode validates this harness only; it does NOT run RustProxy.
All traffic uses loopback by default. Do not point it at a production service.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import secrets
import socket
import socketserver
import sys
import threading
import time
MAX_LINE = 1024
class TCPServer(socketserver.ThreadingTCPServer):
allow_reuse_address = True
daemon_threads = True
request_queue_size = 128
class TCPBackend(socketserver.StreamRequestHandler):
def handle(self) -> None:
self.request.settimeout(10)
backend_id = self.server.backend_id.encode('ascii')
try:
while True:
line = self.rfile.readline(MAX_LINE + 1)
if not line or len(line) > MAX_LINE or not line.endswith(b'\n'):
return
self.wfile.write(backend_id + b'|' + line)
self.wfile.flush()
except (OSError, TimeoutError):
return
def run_server(host: str, port: int, backend_id: str) -> None:
with TCPServer((host, port), TCPBackend) as server:
server.backend_id = backend_id
print(f'TCP mock {backend_id} listening on {server.server_address}', flush=True)
try:
server.serve_forever(poll_interval=0.1)
except KeyboardInterrupt:
pass
def check_one(index: int, host: str, port: int, messages: int, timeout: float, hold_ms: float, nonce: str) -> str:
backend = None
with socket.create_connection((host, port), timeout=timeout) as connection:
connection.settimeout(timeout)
with connection.makefile('rb') as reader:
for seq in range(messages):
payload = f'rp-tcp:{nonce}:{index}:{seq}\n'.encode('ascii')
connection.sendall(payload)
response = reader.readline(MAX_LINE + 1)
try:
observed, echoed = response.split(b'|', 1)
except ValueError as exc:
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
assert echoed == payload, f'session {index}: response mismatch: {response!r}'
assert observed, f'session {index}: empty backend ID'
if backend is None:
backend = observed
assert observed == backend, f'session {index}: TCP backend changed during connection'
if hold_ms and seq != messages - 1:
time.sleep(hold_ms / 1000)
return backend.decode('ascii')
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
hold_ms: float, min_backends: int) -> Counter:
nonce = secrets.token_hex(8)
started = time.monotonic()
counts: Counter[str] = Counter()
with ThreadPoolExecutor(max_workers=parallel) as pool:
futures = [pool.submit(check_one, i, host, port, messages, timeout, hold_ms, nonce)
for i in range(sessions)]
for future in as_completed(futures):
counts[future.result()] += 1
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
print(f'TCP PASS: {sessions} sessions, {sessions * messages} messages, '
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
return counts
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='mode', required=True)
server = sub.add_parser('server', help='run a local TCP mock backend')
server.add_argument('--host', default='127.0.0.1')
server.add_argument('--port', type=int, required=True)
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy listener')
client.add_argument('--host', default='127.0.0.1', help='RustProxy TCP listener host')
client.add_argument('--port', type=int, required=True, help='RustProxy TCP listener port')
client.add_argument('--sessions', type=int, default=24)
client.add_argument('--messages', type=int, default=3)
client.add_argument('--parallel', type=int, default=8)
client.add_argument('--timeout', type=float, default=4.0)
client.add_argument('--hold-ms', type=float, default=20.0, help='pause between messages to keep TCP sessions alive')
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
args = parser.parse_args()
if args.mode == 'server':
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
parser.error('--backend-id must be nonempty ASCII without | or newline')
run_server(args.host, args.port, args.backend_id)
elif args.mode == 'client':
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.hold_ms < 0:
parser.error('invalid port, count, timeout or hold delay')
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
args.timeout, args.hold_ms, args.min_backends)
else:
with TCPServer(('127.0.0.1', 0), TCPBackend) as server:
server.backend_id = 'local-mock'
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
print('SELFTEST PASS: tested mock TCP peer, not RustProxy')
finally:
server.shutdown()
thread.join(timeout=2)
if __name__ == '__main__':
try:
main()
except (OSError, AssertionError, UnicodeError) as exc:
print(f'TCP FAIL: {exc}', file=sys.stderr)
sys.exit(1)
scripts/tests/test_stream_udp.py
-135
View File
@@ -1,135 +0,0 @@
#!/usr/bin/env python3
"""UDP Stream black-box test: sends multiple packets from each stable client socket.
Start separately labeled mock backends and target an existing RustProxy UDP
listener. 'selftest' only validates this Python harness, NOT RustProxy.
By default, the server responds PONG to PING for an explicitly configured probe.
"""
from __future__ import annotations
import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import secrets
import socket
import socketserver
import sys
import threading
import time
MAX_DATAGRAM = 1024
class UDPServer(socketserver.UDPServer):
allow_reuse_address = True
class UDPBackend(socketserver.BaseRequestHandler):
def handle(self) -> None:
data, connection = self.request
if data == b'PING':
reply = b'PONG'
elif data and len(data) <= MAX_DATAGRAM:
reply = self.server.backend_id.encode('ascii') + b'|' + data
else:
return
connection.sendto(reply, self.client_address)
def run_server(host: str, port: int, backend_id: str) -> None:
with UDPServer((host, port), UDPBackend) as server:
server.backend_id = backend_id
print(f'UDP mock {backend_id} listening on {server.server_address}; health probe PING -> PONG', flush=True)
try:
server.serve_forever(poll_interval=0.1)
except KeyboardInterrupt:
pass
def check_one(index: int, host: str, port: int, messages: int, timeout: float, pause_ms: float, nonce: str) -> str:
info = socket.getaddrinfo(host, port, type=socket.SOCK_DGRAM)[0]
family, sock_type, proto, _, address = info
backend = None
# One socket = one stable client identity for the entire UDP Stream session.
with socket.socket(family, sock_type, proto) as connection:
connection.settimeout(timeout)
for seq in range(messages):
payload = f'rp-udp:{nonce}:{index}:{seq}'.encode('ascii')
connection.sendto(payload, address)
response, _ = connection.recvfrom(MAX_DATAGRAM + 128)
try:
observed, echoed = response.split(b'|', 1)
except ValueError as exc:
raise AssertionError(f'session {index}: missing mock backend response: {response!r}') from exc
assert echoed == payload, f'session {index}: response mismatch {response!r}'
assert observed, f'session {index}: empty backend ID'
if backend is None:
backend = observed
assert observed == backend, f'session {index}: UDP backend changed for the same client socket'
if pause_ms and seq != messages - 1:
time.sleep(pause_ms / 1000)
return backend.decode('ascii')
def run_client(host: str, port: int, sessions: int, messages: int, parallel: int, timeout: float,
pause_ms: float, min_backends: int) -> Counter:
nonce = secrets.token_hex(8)
started = time.monotonic()
counts: Counter[str] = Counter()
with ThreadPoolExecutor(max_workers=parallel) as pool:
futures = [pool.submit(check_one, i, host, port, messages, timeout, pause_ms, nonce)
for i in range(sessions)]
for future in as_completed(futures):
counts[future.result()] += 1
assert len(counts) >= min_backends, f'observed {len(counts)} backend(s), required {min_backends}: {dict(counts)}'
print(f'UDP PASS: {sessions} client sockets, {sessions * messages} datagrams, '
f'{len(counts)} backend(s), {time.monotonic() - started:.2f}s; assignments={dict(sorted(counts.items()))}')
return counts
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='mode', required=True)
server = sub.add_parser('server', help='run local UDP echo and optional PING/PONG probe responder')
server.add_argument('--host', default='127.0.0.1')
server.add_argument('--port', type=int, required=True)
server.add_argument('--backend-id', required=True, help='unique ASCII marker (e.g. node-a)')
client = sub.add_parser('client', help='test forwarding via an already-running RustProxy UDP listener')
client.add_argument('--host', default='127.0.0.1', help='RustProxy UDP listener host')
client.add_argument('--port', type=int, required=True, help='RustProxy UDP listener port')
client.add_argument('--sessions', type=int, default=24)
client.add_argument('--messages', type=int, default=3)
client.add_argument('--parallel', type=int, default=8)
client.add_argument('--timeout', type=float, default=4.0)
client.add_argument('--pause-ms', type=float, default=20.0, help='pause between datagrams on each stable client socket')
client.add_argument('--min-backends', type=int, default=1, help='minimum distinct backend IDs observed')
sub.add_parser('selftest', help='test mock peer + harness directly, WITHOUT RustProxy')
args = parser.parse_args()
if args.mode == 'server':
if not args.backend_id.isascii() or not args.backend_id or '|' in args.backend_id or '\n' in args.backend_id:
parser.error('--backend-id must be nonempty ASCII without | or newline')
run_server(args.host, args.port, args.backend_id)
elif args.mode == 'client':
if min(args.port, args.sessions, args.messages, args.parallel, args.min_backends) < 1 or args.port > 65535 or args.timeout <= 0 or args.pause_ms < 0:
parser.error('invalid port, count, timeout or pause delay')
run_client(args.host, args.port, args.sessions, args.messages, args.parallel,
args.timeout, args.pause_ms, args.min_backends)
else:
with UDPServer(('127.0.0.1', 0), UDPBackend) as server:
server.backend_id = 'local-mock'
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
run_client('127.0.0.1', server.server_address[1], 8, 3, 4, 2.0, 5, 1)
print('SELFTEST PASS: tested mock UDP peer, not RustProxy')
finally:
server.shutdown()
thread.join(timeout=2)
if __name__ == '__main__':
try:
main()
except (OSError, AssertionError, UnicodeError) as exc:
print(f'UDP FAIL: {exc}', file=sys.stderr)
sys.exit(1)
scripts/tests/upgrade_performance_test.py
-80
View File
@@ -1,80 +0,0 @@
#!/usr/bin/env python3
"""Run source-upgrade preservation/refusal regressions in disposable directories."""
from pathlib import Path
import shutil
import sqlite3
import sys
import tempfile
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT))
from scripts.deploy.upgrade import upgrade
def fixture(parent):
target = parent / "installed"
target.mkdir()
(target / "Cargo.toml").write_text('[package]\nname = "rust-proxy"\nversion = "0.33.7"\n')
(target / "src").mkdir()
(target / "src" / "obsolete.rs").write_text("// old managed source\n")
(target / "data").mkdir()
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
db.execute("CREATE TABLE preserved(value TEXT)")
db.execute("INSERT INTO preserved VALUES ('configuration and preferences')")
(target / ".env").write_text("LOCAL_FIXTURE=1\n")
(target / "langs").mkdir()
(target / "langs" / "custom.json").write_text('{"custom": true}\n')
return target
def main():
with tempfile.TemporaryDirectory(prefix="rustproxy-upgrade-test-", dir=ROOT.parent) as temporary:
parent = Path(temporary)
target = fixture(parent)
try:
upgrade(ROOT, target, stopped=False)
except ValueError:
pass
else:
raise AssertionError("upgrade must require stopped confirmation")
assert (target / "src" / "obsolete.rs").is_file()
print("PASS upgrade refuses missing --stopped without modifying source")
if sys.platform != "win32":
import fcntl
with (target / "data" / "process.lock").open("w+") as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
try:
upgrade(ROOT, target, stopped=True)
except ValueError:
pass
else:
raise AssertionError("upgrade must refuse a live process lock")
print("PASS upgrade refuses a data directory locked by a live process")
backup = upgrade(ROOT, target, stopped=True)
assert (target / ".env").read_text() == "LOCAL_FIXTURE=1\n"
assert (target / "langs" / "custom.json").read_text() == '{"custom": true}\n'
with sqlite3.connect(target / "data" / "fixture.sqlite") as db:
assert db.execute("SELECT value FROM preserved").fetchone() == ("configuration and preferences",)
assert not (target / "src" / "obsolete.rs").exists()
assert (backup / "src" / "obsolete.rs").is_file()
assert (target / "Cargo.toml").read_bytes() == (ROOT / "Cargo.toml").read_bytes()
assert (target / "src" / "proxy" / "cache_plan.rs").is_file()
print("PASS upgrade preserves SQLite, environment and custom language; backs up old source and replaces managed tree")
if sys.platform != "win32":
shutil.rmtree(target / "src")
(target / "src").symlink_to(backup / "src", target_is_directory=True)
try:
upgrade(ROOT, target, stopped=True)
except ValueError:
pass
else:
raise AssertionError("upgrade must refuse a managed symlink")
assert (backup / "src" / "obsolete.rs").is_file()
print("PASS upgrade refuses managed symlinks without changing their targets")
if __name__ == "__main__":
main()
scripts/tests/vhost_backend_popup_0337_browser_test.py
-268
View File
@@ -1,268 +0,0 @@
#!/usr/bin/env python3
"""Simulated Chromium Vhost backend quick-view, without a Rust server.
Verify both backend triggers, the shared click-to-open upstream H2 protocol
popover without native title flicker, and the two Cache card margins.
Exercise live updates, safe text, keyboard/outside dismissal and mobile sizing.
"""
from pathlib import Path
from panel_html import render_admin_index
from playwright.sync_api import sync_playwright
import json
import re
import time
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
PACKS = {lang: {'id': lang, **json.loads((ROOT / 'langs' / f'{lang}.json').read_text())}
for lang in ('en_US', 'pl_pl')}
def app_bundle():
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert found
code = ROUTER[found.end():]
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
fragments = []
for line in code.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / f'{name}.js').read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
return '\n'.join(fragments)
CONFIG = {
'vhosts': [{
'id': 'api-one', 'enabled': True, 'source': {'type': 'reverse_proxy'},
'frontend': {'hosts': ['api.example.test'], 'listener_ids': []},
'backends': [
{'id': 'origin-http', 'url': 'http://10.12.0.5:8080/api', 'protocol': 'http2', 'enabled': True},
{'id': '<origin-dns>', 'url': 'https://hiddenuser:secret@backend.example.test:8443/path', 'protocol': 'auto', 'enabled': True},
]
}, {'id': 'static-only', 'enabled': True,
'source': {'type': 'web_server', 'web_service_id': 'static-test'},
'frontend': {'hosts': ['static.example.test'], 'listener_ids': []}, 'backends': []}],
'listeners': {'endpoints': []}, 'web_services': [], 'acl_profiles': [],
}
def snapshot(first=200):
return {'backend_pools': [{
'id': 'api-one', 'health_mode': 'on',
'members': [
{'id': 'origin-http', 'status': 'up', 'last_status': first,
'requests': 12, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
{'id': '<origin-dns>', 'status': 'down', 'last_status': 0,
'requests': 0, 'active': 0, 'errors': 0, 'bytes_in': 0, 'bytes_out': 0},
]
}], 'vhost_metrics': {}}
def protocol_snapshot(h2_count):
status = snapshot()
status['backend_pools'][0]['members'][0].update({
'upstream_protocol_observed': 'h2',
'upstream_protocol_last_seen_ms': int(time.time() * 1000),
'upstream_protocol_http1_total': 3,
'upstream_protocol_http2_total': h2_count,
})
return status
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 768, 1440):
page = browser.new_page(viewport={'width': width, 'height': 820})
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
page.evaluate('''([config, status, packs]) => {
languagePacks = new Map(Object.entries(packs));
defaultLanguage = 'en_US'; language = 'pl_pl';
document.documentElement.lang = 'pl-PL';
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-vhosts').hidden = false;
currentPanel = 'vhosts';
envelope = {revision: 12, config};
latestStatus = status;
window.rustProxyVhosts.render();
}''', [CONFIG, snapshot(), PACKS])
vhost = page.locator('[data-vhost-card="api-one"]')
backend = vhost.locator('.vhost-detail.vhost-backend-trigger')
health = vhost.locator('.vhost-live-metric.vhost-backend-trigger')
assert backend.count() == health.count() == 1, width
assert backend.get_attribute('aria-expanded') == 'false'
assert backend.get_attribute('aria-haspopup') == 'dialog'
# Clickable backend/health fields must match their non-button peers,
# including for user-selected interface corner styles.
shapes = page.evaluate('''() => {
const root = document.querySelector('[data-vhost-card="api-one"]');
const peers = [
[root.querySelector('.vhost-detail.vhost-backend-trigger'), root.querySelector('.vhost-detail:not(.vhost-backend-trigger)')],
[root.querySelector('.vhost-live-metric.vhost-backend-trigger'), root.querySelector('.vhost-live-metric:not(.vhost-backend-trigger)')]
];
const checks = ['square', 'subtle', 'rounded', 'extra'].map(corner => {
document.documentElement.dataset.cornerStyle = corner;
return {corner, tiles: peers.map(([trigger, peer]) => {
const left = getComputedStyle(trigger), right = getComputedStyle(peer);
return {radius: left.borderRadius, peerRadius: right.borderRadius,
padding: left.padding, peerPadding: right.padding,
font: left.fontFamily, peerFont: right.fontFamily};
})};
});
document.documentElement.removeAttribute('data-corner-style');
return checks;
}''')
for shape in shapes:
for tile in shape['tiles']:
assert tile['radius'] == tile['peerRadius'] == '0px', (width, shape)
assert tile['padding'] == tile['peerPadding'], (width, shape)
assert tile['font'] == tile['peerFont'], (width, shape)
backend.click()
popup = page.locator('#vhost-backend-popup')
assert popup.count() == 1, width
assert backend.get_attribute('aria-expanded') == 'true'
assert popup.get_attribute('role') == 'dialog'
# The anchored popup is a surface, not a permanently rounded tooltip.
# It must follow live interface corner preference changes without
# dismissing/recreating the open backend (or shared H2) popup.
corners = page.evaluate('''() => {
const node = document.getElementById('vhost-backend-popup');
const saved = document.documentElement.getAttribute('data-corner-style');
const out = {};
for (const style of ['default', 'square', 'subtle', 'rounded', 'extra']) {
document.documentElement.dataset.cornerStyle = style;
out[style] = {
popup: getComputedStyle(node).borderRadius,
row: getComputedStyle(node.querySelector('.vhost-backend-popup-row')).borderRadius,
close: getComputedStyle(node.querySelector('.vhost-backend-popup-close')).borderRadius,
reference: getComputedStyle(document.querySelector('[data-vhost-card="api-one"]')).borderRadius,
sameNode: node === document.getElementById('vhost-backend-popup'),
};
}
if (saved === null) document.documentElement.removeAttribute('data-corner-style');
else document.documentElement.dataset.cornerStyle = saved;
return out;
}''')
expected_radii = {'default': ('0px', '0px'), 'square': ('0px', '0px'),
'subtle': ('5px', '4px'), 'rounded': ('12px', '8px'),
'extra': ('18px', '12px')}
for style, (surface, control) in expected_radii.items():
actual = corners[style]
assert actual == {'popup': surface, 'row': surface, 'close': control,
'reference': surface, 'sameNode': True}, (width, style, actual)
values = popup.evaluate('''node => ({
cards: node.querySelectorAll('.vhost-backend-popup-row').length,
text: node.textContent, html: node.innerHTML,
box: (() => { const r = node.getBoundingClientRect();
return {left: r.left, right: r.right, top:r.top, bottom:r.bottom}; })(),
})''')
assert values['cards'] == 2, (width, values)
assert '10.12.0.5:8080' in values['text'] and 'HTTP 200' in values['text'], (width, values)
assert 'backend.example.test:8443' in values['text'], width
assert 'secret' not in values['text'], width
assert '<origin-dns>' in values['text'] and '&lt;origin-dns&gt;' in values['html'], width
assert 'Brak odpowiedzi HTTP' in values['text'], width
assert 'HTTPS · AUTO' in values['text'], width
assert values['box']['left'] >= -1 and values['box']['right'] <= width + 1, (width, values['box'])
assert values['box']['top'] >= -1 and values['box']['bottom'] <= 821, (width, values['box'])
assert page.evaluate('document.documentElement.scrollWidth - innerWidth') <= 2, width
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', snapshot(503))
assert 'HTTP 503' in popup.inner_text(), width
assert popup.locator('.has-error').count() == 1, width
health.click()
assert popup.count() == 1 and health.get_attribute('aria-expanded') == 'true', width
assert backend.get_attribute('aria-expanded') == 'false', width
page.keyboard.press('Escape')
assert popup.count() == 0 and health.get_attribute('aria-expanded') == 'false', width
assert health.evaluate('(node) => document.activeElement === node'), width
backend.click()
page.locator('#vhost-search').click(force=True)
assert popup.count() == 0, width
# A static-only vhost presents a translated empty state, not stale pool values.
page.locator('[data-vhost-card="static-only"] .vhost-detail.vhost-backend-trigger').click()
assert 'Brak skonfigurowanych backendów.' in popup.inner_text(), width
popup.locator('.vhost-backend-popup-close').click()
assert popup.count() == 0, width
backend.click()
assert popup.count() == 1, width
# The existing backend popover is also used by the H2 policy badge.
protocol = vhost.locator('[data-upstream-protocol-backend="origin-http"]')
assert protocol.count() == 1 and protocol.evaluate('n => n.tagName') == 'BUTTON', width
assert protocol.get_attribute('title') is None, width
protocol.click()
assert backend.get_attribute('aria-expanded') == 'false', width
assert protocol.get_attribute('aria-expanded') == 'true', width
assert 'Brak odpowiedzi backendu' in popup.inner_text(), width
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(8))
assert 'origin-http · H2 compatible → H2' in protocol.inner_text(), width
assert 'Ostatnia wersja HTTP' in popup.inner_text(), width
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '8'], width
page.evaluate('''() => {
window.__popupReference = document.getElementById('vhost-backend-popup');
window.__headingReference = window.__popupReference.querySelector('h4');
window.__popupRemoved = 0;
window.__popupObserver = new MutationObserver(records => {
for (const record of records) for (const item of record.removedNodes) {
if (item === window.__popupReference) window.__popupRemoved++;
}
});
window.__popupObserver.observe(document.body, {childList: true});
}''')
for count in range(9, 19):
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', protocol_snapshot(count))
unchanged = page.evaluate('''() => ({
samePopup: document.getElementById('vhost-backend-popup') === window.__popupReference,
sameHeading: window.__popupReference.querySelector('h4') === window.__headingReference,
removed: window.__popupRemoved,
expanded: document.querySelector('[data-upstream-protocol-backend="origin-http"]').getAttribute('aria-expanded'),
})''')
assert unchanged == {'samePopup': True, 'sameHeading': True, 'removed': 0, 'expanded': 'true'}, (width, unchanged)
assert popup.locator('.vhost-backend-popup-field > span:last-child').all_inner_texts() == ['3', '18'], width
page.evaluate('window.__popupObserver.disconnect()')
assert protocol.get_attribute('title') is None, width
# A stale observation should change the displayed information in
# place, not dismiss/recreate the click popover.
stale = protocol_snapshot(18)
stale['backend_pools'][0]['members'][0]['upstream_protocol_last_seen_ms'] -= 600000
page.evaluate('''s => window.rustProxyVhosts.renderStatus(s)''', stale)
assert 'H2 compatible → ?' in protocol.inner_text(), width
assert popup.locator('.vhost-backend-popup-list').is_hidden(), width
assert page.evaluate('document.getElementById("vhost-backend-popup") === window.__popupReference'), width
page.keyboard.press('Escape')
assert popup.count() == 0 and protocol.get_attribute('aria-expanded') == 'false', width
assert protocol.evaluate('node => document.activeElement === node'), width
# Cache Domains and Stored objects are the only directly adjacent
# table cards and need a visible, small gap at any viewport width.
cache_gap = page.evaluate('''() => {
const panel = document.getElementById('panel-cache');
panel.hidden = false;
const [domains, stored] = panel.querySelectorAll(':scope > article.card');
const gap = stored.getBoundingClientRect().top - domains.getBoundingClientRect().bottom;
panel.hidden = true;
return gap;
}''')
assert 9.5 <= cache_gap <= 10.5, (width, cache_gap)
page.evaluate('window.rustProxyVhosts.reset()')
assert popup.count() == 0, width
assert not errors, (width, errors)
print(f'PASS Vhosts shared backend/H2 popover and Cache gap at {width}px')
page.close()
browser.close()
print('PASS Vhost quick-view with simulated runtime status (not live Rust)')
scripts/tests/web_observability_0330_browser_test.py
-140
View File
@@ -1,140 +0,0 @@
from pathlib import Path
from panel_html import render_admin_index
import re, json
from playwright.sync_api import sync_playwright
root=Path(__file__).resolve().parents[2]
router=(root/'src/admin/router.rs').read_text()
def concatenate(section):
pattern=r'fn '+section+r'\([^)]*\)[^{]*\{'
m=re.search(pattern,router)
assert m, section
text=router[m.end():]
# The array contents are bounded by ].join("\\n") before the next function.
text=text[:re.search(r'\]\s*\.join\("\\n"\)', text).start()]
result=[]
for line in text.splitlines():
line=line.strip()
if 'include_str!("' in line:
path=re.search(r'include_str!\("([^"]+)"\)',line).group(1)
result.append((root/'src/admin'/path).read_text())
elif line=='admin_vhosts_javascript(),':
result.append('(() => {\n'+''.join((root/'src/admin/web/vhosts'/f'{part}.js').read_text()+'\n' for part in ['core','protocol','health','rates','editor','render'])+'})();')
elif line.startswith('"') and line.endswith('",'):
result.append(json.loads(line[:-1]))
return '\n'.join(result)
bundle=concatenate('admin_app_javascript')
assert 'window.rustProxyOverviewSamples =' in bundle
assert bundle.index('window.rustProxyOverviewSamples =') < bundle.index('window.rustProxyOverviewSamples.onChange(')
css=(root/'src/admin/web/style.css').read_text()
html=render_admin_index(root)
with sync_playwright() as p:
browser=p.chromium.launch(headless=True, executable_path='/usr/bin/chromium',args=['--no-sandbox','--allow-file-access-from-files'])
for width in [320,375,800,1440]:
page=browser.new_page(viewport={'width':width,'height':900})
errors=[]
page.on('pageerror',lambda e: errors.append(str(e)))
# Execute in a fully populated DOM without disallowed filesystem/network navigation.
page.set_content(html.replace('/assets/app___RUSTPROXY_ASSET_ID__.js','').replace('/assets/style___RUSTPROXY_ASSET_ID__.css',''))
page.add_style_tag(content=css)
page.add_script_tag(content=bundle)
page.wait_for_timeout(550)
state=page.evaluate('''() => ({ controller:typeof window.rustProxyOverviewSamples,
handle:typeof window.rustProxyOverviewSamples?.onChange,
web:typeof window.rustProxyWebOverview, services:typeof window.rustProxyWebServer,
registry: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
htmlWidth:document.documentElement.scrollWidth,windowWidth:window.innerWidth })''')
print(width,state,'errors',errors)
assert state['controller']=='object' and state['handle']=='function' and state['web']=='object'
assert state['registry']==12
assert not [e for e in errors if 'onChange' in e or 'ReferenceError' in e or 'SyntaxError' in e or 'TypeError' in e],errors
assert state['htmlWidth'] <= state['windowWidth'] + 2
# The selector belongs to the same header row as the core connection;
# hiding it on other panels must not leave an empty dashboard row.
layout = page.evaluate('''() => {
document.getElementById('workspace').hidden = false;
setPanel('overview', false);
const status = document.getElementById('connection-label');
const sample = document.getElementById('overview-samples-toolbar');
const row = document.querySelector('.page-intro-meta-row');
const rect = node => { const r = node.getBoundingClientRect();
return {x:r.x,y:r.y,width:r.width,height:r.height,right:r.right}; };
const overview = { status:rect(status),sample:rect(sample),row:rect(row),
visible: !sample.hidden,inside: row.contains(sample) && row.contains(status),
overflow: document.documentElement.scrollWidth > innerWidth + 2 };
setPanel('web-server', false);
const notOverview = sample.hidden && !status.hidden;
setPanel('overview', false);
return {overview,notOverview,visibleAgain:!sample.hidden};
}''')
assert layout['overview']['inside'] and layout['overview']['visible']
assert layout['notOverview'] and layout['visibleAgain']
assert not layout['overview']['overflow'], (width, layout)
if width >= 800:
assert abs(layout['overview']['sample']['y'] - layout['overview']['status']['y']) <= 20, (width, layout)
assert layout['overview']['sample']['right'] <= layout['overview']['row']['right'] + 2, (width, layout)
# A genuinely sampled WWW metrics payload (not a screenshot-only fixture).
report = page.evaluate('''() => {
Object.defineProperty(window.performance, "now", {configurable:true, value:()=>window.__clock});
window.__clock = 1000;
currentPanel = "overview";
document.getElementById('workspace').hidden = false;
document.getElementById('login').hidden = true;
document.getElementById('panel-overview').hidden = false;
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
function snap(k) {
return {revision:7, web_server_metrics:{
"public-site": {web: {requests:100+k*30, status_2xx:86+k*24, status_3xx:3+k,
status_4xx:8+k*2, status_5xx:3+k*3, response_headers_us:4000000+k*1300000,
response_bytes:100000+k*20000, static_requests:70+k*20, php_requests:25+k*7,
other_requests:5+k*3, methods:[90+k*25,8+k*4,0,0,2+k],
selected_status:[80+k*18,2,2,1,5+k,0,0,0], active_headers:0},
php: {metrics:{requests:25+k*7, response_headers_us:2000000+k*700000,
response_bytes:30000+k*5000,failures:[1+k,1+k,1+k,0,0,0,0,0,0]},
capacity_limit:32, capacity_used:8, capacity_available:24}}}
};
}
const before = snap(0), after = snap(1);
// Saved custom layout from before 0.33.0 must not enable new cards.
const savedLayout = applyDashboardLayout(['requests', 'errors'],
normalizeHiddenDashboardWidgets([], ['requests', 'errors']));
const newCardsHidden = [...document.querySelectorAll('[data-dashboard-widget^="www_"]')]
.every(card=>card.hidden);
for (const n of document.querySelectorAll('[data-dashboard-widget^="www_"]')) n.hidden = false;
window.rustProxyWebOverview.render(before, true);
window.__clock = 4000;
window.rustProxyWebOverview.render(after, true);
let ids = ['www_requests-all-line', 'www_requests-php-line',
'www_latency-web-line', 'www_traffic-php-line', 'www_status_trend-5xx-line'];
const linePaths = ids.map(id=>document.getElementById(id)?.getAttribute('d'));
const statusText = document.getElementById('www_status-value').textContent;
const capacityText = document.getElementById('www_capacity-value').textContent;
const rateText = document.getElementById('www_php_requests-value').textContent;
const bars = document.querySelectorAll('#www_top_services-bars .www-top-row').length;
window.rustProxyOverviewSamples.setLimit(120);
const historyLimit = window.rustProxyOverviewSamples.getLimit();
currentPanel = "web-server";
envelope = {config:{web_services:[{id:'public-site', name:'Public site',document_root:'/srv/www/public'}],vhosts:[]}};
window.rustProxyWebServer.render();
const details = document.querySelector('.www-service-panel');
if (!details || details.tagName !== 'SECTION') throw Error('Missing always-visible WWW metrics');
window.__clock = 7000;
window.rustProxyWebServer.renderLive(before, true);
window.__clock = 10000;
window.rustProxyWebServer.renderLive(after, true);
const value = details.querySelector('[data-web-live-field="requests"]')?.textContent;
const phpCapacity = details.querySelector('[data-web-live-field="php"]')?.textContent;
return {linePaths,statusText,capacityText,rateText,bars,historyLimit,value,phpCapacity,newCardsHidden,
width:document.documentElement.scrollWidth};
}''')
print('sampled WWW',width,report)
assert all(report['linePaths']), report
assert report['statusText'] != '-' and report['capacityText'] == '8 / 32', report
assert report['rateText'] not in ('-', '0.0/s'), report
assert report['bars'] == 1 and report['historyLimit'] == 120 and report['newCardsHidden'], report
assert report['value'] and report['phpCapacity'] == '8 / 32', report
assert report['width'] <= width + 2, report
assert not [e for e in errors if any(x in e for x in ('ReferenceError', 'TypeError', 'SyntaxError'))], errors
page.close()
browser.close()
print('PASS: actual composed admin bundle starts without a JS TypeError')
scripts/tests/web_observability_0330_source_test.py
-84
View File
@@ -1,84 +0,0 @@
#!/usr/bin/env python3
"""Static contracts for 0.33.0 WWW observability (not Rust compilation)."""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
import tomllib
ROOT = Path(__file__).resolve().parents[2]
read = lambda file: (ROOT / file).read_text()
manifest = tomllib.loads(read('Cargo.toml'))
lock = tomllib.loads(read('Cargo.lock'))
version = manifest['package']['version']
assert tuple(map(int, version.split('.'))) >= (0, 33, 0)
assert [p['version'] for p in lock['package'] if p['name'] == 'rust-proxy'] == [version]
registry = json.loads(read('src/admin/web/dashboard-widgets.json'))
widgets = [w for w in registry if w['id'].startswith('www_')]
assert len(widgets) == 12
assert all(w['default_hidden'] is True for w in widgets)
assert len({w['id'] for w in registry}) == len(registry)
html = render_admin_index(ROOT)
english = json.loads(read('langs/en_US.json'))['translations']
polish = json.loads(read('langs/pl_pl.json'))['translations']
for widget in widgets:
assert f'data-dashboard-widget="{widget["id"]}" hidden' in html, widget['id']
assert widget['label_key'] in english, widget['label_key']
assert widget['label_key'] in polish, widget['label_key']
if widget['description_key']:
assert widget['description_key'] in english
assert widget['description_key'] in polish
# Safely assemble the admin app before core.js: the earlier release crashed on
# registering onChange callbacks before rustProxyOverviewSamples was defined.
router = read('src/admin/router.rs')
assert router.index('include_str!("web/app/overview-samples.js")') < router.index('include_str!("web/app/core.js")')
assert '"web/app/web-overview.js"' in router
assert '"web_server_metrics"' in read('src/admin/status_live.rs')
assert 'web_server_metrics_payload(&snapshot)' in read('src/admin/status.rs')
# record_web is only consumed by web_server::serve; expose no wider than the
# ResponseKind enum argument. This is a static visibility contract, not rustc.
metrics = read('src/web_server/metrics.rs')
assert 'pub(super) enum ResponseKind' in metrics
assert 'pub(super) fn record_web(' in metrics
assert 'pub(crate) fn record_web(' not in metrics
assert 'service.web_metrics.record_web(' in read('src/web_server/serve.rs')
families = read('src/prometheus/web_php.rs')
required = {
'rust_proxy_web_server_requests_total', 'rust_proxy_web_server_responses_total',
'rust_proxy_web_server_request_kind_total', 'rust_proxy_web_server_methods_total',
'rust_proxy_web_server_status_code_total', 'rust_proxy_web_server_response_header_duration_microseconds_total',
'rust_proxy_web_server_response_body_bytes_total', 'rust_proxy_web_server_requests_active',
'rust_proxy_web_server_compression_total', 'rust_proxy_php_requests_total',
'rust_proxy_php_responses_total', 'rust_proxy_php_response_header_duration_microseconds_total',
'rust_proxy_php_response_body_bytes_total', 'rust_proxy_php_concurrency_available',
'rust_proxy_php_concurrency_limit', 'rust_proxy_php_concurrency_in_use', 'rust_proxy_php_failures_total'
}
assert all(f'"{name}"' in families for name in required), required
assert 'PHP_FAILURE_NAMES' in families
assert re.search(r'\["GET", "POST", "PUT", "DELETE", "other"\]', families)
assert 'let wl = labels(&[("web_service", service.id.as_str())]);' in families
j = json.loads(read('grafana/rustproxy-overview.json'))
assert j['title'].startswith('RustProxy 0.33.')
ids = [p['id'] for p in j['panels']]
assert len(ids) == len(set(ids))
assert any(v.get('name') == 'web_service' and v.get('multi') and v.get('includeAll') for v in j['templating']['list'])
for title in ('Web Server 0.33.0', 'PHP-FPM / FastCGI', 'Per Web Service', 'WWW / PHP correlation'):
assert any(p['title'] == title and p['type'] == 'row' for p in j['panels'])
extra = [p for p in j['panels'] if p['id'] >= 89]
for i, p in enumerate(extra):
a = p['gridPos']
assert 0 <= a['x'] and a['w'] > 0 and a['x'] + a['w'] <= 24
for q in extra[i + 1:]:
b = q['gridPos']
overlap = a['x'] < b['x'] + b['w'] and b['x'] < a['x'] + a['w'] and a['y'] < b['y'] + b['h'] and b['y'] < a['y'] + a['h']
assert not overlap, (p['id'], q['id'])
production = [p for p in (ROOT / 'src').rglob('*.rs') if p.name != 'tests.rs' and 'tests' not in p.parts]
large = [str(p.relative_to(ROOT)) for p in production if len(p.read_text().splitlines()) > 500]
assert not large, large
print('PASS RustProxy 0.33.0 source version, opt-in widgets, localizations, status API,')
print(' exporter families, Grafana geometry/filtering and Rust module size (STATIC ONLY)')
scripts/tests/web_server_0330_ui_test.py
-153
View File
@@ -1,153 +0,0 @@
#!/usr/bin/env python3
"""Simulated Web Server telemetry against the production admin JS/CSS bundle.
Browser-only test: does not assert live RustProxy, PHP-FPM, or network behavior.
Run: python3 scripts/tests/web_server_0330_ui_test.py
"""
from __future__ import annotations
import json
from pathlib import Path
from panel_html import render_admin_index
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
router = (ROOT / 'src/admin/router.rs').read_text()
match = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', router)
assert match, 'admin_app_javascript definition missing'
portion = router[match.end():]
portion = portion[:re.search(r'\]\s*\.join\("\\n"\)', portion).start()]
fragments = []
for line in portion.splitlines():
line = line.strip()
if 'include_str!("' in line:
path = re.search(r'include_str!\("([^"]+)"\)', line).group(1)
fragments.append((ROOT / 'src/admin' / path).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join((ROOT / 'src/admin/web/vhosts' / f'{part}.js').read_text() for part in ['core', 'protocol', 'health', 'rates', 'editor', 'render']) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
BUNDLE = '\n'.join(fragments)
assert BUNDLE.index('window.rustProxyOverviewSamples =') < BUNDLE.index('window.rustProxyOverviewSamples.onChange(')
WEB = {
'requests': 100, 'status_2xx': 90, 'status_3xx': 2, 'status_4xx': 5, 'status_5xx': 3,
'response_headers_us': 2_500_000, 'response_bytes': 10_000, 'active_headers': 2,
'static_requests': 70, 'php_requests': 25, 'other_requests': 5,
'methods': [80, 10, 5, 2, 3], 'selected_status': [75, 5, 2, 0, 5, 1, 1, 1],
'failures': [0] * 9, 'compression': [5, 3, 2, 1],
}
PHP = {
'requests': 25, 'status_2xx': 22, 'status_3xx': 0, 'status_4xx': 1, 'status_5xx': 2,
'response_headers_us': 1_000_000, 'response_bytes': 2_000,
'failures': [1, 2, 3, 4, 5, 6, 7, 8, 9],
}
def observation(factor: int):
web = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
for key, value in WEB.items()}
php = {key: ([factor * v for v in value] if isinstance(value, list) else factor * value)
for key, value in PHP.items()}
web['active_headers'] = 2
return {'revision': 10, 'web_server_metrics': {
'site-A': {'web': web, 'php': {'metrics': php, 'capacity_limit': 32, 'capacity_used': 18}}
}}
with sync_playwright() as p:
browser = p.chromium.launch(headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 800, 1440):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda err: errors.append(str(err)))
page.set_content(render_admin_index(ROOT))
page.add_style_tag(content=(ROOT / 'src/admin/web/style.css').read_text())
page.add_script_tag(content=BUNDLE)
page.evaluate("""() => {
document.querySelector('#workspace').hidden = false;
document.querySelector('#panel-overview').hidden = false;
document.querySelectorAll('[data-dashboard-widget^="www_"]').forEach(n => n.hidden = false);
currentPanel = 'overview';
window.__clock = 1000;
Object.defineProperty(performance, 'now', { configurable: true, value: () => window.__clock });
}""")
first, second = observation(1), observation(2)
state = page.evaluate("""([one, two]) => {
const api = window.rustProxyWebOverview;
api.render(one);
window.__clock = 4000;
api.render(two);
return {
rps: document.getElementById('www_requests-all-value').textContent,
php: document.getElementById('www_requests-php-value').textContent,
latency: document.getElementById('www_latency-web-value').textContent,
capacity: document.getElementById('www_capacity-value').textContent,
types: document.getElementById('www_kind-value').textContent,
status: document.getElementById('www_status-value').textContent,
bytes: document.getElementById('www_traffic-value').textContent,
top: document.getElementById('www_top_services-value').textContent,
layoutWidth: document.documentElement.scrollWidth,
screenWidth: window.innerWidth,
children: document.querySelectorAll('[data-dashboard-widget^="www_"]').length,
sampleCount: document.getElementById('www_requests-all-line').getAttribute('d'),
limit: window.rustProxyOverviewSamples.getLimit(),
};
}""", [first, second])
assert state['children'] == 12, state
assert state['rps'] != '-' and state['php'] != '-', state
assert 'ms' in state['latency'] and '18' in state['capacity'], state
assert state['types'].endswith('%') and state['status'].endswith('%'), state
assert '/s' in state['top'], state
assert state['layoutWidth'] <= state['screenWidth'] + 2, (width, state)
assert not errors, (width, errors)
# Exercise the always-visible per-service metrics with the same status,
# without any real HTTP/PHP request or extra polling.
details = page.evaluate("""(status) => {
envelope = { revision: 10, config: {
web_services: [{ id: 'site-A', name: 'Example', document_root: '/var/www/site-A',
index: ['index.html'], php: { enabled: true } }],
vhosts: [], php_fpm_profiles: []
}};
currentPanel = 'web-server';
window.rustProxyWebServer.render();
const detail = document.querySelector('#web-service-list .www-service-panel');
if (!detail) return { missing: true };
window.rustProxyWebServer.renderLive(status, false);
return { text: detail.textContent, rows: detail.querySelectorAll('[data-web-live-field]').length,
alwaysVisible: detail.tagName === 'SECTION' && !detail.querySelector('summary') };
}""", second)
assert not details.get('missing') and details['rows'] == 7 and details['alwaysVisible'], (width, details)
assert '18 / 32' in details['text'], (width, details)
# The first reading after an overdue browser interval must establish a
# fresh baseline without adding a fake 0/s dip to the service chart.
continuity = page.evaluate("""([first, second, paused, resumed]) => {
const api = window.rustProxyWebServer;
const chart = document.querySelector('#web-service-list [data-web-live-id="site-A"] polyline');
const rate = document.querySelector('#web-service-list [data-web-live-field="rate"]');
window.__clock = 5000; api.renderLive(first);
window.__clock = 8000; api.renderLive(second);
const before = chart.getAttribute('points');
window.__clock = 100000; api.renderLive(paused);
const during = chart.getAttribute('points'), unavailable = rate.textContent;
window.__clock = 103000; api.renderLive(resumed);
return { before, during, unavailable, after: chart.getAttribute('points'), resumedRate: rate.textContent };
}""", [observation(2), observation(3), observation(10), observation(12)])
assert continuity['before'] == continuity['during'], (width, continuity)
assert continuity['unavailable'] == '-', (width, continuity)
assert continuity['resumedRate'] != '-', (width, continuity)
assert continuity['after'] != continuity['before'], (width, continuity)
assert not errors, (width, errors)
old = page.evaluate("""() => {
currentPanel = 'overview';
window.rustProxyWebOverview.render({ revision: 11, web_server_metrics: {} });
return {
rate: document.getElementById('www_requests-value').textContent,
status: document.getElementById('www_status-value').textContent,
top: document.getElementById('www_top_services-bars').childElementCount,
};
}""")
assert old == {'rate': '-', 'status': '-', 'top': 0}, (width, old)
assert not errors, (width, errors)
print(f'PASS simulated Web Server cards and always-visible service metrics at {width}px: {state["rps"]}, {state["php"]}, layout {state["layoutWidth"]}px')
page.close()
browser.close()
print('PASS production JS bundle + all twelve visible WWW widgets (simulated data only)')
scripts/tests/web_server_0334_browser_test.py
-155
View File
@@ -1,155 +0,0 @@
#!/usr/bin/env python3
"""WWW/Vhost visual parity in production UI files, using simulated status.
Chromium exercises the real assembled admin JS/CSS and responsive layouts.
It does not establish Rust, SSE or PHP-FPM runtime acceptance.
"""
from pathlib import Path
from panel_html import render_admin_index
import json
import re
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[2]
HTML = render_admin_index(ROOT)
CSS = (ROOT / 'src/admin/web/style.css').read_text()
ROUTER = (ROOT / 'src/admin/router.rs').read_text()
def app_bundle():
found = re.search(r'fn admin_app_javascript\([^)]*\)[^{]*\{', ROUTER)
assert found
code = ROUTER[found.end():]
code = code[:re.search(r'\]\s*\.join\("\\n"\)', code).start()]
fragments = []
for line in code.splitlines():
line = line.strip()
include = re.search(r'include_str!\("([^"]+)"\)', line)
if include:
fragments.append((ROOT / 'src/admin' / include.group(1)).read_text())
elif line == 'admin_vhosts_javascript(),':
fragments.append('(() => {\n' + '\n'.join(
(ROOT / 'src/admin/web/vhosts' / (name + '.js')).read_text()
for name in ('core', 'protocol', 'health', 'rates', 'editor', 'render')) + '\n})();')
elif line.startswith('"') and line.endswith('",'):
fragments.append(json.loads(line[:-1]))
return '\n'.join(fragments)
def status(step):
return {'web_server_metrics': {
'site-A': {'web': {
'requests': step * 120, 'response_headers_us': step * 2_400_000,
'status_5xx': step * 4, 'response_bytes': step * 20_480,
'static_requests': step * 80, 'php_requests': step * 30,
'other_requests': step * 10,
}, 'php': {'capacity_used': 3, 'capacity_limit': 16}},
'site-B': {'web': {
'requests': step * 24, 'response_headers_us': step * 100_000,
'status_5xx': 0, 'response_bytes': step * 8_192,
'static_requests': step * 24, 'php_requests': 0,
'other_requests': 0,
}, 'php': None},
}}
with sync_playwright() as playwright:
browser = playwright.chromium.launch(
headless=True, executable_path='/usr/bin/chromium', args=['--no-sandbox'])
for width in (320, 375, 640, 800, 980, 1440, 1920):
page = browser.new_page(viewport={'width': width, 'height': 900})
errors = []
page.on('pageerror', lambda e: errors.append(str(e)))
page.set_content(HTML.replace('/assets/app___RUSTPROXY_ASSET_ID__.js', '')
.replace('/assets/style___RUSTPROXY_ASSET_ID__.css', ''))
page.add_style_tag(content=CSS)
page.add_script_tag(content=app_bundle())
initial = page.evaluate('''(status) => {
document.documentElement.classList.remove('app-loading', 'app-startup-failed');
document.getElementById('login').hidden = true;
document.getElementById('workspace').hidden = false;
document.getElementById('panel-web-server').hidden = false;
currentPanel = 'web-server';
envelope = {revision: 10, config: {web_services: [
{id: 'site-A', name: 'Example service', document_root: '/var/www/a', index: ['index.html'], php: {enabled: true}},
{id: 'site-B', name: 'Static service', document_root: '/var/www/b', index: ['index.html']}
], vhosts: [], php_fpm_profiles: []}};
window.__clock = 1000;
Object.defineProperty(performance, 'now', {configurable: true, value: () => window.__clock});
latestStatus = status;
window.rustProxyWebServer.render();
const list = document.querySelector('#web-service-list');
const cards = [...list.querySelectorAll(':scope > .vhost-card')];
const metricPanels = cards.map(card => card.querySelector('.www-service-panel'));
const grid = metricPanels[0].querySelector('.vhost-live-grid');
const chart = metricPanels[0].querySelector('.vhost-chart');
const svg = chart.querySelector('svg');
const svgBox = svg.getBoundingClientRect(), svgMatrix = svg.getScreenCTM();
const rect = node => node.getBoundingClientRect();
const expectedMetricLabels = ['webLiveRequests','webLiveRate','webLiveErrors',
'webLiveLatency','webLivePhpCapacity','webLiveBytes'];
return {
cards: cards.length, sectionTags: metricPanels.map(panel => panel.tagName),
collapsibles: list.querySelectorAll('details, summary').length,
liveFields: metricPanels.map(panel => panel.querySelectorAll('[data-web-live-field]').length),
labels: [...grid.children].map(tile => tile.querySelector('span')?.dataset.i18n),
tileClasses: [...grid.children].map(tile => tile.className),
values: [...grid.querySelectorAll('strong')].map(el => el.textContent),
phpMissing: metricPanels[1].querySelector('[data-web-live-field="php"]').textContent,
columns: getComputedStyle(grid).gridTemplateColumns.split(' ').length,
chartWidth: rect(chart).width, gridWidth: rect(grid).width,
cardWidth: rect(cards[0]).width, listWidth: rect(list).width,
svgWidth: svgBox.width, svgContentWidth: svg.viewBox.baseVal.width * Math.hypot(svgMatrix.a, svgMatrix.b),
preserveAspect: svg.getAttribute('preserveAspectRatio'),
stroke: getComputedStyle(svg.querySelector('polyline')).vectorEffect,
chartLabel: !!chart.querySelector('.vhost-chart-head .small-label'),
rendered: getComputedStyle(metricPanels[0]).display !== 'none' && rect(chart).height > 60,
fitsViewport: document.documentElement.scrollWidth <= innerWidth + 2,
expectedMetricLabels,
};
}''', status(1))
expected_cols = 2 if width <= 640 else 3 if width <= 980 else 6
assert initial['cards'] == 2, (width, initial)
assert initial['sectionTags'] == ['SECTION', 'SECTION'], (width, initial)
assert initial['collapsibles'] == 0, (width, initial)
assert initial['liveFields'] == [7, 7], (width, initial)
assert initial['labels'] == initial['expectedMetricLabels'], (width, initial)
assert initial['tileClasses'] == ['vhost-live-metric'] * 6, (width, initial)
assert initial['values'][0] != '-' and initial['values'][4] == '3 / 16', (width, initial)
assert initial['phpMissing'] != '-', (width, initial)
assert initial['columns'] == expected_cols, (width, initial)
assert initial['chartWidth'] > 0.95 * initial['gridWidth'], (width, initial)
assert initial['listWidth'] - initial['cardWidth'] < 3, (width, initial)
assert abs(initial['svgWidth'] - initial['svgContentWidth']) <= 2, (width, initial)
assert initial['preserveAspect'] == 'none' and initial['stroke'] == 'non-scaling-stroke', (width, initial)
assert initial['chartLabel'] and initial['rendered'] and initial['fitsViewport'], (width, initial)
sampled = page.evaluate('''([first, second, last]) => {
const api = window.rustProxyWebServer;
const panel = document.querySelector('[data-web-live-id="site-A"]');
const rate = panel.querySelector('[data-web-live-field="rate"]');
window.__clock = 4000; api.renderLive(first, true);
window.__clock = 7000; api.renderLive(second, true);
const before = {rate: rate.textContent, points: panel.querySelector('polyline').getAttribute('points')};
// Service-list redraw must show the current status without opening anything.
latestStatus = last;
api.render();
const updated = document.querySelector('[data-web-live-id="site-A"]');
const total = updated.querySelector('[data-web-live-field="requests"]').textContent;
const split = updated.querySelector('[data-web-live-field="split"]').textContent;
return {before, total, split,
pointsAfterRedraw: updated.querySelector('polyline').getAttribute('points'),
metricsRemainVisible: !!updated.querySelector('.vhost-live-metric'),
overflow: document.documentElement.scrollWidth - innerWidth};
}''', [status(2), status(3), status(4)])
assert sampled['before']['rate'] not in ('-', '0/s', '0.0/s'), (width, sampled)
assert sampled['before']['points'].count(' ') >= 1, (width, sampled)
assert sampled['total'] != initial['values'][0], (width, sampled)
assert sampled['split'] != '-' and sampled['metricsRemainVisible'], (width, sampled)
assert sampled['pointsAfterRedraw'] == sampled['before']['points'], (width, sampled)
assert sampled['overflow'] <= 2, (width, sampled)
assert not errors, (width, errors)
print(f'PASS 0.33.4 WWW live metrics visual parity at {width}px; {expected_cols} columns')
page.close()
browser.close()
print('PASS always-visible full-width WWW/Vhost parity with simulated updates (not Rust runtime)')