This commit is contained in:
Mateusz Gruszczyński committed 2026-11-06 23:16:36 +01:00
1 parent 09414dfb09
commit f4606cce18
36 files changed
+641 -74

No files matched your search

+11
View File
@@ -14,6 +14,17 @@ page has its own stylesheet.
The Overview contains live counters, charts and optional backend, cache, PHP, Docker and Stream cards. Under **Settings → Appearance → Dashboard layout** or the Overview editor, configure visibility, order, card width, density and the number of history samples. A hidden card stops consuming its optional browser-side visualization work but does not turn off its underlying proxy feature.
Quick-health tiles shown by default since 0.35.2:
| Tile | Shows |
|---|---|
| **Healthy backends** | Monitored backend members passing health checks as `up / total`, with the names of failing members. |
| **Error share** | Percentage of HTTP requests that ended with an error during the last sampling interval. |
| **Rejected by limits / s** | Requests and connections refused by `--max-connections` / `--max-inflight`; sustained values mean the limits are too low for the load. |
| **Nearest certificate expiry** | Days until the earliest installed certificate expires; the bar is full at 90 days or more. |
Existing layouts receive these tiles at the end of the Overview; hide or move them in the layout editor.
The unlocked Overview layout exposes a single **Save and lock** action (plus Cancel), removing the redundant lock-and-save button.
**Dashboard Presets** supports up to **16 named layouts per administrator**. Save, load, rename, overwrite and delete presets; a factory layout can be restored. Presets preserve the dashboard arrangement, not the live traffic values. Interface theme, scale and other display settings are also account-scoped and do not alter proxy routing or rewrite shared configuration.
+3 -2
View File
@@ -11,7 +11,7 @@ RustProxy is a single service that combines a reverse proxy, a static/PHP web se
| Vhosts | Exact and wildcard hostnames, aliases, several listeners per Vhost, per-Vhost HTTP/2 and HTTP/3 switches |
| Backends | HTTP and HTTPS origins, HTTP/1.1, automatic ALPN (H2/H1), strict HTTP/2 and H2C, optional upstream certificate verification with custom CA and server name |
| Load balancing | Least connections (default), round robin, least traffic, adaptive and IP hash; member weights, backup and draining members, sticky cookie or IP affinity |
| Health checks | Active TCP or HTTP probes with thresholds and live health status per member |
| Health checks | Active TCP or HTTP probes with thresholds, optional separate probe port, expected response text and live health status per member |
| Connection reuse | Bounded keep-alive pools with idle and age limits, pool recycling and protocol telemetry |
| Custom locations | Prefix, exact and regex paths, path rewriting, per-location backends, Web Services, local responses, redirects and internal error fallbacks |
| WebSockets | HTTP/1.1 Upgrade, RFC 8441 (HTTP/2) and RFC 9220 (HTTP/3) Extended CONNECT |
@@ -45,6 +45,7 @@ RustProxy is a single service that combines a reverse proxy, a static/PHP web se
- Managed ACME with HTTP-01 and DNS-01 (Cloudflare, OVH, RFC 2136), automatic renewal and job history.
- Certificate inventory with expiry, coverage check and bulk assignment to Vhosts.
- Restricted certificate API keys for deploy/reload from external systems such as OPNsense.
- Prometheus metrics for the expiry date and remaining days of every installed certificate, ready for alerting.
![Certificates](images/certificates.png)
@@ -74,7 +75,7 @@ RustProxy is a single service that combines a reverse proxy, a static/PHP web se
## Monitoring and logs
- Overview dashboard with requests per second, bandwidth, status codes, cache hit ratio, protocol mix, system load and backend health.
- Overview dashboard with requests per second, bandwidth, status codes, error share, cache hit ratio, protocol mix, system load, backend health, requests rejected by limits and nearest certificate expiry.
- Customizable dashboard layouts and saved presets.
- Per-Vhost, per-backend, per-Web-Service and per-Stream statistics.
- Live request log in the browser and in the terminal (`--livelogs`).
+8
View File
@@ -44,6 +44,14 @@ scrape_configs:
- targets: ['127.0.0.1:9191']
```
Certificate expiry is exported for every installed certificate, including manually uploaded ones: `rust_proxy_certificate_not_after_timestamp_seconds{certificate="…"}` and `rust_proxy_certificate_days_remaining{certificate="…"}` (negative after expiry). Values are read from the certificate files and refreshed at most every five minutes. Example alert:
```yaml
- alert: RustProxyCertificateExpiresSoon
expr: rust_proxy_certificate_days_remaining < 14
for: 1h
```
When using a bearer token, configure Prometheus `authorization` or `bearer_token_file` securely rather than embedding a clear token in an exported dashboard. Review `grafana/README.md` and `grafana/rustproxy-overview.json` for chart examples; `nagios/README.md` contains external check scripts.
## JSON metrics API for external dashboards
+36 -8
View File
@@ -20,7 +20,7 @@ Results are only comparable when both servers do the same work. A stock Nginx in
- Machine: 2 vCPU virtual machine, Linux 6.18, loopback networking.
- Each server is pinned to one CPU core (`worker_processes 1` for Nginx, `taskset` for RustProxy). The load generator and the origin share the second core.
- Load: `wrk -t1 -c50 -d8s`, HTTP/1.1 keep-alive, two runs per scenario, average reported.
- Load: `wrk -t1 -c50 -d8s`, HTTP/1.1 keep-alive, two or three runs per scenario, average reported.
- Origin: Nginx serving static files with `Cache-Control: public, max-age=600`.
- Nginx 1.24 and RustProxy release builds (`lto = "thin"`, `codegen-units = 1`).
@@ -28,23 +28,50 @@ Results are only comparable when both servers do the same work. A stock Nginx in
Requests per second on one CPU core (higher is better):
| Scenario | Nginx | RustProxy 0.35.0 | RustProxy 0.35.1 | 0.35.1 vs 0.35.0 |
| Scenario | Nginx | RustProxy 0.35.0 | RustProxy 0.35.2 | 0.35.2 vs 0.35.0 |
|---|---:|---:|---:|---:|
| Reverse proxy, 1 KiB response | 54,700 | 12,700 | 14,200 | +12% |
| Reverse proxy, 100 KiB response | 9,600 | 11,400 | 13,600 | +19% |
| Cache hit, 1 KiB response | 51,000 | 19,500 | 23,600 | +21% |
| Web Server, 1 KiB file | 78,500 | 13,400 | 23,900 | +78% |
| Web Server, 100 KiB file | 22,500 | 8,000 | 14,900 | +86% |
| Reverse proxy, 1 KiB response | 61,400 | 14,700 | 15,100 | +3% |
| Reverse proxy, 100 KiB response | 10,600 | 14,500 | 15,500 | +7% |
| Cache hit, 1 KiB response | 53,300 | 22,500 | 25,000 | +11% |
| Web Server, 1 KiB file | 85,000 | 15,200 | 26,000 | +71% |
| Web Server, 100 KiB file | 25,900 | 9,200 | 16,900 | +84% |
Reading the results:
- **Large proxied responses:** RustProxy streams the body directly between connections and is faster than Nginx in its default buffering mode, which spools larger upstream responses to temporary files.
- **Small requests:** Nginx remains ahead, because its fixed per-request cost is lower. In RustProxy this cost is dominated by per-request accounting (live statistics, traffic analytics, request IDs and logs) and by the asynchronous runtime. For most deployments it is far below the capacity of the backends behind the proxy.
- **Static files:** 0.35.1 reads files directly on the request task instead of a background thread pool, which nearly doubles static throughput.
- **Static files:** since 0.35.1 RustProxy reads files directly on the request task instead of a background thread pool, which nearly doubles static throughput.
- **Why small requests and cache hits trail Nginx:** each proxied request crosses the loopback network twice (client and backend), and on a VM about half of the CPU time goes to the kernel for those writes and the wake-ups of the other side. Most of the remaining time is RustProxy's per-request work that Nginx does not do in the same way: request IDs, live per-Vhost statistics, traffic analytics and structured access-log entries, plus the asynchronous runtime. A cache hit saves the backend hop but still pays for this accounting, so its gap is similar. No single function dominates the profile; closing the gap further would require architectural changes rather than tuning.
- **CPU per request:** at the 1 KiB proxy scenario RustProxy uses about 70 µs of CPU time per request on the test machine, Nginx about 17 µs. A single core therefore sustains well over 10,000 proxied requests per second.
Throughput scales with additional CPU cores; RustProxy uses all available cores automatically.
## Memory allocator
Release builds use the operating system's allocator (glibc on most Linux distributions). An alternative allocator, [mimalloc](https://github.com/microsoft/mimalloc), can be compiled in as an optional feature:
```bash
cargo build --release --features mimalloc
```
It is off by default because of how it behaves with a large HOT cache. Measured on the test machine with a 300 MiB HOT cache under constant turnover (random objects of 1–260 KiB, continuous evictions):
| Build | Process memory (RSS) | Throughput under cache turnover |
|---|---:|---:|
| System allocator (default) | 614 MiB | 5,700 req/s |
| mimalloc | 760 MiB (+24%) | 20,300 req/s (3.5×) |
| mimalloc with `MIMALLOC_PURGE_DELAY=0` | 720 MiB (+17%) | 18,600 req/s |
In the standard scenarios from the results table (proxy, cache hits, static files) the two allocators performed within measurement noise. On a production server with a 600 MiB HOT cache the same difference showed as about 800 MiB versus 1.1–1.2 GiB of process memory.
Choose mimalloc only when all of these apply:
- the HOT cache holds many objects that are replaced constantly (high eviction rate on the **Cache** page);
- throughput under that turnover matters more than memory;
- the server has at least 30% RAM headroom above the configured HOT cache size.
Otherwise keep the default. The system allocator returns memory from evicted cache entries to the operating system right away, so process memory stays close to the configured cache size.
## Production tuning
| Setting | Recommendation |
@@ -56,6 +83,7 @@ Throughput scales with additional CPU cores; RustProxy uses all available cores
| Logging | Keep stored log files on fast local storage and use retention limits on busy servers. |
| Limits | `--max-connections` (default 512) and `--max-inflight` (default 256) cap concurrency. Raise them on busy servers with enough memory. |
| `sendfile` | Optional for large cleartext static files on Linux. On the test machine it gave no measurable gain, so keep `off` unless your own measurements show otherwise. |
| Memory allocator | Keep the default system allocator. See [Memory allocator](#memory-allocator) for when the optional `mimalloc` build is worth its extra memory. |
| TLS | TLS 1.3 with session resumption keeps handshake cost low; HTTP/2 or HTTP/3 reduce connection setup for browsers. |
## Reproducing the test
+2
View File
@@ -65,6 +65,8 @@ Connection reuse is bounded by configured idle-pool/recycle limits. The default
Backend pools support scheduling/weighting, standby members, draining and optional session affinity. Check the **Backends** status view for member health, recently observed HTTP response code, upstream protocol and resource usage. A reported status from normal traffic is not necessarily the result of a health probe.
- Health monitoring may use TCP or HTTP checks with a path, Host, interval and thresholds. Avoid write endpoints; use a lightweight `/health` or equivalent.
- **Health check port** (`health_check.port`) probes a different port than the one receiving traffic, like `check port` in HAProxy. Example: a backend serving traffic on `http://10.0.0.5:80` with a status endpoint on port `8079` uses `"health_check": {"mode": "http", "path": "/status", "port": 8079}`. The probe keeps the backend host, scheme, TLS settings and address restrictions; only the port changes. Leave it empty to probe the traffic port. The port can be set for the whole pool or per backend.
- **Expected response text** (`health_check.expect_body`) makes an HTTP probe fail unless the first 64 KiB of the response body contain the given text, like `http-check expect string` in HAProxy. Use it for status pages that answer `200` even when the application is degraded, for example `"expect_body": "\"status\":\"ok\""`. Failed matches appear as `http_body` in backend health details.
- Each member can override the pool health check or disable active/passive monitoring with `health_check_enabled=false`. Disabling checks treats the member as available, not proven healthy.
- New monitored pools can temporarily have no eligible healthy member until rise checks succeed. A failed/incorrect check can produce `503` even if the process is reachable.
- `backup` members act as fallbacks; `drain` prevents new normal selection while letting existing work complete within normal lifetimes. Explicitly pinned backends need careful policy review.