v0.35.2

docs/PERFORMANCE.md

@@ -20,7 +20,7 @@ Results are only comparable when both servers do the same work. A stock Nginx in
 
 - Machine: 2 vCPU virtual machine, Linux 6.18, loopback networking.
 - Each server is pinned to one CPU core (`worker_processes 1` for Nginx, `taskset` for RustProxy). The load generator and the origin share the second core.
-- Load: `wrk -t1 -c50 -d8s`, HTTP/1.1 keep-alive, two runs per scenario, average reported.
+- Load: `wrk -t1 -c50 -d8s`, HTTP/1.1 keep-alive, two or three runs per scenario, average reported.
 - Origin: Nginx serving static files with `Cache-Control: public, max-age=600`.
 - Nginx 1.24 and RustProxy release builds (`lto = "thin"`, `codegen-units = 1`).
 
@@ -28,23 +28,50 @@ Results are only comparable when both servers do the same work. A stock Nginx in
 
 Requests per second on one CPU core (higher is better):
 
-| Scenario | Nginx | RustProxy 0.35.0 | RustProxy 0.35.1 | 0.35.1 vs 0.35.0 |
+| Scenario | Nginx | RustProxy 0.35.0 | RustProxy 0.35.2 | 0.35.2 vs 0.35.0 |
 |---|---:|---:|---:|---:|
-| Reverse proxy, 1 KiB response | 54,700 | 12,700 | 14,200 | +12% |
-| Reverse proxy, 100 KiB response | 9,600 | 11,400 | 13,600 | +19% |
-| Cache hit, 1 KiB response | 51,000 | 19,500 | 23,600 | +21% |
-| Web Server, 1 KiB file | 78,500 | 13,400 | 23,900 | +78% |
-| Web Server, 100 KiB file | 22,500 | 8,000 | 14,900 | +86% |
+| Reverse proxy, 1 KiB response | 61,400 | 14,700 | 15,100 | +3% |
+| Reverse proxy, 100 KiB response | 10,600 | 14,500 | 15,500 | +7% |
+| Cache hit, 1 KiB response | 53,300 | 22,500 | 25,000 | +11% |
+| Web Server, 1 KiB file | 85,000 | 15,200 | 26,000 | +71% |
+| Web Server, 100 KiB file | 25,900 | 9,200 | 16,900 | +84% |
 
 Reading the results:
 
 - **Large proxied responses:** RustProxy streams the body directly between connections and is faster than Nginx in its default buffering mode, which spools larger upstream responses to temporary files.
 - **Small requests:** Nginx remains ahead, because its fixed per-request cost is lower. In RustProxy this cost is dominated by per-request accounting (live statistics, traffic analytics, request IDs and logs) and by the asynchronous runtime. For most deployments it is far below the capacity of the backends behind the proxy.
-- **Static files:** 0.35.1 reads files directly on the request task instead of a background thread pool, which nearly doubles static throughput.
+- **Static files:** since 0.35.1 RustProxy reads files directly on the request task instead of a background thread pool, which nearly doubles static throughput.
+- **Why small requests and cache hits trail Nginx:** each proxied request crosses the loopback network twice (client and backend), and on a VM about half of the CPU time goes to the kernel for those writes and the wake-ups of the other side. Most of the remaining time is RustProxy's per-request work that Nginx does not do in the same way: request IDs, live per-Vhost statistics, traffic analytics and structured access-log entries, plus the asynchronous runtime. A cache hit saves the backend hop but still pays for this accounting, so its gap is similar. No single function dominates the profile; closing the gap further would require architectural changes rather than tuning.
 - **CPU per request:** at the 1 KiB proxy scenario RustProxy uses about 70 µs of CPU time per request on the test machine, Nginx about 17 µs. A single core therefore sustains well over 10,000 proxied requests per second.
 
 Throughput scales with additional CPU cores; RustProxy uses all available cores automatically.
 
+## Memory allocator
+
+Release builds use the operating system's allocator (glibc on most Linux distributions). An alternative allocator, [mimalloc](https://github.com/microsoft/mimalloc), can be compiled in as an optional feature:
+
+```bash
+cargo build --release --features mimalloc
+```
+
+It is off by default because of how it behaves with a large HOT cache. Measured on the test machine with a 300 MiB HOT cache under constant turnover (random objects of 1–260 KiB, continuous evictions):
+
+| Build | Process memory (RSS) | Throughput under cache turnover |
+|---|---:|---:|
+| System allocator (default) | 614 MiB | 5,700 req/s |
+| mimalloc | 760 MiB (+24%) | 20,300 req/s (3.5×) |
+| mimalloc with `MIMALLOC_PURGE_DELAY=0` | 720 MiB (+17%) | 18,600 req/s |
+
+In the standard scenarios from the results table (proxy, cache hits, static files) the two allocators performed within measurement noise. On a production server with a 600 MiB HOT cache the same difference showed as about 800 MiB versus 1.1–1.2 GiB of process memory.
+
+Choose mimalloc only when all of these apply:
+
+- the HOT cache holds many objects that are replaced constantly (high eviction rate on the **Cache** page);
+- throughput under that turnover matters more than memory;
+- the server has at least 30% RAM headroom above the configured HOT cache size.
+
+Otherwise keep the default. The system allocator returns memory from evicted cache entries to the operating system right away, so process memory stays close to the configured cache size.
+
 ## Production tuning
 
 | Setting | Recommendation |
@@ -56,6 +83,7 @@ Throughput scales with additional CPU cores; RustProxy uses all available cores
 | Logging | Keep stored log files on fast local storage and use retention limits on busy servers. |
 | Limits | `--max-connections` (default 512) and `--max-inflight` (default 256) cap concurrency. Raise them on busy servers with enough memory. |
 | `sendfile` | Optional for large cleartext static files on Linux. On the test machine it gave no measurable gain, so keep `off` unless your own measurements show otherwise. |
+| Memory allocator | Keep the default system allocator. See [Memory allocator](#memory-allocator) for when the optional `mimalloc` build is worth its extra memory. |
 | TLS | TLS 1.3 with session resumption keeps handshake cost low; HTTP/2 or HTTP/3 reduce connection setup for browsers. |
 
 ## Reproducing the test