v0.34.0_6

docs/PROXY_1KB_OPT4_DEV_0.34.0.md

@@ -0,0 +1,97 @@
+# RustProxy 0.34.0 DEV / OPT4: HTTP/1.1 small-body follow-up
+
+## Objective and validity
+
+Primary goal: 1,040-byte reverse-proxy response, HTTP/1.1 keep-alive,
+cache disabled; target about 100k RPS and at least 80% of nginx on the **same
+isolated host**. Secondary goal: local static Web Server, benchmarked separately.
+No version bump or feature, metric, logging, ACL, authentication, TLS,
+HTTP/2/3, stream, retry or cache-policy bypass is introduced.
+
+The **provided and compiled OPT3 binary** (SHA-256
+`2efe273340f9cf92c5900c85c0bd1b775c87a9d1eaff1a10ec7635d268b080b1`)
+passed 151 black-box HTTP assertions: 0 failures and 1 optional Zstandard
+case skipped. This is evidence for the **input** binary, not for the new
+OPT4 sources. OPT4 still requires a native Rust build, cargo tests and a
+post-build benchmark. The diagnostic same-host CPU shares resources and is
+not an acceptance benchmark.
+
+## Code changes relative to OPT3
+
+1. `src/proxy/forward/route_cache.rs`: store normal public path/query as
+   `Cow<'_, str>` referencing the independently retained, immutable original
+   URI. Without a path rewrite both public/origin suffixes stay borrowed;
+   strict encoded/rewrite handling remains an owned validated string.
+   Cache refresh tasks explicitly call `.to_string()` only when they actually
+   must retain an owned target across spawned tasks.
+2. `src/proxy/upstream_helpers.rs`: skip allocating a request-body metering
+   adapter **only** when `Body::is_end_stream()` establishes that it contains
+   no DATA frames or trailers. Every nonempty/streaming request retains the
+   original metering, timeout, backend lease accounting and cancellation.
+   The response adapter still retains the exchange/inflight permit and handles
+   completion/response bytes for empty requests.
+3. `src/proxy/types.rs`, `src/proxy/forward/upstream.rs`: consume owned
+   `http::Uri` when preparing the upstream target. HTTP/1.1 returns it directly;
+   H2 takes the same URI into parts for strict authority rewriting. Remove an
+   unnecessary intermediate URI clone; replay still builds independent URIs.
+4. `src/forwarding.rs`: reuse the formatted IPv4 peer address when creating
+   RFC 7239 `Forwarded` as well as X-Real-IP/X-Forwarded-For. Preserve quoted
+   IPv6 bracket and proxy-chain logic for trusted and untrusted peers.
+5. `src/proxy/upstream_helpers.rs`: fast-path byte-safe truncation of short
+   or ASCII host/path log values, keeping the original max-*characters*
+   semantics for non-ASCII UTF-8.
+6. `src/proxy/headers.rs`: avoid nominated-field token parsing for absent,
+   lone `keep-alive`, or lone `close` Connection values. A `close` field on
+   the wire still forces the general parser, so a nominated `Close` header
+   is removed exactly as before. Multi-value, comma lists and malformed tokens
+   retain the previous defensive parser.
+7. `src/proxy/headers.rs`: similarly avoid the generic token parser when
+   checking HTTP upgrade on plain keep-alive, close, or absent Connection.
+   Complex multi-valued requests and explicit Upgrade remain unchanged.
+
+## Tests and limits
+
+- New Rust cases validate empty vs nonempty request-body permits and bytes,
+  H1 URI/authority/query, ASCII/Unicode truncation, and Connection variants
+  including a literal nominated `Close` field. Existing regression coverage
+  still includes sticky backends, cache, ACL and trusted forwarded identity.
+- Source, SQLite migration and upgrade tests are necessary but **not** Rust
+  compilation or performance evidence. The compile check must run before use.
+- Run `./dev.sh check && CARGO_INCREMENTAL=0 ./dev.sh test`, followed by
+  the native black-box acceptance suite (proxy, policy, cache, transform,
+  compression, static serving), plus TLS/H2/H3 and websocket tests.
+
+## Measurements
+
+An initial diagnostic same-host `ab` sample at concurrency 16 gave
+OPT2 33,042.55 RPS and compiled OPT3 36,962.11 RPS, without HTTP errors;
+this single sample is not proof of a reliable uplift. A separate complete
+12-sample run (two 3-second repetitions each at 16/64 clients) produced:
+
+| Concurrency | OPT2 median RPS | OPT3 median RPS | nginx median RPS | OPT3/nginx |
+| --- | ---: | ---: | ---: | ---: |
+| 16 | 21,608 | 29,906 | 92,586 | 32.3% |
+| 64 | 38,060 | 34,781 | 123,809 | 28.1% |
+
+All 12 samples had valid 200 responses and correct 1040-byte body checks.
+OPT3 was slower than OPT2 at concurrency 64; the variability is too large to
+claim a sustained uplift or regression based on two short same-host samples.
+Raw evidence includes run order, response checksum and nginx comparator.
+No result here measures OPT4, which was not built in this environment.
+
+## Next evaluation commands
+
+```bash
+./dev.sh check
+CARGO_INCREMENTAL=0 ./dev.sh test
+bash dev.sh
+python3 scripts/performance/regression.py --binary opt4=/absolute/path/target/debug/rust-proxy --out /tmp/opt4-http
+python3 scripts/performance/ab_1kb_dev.py --binary opt3=/absolute/path/opt3/rust-proxy --binary opt4=/absolute/path/target/debug/rust-proxy --nginx /usr/sbin/nginx --ab /usr/bin/ab --out /tmp/opt4-ab --duration 5 --repeats 3 --concurrency 16,64 --workers 4 --max-idle 64
+```
+
+For authoritative acceptance: pin client/server/origin to nonoverlapping CPU
+sets, align build profile and all flags, compare 30s x7 runs, record median,
+p95/p99, correct 1040 B response checksum and status, CPU, RSS, FDs, cache
+stats, analytic/sink queue drops, errors, and both HTTP/1.1 and optional
+HTTP/2 endpoints. Reject changes on new 4xx/5xx, body mismatches, regressions
+of access controls or material memory/fd increases.