60 lines
6.6 KiB
Markdown
60 lines
6.6 KiB
Markdown
# RustProxy 0.24.5 specification audit
|
|
|
|
## Scope
|
|
|
|
0.24.5 fixes HTTP/3 request-body framing when an H3 request has no declared `Content-Length`, carries no DATA, and is forwarded to an HTTP/1.1 upstream. Before the fix, the H3 receive body remains length-unknown until its stream end is polled; Hyper can therefore serialize body-capable HTTP/1.1 methods such as `DELETE` with `Transfer-Encoding: chunked`. Some upstream servers reject chunked request bodies even when the logical body is empty.
|
|
|
|
## Source/spec matrix
|
|
|
|
| Requirement | Source contract | Status |
|
|
|---|---|---|
|
|
| Preserve HTTP method and URI | normalization changes only the request body; method/URI rewriting remains in the existing proxy path | implemented in source |
|
|
| Detect an actually empty H3 body | for H3 -> H1 with no declared/exact length, `normalize_h3_h1_unknown_body()` polls until EOF or the first semantically significant frame | implemented in source |
|
|
| Represent empty H3 body as zero length | EOF replaces the streaming wrapper with `Body::empty()`, giving Hyper an exact zero-length size hint | implemented in source |
|
|
| Preserve non-empty bodies | the first non-empty DATA/trailer frame is wrapped in `ReplayBody`; the remaining tail stays streaming | implemented in source |
|
|
| Do not change H2 upstream framing | normalization exits before polling unless the selected upstream version is HTTP/1.1 | implemented in source |
|
|
| Do not preconsume declared bodies | requests carrying `Content-Length` are left unchanged | implemented in source |
|
|
| Avoid DELETE-only/Proxmox-only behavior | normalization applies generically to body-capable H3 methods forwarded to H1; GET/HEAD/CONNECT retain Hyper's existing implicit-empty behavior | implemented in source |
|
|
|
|
## Regression coverage added
|
|
|
|
`src/proxy/mod.rs` contains focused async tests for:
|
|
|
|
- empty length-unknown H3 `POST`, `PUT`, `PATCH` and `DELETE` to HTTP/1.1 becoming exact zero-length bodies;
|
|
- non-empty length-unknown H3 `DELETE` preserving its payload through `ReplayBody`;
|
|
- H3 request bodies remaining untouched for HTTP/2 upstreams;
|
|
- H1/H2 downstream request bodies remaining untouched by the H3-specific normalization;
|
|
- declared-length H3 `DELETE` bodies not being preconsumed.
|
|
|
|
## Runtime acceptance
|
|
|
|
Compiler/type/runtime acceptance remains open in this source workspace because the Rust toolchain is unavailable. On the test host, run `./dev.sh check` and `./dev.sh test`, then verify the full H1/H2/H3 method matrix. For the original Proxmox case, compare `/var/log/pveproxy/access.log` and a raw/mock HTTP/1.1 upstream. The H3 bodyless DELETE must arrive as `DELETE` with the same URI and without chunked request framing; a zero-length representation such as `Content-Length: 0` is expected for HTTP/1.1.
|
|
|
|
## Cache RAM hardening included in the same 0.24.5 source release
|
|
|
|
The cache keeps the existing HOT RAM -> COLD disk lifecycle. The changes are admission/accounting safeguards rather than a COLD format redesign:
|
|
|
|
- `cache.max_bytes` is a hard published-HOT charge based on the body backing allocation retained by `Bytes`, directly owned entry metadata and a conservative charge for HOT hash/LRU/variant indexes. `hot_logical_bytes` remains the visible payload total; `hot_allocated_bytes`/`resident_bytes` are the charged allocation view, not a duplicate RSS measurement.
|
|
- response fill uses a `Vec<u8>` whose actual `capacity()` is reserved before growth and transferred zero-copy into `Bytes`; `max_object_bytes` is checked before every chunk append;
|
|
- active fills reserve from `transient_max_bytes` and compare HOT + active fill against HOT capacity plus a small allocation-rounding headroom (about 1/64 HOT, capped at 1 MiB and further bounded for small caches); LRU entries are removed before further fill allocation where possible;
|
|
- one admission can remove at most `hot_eviction_max_objects`; if sufficient space cannot be made safely, the response continues uncached;
|
|
- ordinary HOT -> COLD demotion removes the entry from HOT and moves the same `Entry` into `ColdDraft`. The payload is not cloned. A transient lease follows the draft until persistence completes; the persisted `ColdEntry` has paths/metadata and no `Bytes` body;
|
|
- the COLD writer queue remains count-bounded and is additionally capped by `cold_queue_max_bytes`; COLD reads/promotions and revalidation reads reserve transient bytes before reading full bodies;
|
|
- global `cache.enabled=false` cannot be overridden by a vhost `enabled=true`;
|
|
- HOT/object settings no longer have the old artificial 1 GiB UI/source ceiling. Safety comes from the configured byte budgets rather than an arbitrary maximum.
|
|
|
|
Default memory values are 256 MiB HOT, 16 MiB maximum object, 64 MiB transient and 32 MiB COLD queue, with 4096 HOT entries and a 64-object LRU admission bound. The administration panel can calculate conservative/recommended/performance settings from the lower of detected host RAM and a finite cgroup memory maximum. For example, the recommended profile on an effective 2 GiB VM selects approximately 400 MiB HOT, 96 MiB transient, 16 MiB maximum object, 48 MiB COLD queue, 6400 entries and a 64-object LRU admission limit. Presets are disabled below 256 MiB effective RAM. The panel also checks the configured budgets against current host/cgroup memory headroom and warns without blocking manual configuration. COLD disk capacity is intentionally not changed by RAM presets.
|
|
|
|
Monitoring exposes HOT objects, HOT logical bytes, HOT allocated bytes, HOT limit, transient current/limit, COLD queue current/limit, COLD read/write activity, evicted/expired/rejected counts and existing process RSS/RssAnon/VmData/VmSize diagnostics. Cache RAM gauges and process RSS are overlapping views and must not be summed.
|
|
|
|
### Cache runtime acceptance still required
|
|
|
|
The source-level/offline gates verify configuration, UI/API wiring and static contracts. Because this workspace has no Rust toolchain, compile/type/runtime acceptance must still be performed on the test host. In addition to `./dev.sh check` and `./dev.sh test`, exercise at least:
|
|
|
|
- HOT 8 MiB under sustained traffic and verify HOT allocated stays bounded and RSS plateaus;
|
|
- HOT 2 GiB with a deliberately high preset and verify admission remains bounded;
|
|
- `cold_after_seconds=5`, verify HOT allocated falls after demotion and transient/COLD queue bytes return to zero after writes drain;
|
|
- an object larger than `max_object_bytes`, verify the client receives the full streamed response while the cache fill aborts before full buffering;
|
|
- concurrent COLD hits/writes under a small transient limit, verify memory-pressure rejection/pass-through rather than unbounded RAM growth;
|
|
- all four global/vhost enabled combinations, especially global OFF + vhost ON => effective OFF.
|