RustProxy/docs/CACHE.md

# Shared hot/cold cache

See [CACHE_DIAGNOSTICS.md](CACHE_DIAGNOSTICS.md) for practical HIT/MISS/BYPASS diagnosis.

## Scope

RustProxy implements a bounded reverse-proxy cache with a hot RAM tier and an optional
persistent cold disk tier. Version 0.11.5 aligned request-method handling and most logical
hash behavior with Varnish built-in semantics. Since 0.19.1 the logical primary identity
includes the frontend scheme as well as Host + exact request URL, so `http://host/path`
and `https://host/path` cannot reuse the same stored response. RustProxy still isolates
different vhost/config scopes internally and uses response `Vary` to select representation
variants.

This is not a VCL implementation. Generic Varnish grace, ESI, public unauthenticated
BAN/PURGE and distributed cache state are not implemented. RustProxy 0.22.1 adds bounded
foreground request collapsing, authenticated selective PURGE/BAN and optional proactive
refresh while keeping directive-controlled stale delivery through RFC 5861 and request
`max-stale`; it is not an unconditional availability/grace layer. RustProxy also keeps its
validator-based handling of response `no-cache` instead of reproducing Varnish hit-for-miss
behavior exactly.

## Settings

| Field | Default | Bounds / meaning |
|---|---|---|
| enabled | true | Global master switch; vhosts may disable cache but cannot re-enable a globally disabled cache |
| max_bytes | 268435456 | Hard HOT allocation budget (body backing allocation + charged entry metadata); minimum 64 KiB, no artificial high ceiling |
| max_object_bytes | 16777216 | Maximum logical body accepted by one fill; minimum 1 KiB and at most `max_bytes` |
| max_entries | 4096 | 1..16384 hot objects |
| transient_max_bytes | 67108864 | Hard RAM budget for fill buffers and complete bodies moving between HOT/COLD |
| cold_queue_max_bytes | 33554432 | Byte ceiling for complete demoted objects waiting for the COLD writer; at most `transient_max_bytes` |
| hot_eviction_max_objects | 64 | 1..4096 LRU removals attempted for one HOT admission before the new object passes uncached |
| hot_auto_trim_enabled | true | Periodically trim sufficiently idle HOT objects only while HOT usage is above the target waterline |
| hot_auto_trim_after_seconds | 300 | 5..86400; minimum idle age before an object can be removed by automatic HOT trim |
| hot_auto_trim_target_percent | 80 | 10..95; automatic trim stops once HOT usage reaches this percentage of `max_bytes` |
| cold_enabled | false | Enable persistent disk tier; never implied by hot-cache enable |
| cold_path | cache/cold | Relative path under the private RustProxy data directory |
| cold_max_bytes | 4294967296 | 64 KiB..1 TiB disk budget |
| cold_max_entries | 50000 | 1..1000000 cold objects |
| cold_gc_interval_seconds | 60 | 5..86400; interval between asynchronous pressure-GC checks |
| cold_gc_max_delete | 256 | 1..10000; maximum objects removed by one automatic GC pass |
| cold_after_seconds | 300 | 5..604800; idle hot objects are eligible after this delay |
| cold_on_idle | true | Move idle eligible hot objects to cold storage |
| cold_max_hot_seconds | 3600 | 5..2592000; total RAM-residency age used when `cold_on_age=true` |
| cold_on_age | false | Demote objects after the maximum hot-tier age even if they remain frequently accessed |
| cold_on_lru | true | Move objects displaced by hot-tier LRU pressure to cold storage |
| cold_on_shutdown | true | Best-effort direct hot-to-cold flush during ordinary graceful shutdown/restart |
| default_ttl_seconds | 0 | Only explicit public responses without origin freshness |
| max_ttl_seconds | 86400 | 1..604800; maximum configured retention/freshness cap |
| bypass_query | false | Bypass requests containing a query when enabled |
| bypass_prefixes | [] | Up to 64 normalized absolute path prefixes |
| static_ignore_cookies | true | For configured static extensions only, ignore request Cookie for eligibility/keying |
| expose_status | false | Emit `X-Cache: HIT`, `MISS` or `BYPASS` |
| expose_hits | false | Emit per-object `X-Cache-Hits` |
| expose_cache_status | false | Emit standardized `Cache-Status` (RFC 9211) |
| expose_resident_age | false | Emit `X-Cache-Resident-Age` on reused stored responses |
| collapse_enabled | true | Collapse concurrent foreground fills for the same primary cache key |
| collapse_wait_timeout_ms | 1500 | 10..60000 ms follower wait before fallback |
| collapse_timeout_fallback | origin | `origin` or `stale_if_available` after follower timeout |
| stale_if_error_timeout | true | Allow RFC 5861 stale fallback for timeout failures |
| stale_if_error_connection | true | Allow stale fallback for connection failures |
| stale_if_error_dns | true | Allow stale fallback for DNS failures |
| stale_if_error_tls | true | Allow stale fallback for TLS failures |
| stale_if_error_statuses | [500,502,503,504] | Up to 32 unique eligible HTTP 5xx statuses |
| proactive_refresh_enabled | false | Schedule sampled cached objects for validator refresh before expiry |
| proactive_refresh_before_expiry_seconds | 15 | 1..604800 seconds before expiry |
| proactive_refresh_percent | 10 | 0..100 deterministic percentage of primary keys |
| proactive_refresh_max_inflight | 4 | 1..64 concurrent proactive refreshes |

The hot tier enforces `max_bytes` and `max_entries` before publication. HOT accounting uses the retained body allocation (the fill buffer capacity transferred into `Bytes`) plus directly owned entry metadata and a conservative per-entry charge for the HOT hash/LRU/variant indexes, while `hot_logical_bytes` reports only visible body lengths. Rust container/allocator APIs do not expose every bucket/node allocation exactly, so `hot_allocated_bytes` is intentionally a conservative charged estimate rather than a promise that it equals process RSS. In-progress fills reserve a separate bounded transient budget before growing their buffers. When HOT pressure would exceed the budget plus a small allocator-rounding headroom (roughly 1/64 of HOT, capped at 1 MiB), RustProxy removes LRU objects first; if admission still cannot be made safe within `hot_eviction_max_objects`, the response continues normally without being cached. `max_object_bytes` is checked incrementally before every body-buffer growth, so an oversized response is not fully accumulated before rejection.

Automatic HOT trim is a separate low-pressure maintenance path. Every maintenance pass first checks whether HOT allocation is above `hot_auto_trim_target_percent` of `max_bytes`; if so, only least-recently-used objects that have been idle for at least `hot_auto_trim_after_seconds` are eligible. One pass is capped by `hot_eviction_max_objects`. Active/recent objects are never removed by this mechanism. Eligible objects follow the existing COLD-on-LRU policy when enabled; otherwise they are evicted. This is intended to release stale working-set memory after traffic bursts without periodically purging the cache or reloading vhosts.

When cold storage is enabled,
`cold_on_lru`, `cold_on_idle` and `cold_on_age` independently choose which hot objects are demoted.
`cold_after_seconds` measures time since the last access. `cold_max_hot_seconds` measures total
hot-tier residency, so it can move a frequently accessed object to disk after a configured age.
The administration UI exposes these as separate policies to avoid conflating "old" with "unused".
Demotions use an asynchronous queue bounded by both object count and `cold_queue_max_bytes`. A demoted `Entry` is removed from HOT accounting and moved, not copied, into a `ColdDraft`; its `Bytes` payload remains charged to `transient_max_bytes` until persistence completes and the draft is dropped. The persisted `ColdEntry` contains paths/metadata rather than the body allocation. This prevents the COLD writer queue from becoming a second unbounded RAM cache. If an HTTP response is already using a clone of the same `Bytes`, that last transport-owned reference can keep the backing allocation alive until the response finishes; the cache-owned HOT/COLD reference itself is released after persistence. Process RSS is therefore monitored separately from cache accounting. Supported
glibc/Linux builds ask the allocator to release unused pages after bulk demotion. During
ordinary graceful shutdown, `cold_on_shutdown` performs a best-effort direct flush of remaining hot
objects before the store is closed. A **managed software update** (`update.sh` / `scripts/deploy.py`)
uses a private one-shot marker and, on RustProxy 0.13.2+, forces this hot-to-cold handoff whenever
the cold tier is enabled even if `cold_on_shutdown=false`. The updater gives the process a larger
stop budget and removes the marker before backing up state. If the cold tier is disabled there is
nowhere persistent to hand the hot objects, so RAM-only cache is still lost on process exit. The
body is stored as a raw `.body` file and metadata as JSON with a SHA-256 body digest. A fresh cold hit is integrity-checked and promoted to
RAM. Cold format 3, introduced in 0.21.0, stores objects below the first two hexadecimal
characters of the SHA-256 cache key, for example `cache/cold/7f/<hash>.body` and
`cache/cold/7f/<hash>.json`. Shard directories are created lazily. The previous flat
Cold Cache layout is deliberately not migrated; legacy flat objects are removed and can
be refilled from origin. Format 3 metadata also carries RFC 5861 windows, revalidation
flags, origin validators, last access and hit count.

Automatic Cold Cache GC is asynchronous. When either byte or entry usage reaches about
90% of its configured limit, it selects old/rarely used candidates and cleans toward
about 80%, removing at most `cold_gc_max_delete` objects in one pass. Metadata is
snapshotted in small batches and disk deletion is queued instead of holding request-path
state across a full-cache sort/delete loop.

## Varnish-style identity and variants

The logical primary cache identity is:

- a stable cache-compatibility revision derived only from settings that can change
  cached representation/origin semantics, plus vhost scope (internal isolation),
- normalized frontend scheme (`http`/`https`),
- normalized original Host authority, including an explicit port,
- exact request path and query.

Selected backend-pool member, route/backend IDs and unrelated request headers are not
part of the representation hash. This means two healthy members of the same pool can
serve/fill the same public cache object, while HTTP and HTTPS remain isolated. Headers
such as User-Agent, X-Forwarded-For or browser client hints no longer create unnecessary
variants by themselves.

The response `Vary` header provides the secondary representation identity. For example,
`Vary: Accept-Language` stores independent language variants under the same scheme + Host + URL
primary hash. `Vary: *`, malformed Vary fields or an excessive Vary field set are not
stored. If an origin changes the Vary schema for a URL, incompatible old variants are
removed so an obsolete no-Vary object cannot shadow a newer Vary representation (or
vice versa).

Cookie and Authorization requests still pass by default, matching Varnish built-in
safety behavior. RustProxy's static-cookie optimization can ignore Cookie only for the
configured static file extensions; response `Set-Cookie`, `private`, `no-store` and
`Vary: Cookie` still prevent unsafe shared reuse.

## Request methods and invalidation

GET and HEAD participate in lookup. They share one representation identity. On a
cacheable HEAD miss RustProxy fetches the backend representation as GET, may populate
the shared object, and strips the response body before delivering the original HEAD
request.

POST, PUT, PATCH, DELETE, OPTIONS and TRACE pass to the backend and are not stored.
They do **not** implicitly purge the whole cache. This fixes the pre-0.11.5 behavior
where any successful unsafe request could erase unrelated static objects and destroy
the hit ratio.

Cache invalidation remains explicit:

- authenticated full purge can clear all objects or one vhost scope,
- selective PURGE supports exact URL, URL prefix, `Cache-Tag`, safe regex and primary cache key, with optional host and vhost restrictions,
- exact/prefix matching uses the stored request target including query; a match removes the whole `Vary` family under that primary cache key,
- selective PURGE operates on RAM and Cold Cache and advances a publication generation so no fill that started before the operation can publish afterward, even if the selector depended on response metadata such as `Cache-Tag`,
- BAN records a bounded TTL rule without synchronously scanning the entire store; objects created before a matching active BAN are rejected during lookup/insertion and are removed later by bounded GC,
- new fills created after a BAN rule are allowed, so BAN invalidates the old representation set rather than permanently blocking a URL,
- regex rules and prefix `/` are treated as broad operations and require explicit confirmation,
- only cache-compatibility changes (cache policy, configured header/origin semantics or vhosts) advance the normal cache generation/revision; unrelated logging/TLS/admin/metrics/response-presentation saves retain objects.

There is no public unauthenticated PURGE/BAN endpoint. If an application changes a GET
resource after POST/PUT/PATCH/DELETE, use origin freshness/validators or the authenticated
administration invalidation path; RustProxy does not guess the affected URLs.

## Eligibility and freshness

Hard request bypasses include Cookie (outside the static exception), Authorization,
Upgrade, `If-Match`, `If-Unmodified-Since`, nonempty request bodies, configured path/query
exclusions and request `Cache-Control: no-store`.

A single `Range: bytes=...` request is special rather than a normal bypass. When a complete
fresh cached representation is available, RustProxy evaluates `If-Range` and can return
`206 Partial Content` directly with `Content-Range`, `Accept-Ranges`, the correct sliced
`Content-Length`, validators and normal cache `Age`; an unsatisfied range returns `416`
with `Content-Range: bytes */<length>`. A mismatching `If-Range` returns the complete cached
representation. If origin access is needed, RustProxy forwards the Range request as an
identity, non-transformed stream. Multiple/multipart, duplicate or otherwise unsupported
Range forms are passed through to origin and are never stored as separate range objects.

A fresh cached representation can satisfy `If-None-Match`/`If-Modified-Since` locally.
Request `no-cache`, `max-age=0` and legacy `Pragma: no-cache` require validation when a
safe stored validator exists. `max-age=N` and `min-fresh=N` are freshness constraints:
they reuse the cached response when its current Age/remaining freshness satisfies the
client limit and otherwise drive normal revalidation/fetch behavior. Request `max-stale`
can explicitly allow a retained stale response: bare `max-stale` accepts any retained
staleness, while `max-stale=N` bounds it. Freshness expiry does not itself physically
delete an object; normal RAM LRU/Cold Cache GC may still evict it. `must-revalidate` and
`proxy-revalidate` override client stale permission. With a safe validator, an origin
304 refreshes metadata and reuses the stored body.

Responses with `Set-Cookie`, `private`, `no-store`, `Content-Range`, trailers,
SSE/gRPC, `Vary: *`, invalid freshness metadata or unusable size/lifetime are not
stored. Response `Cache-Control: no-cache` may be retained only with ETag or
Last-Modified and must validate before every reuse. Both origin and transformed response
policy must remain eligible.

`s-maxage` takes precedence over `max-age`, then Expires/Date. `Age` uses the RFC 9111
current-age model and includes origin age; it is not a measure of how long the object has
been in RustProxy. Cache metadata keeps normal TTL, `stale-while-revalidate`,
`stale-if-error`, `must-revalidate` and `proxy-revalidate` separately. A representation
is therefore **fresh**, **stale-but-servable** only under an applicable directive, or
**expired/unusable**.

Inside `stale-while-revalidate=N`, RustProxy may return stale immediately and starts one
deduplicated background refresh for that cache identity. Inside `stale-if-error=N`, stale
is used only after an eligible origin failure. Version 0.22 makes timeout, connection, DNS
and TLS classes independently configurable and uses an explicit configurable 5xx status
allowlist; defaults preserve the previous behavior (`500`, `502`, `503`, `504` plus all four
transport classes). After TTL plus the corresponding stale window, that mode is unavailable.
`must-revalidate` and `proxy-revalidate` still permit storage, but prohibit all stale delivery
after expiry, including RFC 5861 and request `max-stale`.

Optional proactive refresh schedules a deterministic percentage of cache keys before normal
expiry. It reuses the same validator-based refresh logic and background-revalidation guard,
keeps only one schedule per key, and enforces a global concurrent-refresh bound. A newly
completed fill can schedule this work immediately, so sampled objects do not require another
client request to start the pre-expiry timer. The feature is disabled by default.

## Resident age

`X-Cache-Resident-Age` is the RustProxy-specific whole-second clock for how long the
stored representation has remained in this cache. In 0.11.5 that clock is independent
from the HTTP freshness `Age` clock and survives:

- ordinary hot-cache HITs,
- successful backend 304 revalidation,
- hot-to-cold demotion,
- cold-to-hot promotion.

A value of `0` is valid only during the first sub-second of residency. Revalidation or
tier movement must no longer reset a long-lived object to zero.

## Streaming and capacity

RustProxy tees response frames while delivering them; it does not wait for the complete
origin body before sending the first bytes. Only a complete successful body is
published. Oversize, body error, trailer and cancelled-fill paths discard the candidate.
Large buffers grow incrementally and consume multiple bounded fill permits.

The configured resident byte limit accounts for body bytes, estimated header storage
and fixed per-entry overhead; it is not total process RSS. Active HIT references,
in-flight origin responses, compression buffers and runtime allocations also use memory.

With `collapse_enabled=true`, the first foreground MISS for a primary cache key owns the
fill while concurrent followers wait for publication. The fill guard is held through the
actual response-body tee, so followers cannot wake merely because origin headers arrived.
Leader error/cancellation releases waiters. A follower that wakes to a different `Vary`
representation re-enters the single-flight decision instead of immediately stampeding the
origin. After `collapse_wait_timeout_ms`, the configured fallback is either an independent
origin request or an eligible stale response. SWR/background revalidation keeps its separate
existing deduplication guard.

## Diagnostics and API

`GET /api/cache` reports settings, HIT/MISS/BYPASS, hot/cold counts and bytes,
configured COLD byte/object limits, promotions/demotions, errors, fill pressure, object
views and per-domain reason counters. The overview dashboard shows COLD disk occupancy
separately from RAM accounting. Hit ratio is `hits / (hits + misses)` and intentionally
excludes BYPASS traffic.

`POST /api/cache/purge` requires the existing administrator session, same-origin marker
and CSRF token. `?vhost=<id>` purges only that vhost scope; omitting it purges all cache
objects. Purge is audited.

`POST /api/cache/invalidate` performs authenticated selective `purge` or `ban` using
`exact`, `prefix`, `tag`, `regex` or `cache_key`. Optional `host` and `vhost` restrict scope;
BAN accepts a bounded TTL. Regex and prefix `/` require `confirm_wide=true`. The result
reports the selector/scope, affected entry count, elapsed time, status and BAN id when one
was created. `GET /api/cache/bans` lists active rules and `DELETE /api/cache/bans/{id}`
removes one. These operations use the normal administration audit path. The Cache panel
exposes the same controls plus per-object **Purge URL**.

`POST /api/cache/cold/cleanup` starts a selective asynchronous Cold Cache cleanup using
the same candidate/removal machinery as automatic GC. Criteria can select entries unused
for a duration, cap removal to the oldest N objects, and/or require `hits == 0`.
`GET /api/cache/cold/cleanup/{id}` reports job status, removed object count and freed
bytes. Purge-all and purge-vhost remain the full-clear mechanisms; selective invalidation
is deliberately separate.

`X-Cache`, `X-Cache-Hits`, `Cache-Status`, `X-Cache-Reason` and
`X-Cache-Resident-Age` are optional diagnostics. `Cache-Status` is the standardized
RFC 9211 field name; RustProxy does not rename it to the legacy/custom `X-Cache-Status`.
HTTP/2 and HTTP/3 serialize field names in lowercase, so `cache-status:` on the wire is
normal. When the cached representation was produced by the transform pipeline,
RustProxy appends the extension parameter `transformed`, for example:

```http
Cache-Status: RustProxy; hit; ttl=556488; detail="cache_hit"; transformed
```

Origin lookalike values (`Cache-Status`, `X-Cache-Status` and the other RustProxy
cache diagnostics) are removed before RustProxy emits its own diagnostics. HTTP `Age`
remains present on reused cached responses because it is protocol freshness metadata,
not a debug header. The Cache dashboard also marks transformed objects explicitly.

## Pool behavior

Backend selection and health policy still run before an origin fetch. The selected pool
member no longer fragments the cache identity in 0.11.5, so a fresh public object can be
reused regardless of which healthy member would have been selected for a MISS. A pool
with no eligible member still returns 503 before RustProxy can use the cache; this cache
is not a stale-availability layer.

Cookie affinity and protected/authenticated traffic keep their existing privacy bypass
rules. Per-vhost response/security policy also applies to HIT responses.