RustProxy/docs/CACHE.md

Shared hot/cold cache

See CACHE_DIAGNOSTICS.md for practical HIT/MISS/BYPASS diagnosis.

Scope

RustProxy implements a bounded reverse-proxy cache with a hot RAM tier and an optional persistent cold disk tier. Version 0.11.5 aligned request-method handling and most logical hash behavior with Varnish built-in semantics. Since 0.19.1 the logical primary identity includes the frontend scheme as well as Host + exact request URL, so http://host/path and https://host/path cannot reuse the same stored response. RustProxy still isolates different vhost/config scopes internally and uses response Vary to select representation variants.

This is not a VCL implementation. Generic Varnish grace, ESI, public unauthenticated BAN/PURGE and distributed cache state are not implemented. RustProxy 0.22.1 adds bounded foreground request collapsing, authenticated selective PURGE/BAN and optional proactive refresh while keeping directive-controlled stale delivery through RFC 5861 and request max-stale; it is not an unconditional availability/grace layer. RustProxy also keeps its validator-based handling of response no-cache instead of reproducing Varnish hit-for-miss behavior exactly.

Settings

Field	Default	Bounds / meaning
enabled	true	Global master switch; vhosts may disable cache but cannot re-enable a globally disabled cache
max_bytes	268435456	Hard HOT allocation budget (body backing allocation + charged entry metadata); minimum 64 KiB, no artificial high ceiling
max_object_bytes	16777216	Maximum logical body accepted by one fill; minimum 1 KiB and at most max_bytes
max_entries	4096	1..16384 hot objects
transient_max_bytes	67108864	Hard RAM budget for fill buffers and complete bodies moving between HOT/COLD
cold_queue_max_bytes	33554432	Byte ceiling for complete demoted objects waiting for the COLD writer; at most transient_max_bytes
hot_eviction_max_objects	64	1..4096 LRU removals attempted for one HOT admission before the new object passes uncached
cold_enabled	false	Enable persistent disk tier; never implied by hot-cache enable
cold_path	cache/cold	Relative path under the private RustProxy data directory
cold_max_bytes	4294967296	64 KiB..1 TiB disk budget
cold_max_entries	50000	1..1000000 cold objects
cold_gc_interval_seconds	60	5..86400; interval between asynchronous pressure-GC checks
cold_gc_max_delete	256	1..10000; maximum objects removed by one automatic GC pass
cold_after_seconds	300	5..604800; idle hot objects are eligible after this delay
cold_on_idle	true	Move idle eligible hot objects to cold storage
cold_max_hot_seconds	3600	5..2592000; total RAM-residency age used when cold_on_age=true
cold_on_age	false	Demote objects after the maximum hot-tier age even if they remain frequently accessed
cold_on_lru	true	Move objects displaced by hot-tier LRU pressure to cold storage
cold_on_shutdown	true	Best-effort direct hot-to-cold flush during ordinary graceful shutdown/restart
default_ttl_seconds	0	Only explicit public responses without origin freshness
max_ttl_seconds	86400	1..604800; maximum configured retention/freshness cap
bypass_query	false	Bypass requests containing a query when enabled
bypass_prefixes	[]	Up to 64 normalized absolute path prefixes
static_ignore_cookies	true	For configured static extensions only, ignore request Cookie for eligibility/keying
expose_status	false	Emit X-Cache: HIT, MISS or BYPASS
expose_hits	false	Emit per-object X-Cache-Hits
expose_cache_status	false	Emit standardized Cache-Status (RFC 9211)
expose_resident_age	false	Emit X-Cache-Resident-Age on reused stored responses
collapse_enabled	true	Collapse concurrent foreground fills for the same primary cache key
collapse_wait_timeout_ms	1500	10..60000 ms follower wait before fallback
collapse_timeout_fallback	origin	origin or stale_if_available after follower timeout
stale_if_error_timeout	true	Allow RFC 5861 stale fallback for timeout failures
stale_if_error_connection	true	Allow stale fallback for connection failures
stale_if_error_dns	true	Allow stale fallback for DNS failures
stale_if_error_tls	true	Allow stale fallback for TLS failures
stale_if_error_statuses	[500,502,503,504]	Up to 32 unique eligible HTTP 5xx statuses
proactive_refresh_enabled	false	Schedule sampled cached objects for validator refresh before expiry
proactive_refresh_before_expiry_seconds	15	1..604800 seconds before expiry
proactive_refresh_percent	10	0..100 deterministic percentage of primary keys
proactive_refresh_max_inflight	4	1..64 concurrent proactive refreshes

The hot tier enforces max_bytes and max_entries before publication. HOT accounting uses the retained body allocation (the fill buffer capacity transferred into Bytes) plus directly owned entry metadata and a conservative per-entry charge for the HOT hash/LRU/variant indexes, while hot_logical_bytes reports only visible body lengths. Rust container/allocator APIs do not expose every bucket/node allocation exactly, so hot_allocated_bytes is intentionally a conservative charged estimate rather than a promise that it equals process RSS. In-progress fills reserve a separate bounded transient budget before growing their buffers. When HOT pressure would exceed the budget plus a small allocator-rounding headroom (roughly 1/64 of HOT, capped at 1 MiB), RustProxy removes LRU objects first; if admission still cannot be made safe within hot_eviction_max_objects, the response continues normally without being cached. max_object_bytes is checked incrementally before every body-buffer growth, so an oversized response is not fully accumulated before rejection.

When cold storage is enabled, cold_on_lru, cold_on_idle and cold_on_age independently choose which hot objects are demoted. cold_after_seconds measures time since the last access. cold_max_hot_seconds measures total hot-tier residency, so it can move a frequently accessed object to disk after a configured age. The administration UI exposes these as separate policies to avoid conflating "old" with "unused". Demotions use an asynchronous queue bounded by both object count and cold_queue_max_bytes. A demoted Entry is removed from HOT accounting and moved, not copied, into a ColdDraft; its Bytes payload remains charged to transient_max_bytes until persistence completes and the draft is dropped. The persisted ColdEntry contains paths/metadata rather than the body allocation. This prevents the COLD writer queue from becoming a second unbounded RAM cache. If an HTTP response is already using a clone of the same Bytes, that last transport-owned reference can keep the backing allocation alive until the response finishes; the cache-owned HOT/COLD reference itself is released after persistence. Process RSS is therefore monitored separately from cache accounting. Supported glibc/Linux builds ask the allocator to release unused pages after bulk demotion. During ordinary graceful shutdown, cold_on_shutdown performs a best-effort direct flush of remaining hot objects before the store is closed. A managed software update (update.sh / scripts/deploy.py) uses a private one-shot marker and, on RustProxy 0.13.2+, forces this hot-to-cold handoff whenever the cold tier is enabled even if cold_on_shutdown=false. The updater gives the process a larger stop budget and removes the marker before backing up state. If the cold tier is disabled there is nowhere persistent to hand the hot objects, so RAM-only cache is still lost on process exit. The body is stored as a raw .body file and metadata as JSON with a SHA-256 body digest. A fresh cold hit is integrity-checked and promoted to RAM. Cold format 3, introduced in 0.21.0, stores objects below the first two hexadecimal characters of the SHA-256 cache key, for example cache/cold/7f/<hash>.body and cache/cold/7f/<hash>.json. Shard directories are created lazily. The previous flat Cold Cache layout is deliberately not migrated; legacy flat objects are removed and can be refilled from origin. Format 3 metadata also carries RFC 5861 windows, revalidation flags, origin validators, last access and hit count.

Automatic Cold Cache GC is asynchronous. When either byte or entry usage reaches about 90% of its configured limit, it selects old/rarely used candidates and cleans toward about 80%, removing at most cold_gc_max_delete objects in one pass. Metadata is snapshotted in small batches and disk deletion is queued instead of holding request-path state across a full-cache sort/delete loop.

Varnish-style identity and variants

The logical primary cache identity is:

• a stable cache-compatibility revision derived only from settings that can change cached representation/origin semantics, plus vhost scope (internal isolation),
• normalized frontend scheme (http/https),
• normalized original Host authority, including an explicit port,
• exact request path and query.

Selected backend-pool member, route/backend IDs and unrelated request headers are not part of the representation hash. This means two healthy members of the same pool can serve/fill the same public cache object, while HTTP and HTTPS remain isolated. Headers such as User-Agent, X-Forwarded-For or browser client hints no longer create unnecessary variants by themselves.

The response Vary header provides the secondary representation identity. For example, Vary: Accept-Language stores independent language variants under the same scheme + Host + URL primary hash. Vary: *, malformed Vary fields or an excessive Vary field set are not stored. If an origin changes the Vary schema for a URL, incompatible old variants are removed so an obsolete no-Vary object cannot shadow a newer Vary representation (or vice versa).

Cookie and Authorization requests still pass by default, matching Varnish built-in safety behavior. RustProxy's static-cookie optimization can ignore Cookie only for the configured static file extensions; response Set-Cookie, private, no-store and Vary: Cookie still prevent unsafe shared reuse.

Request methods and invalidation

GET and HEAD participate in lookup. They share one representation identity. On a cacheable HEAD miss RustProxy fetches the backend representation as GET, may populate the shared object, and strips the response body before delivering the original HEAD request.

POST, PUT, PATCH, DELETE, OPTIONS and TRACE pass to the backend and are not stored. They do not implicitly purge the whole cache. This fixes the pre-0.11.5 behavior where any successful unsafe request could erase unrelated static objects and destroy the hit ratio.

Cache invalidation remains explicit:

• authenticated full purge can clear all objects or one vhost scope,
• selective PURGE supports exact URL, URL prefix, Cache-Tag, safe regex and primary cache key, with optional host and vhost restrictions,
• exact/prefix matching uses the stored request target including query; a match removes the whole Vary family under that primary cache key,
• selective PURGE operates on RAM and Cold Cache and advances a publication generation so no fill that started before the operation can publish afterward, even if the selector depended on response metadata such as Cache-Tag,
• BAN records a bounded TTL rule without synchronously scanning the entire store; objects created before a matching active BAN are rejected during lookup/insertion and are removed later by bounded GC,
• new fills created after a BAN rule are allowed, so BAN invalidates the old representation set rather than permanently blocking a URL,
• regex rules and prefix / are treated as broad operations and require explicit confirmation,
• only cache-compatibility changes (cache policy, configured header/origin semantics or vhosts) advance the normal cache generation/revision; unrelated logging/TLS/admin/metrics/response-presentation saves retain objects.

There is no public unauthenticated PURGE/BAN endpoint. If an application changes a GET resource after POST/PUT/PATCH/DELETE, use origin freshness/validators or the authenticated administration invalidation path; RustProxy does not guess the affected URLs.

Eligibility and freshness

Hard request bypasses include Cookie (outside the static exception), Authorization, Upgrade, If-Match, If-Unmodified-Since, nonempty request bodies, configured path/query exclusions and request Cache-Control: no-store.

A single Range: bytes=... request is special rather than a normal bypass. When a complete fresh cached representation is available, RustProxy evaluates If-Range and can return 206 Partial Content directly with Content-Range, Accept-Ranges, the correct sliced Content-Length, validators and normal cache Age; an unsatisfied range returns 416 with Content-Range: bytes */<length>. A mismatching If-Range returns the complete cached representation. If origin access is needed, RustProxy forwards the Range request as an identity, non-transformed stream. Multiple/multipart, duplicate or otherwise unsupported Range forms are passed through to origin and are never stored as separate range objects.

A fresh cached representation can satisfy If-None-Match/If-Modified-Since locally. Request no-cache, max-age=0 and legacy Pragma: no-cache require validation when a safe stored validator exists. max-age=N and min-fresh=N are freshness constraints: they reuse the cached response when its current Age/remaining freshness satisfies the client limit and otherwise drive normal revalidation/fetch behavior. Request max-stale can explicitly allow a retained stale response: bare max-stale accepts any retained staleness, while max-stale=N bounds it. Freshness expiry does not itself physically delete an object; normal RAM LRU/Cold Cache GC may still evict it. must-revalidate and proxy-revalidate override client stale permission. With a safe validator, an origin 304 refreshes metadata and reuses the stored body.

Responses with Set-Cookie, private, no-store, Content-Range, trailers, SSE/gRPC, Vary: *, invalid freshness metadata or unusable size/lifetime are not stored. Response Cache-Control: no-cache may be retained only with ETag or Last-Modified and must validate before every reuse. Both origin and transformed response policy must remain eligible.

s-maxage takes precedence over max-age, then Expires/Date. Age uses the RFC 9111 current-age model and includes origin age; it is not a measure of how long the object has been in RustProxy. Cache metadata keeps normal TTL, stale-while-revalidate, stale-if-error, must-revalidate and proxy-revalidate separately. A representation is therefore fresh, stale-but-servable only under an applicable directive, or expired/unusable.

Inside stale-while-revalidate=N, RustProxy may return stale immediately and starts one deduplicated background refresh for that cache identity. Inside stale-if-error=N, stale is used only after an eligible origin failure. Version 0.22 makes timeout, connection, DNS and TLS classes independently configurable and uses an explicit configurable 5xx status allowlist; defaults preserve the previous behavior (500, 502, 503, 504 plus all four transport classes). After TTL plus the corresponding stale window, that mode is unavailable. must-revalidate and proxy-revalidate still permit storage, but prohibit all stale delivery after expiry, including RFC 5861 and request max-stale.

Optional proactive refresh schedules a deterministic percentage of cache keys before normal expiry. It reuses the same validator-based refresh logic and background-revalidation guard, keeps only one schedule per key, and enforces a global concurrent-refresh bound. A newly completed fill can schedule this work immediately, so sampled objects do not require another client request to start the pre-expiry timer. The feature is disabled by default.

Resident age

X-Cache-Resident-Age is the RustProxy-specific whole-second clock for how long the stored representation has remained in this cache. In 0.11.5 that clock is independent from the HTTP freshness Age clock and survives:

• ordinary hot-cache HITs,
• successful backend 304 revalidation,
• hot-to-cold demotion,
• cold-to-hot promotion.

A value of 0 is valid only during the first sub-second of residency. Revalidation or tier movement must no longer reset a long-lived object to zero.

Streaming and capacity

RustProxy tees response frames while delivering them; it does not wait for the complete origin body before sending the first bytes. Only a complete successful body is published. Oversize, body error, trailer and cancelled-fill paths discard the candidate. Large buffers grow incrementally and consume multiple bounded fill permits.

The configured resident byte limit accounts for body bytes, estimated header storage and fixed per-entry overhead; it is not total process RSS. Active HIT references, in-flight origin responses, compression buffers and runtime allocations also use memory.

With collapse_enabled=true, the first foreground MISS for a primary cache key owns the fill while concurrent followers wait for publication. The fill guard is held through the actual response-body tee, so followers cannot wake merely because origin headers arrived. Leader error/cancellation releases waiters. A follower that wakes to a different Vary representation re-enters the single-flight decision instead of immediately stampeding the origin. After collapse_wait_timeout_ms, the configured fallback is either an independent origin request or an eligible stale response. SWR/background revalidation keeps its separate existing deduplication guard.

Diagnostics and API

GET /api/cache reports settings, HIT/MISS/BYPASS, hot/cold counts and bytes, promotions/demotions, errors, fill pressure, object views and per-domain reason counters. Hit ratio is hits / (hits + misses) and intentionally excludes BYPASS traffic.

POST /api/cache/purge requires the existing administrator session, same-origin marker and CSRF token. ?vhost=<id> purges only that vhost scope; omitting it purges all cache objects. Purge is audited.

POST /api/cache/invalidate performs authenticated selective purge or ban using exact, prefix, tag, regex or cache_key. Optional host and vhost restrict scope; BAN accepts a bounded TTL. Regex and prefix / require confirm_wide=true. The result reports the selector/scope, affected entry count, elapsed time, status and BAN id when one was created. GET /api/cache/bans lists active rules and DELETE /api/cache/bans/{id} removes one. These operations use the normal administration audit path. The Cache panel exposes the same controls plus per-object Purge URL.

POST /api/cache/cold/cleanup starts a selective asynchronous Cold Cache cleanup using the same candidate/removal machinery as automatic GC. Criteria can select entries unused for a duration, cap removal to the oldest N objects, and/or require hits == 0. GET /api/cache/cold/cleanup/{id} reports job status, removed object count and freed bytes. Purge-all and purge-vhost remain the full-clear mechanisms; selective invalidation is deliberately separate.

X-Cache, X-Cache-Hits, Cache-Status, X-Cache-Reason and X-Cache-Resident-Age are optional diagnostics. Cache-Status is the standardized RFC 9211 field name; RustProxy does not rename it to the legacy/custom X-Cache-Status. HTTP/2 and HTTP/3 serialize field names in lowercase, so cache-status: on the wire is normal. When the cached representation was produced by the transform pipeline, RustProxy appends the extension parameter transformed, for example:

Cache-Status: RustProxy; hit; ttl=556488; detail="cache_hit"; transformed

Origin lookalike values (Cache-Status, X-Cache-Status and the other RustProxy cache diagnostics) are removed before RustProxy emits its own diagnostics. HTTP Age remains present on reused cached responses because it is protocol freshness metadata, not a debug header. The Cache dashboard also marks transformed objects explicitly.

Pool behavior

Backend selection and health policy still run before an origin fetch. The selected pool member no longer fragments the cache identity in 0.11.5, so a fresh public object can be reused regardless of which healthy member would have been selected for a MISS. A pool with no eligible member still returns 503 before RustProxy can use the cache; this cache is not a stale-availability layer.

Cookie affinity and protected/authenticated traffic keep their existing privacy bypass rules. Per-vhost response/security policy also applies to HIT responses.