RustProxy/docs/CACHE_DIAGNOSTICS.md

Cache diagnostics and behavior

This describes implemented source, not a verified Rust runtime. See TEST_REPORT.md.

0.19.1 cache identity safety

The frontend scheme is part of the primary cache identity. http://host/path and https://host/path therefore cannot share an object, preventing responses that depend on X-Forwarded-Proto or equivalent origin logic from crossing schemes.

0.11.5 cache changes

The cache follows the Varnish built-in model for the behaviors that most directly affect hit ratio, with the 0.19.1 scheme-isolation difference noted above:

• GET and HEAD use the cache; other ordinary methods pass without clearing unrelated objects.
• The logical primary identity is frontend scheme + normalized Host authority + exact URL inside the current vhost/config scope.
• Selected backend-pool member and unrelated request headers do not fragment the primary object.
• Response Vary chooses representation variants under that object.
• GET and HEAD share the same stored representation; a cacheable HEAD miss can fill it from a backend GET.
• X-Cache-Resident-Age survives 304 revalidation and hot/cold tier movement.

RustProxy still deliberately bypasses Cookie and Authorization requests by default, matching Varnish built-in safety behavior. The configured static-extension optimization can ignore Cookie only for selected static paths, subject to response privacy guards.

Why objects used to disappear after POST/PUT/etc.

Before 0.11.5 a successful non-safe request could call a global cache purge. A periodic POST, API heartbeat or form submission could therefore erase every cached static object even though the request was unrelated to those URLs. In 0.11.5 POST/PUT/PATCH/DELETE, OPTIONS and TRACE pass through and do not implicitly purge the shared cache.

Only configuration changes that alter cache/origin compatibility invalidate incompatible cache state; unrelated control-plane saves keep reusable hot/cold objects. Explicit administrator purge still invalidates the selected cache state. Use the purge action when the application really needs explicit invalidation.

Age versus X-Cache-Resident-Age

These are different clocks:

• Age is HTTP freshness age. It can already be thousands of seconds when RustProxy receives an object because the origin/CDN Date/Age contributes to it.
• X-Cache-Resident-Age is RustProxy-specific and counts whole seconds that the stored representation has remained resident in this cache.

In 0.11.5 resident age is preserved across a successful backend 304 and across hot-to-cold/cold-to-hot movement. An immediate first HIT can legitimately show 0 because less than one full second has elapsed; a long-lived object should then keep increasing rather than repeatedly resetting to zero.

Practical HIT test

Enable cache plus X-Cache, X-Cache-Hits and X-Cache-Resident-Age, then make repeated identical requests that read the complete body:

for n in 1 2 3; do
  curl -q -sS -D - -o /dev/null \
    -H 'Host: app.localhost' \
    http://127.0.0.2:8888/cache
  sleep 2
done

For an eligible response, expect MISS then HIT/HIT. X-Cache-Resident-Age should grow between later HITs. A browser is a poorer diagnostic because its own cache may satisfy a navigation without sending any request to RustProxy.

HEAD is now eligible too. It shares the GET representation, but GET is still the best basic test because it proves that the body was completely consumed and published.

Vary test

A response with:

Vary: Accept-Language

stores language variants under one primary scheme + Host + URL object identity. Two English requests can HIT each other; a Polish request must not receive the English body. If the origin later changes between no Vary and Vary: Accept-Language, incompatible older variants are removed rather than used as a fallback.

Vary: * is not cacheable.

Cookie/static behavior

request_cookie is a BYPASS by default. With static_ignore_cookies=true, configured static extensions such as .css, .js, fonts and images ignore the request Cookie for cache eligibility/keying. This is intentionally limited to static paths. A response with Set-Cookie, Cache-Control: private/no-store, or Vary: Cookie still cannot be shared through that optimization.

This setting is useful for raising the asset hit ratio for logged-in users without turning dynamic personalized pages into shared objects.

Conditional requests and reloads

Fresh cached objects can satisfy If-None-Match/If-Modified-Since locally. Request Cache-Control: no-cache, max-age, min-fresh and legacy Pragma: no-cache select validation instead of creating a separate cache key. If a stored validator exists and the backend returns 304, RustProxy refreshes metadata and reuses the body.

Hard browser reloads commonly add revalidation directives, so they may produce cache_revalidated instead of a plain cache_hit. This is different from a cache miss.

Reason tokens

When cache diagnostics are enabled, these fixed reason tokens explain decisions:

Token	Meaning
cache_disabled	Effective cache is disabled.
route_disabled	Selected route explicitly disables cache.
request_method	Method is not GET/HEAD. It passes and does not purge the cache.
request_cookie, request_authorization	Private request state bypasses shared cache. Static Cookie exception may apply.
request_no_store	Client forbids use/storage of shared cache.
request_no_cache, request_max_age, request_min_fresh, request_pragma_no_cache	Client requests validation.
request_body, request_range, request_precondition	Request is outside reusable shared-object semantics.
query_excluded, path_excluded	Configured exclusion matched.
tinyauth, sticky_cookie	Authentication/affinity policy deliberately bypasses shared cache.
response_set_cookie, response_cache_control	Origin response is not safely shareable.
response_vary, response_ttl_size_or_metadata	Vary/freshness/size/metadata prevents storage.
response_headers_policy	Transformed client-response policy is not cacheable.
cache_miss, cache_hit	No reusable variant was found, or a variant was reused.
cache_conditional_hit	Fresh cache metadata satisfied the client validator locally.
cache_revalidated	Backend 304 validated the stored body and it was reused.
cache_revalidated_uncached	Validation succeeded but refreshed policy no longer permits storage.

A MISS only means lookup found no reusable object. The later origin response can still fail storage because of privacy headers, size, incomplete streaming, trailers or a changed policy.

Memory accounting in 0.16.0

The process RSS total is an operating-system measurement and is intentionally shown separately from RustProxy's own cache accounting. Do not add the rows together: they overlap. The process block reports:

• RSS total (VmRSS) - all resident pages currently attributed to the process;
• RSS anonymous (RssAnon) - anonymous resident pages, which typically include heap/stacks and other private allocations;
• RSS file-backed / shared (RssFile / RssShmem) - resident mapped file/shared-memory pages;
• data address space / swap (VmData / VmSwap) - address-space and swapped-page context, not extra resident bytes.

The cache block separately reports memory RustProxy can attribute to the cache subsystem:

• hot logical - visible cached body bytes (body.len());
• hot allocated - the hard-budget charge for retained backing allocations plus entry metadata (resident_bytes remains a compatibility alias);
• transient - bounded cache allocations temporarily outside the hot store;
• fill - active response buffers being assembled before publication;
• cold queue - complete bodies waiting for cold-tier persistence;
• cold I/O - bodies currently being written to or read from the cold tier.

This explains why, for example, a process RSS around 630 MiB can coexist with only about 100 MiB of reported hot cache: the old Pamięć hot card was never a whole-process memory counter. TLS/QUIC/Hyper/Tokio allocations, thread stacks, configuration/routing state, active request bodies and allocator-retained pages are also part of RSS but are not falsely assigned to a RustProxy subsystem when the source cannot measure them directly.

The existing Linux/glibc allocator-release path remains in place: large cache demotion/purge paths call malloc_trim(0) through release_allocator_memory(). Since 0.24.5, HOT allocation, active fills and bodies moving to/from COLD are explicitly budgeted. COLD queue admission is byte-bounded as well as count-bounded. RSS is still a process-wide view containing the same cache memory plus TLS/QUIC/Hyper/Tokio, stacks and allocator retention, so cache gauges must not be added to RSS. Long-running acceptance should verify that HOT stays within hot_limit_bytes, transient stays within transient_limit_bytes, demotion drains transient bytes, and process RSS reaches a practical plateau under representative traffic.

Sustained traffic, vhost saves and RSS

A vhost configuration save can make RSS fall abruptly for two independent reasons. First, the current cache compatibility identity includes vhost configuration; when that identity changes, publication performs a cache purge so representations created under the old origin/configuration contract cannot survive incorrectly. Second, publishing the newly compiled routing snapshot retires the previous snapshot and its upstream clients; once outstanding references finish, idle connection-pool state and related buffers can be released. A visible RSS drop after saving a busy vhost is therefore a useful diagnostic clue, but saving/reloading a vhost is not a memory-maintenance requirement.

The 0.25.0 development runtime adds two maintenance mechanisms that do not reload configuration:

• every 60 seconds on Linux/glibc, RustProxy calls malloc_trim(0) from a blocking maintenance task; this can return allocator pages that are already free, but cannot and does not evict live Rust objects;
• HOT cache maintenance trims only least-recently-used entries that have been idle for the configured delay and only while HOT allocation is above the configured target waterline. The pass is bounded by hot_eviction_max_objects; recent/active entries are not purged.

Upstream HTTP keep-alive is not reset on a timer. It is already bounded per backend by max_idle_per_host and idle_timeout_seconds, so forcing periodic client/snapshot replacement would trade memory symptoms for connection churn and lower proxy efficiency. If RSS remains high while HOT/transient usage and connection-pool bounds are stable, the remaining resident pages may still be active runtime/TLS/QUIC/Hyper/Tokio working set rather than reclaimable cache memory.

Hot and cold tiers

HOT RAM is bounded by allocated/charged max_bytes and max_entries; hot_logical_bytes separately exposes payload length. Fill and COLD handoff/read memory are bounded by transient_max_bytes, and queued demotions also obey cold_queue_max_bytes. Optional cold storage demotes LRU or sufficiently idle hot objects asynchronously. Cold hits verify the body digest and promote the object back to RAM. Version 0.11.5 uses cold metadata format 2 to persist Vary identity and resident age; old-format metadata is not reused.

Cold is not required for normal hot-cache stability. With cold disabled, hot objects leave RAM because of expiry, hot capacity/LRU pressure, explicit purge, configuration generation changes, replacement of the same logical representation, or the optional HOT auto-trim waterline. Auto-trim is not a blind TTL: it acts only when HOT is above its configured target and only on entries idle longer than the configured threshold; it can be disabled independently.

Cache panel and API

The Cache panel shows hot/cold object counts, RAM/disk usage, HIT/MISS/BYPASS, hit ratio, fill pressure, evictions/expiry, per-domain decisions and stored objects. Live polling refreshes the view; it does not itself purge cache objects.

GET /api/cache?offset=0&limit=50 returns aggregate and object data. Add vhost=ID to filter. POST /api/cache/purge?vhost=ID invalidates one vhost; omitting the query purges all cache objects. Mutations remain authenticated, CSRF-protected and audited.

Hit ratio is HIT / (HIT + MISS). BYPASS requests are excluded, so a large number of cookie/authenticated requests can reduce backend offload without mathematically lowering that displayed ratio. For the highest safe reusable-object ratio, give static assets long explicit public TTLs, keep their URLs stable/immutable, leave the static-cookie optimization enabled when appropriate, and avoid unnecessary Vary fields.