206 lines
13 KiB
Markdown
206 lines
13 KiB
Markdown
# Cache diagnostics and behavior
|
|
|
|
This describes implemented source, not a verified Rust runtime. See
|
|
[TEST_REPORT.md](TEST_REPORT.md).
|
|
|
|
## 0.19.1 cache identity safety
|
|
|
|
The frontend scheme is part of the primary cache identity. `http://host/path` and
|
|
`https://host/path` therefore cannot share an object, preventing responses that depend
|
|
on `X-Forwarded-Proto` or equivalent origin logic from crossing schemes.
|
|
|
|
## 0.11.5 cache changes
|
|
|
|
The cache follows the Varnish built-in model for the behaviors that most directly
|
|
affect hit ratio, with the 0.19.1 scheme-isolation difference noted above:
|
|
|
|
- GET and HEAD use the cache; other ordinary methods pass without clearing unrelated objects.
|
|
- The logical primary identity is frontend scheme + normalized Host authority + exact URL inside the current vhost/config scope.
|
|
- Selected backend-pool member and unrelated request headers do not fragment the primary object.
|
|
- Response `Vary` chooses representation variants under that object.
|
|
- GET and HEAD share the same stored representation; a cacheable HEAD miss can fill it from a backend GET.
|
|
- `X-Cache-Resident-Age` survives 304 revalidation and hot/cold tier movement.
|
|
|
|
RustProxy still deliberately bypasses Cookie and Authorization requests by default,
|
|
matching Varnish built-in safety behavior. The configured static-extension optimization
|
|
can ignore Cookie only for selected static paths, subject to response privacy guards.
|
|
|
|
## Why objects used to disappear after POST/PUT/etc.
|
|
|
|
Before 0.11.5 a successful non-safe request could call a global cache purge. A periodic
|
|
POST, API heartbeat or form submission could therefore erase every cached static object
|
|
even though the request was unrelated to those URLs. In 0.11.5 POST/PUT/PATCH/DELETE,
|
|
OPTIONS and TRACE pass through and do not implicitly purge the shared cache.
|
|
|
|
Only configuration changes that alter cache/origin compatibility invalidate incompatible cache
|
|
state; unrelated control-plane saves keep reusable hot/cold objects. Explicit administrator purge
|
|
still invalidates the selected cache state. Use the purge action when the application really needs
|
|
explicit invalidation.
|
|
|
|
## `Age` versus `X-Cache-Resident-Age`
|
|
|
|
These are different clocks:
|
|
|
|
- `Age` is HTTP freshness age. It can already be thousands of seconds when RustProxy
|
|
receives an object because the origin/CDN `Date`/`Age` contributes to it. On an ordinary
|
|
fresh HIT it advances from RustProxy's stored receive time even when the backend never
|
|
supplied an `Age` header. A successful validator revalidation (backend `304`) starts a
|
|
new freshness calculation, so the next `Age` can legitimately be `0`. Requests carrying
|
|
`Cache-Control: no-cache`, `max-age=0` or `Pragma: no-cache` can therefore make repeated
|
|
validated HITs appear with `Age: 0`.
|
|
- `X-Cache-Resident-Age` is RustProxy-specific and counts whole seconds that the stored
|
|
representation has remained resident in this cache.
|
|
|
|
Resident age is preserved across a successful backend 304 and across hot-to-cold/cold-to-hot
|
|
movement. An immediate first HIT can legitimately show `0` because less than one full second
|
|
has elapsed; a long-lived resident object should then keep increasing rather than repeatedly
|
|
resetting to zero. Use this header when the diagnostic question is cache residency rather than
|
|
HTTP freshness.
|
|
|
|
## Practical HIT test
|
|
|
|
Enable cache plus `X-Cache`, `X-Cache-Hits` and `X-Cache-Resident-Age`, then make
|
|
repeated identical requests that read the complete body:
|
|
|
|
```bash
|
|
for n in 1 2 3; do
|
|
curl -q -sS -D - -o /dev/null \
|
|
-H 'Host: app.localhost' \
|
|
http://127.0.0.2:8888/cache
|
|
sleep 2
|
|
done
|
|
```
|
|
|
|
For an eligible response, expect MISS then HIT/HIT. `X-Cache-Resident-Age` should grow
|
|
between later HITs. A browser is a poorer diagnostic because its own cache may satisfy
|
|
a navigation without sending any request to RustProxy.
|
|
|
|
HEAD is now eligible too. It shares the GET representation, but GET is still the best
|
|
basic test because it proves that the body was completely consumed and published.
|
|
|
|
## Vary test
|
|
|
|
A response with:
|
|
|
|
```text
|
|
Vary: Accept-Language
|
|
```
|
|
|
|
stores language variants under one primary scheme + Host + URL object identity. Two English
|
|
requests can HIT each other; a Polish request must not receive the English body. If the
|
|
origin later changes between no `Vary` and `Vary: Accept-Language`, incompatible older
|
|
variants are removed rather than used as a fallback.
|
|
|
|
`Vary: *` is not cacheable.
|
|
|
|
## Cookie/static behavior
|
|
|
|
`request_cookie` is a BYPASS by default. With `static_ignore_cookies=true`, configured
|
|
static extensions such as `.css`, `.js`, fonts and images ignore the request Cookie for
|
|
cache eligibility/keying. This is intentionally limited to static paths. A response
|
|
with `Set-Cookie`, `Cache-Control: private/no-store`, or `Vary: Cookie` still cannot be
|
|
shared through that optimization.
|
|
|
|
This setting is useful for raising the asset hit ratio for logged-in users without
|
|
turning dynamic personalized pages into shared objects.
|
|
|
|
## Conditional requests and reloads
|
|
|
|
Fresh cached objects can satisfy `If-None-Match`/`If-Modified-Since` locally. Request
|
|
`Cache-Control: no-cache`, `max-age`, `min-fresh` and legacy `Pragma: no-cache` select
|
|
validation instead of creating a separate cache key. If a stored validator exists and
|
|
the backend returns 304, RustProxy refreshes metadata and reuses the body.
|
|
|
|
Hard browser reloads commonly add revalidation directives, so they may produce
|
|
`cache_revalidated` instead of a plain `cache_hit`. This is different from a cache miss.
|
|
|
|
## Reason tokens
|
|
|
|
When cache diagnostics are enabled, these fixed reason tokens explain decisions:
|
|
|
|
| Token | Meaning |
|
|
|---|---|
|
|
| `cache_disabled` | Effective cache is disabled. |
|
|
| `route_disabled` | Selected route explicitly disables cache. |
|
|
| `request_method` | Method is not GET/HEAD. It passes and does not purge the cache. |
|
|
| `request_cookie`, `request_authorization` | Private request state bypasses shared cache. Static Cookie exception may apply. |
|
|
| `request_no_store` | Client forbids use/storage of shared cache. |
|
|
| `request_no_cache`, `request_max_age`, `request_min_fresh`, `request_pragma_no_cache` | Client requests validation. |
|
|
| `request_body`, `request_range`, `request_precondition` | Request is outside reusable shared-object semantics. |
|
|
| `query_excluded`, `path_excluded` | Configured exclusion matched. |
|
|
| `tinyauth`, `sticky_cookie` | Authentication/affinity policy deliberately bypasses shared cache. |
|
|
| `response_set_cookie`, `response_cache_control` | Origin response is not safely shareable. |
|
|
| `response_vary`, `response_ttl_size_or_metadata` | Vary/freshness/size/metadata prevents storage. |
|
|
| `response_headers_policy` | Transformed client-response policy is not cacheable. |
|
|
| `cache_miss`, `cache_hit` | No reusable variant was found, or a variant was reused. |
|
|
| `cache_conditional_hit` | Fresh cache metadata satisfied the client validator locally. |
|
|
| `cache_revalidated` | Backend 304 validated the stored body and it was reused. |
|
|
| `cache_revalidated_uncached` | Validation succeeded but refreshed policy no longer permits storage. |
|
|
|
|
A MISS only means lookup found no reusable object. The later origin response can still
|
|
fail storage because of privacy headers, size, incomplete streaming, trailers or a
|
|
changed policy.
|
|
|
|
|
|
## Memory accounting in 0.16.0
|
|
|
|
The process **RSS total** is an operating-system measurement and is intentionally shown separately from RustProxy's own cache accounting. Do not add the rows together: they overlap. The process block reports:
|
|
|
|
- `RSS total` (`VmRSS`) - all resident pages currently attributed to the process;
|
|
- `RSS anonymous` (`RssAnon`) - anonymous resident pages, which typically include heap/stacks and other private allocations;
|
|
- `RSS file-backed / shared` (`RssFile` / `RssShmem`) - resident mapped file/shared-memory pages;
|
|
- `data address space / swap` (`VmData` / `VmSwap`) - address-space and swapped-page context, not extra resident bytes.
|
|
|
|
The cache block separately reports memory RustProxy can attribute to the cache subsystem:
|
|
|
|
- `hot logical` - visible cached body bytes (`body.len()`);
|
|
- `hot allocated` - the hard-budget charge for retained backing allocations plus entry metadata (`resident_bytes` remains a compatibility alias);
|
|
- `transient` - bounded cache allocations temporarily outside the hot store;
|
|
- `fill` - active response buffers being assembled before publication;
|
|
- `cold queue` - complete bodies waiting for cold-tier persistence;
|
|
- `cold I/O` - bodies currently being written to or read from the cold tier.
|
|
|
|
This explains why, for example, a process RSS around 630 MiB can coexist with only about 100 MiB of reported hot cache: the old `Pamięć hot` card was never a whole-process memory counter. TLS/QUIC/Hyper/Tokio allocations, thread stacks, configuration/routing state, active request bodies and allocator-retained pages are also part of RSS but are not falsely assigned to a RustProxy subsystem when the source cannot measure them directly.
|
|
|
|
The existing Linux/glibc allocator-release path remains in place: large cache demotion/purge paths call `malloc_trim(0)` through `release_allocator_memory()`. Since 0.24.5, HOT allocation, active fills and bodies moving to/from COLD are explicitly budgeted. COLD queue admission is byte-bounded as well as count-bounded. RSS is still a process-wide view containing the same cache memory plus TLS/QUIC/Hyper/Tokio, stacks and allocator retention, so cache gauges must not be added to RSS. Long-running acceptance should verify that HOT stays within `hot_limit_bytes`, transient stays within `transient_limit_bytes`, demotion drains transient bytes, and process RSS reaches a practical plateau under representative traffic.
|
|
|
|
### Sustained traffic, vhost saves and RSS
|
|
|
|
A vhost configuration save can make RSS fall abruptly for two independent reasons. First, the current cache compatibility identity includes vhost configuration; when that identity changes, publication performs a cache purge so representations created under the old origin/configuration contract cannot survive incorrectly. Second, publishing the newly compiled routing snapshot retires the previous snapshot and its upstream clients; once outstanding references finish, idle connection-pool state and related buffers can be released. A visible RSS drop after saving a busy vhost is therefore a useful diagnostic clue, but saving/reloading a vhost is not a memory-maintenance requirement.
|
|
|
|
The 0.25.0 development runtime adds two maintenance mechanisms that do not reload configuration:
|
|
|
|
- every 60 seconds on Linux/glibc, RustProxy calls `malloc_trim(0)` from a blocking maintenance task; this can return allocator pages that are already free, but cannot and does not evict live Rust objects;
|
|
- HOT cache maintenance trims only least-recently-used entries that have been idle for the configured delay and only while HOT allocation is above the configured target waterline. The pass is bounded by `hot_eviction_max_objects`; recent/active entries are not purged.
|
|
|
|
Upstream HTTP clients now also have bounded generations. Each backend Hyper client is recycled after 25,000 requests or 3,600 seconds by default (both limits are configurable under vhost keepalive and either can be disabled with `0`). Recycling replaces only the backend client/pool and leaves the vhost snapshot, HOT cache and configuration untouched. HTTP/1 upstream read buffers are capped at 128 KiB by default. This bounds long-lived Hyper/TLS/pool high-water allocations while preserving normal keep-alive reuse inside each generation. The existing 60-second allocator trim can then return already-free glibc pages to the OS.
|
|
|
|
## Hot and cold tiers
|
|
|
|
HOT RAM is bounded by allocated/charged `max_bytes` and `max_entries`; `hot_logical_bytes` separately exposes payload length. Fill and COLD handoff/read memory are bounded by `transient_max_bytes`, and queued demotions also obey `cold_queue_max_bytes`. Optional cold storage demotes LRU
|
|
or sufficiently idle hot objects asynchronously. Cold hits verify the body digest and
|
|
promote the object back to RAM. Version 0.11.5 uses cold metadata format 2 to persist
|
|
Vary identity and resident age; old-format metadata is not reused.
|
|
|
|
Cold is not required for normal hot-cache stability. With cold disabled, hot objects leave
|
|
RAM because of expiry, hot capacity/LRU pressure, explicit purge, configuration generation
|
|
changes, replacement of the same logical representation, or the optional HOT auto-trim
|
|
waterline. Auto-trim is not a blind TTL: it acts only when HOT is above its configured target
|
|
and only on entries idle longer than the configured threshold; it can be disabled independently.
|
|
|
|
## Cache panel and API
|
|
|
|
The Cache panel shows hot/cold object counts, RAM/disk usage, HIT/MISS/BYPASS, hit ratio,
|
|
fill pressure, evictions/expiry, per-domain decisions and stored objects. Live polling
|
|
refreshes the view; it does not itself purge cache objects.
|
|
|
|
`GET /api/cache?offset=0&limit=50` returns aggregate and object data. Add `vhost=ID` to
|
|
filter. `POST /api/cache/purge?vhost=ID` invalidates one vhost; omitting the query purges
|
|
all cache objects. Mutations remain authenticated, CSRF-protected and audited.
|
|
|
|
Hit ratio is `HIT / (HIT + MISS)`. BYPASS requests are excluded, so a large number of
|
|
cookie/authenticated requests can reduce backend offload without mathematically lowering
|
|
that displayed ratio. For the highest safe reusable-object ratio, give static assets
|
|
long explicit public TTLs, keep their URLs stable/immutable, leave the static-cookie
|
|
optimization enabled when appropriate, and avoid unnecessary `Vary` fields.
|