# `/convert` shared cache — RustProxy 0.29.3 and Varnish

Only fully downloaded, decoded and converted lists return a cacheable HTTP 200.
All errors (400, 413, 415, 422, 429, 502, etc.) return `Cache-Control: no-store`.
Authorized debug responses (`?debug=1`) are never shared or cached.

## Example successful response

```http
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 12345
ETag: "<sha256 of converted response bytes>"
Cache-Control: public, max-age=3600, s-maxage=604800, stale-while-revalidate=86400, stale-if-error=604800
X-Content-Type-Options: nosniff
```

- `max-age`: cache freshness for browsers/clients (default 1 hour).
- `s-maxage`: freshness for shared caches such as RustProxy/Varnish (default 7 days).
- `stale-while-revalidate`: permits serving a stored stale object while the proxy refreshes it (1 additional day by default; cache support required).
- `stale-if-error`: permits serving an already-stored stale object if refresh fails (7 additional days by default; cache support required).
- `ETag`: SHA-256 of the **converted bytes**, not the upstream file ETag. Identical converted bodies have matching ETags. `If-None-Match` can produce an HTTP 304, but only after the application checks the complete source on a backend request. On a normal proxy HIT, no application work is performed.
- The upstream source is always fetched as an unconditional GET. Client validators must not be sent to the **source list** because they refer to the **converted** resource.
- GET and HEAD use the same validated body, ETag and length; HEAD does not send response bytes.

Environment variables in `/var/www/adlist_mikrotik/.env`:

```dotenv
CACHE_ENABLED=true
CACHE_MAX_AGE=3600
CACHE_S_MAXAGE=604800
CACHE_STALE_WHILE_REVALIDATE=86400
CACHE_STALE_IF_ERROR=604800
```

Existing `.env` files override `config.py` defaults, so update the actual deployed `.env` after upgrading. Restart the `listapp` service to apply changes.

## RustProxy (important: origin headers alone cannot override BYPASS)

1. Enable the global cache and caching on the `listapp.linuxiarz.pl` vhost.
2. Set `max_ttl_seconds=604800` globally and/or on that vhost. RustProxy 0.29.3 has a **default cap of 86400 seconds**, and an allowed maximum of **604800 seconds**. Longer origin `s-maxage` values cannot override the cap.
3. Set `bypass_query=false`: `/convert` **requires** a query string (`url=...&ip=...`). Check `bypass_prefixes` to make sure `/convert` is not excluded.
4. Allow sufficient `max_object_bytes` for the **converted result**, not the source. The RustProxy default is 16 MiB; objects larger than the limit will not be stored. If necessary, increase the total HOT/transient budgets too.
5. Optionally enable `cold_enabled=true` and size COLD appropriately, so cached objects can survive RAM eviction and regular restarts. COLD is still bounded and not permanent storage.
6. Requests with `Cookie`, `Authorization`, `Cache-Control: no-store`, or various forced revalidation directives may BYPASS / revalidate. Normal unauthenticated periodic scans without these request headers are best for high cache hit rate. If you configure an exception, scope it only to public `/convert` requests and consider the security implications.
7. Enable cache diagnostics `expose_status`, `expose_cache_status`, and optionally `expose_resident_age` to inspect MISS/HIT/BYPASS and reason codes.

A long TTL does not guarantee physical retention: eviction, explicit purge, size limits and settings may remove entries. Source updates are not visible to normal cache users until freshness expires or the cache is purged; choose TTL accordingly.

## Varnish

Varnish can use `Cache-Control: public, s-maxage=...` (subject to VCL policy, cookies, Authorization, cache size and eviction). It must be configured to pass/cache the `/convert` route appropriately. The application does not use an internal Redis response-body cache; the shared proxy stores successful responses.

## Verification

Use the *same full URL* repeatedly, with GET (not HEAD), without cookies or request `Cache-Control: no-cache`. For example:

```bash
BASE='https://listapp.linuxiarz.pl/convert'
for n in 1 2 3; do
  echo "Request $n"
  curl -sS --get \
    --data-urlencode 'url=https://v.firebog.net/hosts/RPiList-Malware.txt' \
    --data-urlencode 'ip=0.0.0.0' \
    -D - -o /dev/null "$BASE" | \
    grep -Ei '^(HTTP/|cache-control:|etag:|age:|x-cache:|cache-status:|content-length:|content-type:)'
  sleep 2
done
```

Expect `HTTP 200` and the `public, s-maxage` header from the application. If RustProxy caching is eligible, a typical sequence is `MISS`, `HIT`, `HIT`, while `Age` increases. `BYPASS` means the proxy did not even attempt a cache fill (check request and proxy configuration); `MISS` followed by repeated `MISS` can mean oversize objects, excluded responses or aborted body transfer. These labels require corresponding diagnostics to be enabled on the proxy.

Run local code checks:

```bash
python3 -m unittest discover -s tests -p 'test_*.py' -v
python3 tests/smoke_cache_routes.py
```

## Duplicate Date and RustProxy 0.29.3

`/convert` deliberately does not set its own `Date` header after
`Response.make_conditional()`. Werkzeug can create a `Date` header there;
Uvicorn also supplies one, and RustProxy 0.29.3 rejects multiple `Date`
headers with `response_ttl_size_or_metadata`.

Verify that **exactly one** `Date` header is returned directly by the
ASGI backend (no reverse proxy in the path). All success responses should
also include a valid `Content-Length`, `ETag` and public `Cache-Control`.
For a full seven days, set RustProxy global `max_ttl_seconds` to `604800`
AND configure the app's `CACHE_S_MAXAGE=604800` in `.env`.
The `unsafe_force` cache switch is not required for `/convert`.