Stored log explorer

RustProxy 0.29.1 provides an authenticated, read-only Logs workspace at /logs. The top bar has one Logs menu with separate Stored logs and Live Logs actions; the two data paths remain independent.

Separation from Live Logs

Live Logs uses the bounded in-memory request history and Tokio broadcast channel. It remains available even when file logging is disabled and is optimized for current routed request activity.

Stored Logs reads only paths derived from the active RustProxy logging configuration: the combined structured access log or per-vhost JSONL directory plus the optional error log. Existing files remain browsable after their writer is disabled. Low-level protocol_error records remain persistence-only and do not enter Live Logs.

Search syntax

The dashboard uses one search field and a page-size selector. Search is case-insensitive and supports field clauses, negation, quoted values, wildcards and numeric comparisons.

Examples:

req: GET AND !country: Poland
status:>=500 AND path:/api/*
error:"connection reset"
client_ip:192.0.2.* AND status:5xx

Rules:

whitespace and explicit AND join clauses;
!field:value excludes matching records;
quotes keep spaces in one value;
* acts as a wildcard;
numeric fields accept >, >=, < and <=;
status:5xx matches a status class;
bare terms search all stored values;
req/request aliases method, ip/client aliases client_ip, and source aliases the internal log-source ID;
OR is not supported and is rejected explicitly.

The API continues to accept the older source, event, status_class, client_ip and vhost parameters for compatibility, but the dashboard no longer exposes those separate controls.

JSON parsing and dynamic columns

JSON/JSONL lines are parsed into record objects. Canonical access-log fields are reconstructed from the active JSON field mapping where possible, so fields such as client_ip remain usable even when their output key was renamed.

The query response includes a bounded list of discovered leaf field names. The browser builds the column picker from those fields. The default concise view is time, event, status, client IP, vhost, record summary and source; any returned JSON field can be selected dynamically. Fields such as matched_by, sni, authority, country, request_id or custom mapped fields therefore appear only when the data actually contains them.

At least one column must remain visible. Stored-log column selection is saved per administrator account and accepts up to 64 bounded field names.

Selecting a row highlights it and expands parsed field/value pairs directly below the row. A nested Raw JSON section shows the normalized complete record.

RAW error logs

RustProxy's built-in RAW error-log format is also normalized for browsing. The explorer extracts timestamp, level, target, message and recognizable key=value fields, while retaining the original line in raw.

RAW is not a lossless structured serialization, so extraction is best-effort. Fields that cannot be separated safely remain visible through the message/raw record rather than being invented.

Vhost-aware protocol records

Ordinary completed requests use event: "request". Low-level failures use event: "protocol_error". When RustProxy can safely identify a configured vhost, the record is persisted through that vhost's existing sink. In per-vhost file mode an unresolved record uses _unmatched.jsonl.

Attribution priority remains routed vhost, HTTP :authority, HTTP Host, TLS/QUIC SNI, then unmatched. Matching reuses normal exact/wildcard frontend resolution and listener binding. Client Host/SNI text never becomes a filesystem path.

Browser security and bounds

The dashboard no longer exposes the configured-file inventory. The Source selector is optional: All configured files is the default, while selecting one source narrows the same bounded query to that log only.

The browser still never supplies a filesystem path. The server derives sources from the active configuration and reads only regular files. On Unix it opens with O_NOFOLLOW; per-vhost enumeration rejects a symlinked directory and accepts only direct regular .jsonl children.

Logrotate-style gzip rotations are discovered next to configured combined access/error files and inside the configured per-vhost directory. Only regular non-symlink files matching the active filename rotation pattern (for example access.jsonl.1.gz) are eligible. The browser still refers to them by server-generated source IDs rather than paths.

Gzip reading is bounded before and after decompression. A compressed source larger than 8 MiB is not decompressed for a query, decompressed output is capped at 64 MiB, and only the bounded tail is parsed/searched. These limits prevent a gzip rotation from becoming an unbounded memory/decompression workload.

The reader remains bounded:

up to 2 MiB of retained tail data from one source;
up to 8 MiB across one query;
up to 20,000 lines across a query;
at most 32 sources in an all-source query;
page size at most 500 and offset at most 10,000;
search input at most 512 bytes and 32 parsed clauses;
dynamic field discovery at most 64 leaf fields.

If more than 32 sources are eligible, the newest 32 are scanned and the response is marked truncated. File reads run in a blocking worker rather than the async proxy runtime.

The explorer is diagnostic rather than a retention/indexing system. RustProxy does not create rollovers or compress files itself; use operating-system log rotation for retention. Existing gzip rotations can be read as described above.

5.6 KiB Raw Blame History