v0.26.2
This commit is contained in:
Mateusz Gruszczyński committed
1 parent
b9d137eee1
commit
32e6875980
51 files changed
+1255
-452
No files matched your search
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Administration API access - 0.26.1
|
||||
# Administration API access - 0.26.2
|
||||
|
||||
## Sign-in boundary
|
||||
|
||||
|
||||
+56
-65
@@ -1,87 +1,78 @@
|
||||
# Stored log explorer
|
||||
|
||||
RustProxy 0.26.1 provides an authenticated, read-only **Logs** workspace at `/logs`. It is a local file-backed explorer for RustProxy logging outputs and is intentionally separate from **Live Logs**.
|
||||
RustProxy 0.26.2 provides an authenticated, read-only **Logs** workspace at `/logs`. The top bar has one **Logs** menu with separate **Stored logs** and **Live Logs** actions; the two data paths remain independent.
|
||||
|
||||
## Separation from Live Logs
|
||||
|
||||
Live Logs uses the bounded in-memory request history and Tokio broadcast channel. It remains available even when file logging is disabled and is optimized for current routed request activity.
|
||||
|
||||
The stored-log explorer does not read that memory ring. It reads only files derived from the active RustProxy logging configuration:
|
||||
Stored Logs reads only paths derived from the active RustProxy logging configuration: the combined structured access log or per-vhost JSONL directory plus the optional error log. Existing files remain browsable after their writer is disabled. Low-level `protocol_error` records remain persistence-only and do not enter Live Logs.
|
||||
|
||||
- the combined structured access-log path, or JSONL files directly inside the configured per-vhost directory;
|
||||
- the configured optional error-log file.
|
||||
## Search syntax
|
||||
|
||||
Existing files remain browsable after their writer is disabled. Low-level `protocol_error` records are sent only through configured persistent structured-log sinks and are deliberately not inserted into Live Logs.
|
||||
The dashboard uses one search field and a page-size selector. Search is case-insensitive and supports field clauses, negation, quoted values, wildcards and numeric comparisons.
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
req: GET AND !country: Poland
|
||||
status:>=500 AND path:/api/*
|
||||
error:"connection reset"
|
||||
client_ip:192.0.2.* AND status:5xx
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- whitespace and explicit `AND` join clauses;
|
||||
- `!field:value` excludes matching records;
|
||||
- quotes keep spaces in one value;
|
||||
- `*` acts as a wildcard;
|
||||
- numeric fields accept `>`, `>=`, `<` and `<=`;
|
||||
- `status:5xx` matches a status class;
|
||||
- bare terms search all stored values;
|
||||
- `req`/`request` aliases `method`, `ip`/`client` aliases `client_ip`, and `source` aliases the internal log-source ID;
|
||||
- `OR` is not supported and is rejected explicitly.
|
||||
|
||||
The API continues to accept the older `source`, `event`, `status_class`, `client_ip` and `vhost` parameters for compatibility, but the dashboard no longer exposes those separate controls.
|
||||
|
||||
## JSON parsing and dynamic columns
|
||||
|
||||
JSON/JSONL lines are parsed into record objects. Canonical access-log fields are reconstructed from the active JSON field mapping where possible, so fields such as `client_ip` remain usable even when their output key was renamed.
|
||||
|
||||
The query response includes a bounded list of discovered leaf field names. The browser builds the column picker from those fields. The default concise view is time, event, status, client IP, vhost, record summary and source; any returned JSON field can be selected dynamically. Fields such as `matched_by`, `sni`, `authority`, `country`, `request_id` or custom mapped fields therefore appear only when the data actually contains them.
|
||||
|
||||
At least one column must remain visible. Stored-log column selection is saved per administrator account and accepts up to 64 bounded field names.
|
||||
|
||||
Selecting a row highlights it and expands parsed field/value pairs directly below the row. A nested **Raw JSON** section shows the normalized complete record.
|
||||
|
||||
## RAW error logs
|
||||
|
||||
RustProxy's built-in RAW error-log format is also normalized for browsing. The explorer extracts timestamp, level, target, message and recognizable `key=value` fields, while retaining the original line in `raw`.
|
||||
|
||||
RAW is not a lossless structured serialization, so extraction is best-effort. Fields that cannot be separated safely remain visible through the message/raw record rather than being invented.
|
||||
|
||||
## Vhost-aware protocol records
|
||||
|
||||
Ordinary completed requests use `event: "request"`. Low-level failures use `event: "protocol_error"`. When RustProxy can safely identify a configured vhost, the record is persisted through that vhost's existing sink. In per-vhost file mode an unresolved record uses `_unmatched.jsonl`.
|
||||
|
||||
Attribution priority is:
|
||||
|
||||
1. vhost already selected by normal routing;
|
||||
2. HTTP `:authority`;
|
||||
3. HTTP `Host`;
|
||||
4. TLS/QUIC SNI;
|
||||
5. no safe match -> `_unmatched`.
|
||||
|
||||
All hostname matching reuses the normal exact/wildcard frontend resolver and listener binding rules. A listener or local port by itself is not sufficient to select a vhost.
|
||||
|
||||
A representative record is:
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "protocol_error",
|
||||
"reason": "tls_handshake_error",
|
||||
"protocol_error": "tls_handshake_error",
|
||||
"error": "...",
|
||||
"error_detail": "...",
|
||||
"client_ip": "192.0.2.10",
|
||||
"local_addr": "192.0.2.20:443",
|
||||
"transport": "tls",
|
||||
"vhost": "app",
|
||||
"sni": "app.example",
|
||||
"tls_sni": "app.example",
|
||||
"authority": null,
|
||||
"matched_by": "sni",
|
||||
"duration_ms": 0
|
||||
}
|
||||
```
|
||||
|
||||
`matched_by` is `router`, `authority`, `host`, `sni` or `none`. Compatibility fields from 0.26.0 remain present so existing consumers do not need an immediate migration.
|
||||
|
||||
SNI, authority and Host are lookup inputs only. RustProxy never derives a filename directly from client-provided identity text. Only the configured internal vhost identifier returned by the shared resolver can select a vhost sink. Invalid internal IDs fall back to `_unmatched` rather than being sanitized into a new path.
|
||||
|
||||
Diagnostic identity values are control-character stripped and bounded before persistence. Existing error-detail redaction and limits remain in force. Raw packets, request bodies and raw malformed request bytes are not captured.
|
||||
|
||||
## Logs Explorer UI
|
||||
|
||||
The workspace supports source, case-insensitive text, event class, HTTP status class, client IP and vhost filters. Text search scans stored values, including protocol diagnostics such as `reason`, `error`, `sni` and `authority`, and refreshes with a short debounce while typing.
|
||||
|
||||
The table provides account-scoped selectable columns. The default view keeps the original concise columns while optional protocol columns expose transport, match source, SNI and authority. At least one column must remain visible.
|
||||
|
||||
The table header stays visible while the result list scrolls. Selecting a row highlights it and expands the complete JSON record inline immediately below that row; selecting it again collapses the details.
|
||||
|
||||
The configured-file inventory is collapsed by default so installations with many per-vhost files do not lose most of the screen to source paths. The source selector still contains the configured sources, including `_unmatched.jsonl` when present.
|
||||
|
||||
For structured access logs, the explorer reconstructs canonical fields from the active JSON field mapping. In particular, `client_ip` remains available to the table and filter even when the output JSON key was renamed by a custom mapping.
|
||||
Attribution priority remains routed vhost, HTTP `:authority`, HTTP `Host`, TLS/QUIC SNI, then unmatched. Matching reuses normal exact/wildcard frontend resolution and listener binding. Client Host/SNI text never becomes a filesystem path.
|
||||
|
||||
## Browser security and bounds
|
||||
|
||||
The browser never supplies a filesystem path. `/api/file-logs` returns opaque source IDs derived from the current configuration, and `/api/file-logs/query` accepts only those IDs or `all`.
|
||||
The dashboard no longer exposes the configured-file inventory. The Source selector is optional: `All configured files` is the default, while selecting one source narrows the same bounded query to that log only.
|
||||
|
||||
The reader is bounded:
|
||||
The browser still never supplies a filesystem path. The server derives sources from the active configuration and reads only regular files. On Unix it opens with `O_NOFOLLOW`; per-vhost enumeration rejects a symlinked directory and accepts only direct regular `.jsonl` children.
|
||||
|
||||
- up to 2 MiB scanned from the tail of one source;
|
||||
The reader remains bounded:
|
||||
|
||||
- up to 2 MiB scanned from one source;
|
||||
- up to 8 MiB across one query;
|
||||
- up to 20,000 lines across the complete query, shared across selected sources;
|
||||
- at most 32 sources in an `all` query;
|
||||
- page size at most 500 and offset at most 10,000.
|
||||
- up to 20,000 lines across a query;
|
||||
- at most 32 sources in an all-source query;
|
||||
- page size at most 500 and offset at most 10,000;
|
||||
- search input at most 512 bytes and 32 parsed clauses;
|
||||
- dynamic field discovery at most 64 leaf fields.
|
||||
|
||||
If more than 32 configured files are eligible for an `all` query, the newest 32 are scanned and the response is marked truncated. A specific source can always be selected directly.
|
||||
If more than 32 sources are eligible, the newest 32 are scanned and the response is marked truncated. File reads run in a blocking worker rather than the async proxy runtime.
|
||||
|
||||
File reads run in a blocking worker rather than the async proxy runtime. Only regular files are accepted. On Unix the reader opens with `O_NOFOLLOW`; a symlink is not followed. Per-vhost enumeration also rejects a configured directory that is itself a symlink and only lists direct regular `.jsonl` children.
|
||||
|
||||
The endpoints are private administration routes and inherit the existing panel session/Host/Origin access controls. There is no public-proxy route to the file reader.
|
||||
|
||||
The explorer is diagnostic rather than a retention system. RustProxy does not add rollover, indexing or compression; use the operating-system log rotation policy for retention.
|
||||
The explorer is diagnostic rather than a retention/indexing system. RustProxy does not add rollover, indexing or compression; use operating-system log rotation for retention.
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Installing and updating RustProxy 0.26.1
|
||||
# Installing and updating RustProxy 0.26.2
|
||||
|
||||
The scripts prepare a Linux/systemd installation. This source archive is **not a
|
||||
verified production build**: Rust/Cargo, real CA operations, live systemd startup
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Deployment and configuration migration history
|
||||
|
||||
## 0.26.1 -> 0.26.2
|
||||
|
||||
No SQLite or saved-configuration migration is required. Version 0.26.2 changes only administration/log-explorer behavior and account-scoped view preferences. Existing JSONL and RAW log files remain readable, and old Stored Logs filter query parameters remain accepted by the API for compatibility even though the dashboard now uses one query field.
|
||||
|
||||
## 0.26.0 -> 0.26.1
|
||||
|
||||
No SQLite or saved-configuration migration is required. Version 0.26.1 changes persistent protocol-error attribution and the Stored Logs UI/API normalization only. Existing JSONL files remain readable, Live Logs behavior is unchanged, and unresolved protocol errors continue to use `_unmatched.jsonl`.
|
||||
|
||||
+3
-2
@@ -15,7 +15,8 @@ This is the canonical index for the current project documentation. The authentic
|
||||
- [0.25.1 release audit](V0.25.1_AUDIT.md) - historical Traffic/module-split release scope and compiler-regression guards.
|
||||
- [0.25.0 specification audit](V0.25.0_AUDIT.md) - Web Server foundation point-by-point implementation status.
|
||||
- [0.25.0 pre-production security audit](V0.25.0_SECURITY_PREPROD_AUDIT.md) - threat-oriented source/configuration review, fixes made in-session and mandatory runtime security gates.
|
||||
- [0.26.1 implementation note](V0.26.1_IMPLEMENTATION.md) - vhost-aware protocol logging, shared resolver and Stored Logs UX changes.
|
||||
- [0.26.2 implementation note](V0.26.2_IMPLEMENTATION.md) - unified Logs navigation, field-query search, dynamic JSON columns and RAW parsing.
|
||||
- [0.26.1 implementation note](V0.26.1_IMPLEMENTATION.md) - historical vhost-aware protocol logging and Stored Logs UX milestone.
|
||||
- [TEST_REPORT](TEST_REPORT.md) - verification commands, results and environment limitations.
|
||||
|
||||
## Functional documentation
|
||||
@@ -61,7 +62,7 @@ This is the canonical index for the current project documentation. The authentic
|
||||
|
||||
## Historical or version-labelled context
|
||||
|
||||
Version-labelled `V*.md` files are retained as historical implementation/audit records. Their version number describes the audited milestone; they are not current-release documentation. The current release implementation note is [V0.26.1_IMPLEMENTATION.md](V0.26.1_IMPLEMENTATION.md); the latest retained audit is [V0.25.5_AUDIT.md](V0.25.5_AUDIT.md); [V0.25.4_AUDIT.md](V0.25.4_AUDIT.md) records the prior retained-history restore milestone and [V0.25.3_AUDIT.md](V0.25.3_AUDIT.md) records the prior request-policy/history milestone and [V0.25.2_AUDIT.md](V0.25.2_AUDIT.md) records the prior backup/session hardening milestone and [V0.25.1_AUDIT.md](V0.25.1_AUDIT.md) records the prior Traffic/module-split milestone and [V0.25.0_AUDIT.md](V0.25.0_AUDIT.md) remains the Web Server foundation audit.
|
||||
Version-labelled `V*.md` files are retained as historical implementation/audit records. Their version number describes the audited milestone; they are not current-release documentation. The current release implementation note is [V0.26.2_IMPLEMENTATION.md](V0.26.2_IMPLEMENTATION.md); [V0.26.1_IMPLEMENTATION.md](V0.26.1_IMPLEMENTATION.md) is retained as the prior Stored Logs/protocol-attribution milestone; the latest retained audit is [V0.25.5_AUDIT.md](V0.25.5_AUDIT.md); [V0.25.4_AUDIT.md](V0.25.4_AUDIT.md) records the prior retained-history restore milestone and [V0.25.3_AUDIT.md](V0.25.3_AUDIT.md) records the prior request-policy/history milestone and [V0.25.2_AUDIT.md](V0.25.2_AUDIT.md) records the prior backup/session hardening milestone and [V0.25.1_AUDIT.md](V0.25.1_AUDIT.md) records the prior Traffic/module-split milestone and [V0.25.0_AUDIT.md](V0.25.0_AUDIT.md) remains the Web Server foundation audit.
|
||||
|
||||
Older context documents such as [INTEGRATIONS_040](INTEGRATIONS_040.md), [MANAGEMENT](MANAGEMENT.md), [SECURITY_AUDIT](SECURITY_AUDIT.md), [ACME_RFC2136](ACME_RFC2136.md) and older version audits remain available for traceability but should not override the current functional/technical documents above.
|
||||
|
||||
|
||||
+7
-5
@@ -1,16 +1,18 @@
|
||||
# Verification report - RustProxy 0.26.1
|
||||
# Verification report - RustProxy 0.26.2
|
||||
|
||||
## Scope
|
||||
|
||||
RustProxy 0.26.1 extends the persistent protocol diagnostics introduced in 0.26.0 with safe per-vhost attribution and updates the Stored Logs explorer. `protocol_error` records can now carry canonical `reason`, `error`, `sni`, `authority`, `matched_by` and resolved `vhost` diagnostics while retaining the 0.26.0 compatibility fields. Attribution reuses the normal exact/wildcard frontend resolver and never derives a filename directly from client Host/SNI input. Unresolved traffic remains `_unmatched` in per-vhost file mode, and protocol errors continue to bypass the Live Logs history/broadcast path.
|
||||
RustProxy 0.26.2 keeps the 0.26.1 vhost-aware persistent protocol diagnostics and focuses on the administration log workflow. Stored Logs and Live Logs now share one top-bar Logs menu while retaining independent data paths.
|
||||
|
||||
Stored Logs now supports account-scoped selectable columns, sticky headers, highlighted rows with inline JSON details, a collapsed configured-file inventory, custom-field recovery for canonical `client_ip`, and value-based keyword search including protocol diagnostics.
|
||||
Stored Logs replaces the fixed event/status/client/vhost controls with an optional source selector, one bounded field-query input and page size. The query parser supports `field:value`, explicit `AND`, `!` negation, quoted values, wildcards, numeric comparisons and aliases such as `req` -> `method`. JSON/JSONL records expose a bounded discovered-field list for dynamic columns; protocol-specific fields such as `matched_by` and `sni` are no longer hard-coded table columns. RAW error-log lines are best-effort parsed into timestamp, level, target, message and detected `key=value` fields while preserving `raw`.
|
||||
|
||||
The dashboard removes the Configured files inventory but keeps an optional source selector with `All configured files` as the default, moves the column picker to the table toolbar, expands selected rows as parsed fields plus optional raw JSON, and uses `↑`/`↓` for backend sort direction. SQLite schema 8 and the saved proxy configuration format remain unchanged.
|
||||
|
||||
## Source-level regression coverage
|
||||
|
||||
0.26.1-specific Rust tests cover shared logging-vhost resolution and precedence, authority/Host normalization, unknown/missing SNI, safe vhost sink names, bounded protocol identity fields, persistent protocol fields under custom JSON mappings, Live Logs separation, Stored Logs canonical client-IP recovery and protocol-diagnostic text matching. Administration preference tests cover the new Stored Logs column list and backward-compatible defaults.
|
||||
0.26.2-specific Rust tests cover field aliases, explicit AND, negation, quoted values, wildcard/status/numeric matching and built-in RAW error parsing. Retained 0.26.1 tests continue to cover canonical field recovery, safe source derivation, symlink refusal, bounded reads and protocol diagnostic normalization.
|
||||
|
||||
The HTTP/TLS/H2/H3 sources carry the resolved identity through the persistent logging path. TLS ClientHello SNI is retained for later handshake and HTTP/2 failures; HTTP/3 reads the completed QUIC/Rustls handshake identity for later request/connection diagnostics. Failures before a usable identity exists remain unmatched.
|
||||
Administration preference validation now permits up to 64 bounded, control-free Stored Logs field names so dynamically discovered JSON fields can be persisted per account. Browser JavaScript is syntax-checked, and the language/documentation/source-contract suites cover the new UI strings and release identity.
|
||||
|
||||
## Verification status in this workspace
|
||||
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# RustProxy 0.26.2 implementation note
|
||||
|
||||
## Scope
|
||||
|
||||
Version 0.26.2 is a UI/query follow-up to the Stored Logs explorer. It does not change SQLite schema 8 or the saved proxy configuration format.
|
||||
|
||||
The release:
|
||||
|
||||
- combines Stored Logs and Live Logs under one top-bar Logs menu;
|
||||
- replaces the fixed Stored Logs filter matrix with an optional source selector, one bounded field-query input and a page-size selector;
|
||||
- discovers selectable table columns from parsed record fields;
|
||||
- parses RustProxy RAW error-log lines into basic fields;
|
||||
- removes the configured-file inventory while retaining the source selector as an optional index-like scope;
|
||||
- moves column configuration to the upper-right table toolbar;
|
||||
- uses arrow symbols for backend sort direction.
|
||||
|
||||
## Stored Logs query syntax
|
||||
|
||||
The `q` parameter remains bounded and now accepts clauses separated by whitespace or explicit `AND`.
|
||||
|
||||
Supported forms include:
|
||||
|
||||
```text
|
||||
req: GET AND !country: Poland
|
||||
status:>=500 AND path:/api/*
|
||||
error:"connection reset"
|
||||
```
|
||||
|
||||
`req`/`request` aliases `method`, `ip`/`client` aliases `client_ip`, and `source` aliases the internal source ID. A leading `!` negates a clause. Quoted text keeps spaces together. `*` is a wildcard. Numeric fields accept `>`, `>=`, `<` and `<=`; status values also accept forms such as `5xx`.
|
||||
|
||||
All clauses are ANDed. `OR` is deliberately rejected rather than being interpreted ambiguously. Existing `source`, `event`, `status_class`, `client_ip` and `vhost` query parameters remain accepted by the API for compatibility. The dashboard emits `source` only when one specific log is selected; `All configured files` leaves the source parameter unset.
|
||||
|
||||
## Parsed records and dynamic columns
|
||||
|
||||
JSON/JSONL lines are normalized as before, including recovery of canonical fields from configured JSON mappings. The query response also returns a bounded list of discovered leaf field names. The browser uses those names to build the column picker dynamically.
|
||||
|
||||
The default concise columns remain time, event, status, client IP, vhost, record summary and source. Protocol-specific fields such as `matched_by` and `sni` are no longer hard-coded columns; they appear automatically when present in returned JSON and can be selected like any other discovered field.
|
||||
|
||||
Stored-log column preferences remain account-scoped. Validation now accepts up to 64 bounded, control-free field names so discovered JSON fields can be persisted without widening any filesystem or proxy configuration authority.
|
||||
|
||||
## RAW error-log parsing
|
||||
|
||||
The built-in RAW error format is parsed into:
|
||||
|
||||
- `timestamp`;
|
||||
- `level`;
|
||||
- `target`;
|
||||
- message text before the first recognized `key=value` field;
|
||||
- subsequent detected `key=value` fields.
|
||||
|
||||
Space-separated text following a `key=value` token is attached to that field until another valid field token begins. Because RAW is intentionally not a fully self-describing serialization, the original line is retained in `raw` and remains available in the expanded record view.
|
||||
|
||||
## UI details
|
||||
|
||||
Selecting a row shows parsed field/value pairs inline and offers the normalized raw JSON in a collapsible section. Column configuration sits in the table toolbar instead of the filter row.
|
||||
|
||||
The previous Configured files block remains removed. Stored Logs keeps an optional Source selector as an index-like scope: `All configured files` is the default, and choosing one configured log limits the query to that source. The server continues to derive every readable path from active RustProxy logging configuration; the browser submits only a source ID, never a filesystem path.
|
||||
|
||||
Stored Logs and Live Logs remain technically independent. The combined Logs top-bar control changes navigation only; Live Logs still reads the bounded in-memory request history, while Stored Logs reads bounded tails of configured files.
|
||||
|
||||
## Security and bounds
|
||||
|
||||
Existing file-reader protections remain unchanged: regular-file checks, Unix `O_NOFOLLOW`, configured-source derivation, blocking-worker I/O, bounded scan bytes/lines/source count, bounded result offsets and page size.
|
||||
|
||||
Search parsing is bounded to 512 input bytes and at most 32 clauses. Field names and individual values are bounded before matching. Dynamic column discovery is capped at 64 leaf fields.
|
||||
|
||||
## Regression coverage
|
||||
|
||||
Source tests cover field aliases, negation, quoted values, status/numeric matching, wildcard matching and RAW error parsing in addition to the retained 0.26.1 stored-log normalization tests.
|
||||
|
||||
A Rust-enabled release host must still run `cargo fmt --check`, Clippy with warnings denied, all-target/all-feature tests and the complete `dev.sh` release build before the package is considered compiler-verified.
|
||||
Reference in new issue
Block a user