56 lines
4.3 KiB
Markdown
56 lines
4.3 KiB
Markdown
# RustProxy 0.31.5 implementation and verification
|
|
|
|
## Scope
|
|
|
|
0.31.5 adds an operator workflow from traffic/log inspection to a bounded User-Agent blocklist without changing SQLite schema 9 or existing saved configuration requirements.
|
|
|
|
## User-Agent blocking
|
|
|
|
`user_agent_blocking` is available globally and on each vhost. Both levels use the same bounded policy:
|
|
|
|
- disabled by default;
|
|
- at most 128 patterns, 256 bytes per pattern and 16 KiB total pattern text;
|
|
- case-insensitive ASCII matching;
|
|
- a pattern without `*` matches a substring;
|
|
- `*` is a wildcard and may be used at the start, middle or end;
|
|
- a request with no `User-Agent` header is not denied by this rule;
|
|
- global and per-vhost policies are cumulative.
|
|
|
|
The check runs after vhost lookup but before redirects, cache/origin processing and before an upstream is opened. A match returns HTTP 403. The global policy is edited under **Settings -> Security**; the vhost policy is edited under **Vhosts -> Security**.
|
|
|
|
This is an operator blocklist, not a browser identity/authentication mechanism. User-Agent values are client-controlled and can be changed trivially.
|
|
|
|
## Traffic copy workflow
|
|
|
|
Traffic dimension labels are rendered as copy controls. The copied value is the raw displayed dimension value rather than a formatted count, so IP addresses, User-Agent values, paths, hosts, methods, status values and other traffic dimensions can be pasted directly into investigation notes or configuration fields.
|
|
|
|
## Stored Logs repeated-value aggregation
|
|
|
|
`GET /api/file-logs/query` accepts optional `aggregate_field`, `aggregate_min_count` and `aggregate_limit` parameters. Aggregation is computed across matching records in the same bounded scan used by Stored Logs, before page offset/limit are applied.
|
|
|
|
The field may be a canonical field such as `user_agent`, `client_ip`, `host` or `path`, or another parsed/nested field accepted by the Stored Logs field resolver. Scalar strings, booleans and numbers are counted; bounded array scalar values are counted individually; objects are not flattened into invented text values.
|
|
|
|
Safety limits are retained: the scan remains limited to 8 MiB/20,000 lines/32 sources, aggregate output is at most 200 rows, and aggregate cardinality is capped at 4,096 distinct values. The response reports when aggregation was truncated by those limits.
|
|
|
|
## Consolidated per-vhost Stored Logs
|
|
|
|
Each per-vhost active JSONL file and matching `.jsonl.*.gz` rotation receives the same logical `group_id`, for example `vhost:example.com`. Individual physical files still have unique source IDs and metadata.
|
|
|
|
When `consolidate_vhosts=true` and `source=vhost:<id>`, the query scans all discovered members in that group subject to the existing global source/byte/line bounds. The panel enables this behavior by default and presents one logical vhost option with the member count and summed discovered size. Turning consolidation off exposes the physical files separately again.
|
|
|
|
Combined access/error logs keep their existing source behavior; this switch only changes per-vhost grouping.
|
|
|
|
## Compatibility and security
|
|
|
|
- Existing configuration files deserialize with `user_agent_blocking` disabled through serde defaults.
|
|
- No SQLite migration or cache-format migration is introduced.
|
|
- Stored Logs still accept server-generated source IDs only, never arbitrary filesystem paths.
|
|
- Existing symlink rejection, gzip input/output caps and blocking-worker isolation remain in force.
|
|
- User-Agent denial occurs before origin contact, so a blocked request cannot cause a new upstream connection.
|
|
|
|
## Verification status
|
|
|
|
Rust-independent checks cover language packs, JavaScript syntax/UI source contracts, OpenAPI/source-version consistency, Stored Logs grouping/aggregation source contracts and existing migration/release smoke tests. The source also contains Rust unit/regression tests for User-Agent matching and Stored Logs aggregation/grouping.
|
|
|
|
The validation environment used for this source package does not contain Rust/Cargo. Therefore `cargo fmt`, `cargo clippy`, Rust compilation and Rust unit/integration execution remain a release gate. Run `./dev.sh check` and `./dev.sh test` on the supported Rust toolchain before production deployment, then verify real HTTP 403 behavior, vhost/global precedence, clipboard actions and gzip consolidation against representative production logs.
|