RustProxy/docs/IMPLEMENTATION_0.31.5.md

RustProxy 0.31.5 implementation and verification

Scope

0.31.5 adds an operator workflow from traffic/log inspection to a bounded User-Agent blocklist without changing SQLite schema 9 or existing saved configuration requirements.

User-Agent blocking

user_agent_blocking is available globally and on each vhost. Both levels use the same bounded policy:

• disabled by default;
• at most 128 patterns, 256 bytes per pattern and 16 KiB total pattern text;
• case-insensitive ASCII matching;
• a pattern without * matches a substring;
• * is a wildcard and may be used at the start, middle or end;
• a request with no User-Agent header is not denied by this rule;
• global and per-vhost policies are cumulative.

The check runs after vhost lookup but before redirects, cache/origin processing and before an upstream is opened. A match returns HTTP 403. The global policy is edited under Settings -> Security; the vhost policy is edited under Vhosts -> Security.

This is an operator blocklist, not a browser identity/authentication mechanism. User-Agent values are client-controlled and can be changed trivially.

Traffic copy workflow

Traffic dimension labels are rendered as copy controls. The copied value is the raw displayed dimension value rather than a formatted count, so IP addresses, User-Agent values, paths, hosts, methods, status values and other traffic dimensions can be pasted directly into investigation notes or configuration fields.

Stored Logs repeated-value aggregation

GET /api/file-logs/query accepts optional aggregate_field, aggregate_min_count and aggregate_limit parameters. Aggregation is computed across matching records in the same bounded scan used by Stored Logs, before page offset/limit are applied.

The field may be a canonical field such as user_agent, client_ip, host or path, or another parsed/nested field accepted by the Stored Logs field resolver. Scalar strings, booleans and numbers are counted; bounded array scalar values are counted individually; objects are not flattened into invented text values.

Safety limits are retained: the scan remains limited to 8 MiB/20,000 lines/32 sources, aggregate output is at most 200 rows, and aggregate cardinality is capped at 4,096 distinct values. The response reports when aggregation was truncated by those limits.

Consolidated per-vhost Stored Logs

Each per-vhost active JSONL file and matching .jsonl.*.gz rotation receives the same logical group_id, for example vhost:example.com. Individual physical files still have unique source IDs and metadata.

When consolidate_vhosts=true and source=vhost:<id>, the query scans all discovered members in that group subject to the existing global source/byte/line bounds. The panel enables this behavior by default and presents one logical vhost option with the member count and summed discovered size. Turning consolidation off exposes the physical files separately again.

Combined access/error logs keep their existing source behavior; this switch only changes per-vhost grouping.

Compatibility and security

• Existing configuration files deserialize with user_agent_blocking disabled through serde defaults.
• No SQLite migration or cache-format migration is introduced.
• Stored Logs still accept server-generated source IDs only, never arbitrary filesystem paths.
• Existing symlink rejection, gzip input/output caps and blocking-worker isolation remain in force.
• User-Agent denial occurs before origin contact, so a blocked request cannot cause a new upstream connection.

Verification status

Rust-independent checks cover language packs, JavaScript syntax/UI source contracts, OpenAPI/source-version consistency, Stored Logs grouping/aggregation source contracts and existing migration/release smoke tests. The source also contains Rust unit/regression tests for User-Agent matching and Stored Logs aggregation/grouping.

The validation environment used for this source package does not contain Rust/Cargo. Therefore cargo fmt, cargo clippy, Rust compilation and Rust unit/integration execution remain a release gate. Run ./dev.sh check and ./dev.sh test on the supported Rust toolchain before production deployment, then verify real HTTP 403 behavior, vhost/global precedence, clipboard actions and gzip consolidation against representative production logs.