RustProxy/ROADMAP.md

# RustProxy roadmap

## 0.27.0 - Compression policy and persistent Docker publications

Current direction: 0.26.7 -> 0.27.0

- [x] Enable zstd/Brotli/gzip downstream compression by default, keep deflate opt-in, honor client q-values and use deterministic `zstd -> br -> gzip -> deflate` tie priority.
- [x] Allow authenticated/cookie/private/no-store responses to compress while retaining explicit no-transform/range/SSE/upgrade/existing-encoding exclusions.
- [x] Add inherited dynamic-compression overrides for vhosts, Docker publications and Web Server services, including per-scope codec filters under the global codec ceiling; hide/disable codec controls while a scope is explicitly OFF without clearing saved overrides.
- [x] Default fresh installations to `Server: RustProxy` without rewriting existing saved policies.
- [x] Remove Debian package-build metadata from this source tree; packaging is maintained separately at `rustproxy.linuxiarz.pl`.
- [x] Keep published Docker hostnames materialized while a service is missing, return 503 until it returns, and reuse last-known inventory during temporary agent communication failures.
- [x] Render cache occupancy progress fill consistently across Chromium/WebKit and Firefox.
- [x] Keep SQLite schema 8 and additive/defaulted saved configuration compatibility with 0.26.x.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.7 - Cache occupancy and backend rotation continuity

Current direction: 0.26.6 -> 0.26.7

- [x] Show HOT/COLD cache occupancy progress against configured entry and byte limits on `/cache`.
- [x] Preserve compatible upstream client generations across unrelated runtime snapshot rebuilds so rotation age/counters remain continuous.
- [x] Keep request-count and age-based backend client recycling covered by tests and visible diagnostics.
- [x] Refresh container documentation and include all supported Certbot DNS plugins in the shipped OCI image.
- [x] Review current installation/cache/backend lifecycle documentation for stale behavior statements.
- [x] Keep existing 0.26.6 custom `Age` rules loadable while isolating response-side Age overrides from cache freshness/storage semantics.
- [x] Keep SQLite schema 8 and saved proxy configuration compatibility with 0.26.6.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.6 - Access-policy composition and administration fixes

Current direction: 0.26.5 -> 0.26.6

- [x] Keep backend table visible columns, sort key and sort direction synchronized with account-scoped user preferences.
- [x] Clear the certificate-issuance form after the ACME issuance job succeeds while preserving non-secret values after failed jobs.
- [x] Document the sibling Python modules required by `scripts/migrate_npm.py` and fix the container helper example.
- [x] Treat explicit IP allows and allow-only countries as alternative allow sources inside one ACL policy scope, with explicit denies taking precedence and separate scopes remaining cumulative.
- [x] Keep SQLite schema 8 and saved proxy configuration compatibility with 0.26.5.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.5 - Scaled controls and listener navigation

Current direction: 0.26.4 -> 0.26.5

- [x] Keep compact toolbar selects and inputs on the same interface-scaled control height.
- [x] Reuse the interface-scale field variables for ordinary form controls so inputs/selects resize consistently with the UI scale setting.
- [x] Route the Streams **Add TCP/UDP listener** shortcut to Settings -> Listeners instead of opening a Stream editor.
- [x] Keep SQLite schema 8 and saved proxy configuration compatibility with 0.26.4.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.4 - Administration consistency and backend rotation reliability

Current direction: 0.26.3 -> 0.26.4

- [x] Add 404-host forms to the session-only draft workflow.
- [x] Complete Polish translations for the Button style appearance setting.
- [x] Align toolbar search/filter input height with the active button style.
- [x] Reuse the vhost Enabled control styling in 404 hosts, Redirects and Streams.
- [x] Replace the Overview-card checkbox list with compact visibility/reorder tiles.
- [x] Run age-based backend client recycling independently of request arrival.
- [x] Keep SQLite schema 8 and saved proxy configuration compatibility with 0.26.3.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.3 - Administration appearance and draft workflow polish

Current direction: 0.26.2 -> 0.26.3

- [x] Make interface scaling global across navigation, workspace, dialogs, overlays and the draft dock; extend the range to 70-125% and make form-control sizing respond more visibly across that range.
- [x] Make Overview telemetry cards denser without changing responsive breakpoints.
- [x] Expand chart themes to eight full-palette presets affecting colors, surfaces, grid and line treatment.
- [x] Restyle the draft dock as a square-corner RustProxy panel without rounded/glass styling.
- [x] Add a collapse-to-bar state and per-draft age indicator to the session-only draft dock.
- [x] Add Settings search without rebuilding forms, preserving unsaved values while categories are filtered.
- [x] Add chart-style palette previews beside the eight chart presets.
- [x] Add independent Normal/Compact/Ultra table-log density and Overview dashboard density preferences.
- [x] Add per-account Overview card visibility and ordering controls.
- [x] Add per-section Appearance resets plus a full Appearance reset while keeping unrelated account preferences intact.
- [x] Add an account-scoped Normal/Compact button-style selector, with Compact matching the smaller source-style control dimensions independently from global scale, including generated vhost actions and modal tabs.
- [x] Retain sticky Settings category navigation and unsaved-category indicators.
- [x] Keep SQLite schema 8 and saved proxy configuration compatibility with 0.26.2.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.2 - Unified logs search and dynamic fields

Completed direction: 0.26.1 -> 0.26.2

- [x] Add independent in-session editor drafts for Web Server, Vhosts, Redirects and Streams, with minimize/restore/discard actions in a bottom-right dock.
- [x] Consolidate theme/language/display controls under **Settings -> Appearance**, replace density modes with an interface-scale slider and persist appearance choices in administrator preferences.
- [x] Add 12/24-hour timestamp formatting, chart presentation presets and a denser expanded Stored Logs record view.

- [x] Merge Stored Logs and Live Logs into one top-bar Logs menu.
- [x] Replace fixed Stored Logs filters with one bounded field-query syntax supporting AND, negation, quotes, wildcards and numeric comparisons.
- [x] Discover selectable Stored Logs columns from parsed record fields and keep arbitrary safe field choices in account preferences.
- [x] Parse the built-in RAW error-log line format into basic structured fields while retaining the raw record.
- [x] Remove the configured-file inventory/source selector from the Logs workspace and keep bounded all-source scanning by default.
- [x] Move Stored Logs column configuration to the table toolbar and use arrow symbols for backend sort direction.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.26.1 - Vhost-aware protocol diagnostics and Logs Explorer UX

Current direction: 0.26.0 -> 0.26.1

- Attribute persistent protocol errors through one shared vhost resolver using routed vhost, authority/Host and SNI in that order.
- Keep unresolved traffic in `_unmatched.jsonl` and keep low-level protocol errors out of Live Logs.
- Add bounded protocol diagnostic identity fields for Stored Logs filtering and inspection.
- Add selectable Stored Logs columns, sticky headers, highlighted selection and inline details.
- Hide configured-file inventory by default and repair mapped client-IP/search behavior.
- Preserve schema 8 and the existing saved proxy configuration format.

## 0.26.0 - Persistent protocol diagnostics and stored-log explorer

Completed direction: 0.25.7 -> 0.26.0

- [x] Persist malformed HTTP, TLS/QUIC handshake failures, HTTP/2/HTTP/3 protocol failures, invalid h2c attempts and connection-limit rejects through the existing bounded logging sink.
- [x] Keep low-level protocol failures out of the Live Logs memory ring so hostile scans cannot displace ordinary live request history.
- [x] Add a local read-only **Logs** explorer with bounded server-side filtering and pagination over configured access/error files.
- [x] Prevent arbitrary-path browsing and symlink following; enumerate only paths derived from active RustProxy logging configuration.
- [x] Add `/logs` dashboard navigation plus a Logging-settings launcher while retaining the existing Live Logs dialog unchanged.
- [x] Keep schema 8 and configuration compatibility with 0.25.7.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.7 - Vhost operations and backend observability

Current direction: 0.25.6 -> 0.25.7

- [x] Add a fast vhost enable/disable action to the vhost cards using the existing revisioned save path.
- [x] Add backend-table search, sorting and account-scoped persisted column visibility preferences to the main dashboard.
- [x] Style scrollbars for dark and light dashboard themes.
- [x] Export retry and upstream-client lifecycle metrics for ordinary and Docker backends.
- [x] Add Grafana panels for backend client generation, age, retry activity and rotations.
- [x] Make the vhost editor Enabled control visually distinct from ordinary settings.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.6 - Backend memory diagnostics visibility

Current direction: 0.25.5 -> 0.25.6

- [x] Show upstream client generation, current-generation request count, generation age and total rotations in the main Backends diagnostics table.
- [x] Reuse the existing `/api/events` status snapshot refresh path; do not add a second diagnostics polling loop.
- [x] Keep the 0.25.5 client-generation recycling defaults and configuration compatibility unchanged.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.5 - Traffic country consistency and Revision History styling

Current direction: 0.25.4 -> 0.25.5

- [x] Use trusted `CF-IPCountry` when verified Cloudflare client-IP mode is active, with local GeoIP fallback for missing/invalid country headers.
- [x] Keep country analytics, retained country logging and country policy evaluation on the same verified country source.
- [x] Align the Revision History dialog with the shared square-corner modal styling and remove duplicated history-only dialog CSS.
- [x] Add Cloudflare-country trust regression coverage and release/source-contract checks.
- [x] Extend Prometheus with a live Traffic analytics collector, fixed-cardinality User-Agent/method/source families, country/top-route views and built-in Web Server inventory/source metrics.
- [x] Extend the bundled Grafana dashboard with a Traffic vhost filter, Web Server request rate, countries, User-Agent families, serving sources, methods, status classes, cache outcomes, capture-state gauges and configured routes.
- [x] Bound backend Hyper client lifetime with configurable request/age recycling and an upstream HTTP/1 buffer ceiling so sustained traffic can release pool/TLS high-water allocations without a vhost save.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.4 - Configuration history restore

Current direction: 0.25.3 -> 0.25.4

- [x] Restore an older retained configuration through the normal validation and hot-publish path.
- [x] Save restore as a new revision without deleting or rewinding existing history.
- [x] Require current-revision CAS plus administrator session/CSRF and audit restore source/new revision.
- [x] Add restore action and confirmation to the existing revision-history modal.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.3 - Request policy, correlation and configuration history

Current direction: 0.25.2 -> 0.25.3

- [x] Add per-vhost HTTP method allow/block policy.
- [x] Add per-location HTTP method policy, reusable ACL profile, inline IP ACL and country allow/block policy.
- [x] Enforce host and location policies before upstream selection/connection.
- [x] Generate one SHA-256 request ID per exchange and forward it unchanged to the backend as `X-Request-ID`.
- [x] Persist bounded manual-configuration revision history in SQLite schema 8.
- [x] Add a calendar/day revision-history modal with change preview in the existing dashboard style.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on a Rust release host.

## 0.25.2 - Live backup and resumed-session hardening

Current direction: 0.25.1 -> 0.25.2

- [x] Allow encrypted full-backup creation while RustProxy owns the data-directory process lock.
- [x] Snapshot SQLite consistently and stage durable files before archive encryption.
- [x] Preserve only validated relative symlinks that resolve inside the staged data tree, including Certbot `live` -> `archive` links.
- [x] Keep restore offline, validated, staged and atomically activated.
- [x] Recover transient invalid-origin state in restored/resumed admin tabs with a one-shot same-origin reload, without weakening server Host/Origin/CSRF checks.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on the release host.

## 0.25.1 - Traffic UX and source modularization

Current direction: 0.25.0 -> 0.25.1

- [x] Split Traffic into Overview, Clients, Requests and Delivery sub-tabs.
- [x] Chart every bounded Traffic dimension with donut or horizontal bar visualizations suited to the data shape.
- [x] Keep every Rust source file at or below 500 lines and keep unit tests outside implementation files.
- [x] Split oversized dashboard source bundles and the Docker Agent into bounded source modules while preserving behavior and URLs.
- [x] Guard the Web Server split against dangling attributes and missing `htaccess`, `server_protocol` and `anyhow::Context` imports.
- [ ] Run compiler-enabled `cargo fmt --all`, `cargo check --locked --all-targets` and `cargo test --locked --all-targets` on the release host.

## 0.25.0 - Web Server foundation

Current direction: 0.24.10 -> 0.25.0

- [x] Add independent reusable Web Service configuration with name, document root, index list and directory-listing switch.
- [x] Add Web Service CRUD through the dashboard/API and persist it through the existing revisioned configuration store.
- [x] Add `Reverse Proxy` / `Web Server` source selection to vhost fallback and custom locations without storing Web Server filesystem settings in vhosts.
- [x] Allow one Web Service to be referenced by multiple vhosts and one vhost to mix Web Server and Reverse Proxy locations.
- [x] Add `GET`, `HEAD`, MIME, `Content-Length`, streaming, index lookup, directory slash redirect and optional directory listing.
- [x] Add repeatable per-request directory listing, per-directory index-before-listing behavior, Web Service MIME overrides/default MIME and a default static `Cache-Control` policy.
- [x] Organize Web Service editing into General, Routing, MIME, Cache & headers, Delivery, PHP-FPM and Security tabs.
- [x] Add Web Service `try_files`, internal/redirect regex rewrites, ordered per-extension/MIME Cache-Control rules, fixed response headers, precompressed `.br`/`.gz` delivery and forced-download extensions.
- [x] Add bounded FastCGI/PHP-FPM execution over Unix sockets or loopback TCP with `index.php`/front-controller routing and no source fallback on upstream failure; expose advanced bounded limits, Authorization forwarding, custom FastCGI parameters and controlled `PHP_VALUE` overrides with in-panel syntax documentation.
- [x] Add explicit symlink policy: deny by default, with Linux-only inside-root following for static files; PHP-FPM remains restricted to deny.
- [x] Add a bounded `.htaccess` compatibility layer for common Apache migration rules, including rewrite conditions/rules, directory index/listing, MIME, headers, Expires and PHP AddHandler integration, with a dashboard documentation modal and fail-closed security-sensitive directives.
- [x] Add one-shot Web Service presets for Static, SPA, PHP generic, WordPress, Drupal, Laravel, Symfony, Joomla, PrestaShop and Downloads, with optional deterministic hardening for common exploit probes/sensitive paths and fingerprint-header stripping.
- [x] Expose vhost-wide request/active/error/EWMA/traffic metrics for Web Server-only vhosts and fix custom logging mappings for route/cache/transform effective fields.
- [x] Add bounded per-vhost Traffic analytics, User-Agent logging and opt-in country/GeoIP logging with top-dimension views, effective-client-IP rankings and donut charts for the main dimensions.
- [x] Keep Web Service dashboard rendering stable when configuration loads before the deferred Web Server UI module and use the existing generic/custom 403 error-page pipeline when directory listing is disabled.
- [x] Add automatic allocator release and idle HOT-cache waterline trimming so sustained traffic does not require a vhost save/reload to return reclaimable RSS.
- [x] Add metadata-based `ETag`/`Last-Modified`, conditional `304`, single `Range`/`If-Range`, `206` and `416`.
- [x] Enforce decoded-path validation and document-root confinement; Unix traversal uses `openat` + `O_NOFOLLOW` and does not serve symlinks.
- [x] Keep Web Server inside the existing routing, ACL/auth, custom-header/error-page/security and H1/H2/H3 response pipeline; do not create separate routing/error/cache subsystems.
- [x] Add unit/source-contract coverage, OpenAPI/UI language coverage and Web Server documentation.
- [x] Clean README/Docs/CHANGELOG/ROADMAP material, add a canonical docs index, authenticated Markdown Reader, 0.25 specification audit and open-work TODO.
- [ ] Run compiler-enabled `./dev.sh fmt`, `./dev.sh check`, `./dev.sh test` plus live H1/H2/H3/static-file acceptance on the release host.

## Web Server follow-up

Future Web Server work must retain the 0.25 split between reusable **Web Service** configuration and vhost/location source selection.

- Multipart byte ranges if operational demand justifies them.
- Named locations without introducing a second URL router.
- Opt-in PHP `PATH_INFO` script/path splitting with an unambiguous confinement rule.
- Race-resistant filesystem confinement for non-Unix targets before enabling local file serving there.
- Real PHP-FPM release-host acceptance over Unix socket and loopback TCP.

See [`docs/TODO.md`](docs/TODO.md) for unresolved work and release acceptance, and [`docs/ROADMAP_HISTORY.md`](docs/ROADMAP_HISTORY.md) for completed historical milestones.