RustProxy/README.md

# RustProxy 0.26.0

Independent Rust reverse proxy and static Web Server with a dashboard, SQLite configuration,
streaming HTTP/TLS/WebSocket forwarding, raw TCP/UDP stream hosts, IP-bound vhosts and backend pools.

**0.26.0 release source.**

## Layer-4 streams

Raw TCP/UDP forwarding has its own **Streams** page in the primary navigation, alongside **Vhosts**, **Redirects** and **404 hosts**. Streams use the same card/modal workflow as vhosts, but each stream owns a dedicated Layer-4 listener and one upstream and intentionally bypasses HTTP ACL/cache/header/compression logic. The page exposes live per-stream connections/sessions, rejects, errors, transferred bytes and RX/TX rate charts. Listener changes require a restart. See [TCP/UDP streams](docs/STREAMS.md).

Named HTTP/HTTPS listener bind addresses use the same chip-style editor as vhost hostnames, including multi-address listeners.

## What changed in 0.26.0

- Persist malformed HTTP, TLS ClientHello/handshake failures and timeouts, invalid h2c traffic, HTTP/2 protocol failures and HTTP/3/QUIC failures that can happen before a normal routed request exists.
- Keep those low-level `protocol_error` events out of the Live Logs memory ring and send them only through enabled structured-log sinks, preventing scanner noise from displacing ordinary live requests.
- Add the authenticated **Logs** workspace at `/logs`, reachable from the top bar and Settings -> Logging, for bounded filtering and JSON inspection of configured access/error files.
- Keep file-backed Logs independent from Live Logs: existing configured files can be viewed even when their writers are currently disabled.
- Harden access-log creation and stored-log reads with regular-file checks and Unix `O_NOFOLLOW`; the file browser never accepts an arbitrary filesystem path.
- Extend standard structured records with `event`, `protocol_error`, `error_detail`, `local_addr` and `tls_sni`.
- Keep SQLite schema 8 and the existing saved configuration format; upgrading from 0.25.7 requires no data migration.

The detailed 0.25.5 implementation audit remains available as historical baseline evidence. Current 0.26.0 verification status is maintained in [docs/TEST_REPORT.md](docs/TEST_REPORT.md).

## Documentation

- `/docs` - authenticated project documentation Reader plus the API/OpenAPI tab.
- [docs/README.md](docs/README.md) - canonical documentation index.
- [docs/WEB_SERVER.md](docs/WEB_SERVER.md) - Web Server configuration, HTTP behavior and filesystem security.
- [docs/FILE_LOGS.md](docs/FILE_LOGS.md) - stored-log explorer, protocol-error records, bounds and filesystem security.
- [docs/PHP_FPM.md](docs/PHP_FPM.md) - implemented FastCGI/PHP-FPM integration, safety model and runtime requirements.
- [docs/V0.25.5_AUDIT.md](docs/V0.25.5_AUDIT.md) - 0.25.5 Cloudflare country analytics and Revision History styling release audit.
- [docs/V0.25.4_AUDIT.md](docs/V0.25.4_AUDIT.md) - historical 0.25.4 configuration-history restore release audit and compiler gate.
- [docs/V0.25.3_AUDIT.md](docs/V0.25.3_AUDIT.md) - historical method-policy/location-ACL/request-ID/config-history release audit.
- [docs/V0.25.2_AUDIT.md](docs/V0.25.2_AUDIT.md) - historical 0.25.2 live-backup/symlink/origin recovery audit.
- [docs/V0.25.0_AUDIT.md](docs/V0.25.0_AUDIT.md) - Web Server foundation specification audit.
- [docs/TEST_REPORT.md](docs/TEST_REPORT.md) - verification results and environment limitations.
- [docs/TODO.md](docs/TODO.md) - unresolved follow-up only.

## Release history

Historical user/operator-visible changes are maintained in [CHANGELOG.md](CHANGELOG.md). Detailed historical implementation notes are retained in [docs/CHANGELOG_TECHNICAL_ARCHIVE.md](docs/CHANGELOG_TECHNICAL_ARCHIVE.md), and completed roadmap milestones are retained in [docs/ROADMAP_HISTORY.md](docs/ROADMAP_HISTORY.md).

## Core capabilities and references

Managed certificates imported into the library renew automatically unless disabled. Initial issuance/import remains explicit and staging-first; imported PEM alone is not auto-managed. See [renewal and recovery](docs/RENEWAL.md).

Read [installation and update](docs/INSTALL.md), [host access and locations](docs/VHOST_ACCESS.md),
[migration](docs/MIGRATION.md), [NPM migration](docs/NPM_MIGRATION.md), [ACME providers](docs/ACME_PROVIDERS.md),
[Prometheus](docs/PROMETHEUS.md), [cache diagnosis](docs/CACHE_DIAGNOSTICS.md),
[HTTP/3 scope](docs/HTTP3.md), [TCP/UDP streams](docs/STREAMS.md) and [container deployment](docs/CONTAINER.md). Cloudflare Tunnel remains deferred. Docker discovery is available through the standalone agent; see [Docker discovery](docs/DOCKER_DISCOVERY.md).
Examples are disposable full configurations: applying one replaces, not merges,
an existing document.

## Requirements and first build

### Debian 13+ one-line install/update

The bootstrap installs the compiler/build dependencies and Rust, clones this repository and
builds the locked release. With no argument it runs in `auto` mode: a host without a managed
RustProxy installation is installed, while an existing `/opt/rustproxy/install.json` triggers
the safe `update.sh` path with backup, health check and rollback. First install starts HTTP on
`0.0.0.0:80`, HTTPS on `0.0.0.0:443` and keeps the administration panel on loopback HTTP
`127.0.0.1:9090`. Updates preserve the saved listener and service settings. Use an SSH tunnel for
initial remote access, then enable Administration HTTPS before exposing a non-loopback admin bind.

```bash
# automatic: install on a new host, update an existing managed installation
wget -qO- https://git.linuxiarz.pl/gru/RustProxy/raw/branch/master/install-debian13.sh | bash

# explicit update
wget -qO- https://git.linuxiarz.pl/gru/RustProxy/raw/branch/master/install-debian13.sh | bash -s -- update
```

Manual build prerequisites used by that bootstrap:

```bash
apt install -y build-essential gcc git ca-certificates python3 pkg-config wget sudo python3-certbot-dns-rfc2136 python3-certbot-dns-cloudflare
wget -qO- --https-only --secure-protocol=TLSv1_2 https://sh.rustup.rs | sh
```

Rust stable (declared minimum 1.88), Cargo, C compiler/linker, Bash and Python 3.10+.
Linux is the primary deployment target;
macOS/WSL2 need normal native build tools. Detailed OS diagnostics currently use
Linux procfs and cgroup v2. Unsupported measurements remain null/N/A.

The first build needs access to the Rust toolchain and crate registry. `dev.sh`
never installs system software or runs sudo. Rustls handles runtime TLS; zstd's
native dependency still needs a C build toolchain.

The Debian bootstrap intentionally builds with one Cargo job, LTO disabled and
`codegen-units=8`. This avoids `rustc` being killed by the OOM killer on small VPSes
while leaving the repository's normal release profile unchanged. For a manual
low-memory build use:

```bash
CARGO_BUILD_JOBS=1 CARGO_PROFILE_RELEASE_LTO=false \
  CARGO_PROFILE_RELEASE_CODEGEN_UNITS=8 cargo build --release --locked
```

The 0.26.0 source archive preserves the resolved `Cargo.lock` supplied with the source tree and
updates only this package's own version entry. This packaging environment has no Cargo, so the
dependency graph was not re-resolved here. On the compiler-enabled deployment host, run the
locked check/test/release gates below and retain the verified lockfile. `dev.sh` can generate a
lockfile only when one is genuinely absent; it does not replace normal locked verification.

```bash
cd RustProxy
bash dev.sh doctor
bash dev.sh check
bash dev.sh test
bash dev.sh
```

| Service | Development default |
|---|---|
| Default HTTP | `http://127.0.0.1:8080` |
| Default HTTPS | `https://localhost:8443` |
| Administration (default) | `http://127.0.0.1:9090`; HTTPS/remote bind is opt-in in Settings |
| Optional protocol fixture | `127.0.0.1:9000`, only with `RUST_PROXY_DEMO=1` |

In a second terminal, print the one-time setup capability and use the browser,
or initialize an administrator locally:

```bash
bash dev.sh admin setup-token
# Alternative: interactive account creation, no default credentials.
bash dev.sh admin init
```

There is no default username/password. Normal startup serves a neutral informational
page for an unconfigured address and does not seed sample vhosts. The self-signed
bootstrap certificate is retained under `data/certs` and is not publicly trusted.
Assign intended certificates explicitly. Public frontend TLS does not implicitly expose the separate administration listener. Administration remains loopback HTTP by default; enable its own HTTPS setting before choosing a remote bind. Existing databases are never reset by this default.

Use the protocol fixture only explicitly when running the documented test suites;
`RUST_PROXY_DEMO=1 bash dev.sh` seeds a new disposable database. The separate
`bash dev.sh mock` helper remains available for cache/backend testing.

For a system service, build as an ordinary user and follow the complete procedure
in [INSTALL.md](docs/INSTALL.md):

```bash
bash dev.sh release
bash install.sh --plan --binary target/release/rust-proxy
sudo bash install.sh --binary target/release/rust-proxy
# A subsequent release:
sudo bash update.sh --binary target/release/rust-proxy
```

Copy `.env.example` to `.env` for startup defaults; it is trusted shell code.
`RUST_PROXY_DEMO=0` disables the demo. `RUST_PROXY_TLS=0` removes the startup TLS
default used when bootstrapping listeners.

The release checker reads its compile-time defaults from repository-root `UPDATE_CHECK.toml`,
which is embedded in the final binary. It checks the configured `Cargo.toml` URL immediately
on startup and then every 3600 seconds by default. A failed request is ignored until the next
interval; it never blocks startup or proxy traffic and it never installs updates. On Unix-like
systems the checker keeps its bundled public roots and also imports certificates from
`SSL_CERT_FILE`, `CURL_CA_BUNDLE` and common Linux/BSD CA bundles, including the
FreeBSD/OPNsense `ca_root_nss` path. It follows at most five credential-free HTTP(S)
redirects and never downgrades an HTTPS check to HTTP. Failed checks stay invisible in the dashboard, preserving the existing footer
behavior, but are emitted as application `ERROR` events so the optional error-file sink can
capture them. Operators may override the embedded values with global CLI options or equivalent
environment variables:

```text
--update-repository-url         RUST_PROXY_UPDATE_REPOSITORY_URL
--update-version-url            RUST_PROXY_UPDATE_VERSION_URL
--update-check-interval-secs    RUST_PROXY_UPDATE_CHECK_INTERVAL_SECS
```

Set the interval to `0` to disable automatic checks. Debian/systemd installs may place the
environment overrides in `/etc/rustproxy/environment`; source/development runs may use the
normal process environment.

Persisted frontend sockets live only in
`listeners.endpoints` and are edited under Settings -> Listeners. Listener address/TLS/H2/H3
capability changes need a process restart; vhost H2/H3 switches only narrow those
capabilities and routing/pool policies otherwise use validated hot publication.

## Private vhosts and backend pools

Use **Settings -> Listeners** to configure frontend sockets, then use **Vhosts** to
select those named listeners and configure backend members. `examples/vhosts-private.json`
binds its named listener only to `127.0.0.2` and uses two
explicit loopback origins. Replace these with addresses assigned to your machine
and your real services. `examples/vhosts-balanced.json` shows an opt-in public
80/443 deployment; do not apply it unintentionally.

A private vhost is selected by hostname **and the accepted socket's actual local
IP, port and TLS state**. Sending its Host/X-Forwarded-* values to a different IP
does not select it. A shared wildcard socket may accept TCP at multiple addresses,
but each vhost keeps its original routing constraints. IP binding is not a firewall
or authentication: NAT/another proxy can still forward traffic to that private IP.
Keep network restrictions and application authentication.

New health-checked members remain in `checking` until the configured successes
make them eligible. No healthy member means 503; a healthy backup may be selected.
HTTP checks measure time to response headers, not whole-body throughput. Affinity
cookies bypass shared cache; they contain only an opaque routing token and are not
login sessions. Existing cookie sessions may finish on a draining healthy member.
The keepalive controls are connection reuse, **not Keepalived/VRRP**.

Save pool/routing/security changes live. New/removed sockets and TLS ALPN changes
need a process restart; active vs desired endpoints are visible in Settings/API.
Old top-level routes are converted once into vhosts during validated startup/import.
There is no separate legacy routing UI or dispatcher; path-to-origin ownership is
retained. Read the migration guide before updating an existing database. See [VHOSTS.md](docs/VHOSTS.md)
for algorithms, defaults, bounds, caveats and the complete configuration contract.

## Standard ports without running as root

Settings -> Listeners accepts multiple named endpoints and socket addresses such as `0.0.0.0:80`,
`0.0.0.0:443`, `[::]:443` and custom ports. Presets change only the port: they do
not silently replace a loopback interface with a public one. The admin listener is independent of these controls and stays loopback by default. In Settings -> Administration, enabling HTTPS permits a persisted non-loopback bind using either the application-managed certificate or a selected certificate. Plain remote HTTP remains available only with the explicit `--allow-remote-admin` startup flag.

Linux low-port binding can require `CAP_NET_BIND_SERVICE`. Use a dedicated service
account and the example in [deploy/README.md](deploy/README.md), not a root-run
application. Port conflicts and OS permissions cannot be proved by merely saving
configuration; bind failures are reported on restart. See
[settings, defaults and rollback](docs/SETTINGS.md).

## Cache and response controls

The shared cache is enabled by default. Normal fresh `200 GET` and cacheable HEAD
requests use `s-maxage`, `max-age` or `Expires`; a configurable default TTL applies only
to explicit `public` responses without an origin freshness lifetime and defaults to
zero. GET/HEAD share the same frontend scheme + Host + exact URL object, response `Vary` selects
representation variants, and unrelated request headers or the selected backend-pool
member do not fragment that object. Since 0.19.1 HTTP and HTTPS use distinct primary
cache identities so scheme-dependent origin responses cannot cross protocols. Non-GET/HEAD methods pass without globally
purging unrelated cached objects.

Cookies (except the configured static-asset exception), authorization, ranges,
`If-Match`/`If-Unmodified-Since`, private/no-store responses, Set-Cookie, `Vary: *`,
streaming events and failed/incomplete bodies are not reused. `If-None-Match` and
`If-Modified-Since` can be answered from a fresh cache entry as local conditional HITs.
Request `no-cache`/freshness constraints revalidate a stored validator instead of
blanket-bypassing the cache. Response `no-cache` with a usable validator may be retained,
but must validate before every reuse. Request/response `no-store` remains a hard exclusion.

Default cache memory limits are 256 MiB HOT allocated/charged data, 16 MiB per object, 64 MiB transient RAM, a 32 MiB COLD writer queue, 4096 HOT entries and eight fill permits. The panel also offers memory-aware conservative/recommended/performance presets derived from the lower of detected host RAM and a finite cgroup memory limit; automatic presets are disabled below 256 MiB effective RAM. `max_object_bytes` remains an independent per-object ceiling and cannot exceed the global HOT budget. If it is configured above the transient budget, the panel warns that objects near that size will pass through uncached because a fill cannot reserve enough bounded transient RAM. Optional cold cache defaults to disabled; when
enabled it uses a bounded disk path under the private data directory and asynchronously
demotes LRU/idle hot objects. Cold objects survive process restart for the same
configuration revision and are promoted to RAM on a fresh hit. Large fill buffers grow
incrementally and consume multiple permits. Actual process RSS also includes in-flight
responses, compression buffers and runtime allocations. HIT/MISS/BYPASS, hit count and
standardized RFC 9211 `Cache-Status` diagnostics are independently optional; required Age remains on cache hits. Optional `X-Cache-Resident-Age` tracks RustProxy residency and now survives revalidation/tier movement.
Purge is authenticated, CSRF-protected, audited and invalidates both tiers plus older
fills. See [cache behavior and limits](docs/CACHE.md) and
[cache diagnostics](docs/CACHE_DIAGNOSTICS.md).

Response compression is streamed and negotiated with Accept-Encoding; enabled
algorithms are gzip, Brotli, Zstandard and deflate. Privacy-sensitive, ranged,
upgraded, pre-encoded and no-transform responses are excluded. Strong ETags are
weakened after transformation, and obsolete content digests are removed. Server
policy is applied after generic response rules. Protected framing/identity headers
cannot be edited through generic rules.

With the separate demo enabled, `/cache-demo` returns an explicitly public 30-second
object. Enable cache and X-Cache in Settings, then issue this twice without cookies:

```bash
curl -i -H 'Host: localhost' http://127.0.0.1:8080/cache-demo
```

The first eligible request is MISS, then HIT while fresh. The origin timestamp
stays unchanged on HIT. Unlike proxy/system measurements, this endpoint is explicitly
demo content. The reserved admin cookie is removed before eligibility; application cookies still
trigger privacy bypass. Use anonymous curl requests to check cache behavior.

## Adding a language

Create `langs/de_DE.json` with metadata and string translations, then rebuild and
restart. No JavaScript language switch or registry edits are needed:

```json
{
  "meta": {
    "locale": "de-DE", "name": "German", "native_name": "Deutsch", "direction": "ltr"
  },
  "translations": { "settings": "Einstellungen", "overview": "Ubersicht" }
}
```

Missing keys fall back to `en_US.json`. Translation strings are rendered as text,
not HTML; locale and direction metadata also control formatting/RTL. The two
bundled languages are English and Polish. New packs are compile-time assets, not
untrusted runtime plugins. See [langs/README.md](langs/README.md).

## Updating an installed source tree

Read [MIGRATION.md](docs/MIGRATION.md). Back up actual data/certificates, stop the
old server and extract this archive into a separate, non-nested directory. From
the new source directory:

```bash
rust-proxy --data-dir /actual/RustProxy/data backup create \
  --output /private/rustproxy-before-upgrade.rpb
bash upgrade.sh /actual/RustProxy --stopped
# Also pass --data-dir /actual/private/data when it is external.
cd /actual/RustProxy
bash dev.sh test
bash dev.sh
```

The supported operator backup is the password-encrypted `rust-proxy backup create`
command. `scripts/backup.py` is retained only as an offline release-test helper and
must not be used as a Secret Store backup. See
[Secret Store and encrypted backup/restore](docs/SECRET_STORE_BACKUP.md).

`.env`, data, unmanaged files and previous managed sources are preserved. Extra
language JSON files survive the update; shipped files replace their old names,
whose previous bytes remain in the backup. Unknown custom source edits are not
merged. Do not overlay individual source files onto an old tree.

RustProxy 0.25.3 upgrades SQLite to schema version 8 by adding retained manual-configuration
history. Existing schema-7 data is migrated automatically; the current configuration is seeded
as the first retained history row and future successful saves keep the newest 512 revisions.
Vhost/location method and access-policy fields are additive and default to disabled/inherited
behavior when omitted. Once a 0.25.3 configuration containing the new fields is saved, an older
binary may reject unknown configuration fields. Roll back using the pre-upgrade data backup as
well as the old source, not just the binary.

## Documentation and checks

[documentation index](docs/README.md), [API](docs/API.md), [architecture](docs/ARCHITECTURE.md),
[security](docs/SECURITY.md), [transport](docs/TRANSPORT.md),
[vhosts and pools](docs/VHOSTS.md), [Web Server](docs/WEB_SERVER.md), [settings](docs/SETTINGS.md), [cache](docs/CACHE.md),
[roadmap](ROADMAP.md), [changelog](CHANGELOG.md), [verification](docs/TEST_REPORT.md).

```bash
# Rust-independent checks:
python3 scripts/check_source.py
python3 scripts/release_smoke.py
# Runs those checks first, then the Rust suite when Cargo is available:
bash dev.sh test
# Source-only package, with no Git checkout required:
python3 make_zip.py /tmp/RustProxy-0.26.0.zip
```

## New management and test-origin guide

See [MANAGEMENT.md](docs/MANAGEMENT.md) for the certificate staging workflow,
page/status behavior and logs; [MOCK_SERVER.md](docs/MOCK_SERVER.md) provides a
ready-to-use cache/health test setup. The example `examples/mock-vhost.json` is a
complete minimal document for a fresh test installation, not a merge patch.

## Authentication and mock development

In **Vhosts -> Authentication**, enable Tinyauth and supply its base URL (or the
complete `/api/auth/nginx` endpoint) and public login URL. Private verifiers need
the explicit private-network permission; optionally constrain allowed networks.
Default identity headers are Remote-User and Remote-Email. No embedded provider
password or disabled TLS verification is required. Do not protect the login host
with itself. See INTEGRATIONS_040.md for exact status/redirect behavior and limits.

```bash
# Separate terminals, after building the real proxy on a toolchain-enabled host:
bash dev.sh mock --port 9001 --name origin-a --ttl 30
bash dev.sh mock-auth --port 9300
```

The fake-auth server accepts the intentionally insecure cookie `mock_auth=allow`;
it is a loopback-only development peer, never an authentication product.

### Docker vhosts operational controls

Docker vhosts now publish/unpublish immediately per service. Each Docker agent has a configurable inventory polling interval, while the optional Live view refreshes cached runtime state only. Docker-backed applications can use ACL profiles/inline ACLs and per-vhost proxy connect/header/read/send timeouts, matching the ordinary vhost proxy controls.