128 lines
10 KiB
Markdown
128 lines
10 KiB
Markdown
# TCP/UDP Stream pools (v0.32.0)
|
||
|
||
Layer-4 forwarding runs on dedicated TCP/UDP listeners and never traverses HTTP Vhosts, ACL, cache, headers, compression, TLS termination or authentication. The existing **Streams** page provides a card/modal editor and per-Stream RX/TX chart. Changes to Stream listeners and backend pool configuration remain **desired state** until the next controlled RustProxy restart; saving Appearance preferences does not restart the proxy.
|
||
|
||
## Compatible configuration
|
||
|
||
The legacy document still works without modification:
|
||
|
||
```json
|
||
{"streams":[{"id":"ssh","name":"SSH relay","enabled":true,"protocol":"tcp","listen":"0.0.0.0:2222","upstream":"10.0.0.20:22","connect_timeout_seconds":5,"idle_timeout_seconds":300,"max_connections":512}]}
|
||
```
|
||
|
||
Without `backends`, `upstream` is treated as one enabled backend with ID `primary`, weight 1, no active health probe and detailed metrics disabled by default (opt in explicitly). The legacy field is retained in serialized configuration and is set to the first backend address when a pool is explicitly configured. Existing listeners continue to require restart on change.
|
||
|
||
A TCP pool can be configured as follows (the same backend fields apply to UDP):
|
||
|
||
```json
|
||
{
|
||
"id":"ssh", "name":"SSH pool", "enabled":true, "protocol":"tcp",
|
||
"listen":"0.0.0.0:2222", "upstream":"10.0.0.20:22",
|
||
"balance":"round_robin", "metrics_enabled":true,
|
||
"connect_timeout_seconds":5, "idle_timeout_seconds":300, "max_connections":512,
|
||
"backends":[
|
||
{"id":"node-a","name":"Server A","address":"10.0.0.20:22","enabled":true,"drain":false,"weight":2,"max_sessions":256,"metrics_enabled":true,"health":{"mode":"tcp_connect","interval_seconds":10,"timeout_ms":1000,"rise":2,"fall":2}},
|
||
{"id":"node-b","name":"Server B","address":"10.0.0.21:22","enabled":true,"drain":false,"weight":1,"max_sessions":256,"metrics_enabled":false,"health":{"mode":"tcp_connect","interval_seconds":10,"timeout_ms":1000,"rise":2,"fall":2}}
|
||
]
|
||
}
|
||
```
|
||
|
||
`balance` is `round_robin` (default), `least_connections` (TCP), or `least_sessions` (UDP). The least-load variants use an active-assignment-to-weight ratio. Each new TCP connection selects one backend for its entire connection; a UDP client session selects once and remains pinned for its lifetime (idle timeout). `drain: true`, `enabled: false`, `Unhealthy` state, or a reached `max_sessions` prevents new assignments. In-flight TCP relays and UDP sessions retain their selected backend; a controlled process restart still follows the existing shutdown grace period and may end in-flight sessions.
|
||
|
||
## Health monitoring
|
||
|
||
Each backend has independent `enabled`, `health` and `metrics_enabled` controls. Disabled and draining backends can continue independent health monitoring if probes are explicitly configured; it does not get new sessions. Probe concurrency is globally limited to 16, at most one health task per enabled/configured backend, with lifecycle tied to the listener cancellation token. Timeouts, probe intervals and rise/fall thresholds are bounded by validation. TCP checks use a timed TCP connect without application payload.
|
||
|
||
UDP checks cannot use a local UDP `connect()` as evidence that the remote service exists. For a validated active UDP probe define both `payload_hex` and an exact, nonempty `expect_hex` response (up to 128 bytes each), e.g.:
|
||
|
||
```json
|
||
{"mode":"udp_probe","interval_seconds":10,"timeout_ms":1000,"rise":2,"fall":2,"payload_hex":"50494e47","expect_hex":"504f4e47"}
|
||
```
|
||
|
||
**Do not enable UDP probing without the service owner's approval.** Unknown application protocols must leave checks `off`; `udp_probe` without an expected reply is `Unverified`, sends no datagrams and does not report Healthy. A backend without active checks is `Unverified` (routing still allowed), while probed backends show `Checking`, `Healthy`, `Unhealthy` or `Disabled`; hysteresis prevents state flapping. An Unhealthy backend stops receiving new sessions, but no ongoing TCP connection is reassigned. If the pool has no eligible backend, new sessions or datagrams are rejected and the legacy per-Stream `rejected` counter advances.
|
||
|
||
## Telemetry and Dashboard
|
||
|
||
The existing `/api/status` response and `/api/status?view=live` SSE/fallback add a bounded `backends` array under `streams.metrics[stream_id]`. It includes stable configured IDs, status, active sessions, health counts/last errors and (if enabled) optional completed/created/assignment, bytes, packets and error/reject metrics. `bytes_in` remains client → upstream, `bytes_out` upstream → client. A UDP datagram rejected before a session is established remains counted in the original per-Stream incoming counters, not backend delivered-byte counters.
|
||
|
||
In the editor, **Stream → Detailed metrics → Default (disabled)** means the absence of a Stream override resolves to `false` (it does **not** inherit from Prometheus). At the backend level, **Use Stream setting** resolves to that Stream's value; backend **Enabled**/**Disabled** override it. Prometheus retains `include_streams` as the separate global export gate. `metrics_enabled` defaults to `false` for detailed per-backend accounting (no new overhead on legacy streams). Stream-level `true` enables the default for its backends; backend-level `true`/`false` explicitly overrides the Stream value, while `null` inherits. This choice never changes session routing or health checks. Basic per-Stream counters and backend active gauges stay available. Backend label cardinality is bounded to IDs declared in config, never client IPs. History is a limited browser-memory window only; metrics are not persisted as periodic SQLite samples.
|
||
|
||
Under **Settings → Appearance → Dashboard layout**, toggle/reorder Stream overview, traffic, sessions and backend-health cards alongside existing telemetry cards. All Stream cards default hidden for existing accounts, including migrated 0.31.x layouts. An explicitly enabled card stays visible even when no Stream is configured, enabled or running; it displays a neutral **No stream data** state in that case rather than ignoring the Appearance setting. They use the existing API status/SSE/fallback and one dashboard refresh scheduler. Hiding a card avoids its chart computations, and changing visibility does not alter Stream runtime, existing widgets or global Prometheus settings. Widget descriptions in Appearance use native `title` tooltips on hover and accessible button descriptions (`aria-description`) instead of floating elements that expand the layout. On the main Overview page, secondary values and RX/TX chart legend are always inline under the metric/graph (not tooltips), with the original six-column desktop layout and taller tiles for readability.
|
||
|
||
## Safety and operational limits
|
||
|
||
At most 128 Stream hosts and 32 backends per host, weights 1–16, 1–65535 stream/session concurrency and optional per-backend session caps. Validated concrete backend IPs and nonzero ports are required. Duplicate backend IDs, local listener loops, conflicting transport health modes and invalid probes are rejected before configuration is published. UDP client queues are bounded at 64 datagrams. The existing listener restart indicator remains authoritative, and no new public administration routes or polling tasks are added.
|
||
|
||
## Standalone TCP and UDP forwarding tests
|
||
|
||
The ready-to-use launcher and individual TCP/UDP test clients are all in `scripts/`.
|
||
Start four loopback-only mock backends (TCP 9101/9102; UDP 9201/9202):
|
||
|
||
```bash
|
||
./scripts/start_stream_backends.sh
|
||
# Optional, only on an isolated test network: ./scripts/start_stream_backends.sh 0.0.0.0
|
||
```
|
||
|
||
Point your corresponding Stream backend pools at these ports, restart RustProxy
|
||
(the Stream configuration's existing restart requirement), then run:
|
||
|
||
```bash
|
||
python3 scripts/test_stream_tcp.py client --port 2222 --sessions 32 --messages 5 --min-backends 2
|
||
python3 scripts/test_stream_udp.py client --port 5353 --sessions 32 --messages 5 --min-backends 2
|
||
```
|
||
|
||
Stop the launcher with `Ctrl+C`. Its four children are stopped together. If you
|
||
prefer launching backend peers separately or need different ports, use the examples below.
|
||
|
||
These are **black-box clients for a real running RustProxy** plus small, intentionally
|
||
loopback-only mock services. Do not aim them at live/production backends. Use
|
||
separate terminals for each mock, and configure the Stream editor with these
|
||
backend addresses. **Restart RustProxy after modifying Stream backends** (the
|
||
existing Stream lifecycle still applies).
|
||
|
||
TCP example (Stream listener `127.0.0.1:2222`, backend pool `127.0.0.1:9121` and
|
||
`127.0.0.1:9122`):
|
||
|
||
```bash
|
||
# Terminal 1
|
||
python3 scripts/test_stream_tcp.py server --port 9121 --backend-id node-a
|
||
# Terminal 2
|
||
python3 scripts/test_stream_tcp.py server --port 9122 --backend-id node-b
|
||
# Terminal 3 - after enabling the TCP Stream listener in RustProxy
|
||
python3 scripts/test_stream_tcp.py client --port 2222 --sessions 32 --messages 5 --min-backends 2
|
||
```
|
||
|
||
UDP example (Stream listener `127.0.0.1:5353`, pool `127.0.0.1:9131` and
|
||
`127.0.0.1:9132`):
|
||
|
||
```bash
|
||
# Terminal 1
|
||
python3 scripts/test_stream_udp.py server --port 9131 --backend-id node-a
|
||
# Terminal 2
|
||
python3 scripts/test_stream_udp.py server --port 9132 --backend-id node-b
|
||
# Terminal 3 - after enabling the UDP Stream listener in RustProxy
|
||
python3 scripts/test_stream_udp.py client --port 5353 --sessions 32 --messages 5 --min-backends 2
|
||
```
|
||
|
||
Each client verifies exact round-trip payloads and logs per-backend session
|
||
assignments. TCP verifies the selected backend stays unchanged through several
|
||
messages on one connection; UDP verifies the same over datagrams from one
|
||
client socket. Use `--min-backends 1` for a single backend, drain, or failover
|
||
case. Compare the reported assignments with Stream metrics and Prometheus.
|
||
`--parallel`, `--sessions`, `--messages`, and `--timeout` control test load.
|
||
UDP mock services also respond to the **explicit** health probe `PING`/`PONG`
|
||
(`payload_hex: 50494e47`, `expect_hex: 504f4e47`); do not apply this payload
|
||
arbitrarily to real application servers.
|
||
|
||
To check the scripts themselves *without RustProxy*, run:
|
||
|
||
```bash
|
||
python3 scripts/test_stream_tcp.py selftest
|
||
python3 scripts/test_stream_udp.py selftest
|
||
```
|
||
|
||
Selftests only exercise the Python mock peer/harness; they do not verify the
|
||
RustProxy forwarding path, balance algorithms or real health check lifecycle.
|
||
Run the `client` cases against your compiled proxy and manually repeat after
|
||
disabling, draining, or stopping one mock backend to verify failover behavior.
|