150 lines
7.4 KiB
Markdown
150 lines
7.4 KiB
Markdown
# Administrator system diagnostics endpoint
|
|
|
|
RustBox can expose a single administrator-only JSON endpoint for operational and security diagnostics. It is intended for local troubleshooting, monitoring agents and cluster operations. It is **disabled by default** and, when enabled without changing the allowlist, accepts only loopback clients.
|
|
|
|
The endpoint is separate from the user-facing `/api/diagnostics/*` routes. It does not require a RustBox mailbox session. Instead, access requires **both**:
|
|
|
|
1. the effective client IP to match `diagnostics_endpoint.allowed_networks`, and
|
|
2. the configured diagnostics token or Basic Auth credentials.
|
|
|
|
Requests from a non-allowed address receive HTTP `403`. Requests from an allowed address with missing/invalid diagnostics credentials receive HTTP `401`. Critical backend probe failures return HTTP `503`; healthy or degraded reports return HTTP `200` with `status` set to `ok` or `degraded`.
|
|
|
|
## Configuration
|
|
|
|
```toml
|
|
[diagnostics_endpoint]
|
|
enabled = false
|
|
path = "/api/system/diagnostics"
|
|
allowed_networks = ["127.0.0.1/32", "::1/128"]
|
|
auth = "token" # token | basic
|
|
# token = "replace-with-at-least-24-random-characters"
|
|
# basic_username = "rustbox-monitor"
|
|
# basic_password = "replace-with-a-long-random-password"
|
|
include_user_identifiers = false
|
|
probe_timeout_secs = 5
|
|
security_window_secs = 900
|
|
```
|
|
|
|
`path` must remain below `/api/system/` so it cannot collide with normal application API routes.
|
|
|
|
### Token authentication
|
|
|
|
Prefer keeping the token outside TOML, for example in `.env` or an orchestrator secret:
|
|
|
|
```bash
|
|
DIAGNOSTICS_ENDPOINT__ENABLED=true
|
|
DIAGNOSTICS_ENDPOINT__AUTH=token
|
|
DIAGNOSTICS_ENDPOINT__TOKEN='a-long-random-secret-at-least-24-characters'
|
|
```
|
|
|
|
Example request:
|
|
|
|
```bash
|
|
curl -fsS \
|
|
-H "Authorization: Bearer $DIAGNOSTICS_TOKEN" \
|
|
http://127.0.0.1:3001/api/system/diagnostics
|
|
```
|
|
|
|
`X-RustBox-Diagnostics-Token` is accepted as an alternative to the Bearer header for monitoring systems that cannot set an `Authorization` header.
|
|
|
|
### Basic authentication
|
|
|
|
```toml
|
|
[diagnostics_endpoint]
|
|
enabled = true
|
|
allowed_networks = ["127.0.0.1/32", "::1/128"]
|
|
auth = "basic"
|
|
```
|
|
|
|
```bash
|
|
DIAGNOSTICS_ENDPOINT__BASIC_USERNAME='rustbox-monitor'
|
|
DIAGNOSTICS_ENDPOINT__BASIC_PASSWORD='a-long-random-monitoring-password'
|
|
```
|
|
|
|
Basic Auth must only be used over HTTPS/TLS (directly or at a trusted reverse proxy), because the credentials are not encrypted by the Basic scheme itself.
|
|
|
|
## Networks and reverse proxies
|
|
|
|
The network allowlist accepts IPv4/IPv6 addresses and CIDRs. Single addresses may also be specified in TOML.
|
|
|
|
Example monitoring network:
|
|
|
|
```toml
|
|
[diagnostics_endpoint]
|
|
enabled = true
|
|
allowed_networks = [
|
|
"127.0.0.1/32",
|
|
"::1/128",
|
|
"10.42.20.0/24",
|
|
"2001:db8:42::/64",
|
|
]
|
|
auth = "token"
|
|
```
|
|
|
|
For a reverse proxy, configure `trusted_proxies` separately. RustBox only accepts `X-Forwarded-For` for diagnostics when the directly connected proxy is trusted. If proxy headers arrive from an untrusted peer, diagnostics fails closed with HTTP `403` rather than treating the proxy address as the real client.
|
|
|
|
```toml
|
|
trusted_proxies = "127.0.0.1/32,10.42.0.0/16"
|
|
|
|
[diagnostics_endpoint]
|
|
enabled = true
|
|
allowed_networks = ["10.42.20.0/24"]
|
|
auth = "token"
|
|
```
|
|
|
|
A trusted proxy should provide a valid `X-Forwarded-For`. `Forwarded` or `X-Real-IP` without `X-Forwarded-For` is deliberately rejected for this endpoint. In production, also restrict the diagnostics path at the firewall, ingress or reverse-proxy layer.
|
|
|
|
## Returned data
|
|
|
|
The report contains:
|
|
|
|
- application version, process uptime/start time and PID; host-wide CPU, memory, load-average and other machine statistics are deliberately not collected;
|
|
- effective non-secret configuration and feature switches;
|
|
- IMAP/SMTP/Sieve endpoint configuration without credentials;
|
|
- SQL backend, engine version, query latency, account/schema versions, SQLite file size or SQL pool size/idle connection count;
|
|
- session persistence health and aggregate active session/user/browser counts for SQL or Redis;
|
|
- local-node cached sessions, timestamps and mail endpoints; e-mail/user hash identifiers appear only when `include_user_identifiers=true`;
|
|
- filesystem/S3 configuration plus a live storage health probe;
|
|
- Tantivy/OpenSearch configuration; OpenSearch adds cluster health, node/shard counts and index document count;
|
|
- active IMAP IDLE task/subscriber counts;
|
|
- request counters and a bounded in-memory security window for failed logins, access denials, rate-limit responses and server errors;
|
|
- configuration warnings for risky settings such as disabled TLS verification, HTTP OpenSearch/S3, global diagnostics allowlists or production browser security headers being disabled.
|
|
|
|
## Secret handling
|
|
|
|
The endpoint never returns diagnostics credentials, mailbox passwords, session encryption keys, Redis passwords, S3 access/secret/session keys, OpenSearch passwords or full credential-bearing DSNs. URLs are reduced to scheme/host/port/path and query strings/userinfo are removed. Secret fields are represented only as `...Configured: true|false` where useful.
|
|
|
|
`include_user_identifiers=false` is the default. With that setting, local session details omit e-mail addresses and stable user hashes while aggregate user/session counts remain visible.
|
|
|
|
All endpoint responses, including `401` and `403`, carry `Cache-Control: no-store`, `Pragma: no-cache`, `X-Content-Type-Options: nosniff` and `Cross-Origin-Resource-Policy: same-origin` independently of the general browser security-header switch.
|
|
|
|
## Security signals
|
|
|
|
The telemetry buffer is bounded and in-memory; it stores only timestamp, event category, source IP and request path. It does **not** store credentials, Authorization headers, cookies, tokens, request bodies or mail content.
|
|
|
|
Within `security_window_secs` the report includes counts and top source IPs for:
|
|
|
|
- rejected login attempts;
|
|
- HTTP `429` rate limiting;
|
|
- HTTP `403` access denials;
|
|
- HTTP `5xx` server failures;
|
|
- denied diagnostics access.
|
|
|
|
`bruteForceSignal` becomes true after repeated login rejection/rate-limit activity, while `diagnosticsAttackSignal` flags repeated attempts against the management endpoint. These are preliminary operational signals, not an IDS or an account-lockout mechanism. In a multi-node deployment the counters are node-local; aggregate application/security logs in a central monitoring/SIEM system for cluster-wide detection.
|
|
|
|
## Process and host metrics
|
|
|
|
The diagnostics endpoint deliberately avoids host-wide metrics and platform-specific process probes. It does not read `/proc`, query system CPU/RAM/load averages, enumerate host resources or add a system-information dependency. The portable process section is limited to the RustBox PID, while process uptime/start time and request/security counters come from RustBox's own in-memory runtime state. This keeps the endpoint low-overhead and portable across supported operating systems.
|
|
|
|
## Probe cost
|
|
|
|
The endpoint performs live backend checks, so it should not be polled at sub-second frequency:
|
|
|
|
- SQL executes lightweight health/version/schema queries;
|
|
- Redis uses `PING` plus bounded `SCAN`/`MGET` of at most 5000 session records and reports `truncated=true` if the cap is hit;
|
|
- S3/filesystem health writes and deletes a tiny temporary `_health/...` object;
|
|
- OpenSearch queries cluster health and the configured index count;
|
|
- each backend probe has its own `probe_timeout_secs` timeout.
|
|
|
|
A polling interval around tens of seconds is appropriate for typical monitoring. Use the normal `/api/health` or `/api/ready` endpoints for very frequent liveness/readiness checks.
|