# Administrator system diagnostics endpoint

RustBox can expose a single administrator-only JSON endpoint for operational and security diagnostics. It is intended for local troubleshooting, monitoring agents and cluster operations. It is **disabled by default** and, when enabled without changing the allowlist, accepts only loopback clients.

The endpoint is separate from the user-facing `/api/diagnostics/*` routes. It does not require a RustBox mailbox session. Instead, access requires **both**:

1. the effective client IP to match `diagnostics_endpoint.allowed_networks`, and
2. the configured diagnostics token or Basic Auth credentials.

Requests from a non-allowed address receive HTTP `403`. Requests from an allowed address with missing/invalid diagnostics credentials receive HTTP `401`. Critical backend probe failures return HTTP `503`; healthy or degraded reports return HTTP `200` with `status` set to `ok` or `degraded`.

## Configuration

```toml
[diagnostics_endpoint]
enabled = false
path = "/api/system/diagnostics"
allowed_networks = ["127.0.0.1/32", "::1/128"]
auth = "token" # token | basic
# token = "replace-with-at-least-24-random-characters"
# basic_username = "rustbox-monitor"
# basic_password = "replace-with-a-long-random-password"
include_user_identifiers = false
probe_timeout_secs = 5
security_window_secs = 900
```

`path` must remain below `/api/system/` so it cannot collide with normal application API routes.

### Token authentication

Prefer keeping the token outside TOML, for example in `.env` or an orchestrator secret:

```bash
DIAGNOSTICS_ENDPOINT__ENABLED=true
DIAGNOSTICS_ENDPOINT__AUTH=token
DIAGNOSTICS_ENDPOINT__TOKEN='a-long-random-secret-at-least-24-characters'
```

Example request:

```bash
curl -fsS \
  -H "Authorization: Bearer $DIAGNOSTICS_TOKEN" \
  http://127.0.0.1:3001/api/system/diagnostics
```

`X-RustBox-Diagnostics-Token` is accepted as an alternative to the Bearer header for monitoring systems that cannot set an `Authorization` header.

### Basic authentication

```toml
[diagnostics_endpoint]
enabled = true
allowed_networks = ["127.0.0.1/32", "::1/128"]
auth = "basic"
```

```bash
DIAGNOSTICS_ENDPOINT__BASIC_USERNAME='rustbox-monitor'
DIAGNOSTICS_ENDPOINT__BASIC_PASSWORD='a-long-random-monitoring-password'
```

Basic Auth must only be used over HTTPS/TLS (directly or at a trusted reverse proxy), because the credentials are not encrypted by the Basic scheme itself.

## Networks and reverse proxies

The network allowlist accepts IPv4/IPv6 addresses and CIDRs. Single addresses may also be specified in TOML.

Example monitoring network:

```toml
[diagnostics_endpoint]
enabled = true
allowed_networks = [
  "127.0.0.1/32",
  "::1/128",
  "10.42.20.0/24",
  "2001:db8:42::/64",
]
auth = "token"
```

For a reverse proxy, configure `trusted_proxies` separately. RustBox only accepts `X-Forwarded-For` for diagnostics when the directly connected proxy is trusted. If proxy headers arrive from an untrusted peer, diagnostics fails closed with HTTP `403` rather than treating the proxy address as the real client.

```toml
trusted_proxies = "127.0.0.1/32,10.42.0.0/16"

[diagnostics_endpoint]
enabled = true
allowed_networks = ["10.42.20.0/24"]
auth = "token"
```

A trusted proxy should provide a valid `X-Forwarded-For`. `Forwarded` or `X-Real-IP` without `X-Forwarded-For` is deliberately rejected for this endpoint. In production, also restrict the diagnostics path at the firewall, ingress or reverse-proxy layer.

## Returned data

The report contains:

- application version, process uptime/start time and PID; host-wide CPU, memory, load-average and other machine statistics are deliberately not collected;
- effective non-secret configuration and feature switches;
- IMAP/SMTP/Sieve endpoint configuration without credentials;
- SQL backend, engine version, query latency, account/schema versions, SQLite file size or SQL pool size/idle connection count;
- session persistence health and aggregate active session/user/browser counts for SQL or Redis;
- local-node cached sessions, timestamps and mail endpoints; e-mail/user hash identifiers appear only when `include_user_identifiers=true`;
- filesystem/S3 configuration plus a live storage health probe;
- Tantivy/OpenSearch configuration; OpenSearch adds cluster health, node/shard counts and index document count;
- active IMAP IDLE task/subscriber counts;
- request counters and a bounded in-memory security window for failed logins, access denials, rate-limit responses and server errors;
- configuration warnings for risky settings such as disabled TLS verification, HTTP OpenSearch/S3, global diagnostics allowlists or production browser security headers being disabled.

## Secret handling

The endpoint never returns diagnostics credentials, mailbox passwords, session encryption keys, Redis passwords, S3 access/secret/session keys, OpenSearch passwords or full credential-bearing DSNs. URLs are reduced to scheme/host/port/path and query strings/userinfo are removed. Secret fields are represented only as `...Configured: true|false` where useful.

`include_user_identifiers=false` is the default. With that setting, local session details omit e-mail addresses and stable user hashes while aggregate user/session counts remain visible.

All endpoint responses, including `401` and `403`, carry `Cache-Control: no-store`, `Pragma: no-cache`, `X-Content-Type-Options: nosniff` and `Cross-Origin-Resource-Policy: same-origin` independently of the general browser security-header switch.

## Security signals

The telemetry buffer is bounded and in-memory; it stores only timestamp, event category, source IP and request path. It does **not** store credentials, Authorization headers, cookies, tokens, request bodies or mail content.

Within `security_window_secs` the report includes counts and top source IPs for:

- rejected login attempts;
- HTTP `429` rate limiting;
- HTTP `403` access denials;
- HTTP `5xx` server failures;
- denied diagnostics access.

`bruteForceSignal` becomes true after repeated login rejection/rate-limit activity, while `diagnosticsAttackSignal` flags repeated attempts against the management endpoint. These are preliminary operational signals, not an IDS or an account-lockout mechanism. In a multi-node deployment the counters are node-local; aggregate application/security logs in a central monitoring/SIEM system for cluster-wide detection.

## Process and host metrics

The diagnostics endpoint deliberately avoids host-wide metrics and platform-specific process probes. It does not read `/proc`, query system CPU/RAM/load averages, enumerate host resources or add a system-information dependency. The portable process section is limited to the RustBox PID, while process uptime/start time and request/security counters come from RustBox's own in-memory runtime state. This keeps the endpoint low-overhead and portable across supported operating systems.

## Probe cost

The endpoint performs live backend checks, so it should not be polled at sub-second frequency:

- SQL executes lightweight health/version/schema queries;
- Redis uses `PING` plus bounded `SCAN`/`MGET` of at most 5000 session records and reports `truncated=true` if the cap is hit;
- S3/filesystem health writes and deletes a tiny temporary `_health/...` object;
- OpenSearch queries cluster health and the configured index count;
- each backend probe has its own `probe_timeout_secs` timeout.

A polling interval around tens of seconds is appropriate for typical monitoring. Use the normal `/api/health` or `/api/ready` endpoints for very frequent liveness/readiness checks.