v0.24.9

docs/ACME_RFC2136.md

@@ -96,16 +96,17 @@ can leave challenge TXT records: inspect DNS and clean only the failed challenge
 values, not unrelated records. Only the latest job status lives in memory; restart
 loses that status, not the private Certbot files.
 
-TSIG is supplied through stdin to the helper, never argv, SQLite, exports or job
-responses. The browser clears the secret field after submission and does not put
-it in local/session storage. Private credentials/account files are deliberately
-retained for renewal; protect them and private logs as secrets. Keep unused staged
-pairs under operator control; active files are never silently deleted.
+TSIG/API credentials are supplied through stdin to the helper, never argv, exports
+or job responses. RustProxy encrypts DNS-provider credentials in the Secret Store;
+Certbot-compatible `.ini` files are recreated with private permissions only for the
+external operation and removed afterwards. The browser clears submitted secret
+fields and does not put them in local/session storage. Protect the Certbot account
+state and private logs as secrets.
 
-The existing `dev.sh backup` includes SQLite and configured PEM files, **not the
-ACME account/profile directory**. Back up `certs/acme` separately while no issuance
-is active, preserving ownership/modes and encryption appropriate to private keys.
-Source upgrade leaves the data directory alone. Do not ship profiles in source ZIPs.
+Use `rust-proxy backup create` for current installations. The encrypted full backup
+includes the durable ACME profile/account state, certificates and Secret Store key
+material needed for renewal. The historical `dev.sh backup` helper is not sufficient
+after Secret Store migration. See SECRET_STORE_BACKUP.md.
 
 ## API and local helper
 

docs/API_ACCESS.md

@@ -1,4 +1,4 @@
-# Administration API access - 0.24.8
+# Administration API access - 0.24.9
 
 ## Sign-in boundary
 

docs/ERROR_LOGGING.md

@@ -0,0 +1,39 @@
+# Optional error-file logging
+
+RustProxy can write an additional local file containing only `ERROR` events. This is an optional sink layered on the existing tracing/logging pipeline; journald/systemd and other configured sinks remain independent.
+
+## Configuration
+
+The settings document contains:
+
+```json
+{
+  "error_log": {
+    "enabled": true,
+    "path": "/var/log/rustproxy/error.log",
+    "format": "JSON"
+  }
+}
+```
+
+`format` is `JSON` or `RAW`. The feature is disabled by default. The administration Settings UI exposes the same enable/path/format controls and applies changes at runtime.
+
+The sink accepts only `ERROR` events. JSON is one object per line; RAW is one human-readable line per event.
+
+## Runtime behavior
+
+Events are handed to a bounded queue and written by one asynchronous worker. The request path does not perform the file write. When the queue is full, new error-file events are dropped and counted instead of allowing unbounded memory growth.
+
+Filesystem errors do not stop RustProxy. The worker reports failures through the existing logger, rate-limits repeated diagnostics and retries opening the file. Changing the path or disabling the sink closes the current file; a format change takes effect without an application restart.
+
+Created log files use mode `0600` on POSIX. Symlink/non-regular targets are refused.
+
+## Secret redaction
+
+The common logging pipeline redacts sensitive keys and values before they reach the error-file sink. This includes authorization credentials, provider tokens/passwords, backup passwords, private-key PEM material and Secret Store/master-key related values. Structured secret fields are replaced with `[REDACTED]`.
+
+Error messages from dependencies should still be treated as untrusted diagnostic input; do not deliberately embed secrets in error text.
+
+## Rotation
+
+RustProxy does not implement file rollover, retention or compression. Use the system `logrotate`. The Debian package configuration in `debian/rustproxy.logrotate` already covers `/var/log/rustproxy/*.log` and uses `copytruncate`, so the default error-log path is included without a RustProxy restart.

docs/INSTALL.md

@@ -1,4 +1,4 @@
-# Installing and updating RustProxy 0.24.8
+# Installing and updating RustProxy 0.24.9
 
 The scripts prepare a Linux/systemd installation. This source archive is **not a
 verified production build**: Rust/Cargo, real CA operations, live systemd startup

docs/MIGRATION.md

@@ -199,10 +199,10 @@ configuration publication. See CACHE_DIAGNOSTICS.md for explaining BYPASS.
 
 ACME/RFC2136 is optional and adds external Python/Certbot/plugin requirements.
 No DNS update or CA request happens simply by upgrading. Issuance is explicit,
-staging-first and does not activate a certificate without import. Existing
-`dev.sh backup` does not include the private `certs/acme` account/TSIG profiles;
-back those up separately when issuance is idle. No automatic renewal is included.
-See ACME_RFC2136.md for limitations and restoration responsibilities.
+staging-first and does not activate a certificate without import. Current encrypted full backup includes the durable ACME profile/account state and
+Secret Store material required for DNS-provider renewal; the historical `dev.sh backup`
+helper does not. Use `rust-proxy backup create` for current installations. Automatic
+renewal behavior is documented in RENEWAL.md.
 
 
 ## Reported auth module compilation failure
@@ -230,16 +230,20 @@ source-tree update is preferable to extracting individual files over a checkout.
 
 ## Recommended source update
 
-Extract the new archive beside, not inside, the installed project. First back up
-the existing database using the new backup script. The destination must not exist:
+Extract the new archive beside, not inside, the installed project. On installations
+that provide the current encrypted backup command, stop RustProxy and create the
+backup outside the data directory before replacing source files:
 
 ```bash
+rust-proxy --data-dir /home/mg/RustProxy/data backup create \
+  --output /home/mg/RustProxy-backup-before-upgrade.rpb
 unzip RustProxy-0.4.0.zip -d /home/mg/rustproxy-040-staging
 cd /home/mg/rustproxy-040-staging/RustProxy
-python3 scripts/backup.py --data-dir /home/mg/RustProxy/data \
-  /home/mg/RustProxy-backup-before-0.4.0
 ```
 
+`scripts/backup.py` is a legacy release-test helper and is not a complete Secret
+Store backup. For current backup/restore behavior see `SECRET_STORE_BACKUP.md`.
+
 A missing database means there is no database to back up. Do not initialize an empty
 replacement accidentally. For data stored elsewhere, use the actual directory.
 
@@ -283,25 +287,15 @@ never overwrite an existing configuration, including an empty existing one.
 
 ## Backup contents and restore
 
-`./dev.sh backup NEW_DIRECTORY` uses SQLite's online backup API, so committed WAL
-content is included. The result contains a private `config.sqlite3`, `backup.json`
-with hashes/revision metadata and any configured/default certificate files under
-`certs/`. Unreferenced staged certificate files are not included. Do not rotate or
-manually replace certificate files while taking a backup. This is a consistent
-SQLite snapshot plus file copies, not one cross-filesystem transaction.
+Current RustProxy releases use the password-encrypted full-backup format documented
+in `SECRET_STORE_BACKUP.md`. The supported interface is `rust-proxy backup create`
+and CLI-only `rust-proxy backup restore`; it includes the encrypted Secret Store,
+key material required to recover it, certificates and other durable application
+state while excluding cache, logs and runtime files.
 
-Backups contain credential hashes, session records and private keys. Keep them
-private, encrypted at rest as appropriate, and outside public web directories.
-They do not contain `.env`, source edits or a generated dependency lockfile; preserve
-those separately. Existing backup destinations and certificate symlinks are refused.
-
-To restore, stop the server, verify backup hashes and keep the current data directory
-as a separate recovery copy. Restore into a **new private directory**, including
-`config.sqlite3` and `certs/` together, then point `RUST_PROXY_DATA_DIR` to it.
-Use directory mode 700 and file mode 600. Do not copy stale `-wal` or `-shm` files
-from another database beside the restored snapshot. Start the matching version and
-run `./dev.sh admin logout-all` or reset the password to revoke restored sessions.
-Verify routes, TLS identities and administration before resuming traffic.
+The historical `./dev.sh backup` / `scripts/backup.py` snapshot helper described by
+older release notes is not sufficient after Secret Store migration and is retained
+only for offline compatibility/release tests.
 
 ## Downgrade
 

docs/README.md

@@ -12,6 +12,8 @@ This index separates current operational documentation from historical/version-l
 - [CACHE.md](CACHE.md) and [CACHE_DIAGNOSTICS.md](CACHE_DIAGNOSTICS.md) - cache semantics and diagnostics.
 - [PROMETHEUS.md](PROMETHEUS.md), [HTTP3.md](HTTP3.md), [STREAMS.md](STREAMS.md) - monitoring and transport features.
 - [SECURITY.md](SECURITY.md) - current security model and limitations.
+- [SECRET_STORE_BACKUP.md](SECRET_STORE_BACKUP.md) - encrypted credentials and password-protected full backup/restore.
+- [ERROR_LOGGING.md](ERROR_LOGGING.md) - optional asynchronous ERROR-only file sink and external log rotation.
 - [TEST_REPORT.md](TEST_REPORT.md) - release verification status and explicit environment limitations.
 - [0.24.8 dashboard cache-memory audit](V0.24.8_AUDIT.md) - overview HOT/transient/COLD occupancy histories and compact RSS accounting help.
 - [0.24.7 transform resource-control audit](V0.24.7_AUDIT.md) - configurable transformation memory/concurrency budgets and resource presets.

docs/SECRET_STORE_BACKUP.md

@@ -0,0 +1,58 @@
+# Secret Store and encrypted full backup/restore
+
+RustProxy keeps application credentials in a central encrypted Secret Store. Provider and ACME secrets are not stored as plaintext configuration values in SQLite.
+
+## Secret Store
+
+Secret records live in the `secret_store` SQLite table and contain an identifier, provider/type, encryption version, nonce, ciphertext, metadata and timestamps. Values are encrypted with AES-256-GCM using a random local data key stored at:
+
+```text
+<data-dir>/secrets/master.key
+```
+
+The key file is created with mode `0600`; its directory is private. The application loads it during normal startup, so normal operation does not require an interactive password.
+
+Existing ACME DNS credential files are migrated on startup for Cloudflare, OVH and RFC2136. After successful import, the legacy plaintext credential file is removed. Certbot-compatible credential files are recreated only as short-lived private files around the external operation and removed afterwards.
+
+Do not copy only `config.sqlite3` as a Secret Store backup: the local key material is also required. Use the encrypted full-backup command below.
+
+## Create a full backup
+
+Stop RustProxy first so the data directory lock is available. The output file must be outside the RustProxy data directory and must not already exist.
+
+```bash
+rust-proxy --data-dir /var/lib/rustproxy backup create \
+  --output /private/rustproxy-backup.rpb
+```
+
+When run from a TTY, RustProxy asks for the backup password without putting it in process arguments or shell history. Press Enter at the password prompt to generate a strong random password. A generated password is printed exactly once and is required for restore.
+
+For non-interactive `backup create`, RustProxy generates the password automatically and prints it once to standard output. Protect that output as a secret.
+
+The encrypted backup contains the durable application state needed for recovery, including:
+
+- a consistent SQLite snapshot,
+- configuration and Secret Store records,
+- backup-wrapped Secret Store key material,
+- certificates and private keys,
+- ACME state and durable provider data,
+- other durable RustProxy data accepted by the backup manifest.
+
+It excludes rebuildable/runtime data such as cache, logs, temporary files, sockets, PID/lock files and SQLite WAL/SHM sidecars.
+
+The outer container derives a backup key with Argon2id and encrypts authenticated chunks with AES-256-GCM. The backup password is independent from the local Secret Store key.
+
+## Restore
+
+Restore is CLI-only and requires an interactive TTY for the password:
+
+```bash
+rust-proxy --data-dir /var/lib/rustproxy backup restore \
+  /private/rustproxy-backup.rpb
+```
+
+Before replacing the destination, RustProxy verifies the encrypted container, password/authentication tags, manifest, checksums, backup format, RustProxy/schema compatibility, restored SQLite database and decryptability of all Secret Store records.
+
+Files are first restored into a private staging directory. Only after validation succeeds is the staged data directory atomically activated; activation failure attempts to roll back to the previous directory. Sensitive restored files use restrictive permissions.
+
+A lost backup password cannot be recovered by RustProxy.

docs/TEST_REPORT.md

@@ -1,11 +1,20 @@
-# Verification report - RustProxy 0.24.8
+# Verification report - RustProxy 0.24.9
 
 ## Scope
 
-RustProxy 0.24.8 expands the overview cache diagnostics with HOT RAM occupancy against its limit, transient cache RAM histories for total/fill/COLD queue/COLD I/O, and COLD storage occupancy against its disk budget. The process/cache RSS accounting explanation is now behind an accessible information control. It retains the 0.24.7 transform resource controls and the 0.24.6 cache/background-worker memory hardening.
+RustProxy 0.24.9 implements the Secret Store + password-encrypted full backup/restore and optional ERROR-only file logging specifications. Secret Store adds schema version 7, encrypts ACME DNS-provider credentials with a local master key, migrates legacy provider `.ini` files, and recreates plaintext credential files only around the external Certbot operation. Full backup/restore uses Argon2id plus authenticated AES-256-GCM encryption, an explicit manifest, Secret Store key wrapping, staged validation and atomic data-directory activation. Error-file logging is a separate, disabled-by-default asynchronous bounded sink with JSON/RAW formats, central secret redaction and external `logrotate` rotation.
+
+The release also retains the 0.24.8 overview cache diagnostics, 0.24.7 transform resource controls and 0.24.6 cache/background-worker memory hardening described by the earlier audit material below.
 
 ## Source contracts reviewed
 
+- SQLite schema version 7 includes the generic encrypted `secret_store` table; stored secret values are AES-256-GCM ciphertext with per-record nonces and the application master key lives outside SQLite in a private file.
+- Cloudflare, OVH and RFC2136 ACME credentials are validated, imported into Secret Store, decrypted only when required, and legacy provider `.ini` files are removed after successful migration. Certbot credential files are temporary and removed in `finally` paths.
+- `rust-proxy backup create` writes a new `.rpb` file outside the data directory, derives an independent backup key with Argon2id, encrypts authenticated chunks with AES-256-GCM, excludes cache/log/runtime data and wraps the Secret Store master key inside the encrypted backup.
+- `rust-proxy backup restore` requires an interactive password, decrypts into private staging, validates manifest/checksums/schema/configuration/Secret Store decryptability before activation, then switches the data directory by rename with rollback on activation failure.
+- `error_log` is independent from normal access logging, defaults disabled, accepts only `ERROR`, supports JSON/RAW, uses a bounded queue and a single asynchronous writer, handles filesystem failures without terminating RustProxy, and relies on system `logrotate` rather than application rollover.
+- The stderr/journald path, access-log sinks and the optional error-file sink share secret-redaction helpers for authorization/provider/password/private-key material.
+- The release checker preserves the existing dashboard contract while using bundled roots plus common system CA bundles (`SSL_CERT_FILE`, `CURL_CA_BUNDLE`, FreeBSD/OPNsense and Linux paths), follows at most five credential-free HTTP(S) redirects without HTTPS-to-HTTP downgrade, and reports network/TLS/HTTP/parser failures at `ERROR` level so the optional error-file sink receives them.
 - The overview Cache RAM block keeps separate histories for HOT allocated bytes, transient RAM, fill bytes, COLD queue bytes, COLD I/O bytes and COLD disk bytes; the disk series is never included in process RAM accounting.
 - HOT and transient histories are rendered against their runtime limits, while COLD storage is rendered against the configured COLD byte budget.
 - The long `memoryAccountingHint` remains translated and available on demand through the Process information control instead of consuming permanent dashboard height.
@@ -32,10 +41,13 @@ RustProxy 0.24.8 expands the overview cache diagnostics with HOT RAM occupancy a
 | Browser JavaScript `node --check` | PASS for all administration JS bundles |
 | OpenAPI/language JSON parse | PASS |
 | Shell `bash -n` | PASS |
-| Python syntax / bytecode compile | PASS |
+| Python syntax / bytecode compile, including ACME provider worker | PASS |
+| Secret Store / encrypted backup / error-log / portable update-check source-contract regression checks | PASS |
+| SQLite schema 7 migration smoke (`secret_store`) | PASS |
 | Docker agent `go test ./...` / `go vet ./...` / `go build ./...` | PASS |
 | `python3 scripts/check_source.py` | PASS |
 | `python3 scripts/release_smoke.py` | PASS |
+| FreeBSD/OPNsense live release-check execution | NOT RUN in this workspace; source contract and platform CA paths are covered, target runtime confirmation is still required |
 | `./dev.sh fmt` / `cargo fmt --all` | BLOCKED: Rust/Cargo toolchain unavailable in this workspace |
 | `./dev.sh check` / `cargo check --locked --all-targets` | BLOCKED: Rust/Cargo toolchain unavailable in this workspace |
 | `./dev.sh test` Rust phase | BLOCKED: Rust/Cargo toolchain unavailable in this workspace |