RustProxy/docs/MIGRATION.md

Deployment and configuration migration history

0.25.6 -> 0.25.7

There is no SQLite schema or saved-configuration migration. Version 0.25.7 reuses the existing admin_preferences document to persist backend-table visible columns and sort order per administrator account; search text remains transient. It also adds vhost enable/disable UI actions, themed scrollbars and additional Prometheus/Grafana observability. Existing vhost enabled values and backend configuration remain unchanged until an administrator explicitly uses the new action.

0.25.5 -> 0.25.6

There is no SQLite schema or saved-configuration migration. Version 0.25.6 adds the already-exposed upstream client generation/rotation diagnostics to the live Backends table and keeps the 0.25.5 backend client recycling defaults unchanged.

0.25.4 -> 0.25.5

There is no SQLite schema or saved-configuration migration. Existing cloudflare_client_ip vhost settings remain opt-in. When enabled and the direct peer is a verified Cloudflare edge, RustProxy now prefers a valid CF-IPCountry value for country analytics/logging and country policy evaluation, falling back to the installed GeoIP database when that header is unavailable or invalid.

0.25.3 -> 0.25.4

There is no SQLite schema or saved-configuration migration. Schema 8 and the existing 512-row configuration_history retention remain unchanged. Version 0.25.4 adds an authenticated restore operation that validates a retained document and saves it as a new revision; existing history is not rewound or overwritten.

0.25.2 -> 0.25.3

SQLite schema 7 is upgraded automatically to schema 8. The migration adds configuration_history; on first open the current revision is seeded into history, and subsequent successful configuration saves retain the newest 512 revisions for the dashboard revision-history view. No manual SQL is required.

New vhost/location access fields are additive. Missing method_policy, location acl_profile_id, location country policy and location inline ACL fields retain the prior behavior. RustProxy-generated request IDs are now SHA-256 values and are forwarded to origins as protected X-Request-ID; custom request-header rules cannot replace that correlation identifier.

0.25.1 -> 0.25.2

There is no SQLite schema or saved-configuration migration. Full backup creation may now run while the service is active; restore still requires RustProxy to be stopped. Backup archive version 2 preserves validated internal relative symlinks such as Certbot live/ entries pointing into the profile archive/ tree. Existing archive-version-1 backups remain accepted for restore.

The administration server keeps the same strict Host/Origin/request-marker/CSRF checks. The browser shell only adds a bounded one-shot same-origin reload when a restored or resumed authenticated tab receives auth.invalid_origin.

0.25.0 -> 0.25.1

There is no SQLite schema or saved-configuration migration. 0.25.1 reorganizes the Traffic dashboard and source tree only; existing Web Services, vhosts, traffic settings, cache state and administrator data remain compatible. Browser asset URLs remain /assets/app.js and /assets/vhosts.js even though their source is now split into bounded modules.

When updating a source checkout by extracting the archive over an older tree, remove the obsolete source monoliths src/admin/web/app.js and src/admin/web/vhosts.js if they remain. Checkouts older than the Web Server module split may also retain obsolete src/web_server.rs; remove it as well. Do not remove src/config.rs, src/docker.rs, src/runtime.rs, src/balance.rs, scripts/deploy.py, scripts/migrate_npm.py or docker-agent/main.go: these are intentional module/CLI entry points in 0.25.1. Build/cache artifacts such as target/ and __pycache__/ may be deleted safely, but .env, data/, databases and local certificates are runtime/operator state and must not be deleted as part of a source refresh.

0.24.10 -> 0.25.0

There is no SQLite schema migration for the first Web Server stage. web_services is an additive collection inside the existing revisioned configuration document, and missing vhost/location source fields continue to deserialize as Reverse Proxy. Existing 0.24.10 configurations therefore remain valid without manual conversion.

A Web Service must be created separately before a vhost/location can reference it. Document root, index files and directory-listing settings belong to that reusable Web Service and are not migrated into individual vhosts.

Historical deployment changes

0.15.1 -> 0.16.0

There is no SQLite schema migration. Existing vhosts, backends, cache files, administrator accounts and sessions are retained.

Custom locations are now interpreted only as overrides. An untouched legacy synthetic root prefix / location with no custom backend/cache/ACL/header/rewrite/local-response behavior is normalized away and replaced by the internal main-vhost fallback. A deliberately customized / location remains a real override. No manual edit is required for the normal legacy root entry.

Publishing the administration listener through a vhost remains explicit-risk functionality. For reserved session-cookie passthrough the public request must be HTTPS, the external hostname must be listed in administration.allowed_hosts, the backend must directly address the configured administration listener by IP, and that backend must have accept_local_listener_risk=true. Ordinary upstreams continue to have rustproxy_session removed in both directions.

Access-log JSON may contain the additional fields http_version, transport, websocket_bootstrap, websocket_phase, websocket_error, h3_reset_code and upstream_status. Cache/system metrics add memory-breakdown gauges; existing consumers should ignore unknown fields as usual.

0.15.0 -> 0.15.1

There is no schema or configuration-format migration. This patch changes only administration UI language-pack placement, Custom locations editing/help/preview and dashboard visualizations. Existing vhosts, locations, cache settings and SQLite data remain unchanged.

0.14.0 -> 0.15.0

There is no new SQLite schema version in 0.15.0. Renewal observations remain in the existing schema-v3 certificate_renewals table. The application no longer contains a runtime JSON importer.

When upgrade.sh / scripts/upgrade.py is used while RustProxy is stopped, an obsolete <data-dir>/certs/renewals.json observation sidecar is validated as a regular file and removed as the final source-upgrade step. It is not imported and no second legacy copy is retained. If the source-upgrade transaction fails before that final removal, the old sidecar remains untouched.

If a site replaces source manually instead of using the stopped upgrader, the old sidecar is simply ignored. Back it up and remove it. An empty renewal-observation table is safe: the ordinary scheduler reconstructs non-secret status/timestamps on its first 5–10 minute startup check, while certificates, keys, ACME account material and renewal profiles remain unchanged.

0.13.1 -> 0.13.2

Back up config.sqlite3 and the configured certificate files before updating. On first 0.13.2 open, RustProxy applies the additive SQLite schema-v3 migration and creates certificate_renewals; existing configuration, administrator accounts, sessions, audit records and revisions are retained. The table stores only non-secret renewal observation state (timestamps/status/message). If it is empty and the older private certs/renewals.json exists, validated observations are imported once and the sidecar is removed only after the database write succeeds. That one-time runtime compatibility path existed in 0.13.2/0.14.0 and is removed in 0.15.0; use the 0.15.0 stopped upgrader described above instead.

The managed update.sh path for 0.13.2+ also marks the stop as a software update so an enabled cold tier receives remaining hot cache objects even when ordinary cold_on_shutdown=false. This is a best-effort persistent handoff bounded by the managed stop timeout; no cache persistence is possible when cold storage is disabled.

The raw JSON editor moved to Settings -> Manual config. /configuration remains a browser compatibility alias, so old bookmarks do not require migration.

0.8.0 -> 0.8.1 cache behavior

The source default for cache.enabled changes to true. Existing saved configurations that explicitly contain enabled: false remain disabled; the new default applies to new or missing cache settings and is not a silent rewrite of an administrator choice. cache.max_object_bytes may now be configured up to 1 GiB when cache.max_bytes is at least as large. Browser conditional requests/no-cache refreshes now use conditional HIT or backend revalidation instead of blanket BYPASS; no-store remains a hard exclusion.

• Existing vhosts deserialize new traffic-policy fields with conservative/default-off behavior except http2_enabled and http3_enabled, which default true to preserve prior protocol behavior.
• The pre-production inline/global listener format is intentionally no longer readable. Recreate development listener configuration under listeners.endpoints and bind each vhost with frontend.listener_ids. No automatic vhost migration is performed.
• Existing backend TLS settings are preserved. Only newly created panel backend rows use the 0.13.0 default tls_verify:false; enable verification explicitly for origins that require certificate identity checks.
• First installs fetch verified GeoIP country files. Existing installations upgraded with update.sh keep their data directory; install the two verified files under data/geoip before enabling country rules if the installation predates 0.8.0.

No database schema migration is introduced. The release changes deployment defaults and control-plane startup behavior:

• admin setup-token can create the one-time setup token before first server start.
• New managed systemd installations default to 0.0.0.0:80 and 0.0.0.0:443. Existing installations keep the saved service unit during update.sh; update does not silently add port 443.
• Wildcard administration is opt-in only. Direct server startup requires --admin-listen 0.0.0.0:9090 serve --allow-remote-admin. The installer uses the equivalent --admin and --allow-remote-admin options. This is plain HTTP; keep it behind a firewall/private network or use a tunnel.

Back up private state before updating and run the real Rust check/test suite on the target build host.

---

0.7.1 -> 0.7.2 build fix

This is a distinct source release. It removes the absent-SWAGGER-environment panic; it does not migrate the database, alter vhosts or reset any account/certificate. Back up private data as usual and update the complete source tree, not just build.rs. The patch changes Cargo dependencies and adds a required build helper, so copying only the Rust file would leave an inconsistent package.

The existing dev.sh Python 3.10+ requirement now also applies to direct Cargo builds. No .cargo/config.toml or manual SWAGGER_UI_DOWNLOAD_URL export is required. The default pinned tag is fetched on the first build; a reviewed local tag archive remains supported for restricted build hosts. See API_ACCESS.md. No default to an older Swagger distribution is introduced.

Use the existing stopped-source update.sh --source-tree workflow or build a new binary and use the managed-service updater as documented in INSTALL.md. dev.sh backs up an older Cargo.lock before regenerating it; do not fake the new lockfile by replacing its root version text. A valid new lockfile is not included. Rebuild/restart after successful tests; the old process cannot load embedded changes. There is no need to clear application data or certificates, or to run cargo clean just to remove this panic. Test the real admin session and /docs after building.

---

0.7.0 -> 0.7.1 access changes

No database/configuration schema migration is introduced by this patch. Back up existing data/certificates and use the existing install/update instructions. The reported release is source-only; compile and test before installing a binary.

Operational JS and full translation packs are now session-protected. Old tabs should be reloaded after upgrading. Automation reading /api/auth/status.version or /healthz.version must use authenticated /api/status instead. Dedicated Prometheus scraping retains its own CIDR/token policy and remains independent of browser sessions. No reverse-proxy vhost auth policy is relaxed by this patch.

In 0.7.1 the Swagger pin depended on .cargo/config.toml. That requirement is removed in 0.7.2 (see the build notes above). Source-tree updates still back up and replace that managed directory; retain any unrelated local Cargo settings from your private backup. Read API_ACCESS.md for current build inputs.

Run python3 scripts/admin_audit.py --url http://127.0.0.1:9090 after deployment. It prints only status checks, no response bodies or credentials.

---

Updating, backup and recovery

0.7.0: renewal and service deployment

See INSTALL.md for separate installer/updater commands and complete code-plus-data rollback. Older source deployments retain update.sh --source-tree. Back up the entire private state, including Certbot profiles, before any update. Do not overlay a new binary onto a live, untested database without a recovery plan.

Managed 0.6 certificate issuer metadata defaults to automatic renewal in 0.7. The scheduler starts after a 5-10 minute delay and reuses existing consented profiles. Imported PEM without issuer metadata is not scheduled. Review auto-renew flags, service PATH, DNS permissions and HTTP-01 reachability before starting. The new process does not implicitly switch staging certificates to production.

Missing WebSocket flags preserve existing enabled behavior. New UI-authored vhosts default to disabled. Existing backend members remain in the normal pool; new dedicated location origins default to pool_eligible:false. Basic Auth is opt-in, requires actual TLS by default and combines with (rather than bypasses) IP ACLs.

Normal dev/server startup no longer starts the demonstration origin. Existing sample vhosts are not silently deleted: remove or replace them deliberately, or explicitly enable the protocol fixture in an isolated old test installation. New databases show a neutral informational landing page with no sample backend.

New fields are not a promise of backward parser compatibility. Downgrade by restoring both old code and the pre-upgrade private-data snapshot. No verified Rust build is supplied; run check/test and server acceptance before rollout.

0.6.0: historical vhost migration note

0.6.0 originally shipped a one-way importer from the pre-vhost development format. That importer is removed in 0.8.0. Current builds accept only vhosts, named listeners.endpoints and frontend.listener_ids. Development databases containing top-level route/backend documents or inline frontend listeners must be recreated or manually rewritten before starting 0.8.0. No automatic compatibility conversion runs.

The dedicated Prometheus listener remains opt-in/disabled by default. Settings -> Prometheus controls it. A restart is required for enable/bind/token-variable changes. Certificate provider metadata remains additive and secrets remain outside exported configuration.

0.5.0 behavior changes

Back up the database and configured PEM pairs before upgrading. The managed updater replaces the full source tree, not live data. Build and test on staging. No verified binary or fabricated Cargo.lock is supplied.

TLS bindings with omitted http3 now enable UDP with defaults; older versions omitted disabled options when serializing, so review ALL TLS bindings. To retain TCP-only operation, explicitly save http3: null on the named TLS listener before restarting the new process. Shared/wildcard IP-port bindings must agree on all protocol flags and H3 limits. A UDP bind failure is not silently ignored. HTTP/1.1 and optional HTTP/2 remain on TCP.

diagnostic_headers.hide_mock_origin was removed and is no longer accepted or exported; use explicit response-header rules if needed. X-Mock-Origin is a normal upstream field; remove it with ordinary response-header rules when desired. Request ID remains off by default. The removed 0.17 diagnostic_headers.revision field is no longer part of the configuration schema. Custom Server values are NOT automatically rewritten to the new binary version.

Cache contents are process-local. Reserved admin-cookie removal now occurs before cache eligibility; application cookies still bypass. Domain counters reset on configuration publication. See CACHE_DIAGNOSTICS.md for explaining BYPASS.

ACME/RFC2136 is optional and adds external Python/Certbot/plugin requirements. No DNS update or CA request happens simply by upgrading. Issuance is explicit, staging-first and does not activate a certificate without import. Current encrypted full backup includes the durable ACME profile/account state and Secret Store material required for DNS-provider renewal; the historical dev.sh backup helper does not. Use rust-proxy backup create for current installations. Automatic renewal behavior is documented in RENEWAL.md.

Reported auth module compilation failure

0.0.1 used src/auth.rs. 0.0.2 moved the implementation into src/auth/mod.rs and src/auth/store.rs. Overlay extraction can retain both layouts and cause Rust E0761. Missing Store::session_at, admin_record, end_session and revoke_all_sessions messages can follow because the implementation module did not load. The 0.0.2 archive itself did not contain both auth layouts.

0.4.0 retains an explicit #[path = "auth/mod.rs"] declaration, including for direct Cargo invocations. Preflight also detects mixed source trees. It recognizes the exact original auth file by SHA-256 and moves it to .upgrade-backup/legacy-auth/auth.rs.0.0.1. Unknown custom source is left untouched and the development script stops with instructions. Never delete custom files just to silence a compiler error.

bash dev.sh doctor
bash dev.sh repair

These commands do not require Rust. Repair is deliberately limited; a safe whole source-tree update is preferable to extracting individual files over a checkout.

Recommended source update

Extract the new archive beside, not inside, the installed project. On installations that provide the current encrypted backup command, stop RustProxy and create the backup outside the data directory before replacing source files:

rust-proxy --data-dir /home/mg/RustProxy/data backup create \
  --output /home/mg/RustProxy-backup-before-upgrade.rpb
unzip RustProxy-0.4.0.zip -d /home/mg/rustproxy-040-staging
cd /home/mg/rustproxy-040-staging/RustProxy

scripts/backup.py is a legacy release-test helper and is not a complete Secret Store backup. For current backup/restore behavior see SECRET_STORE_BACKUP.md.

A missing database means there is no database to back up. Do not initialize an empty replacement accidentally. For data stored elsewhere, use the actual directory.

Stop RustProxy using its terminal or service manager. Then:

bash upgrade.sh /home/mg/RustProxy --stopped
# External data directory: also pass --data-dir /actual/private/data
cd /home/mg/RustProxy
bash dev.sh test
bash dev.sh

The updater checks a process lock on POSIX, requires the explicit --stopped acknowledgement and rejects nested source/target paths and managed symlinks. It preserves .env, data, .git, target and unmanaged files. Source, scripts, tests, examples and documentation are replaced as whole managed paths. Old managed files remain in .upgrade-backup/before-0.4.0-<identifier>. An interrupted copy exception triggers source rollback. Abrupt power loss is not a transactional filesystem update; inspect the backup and source tree before restarting.

The updater does not evaluate the old .env or discover arbitrary external data paths. Pass --data-dir when necessary, and stop every process yourself before changing source. It does not perform a database migration or backup automatically. Custom source edits are backed up, not merged.

Database compatibility

RustProxy 0.13.2 advances the SQLite schema from version 2 to version 3 with one additive certificate_renewals table for non-secret ACME observation state. The existing 0.0.1-to-0.0.2 auth migration still runs when needed. Configuration-document additions remain defaulted/non-destructive. Existing accounts, policies, revisions and configuration remain intact. Starting with 0.15.0 the application reads renewal observations only from this table; historical JSON observation state is handled, when present, by the stopped source-upgrade remove described above rather than by the running service.

The generated default certificate is created on first TLS-enabled startup, in <data-dir>/certs. Its key is never overwritten by startup. A partial or invalid pair causes a clear startup error; it is not silently discarded. Demo settings never overwrite an existing configuration, including an empty existing one.

Backup contents and restore

Current RustProxy releases use the password-encrypted full-backup format documented in SECRET_STORE_BACKUP.md. The supported interface is rust-proxy backup create and CLI-only rust-proxy backup restore; it includes the encrypted Secret Store, key material required to recover it, certificates and other durable application state while excluding cache, logs and runtime files.

The historical ./dev.sh backup / scripts/backup.py snapshot helper described by older release notes is not sufficient after Secret Store migration and is retained only for offline compatibility/release tests.

Downgrade

Older versions reject unknown configuration fields such as tls, protocol or tls_ca_pem. Restoring old source alone is therefore not a safe downgrade after saving new configuration. Restore the matching pre-upgrade database and environment from a verified backup, with the server stopped. Do not edit password hashes or schema versions manually.

0.0.6 -> 0.1.0 document and source changes

No new SQL schema version is introduced. Old documents default to inherited listeners, disabled cache/compression and preserve-Server behavior. New optional route cache policy does not change existing route behavior. Existing account, revision, TLS and ACL state is retained.

Managed paths now include build.rs, langs, deploy and make_zip.py. Extra *.json language packs from the installed langs directory are copied into the new tree when the new distribution does not contain the same filename. Shipped packs are updated, with old bytes preserved in the source backup. Invalid custom packs can correctly block a build; review them with scripts/langs_check.py. No symlinked language pack is followed. The updater never copies a database from the package.

The old lockfile is preserved in the managed source backup and a new dependency graph is generated by Cargo on first build. No 0.1.0 resolved Cargo.lock is included because Cargo was unavailable here. Keep the actual newly resolved file afterward.

Saving any 0.1.0 document serializes its new fields. Older binaries use strict unknown-field rejection, so rolling back only the source/binary may fail startup. Restore the pre-upgrade database/certificates into a new private directory and keep the newer data separately. Do not directly edit live SQLite to remove fields.

A saved invalid listener address or missing low-port permission can prevent startup. The online config CLI cannot repair an offline server. Keep a pre-change backup and use the documented stopped-server restore path; remove persisted listener overrides through Settings while the service is still running before changing deployment assumptions. The cache itself is RAM only and is not backed up.

0.1.0 -> 0.2.0: compression, vhosts and pool behavior

The SizeAbove u16/u64 mismatch is fixed by replacing that predicate with a custom u64 implementation. Do not use try_into().unwrap() or truncate the setting to work around the old source: configured thresholds above 65535 must remain valid. This source change removes that call, but is not a successful compiler result; run bash dev.sh check and bash dev.sh test on a Rust-enabled staging machine.

SQL schema version stays 2; accounts, ACLs, certificates and revisions are retained. New vhosts defaults empty; named TLS listeners default to H2/H3 capability when those fields are omitted. Legacy routes continue on the global listeners and are never silently migrated or exposed on additional IPs. The editor's explicit migration action converts one unambiguous route group in a single revision and keeps the migrated route group's existing health behavior.

New vhosts default to TCP health checks and remain unavailable until rise checks succeed. Plan the correct path/status/timeouts before migration. Health/counter/affinity observations are ephemeral. They survive compatible edits, not restarts; new/changed/re-enabled origins require new checks. Cache and idle upstream pools are not carried across config publication. Admitted exchanges finish with their old snapshot and are not replayed on a different backend.

HTTPS origins previously configured http2 now retain HTTP/1.1 ALPN compatibility. Plaintext http2 is still strict h2c. Per-listener IP/port/TLS/HTTP2 changes require restart. During an address move, hot routing can deny the previous destination before the new socket is bound. Plan a maintenance window, verify assigned IPs, privileges/certificates and retain an offline-restorable backup.

After saving any 0.2.0 document, 0.1.0 can reject new fields even without a vhost, because https_http2 is serialized. Downgrade requires matching pre-0.2.0 data, source and environment. Never remove fields by editing a live SQLite database. The archive contains no fabricated lockfile, executable, database or private key. Generate and retain the real dependency graph before acceptance/deployment.

0.2.0 -> 0.3.0

The reported json_internal! recursion error is addressed by splitting status_payload into small JSON values and inserting feature flags through a map. Do not overlay only this function: this release also adds the query Axum feature, src/pages.rs, management.js, new configuration fields and matching UI assets. The earlier u64 compression predicate remains in place. Compilation has not been executed in the packaging environment; this is not a successful-build assertion.

Existing database documents deserialize with empty/default pages, vhost cache and certificate_id fields. No SQL schema migration is added. Per-vhost certificate selection rewrites the effective SNI map in a snapshot, not the files. Upload is a staging action: Save must publish a validated configuration before traffic uses it. Canceling the editor leaves an unbound pair, not an active certificate change.

Use the complete backup/whole-source update procedure above. The existing backup copies configured keys and fallback files. Unbound upload staging files are not in the configuration and therefore are not part of that configured-certificate backup. Retain them separately only when needed. Never delete a filename still referenced by the configuration or currently serving snapshot; stop the process before manual cleanup and preserve a backup of the full private certificate directory.

New defaults do not activate custom pages or vhost cache overrides. An explicitly disabled global cache can still be overridden by one vhost; the global byte/entry/object limits still bound it. Current releases rotate cache compatibility only for configuration changes that can alter cached/origin semantics; unrelated control-plane publication preserves reusable objects. The dedicated per-vhost Purge action remains selective.

Downgrade requires restoring the earlier database/configuration together with its matching source: 0.2.0 rejects the new fields. Do not merely relabel a new document. The source archive contains no fabricated lockfile. Resolve and retain Cargo.lock on the first compiler-enabled build and run all tests before restarting production.

0.3.0 -> 0.4.0

Use the full-source update above, not an overlay of a few Rust files. This revision adds Quinn/h3 dependencies, new modules and a new embedded integrations.js asset. No successful compiler run or dependency resolution is claimed. Generate and retain a real Cargo.lock, then run check and all tests before acceptance.

SQL schema stays 2. Proxy sockets are persisted only in listeners.endpoints; a new database is seeded from startup CLI listener addresses. Existing development documents using the removed listener format are rejected rather than migrated. Tinyauth and vhost Server policy continue to use their normal defaults. New diagnostic defaults remove x-request-id from client responses unless it is enabled explicitly where needed. Access logs retain request IDs. The old revision diagnostic field/header behavior was removed in 0.18.2. Forwarding headers are rebuilt from actual peers unless known proxy CIDRs are explicitly configured.

Reference named listener IDs per vhost instead of duplicating bindings. There is no inline/global compatibility conversion path in 0.8.0. Scope names have no firewall meaning. Changed sockets, ALPN or H3 limits require restart; opening UDP is necessary for H3. Saving intent does not prove that an OS bind or UDP firewall rule will succeed.

The certificate directory/fallback is prepared even with HTTP-only startup, so library uploads can be staged before enabling TLS. The generated fallback is self-signed development material and does not automatically open an HTTPS socket. Import the PEM pair in Certificates, save inventory, then select its ID in SSL. Unbound staged files are not automatically backed up; preserve private files needed for rollback and never remove any currently referenced certificate/key.

Tinyauth-protected vhosts bypass shared cache regardless of cache toggles. Failures deny access instead of serving cached/static content. Configure its verifier URL, public login URL and explicit private-origin permission before enabling it on an existing service. Test login, refusal, logout/revocation and WebSockets on staging.

The examples are complete replacement documents for disposable test databases, not merge patches. Saving a 0.4 document serializes new fields; older binaries may reject them. Downgrade requires matching pre-upgrade database, certificates, environment and source, with the process stopped. The source backup alone is not a database/configuration rollback.