v0.2.0

README.md

@@ -1,4 +1,4 @@
-# RustBox 0.1.2
+# RustBox 0.2.0
 
 RustBox is a webmail derived from `c0h1b4/oxi`. The consumed Oxi revision is recorded in `UPSTREAM_BASE`.
 
@@ -34,6 +34,23 @@ rustbox/
 
 `build.rs` combines the CSS modules from `src/web/css/manifest.txt` and JavaScript modules from `src/web/js/manifest.txt`. Cargo embeds the generated CSS/JS bundles, HTML and SVG sprite in the RustBox executable. There is no standalone source `src/web/app.js`; `src/web/` is the single source of truth and Cargo generates the embedded bundles in `OUT_DIR`.
 
+## Storage and search in 0.2.0
+
+RustBox separates three concerns: SQL metadata, blob storage and search. The SQL engine does **not** decide whether `data_dir` may be used. A single-node PostgreSQL/MySQL deployment may still use filesystem blobs and local Tantivy. In cluster mode, local node state is rejected unless the configured backend is explicitly shared.
+
+Blob backends:
+
+- `storage.backend="filesystem"` — local disk in single-node mode; set `storage.shared=true` only for a real shared POSIX mount such as NFS, CephFS or GlusterFS.
+- `storage.backend="s3"` — AWS S3 or compatible APIs such as MinIO/Ceph RGW.
+
+Search backends:
+
+- `search.backend="local_tantivy"` — single-node.
+- `search.backend="opensearch"` — cluster-capable. Index mutations are written first to a durable per-tenant SQL outbox and processed with retry/backoff/dead-letter handling.
+- `search.backend="disabled"` — no full-text index.
+
+In a cluster using shared SQL + S3/shared POSIX + OpenSearch/disabled search + `pgp_enabled=false`, `data_dir` is not required as persistent node state. See `docker/README.md` for ready Compose variants.
+
 ## Configure mail servers
 
 Mail endpoints are configured by the administrator, not on the login screen. Users enter only email and password.
@@ -71,64 +88,67 @@ If the mail host settings are omitted, RustBox uses `localhost` with IMAP `993`
 
 Mail-size limits are intentionally separate. `max_attachment_size_mb` applies to each raw uploaded file, while `max_message_size_mb` is checked against the final RFC822/MIME byte stream immediately before SMTP submission. The latter therefore includes message bodies, headers, MIME boundaries, base64 expansion, inline CID parts and OpenPGP transformations. The composer displays only an estimate; the server-side final-MIME check is authoritative.
 
-## Database, cache, sessions and cluster traffic
+## Database, storage, search and cluster traffic
 
-RustBox 0.1.2 uses one SQL backend for the whole application. `database_backend = "sqlite" | "mysql" | "pgsql"` selects where account data is stored and, by default, where persistent login sessions are stored. SQLite defaults to `data_dir/rustbox.sqlite`; MySQL and PostgreSQL require `database_url`.
+`database_backend = "sqlite" | "mysql" | "pgsql"` selects only the authoritative SQL database. It does not force or forbid local blob/search storage. In particular, PostgreSQL/MySQL are valid in a single-node installation together with `storage.backend="filesystem"` and `search.backend="local_tantivy"`.
 
-Mailbox metadata and cached message bodies are stored in that SQL backend. With a shared MySQL/PostgreSQL database the persistent mailbox cache is therefore shared by all application nodes; Redis is not used for mailbox cache data.
+`session_backend = "database"` stores sessions in the selected SQL database. `session_backend = "redis"` moves only session persistence to Redis/Redis Sentinel. Mailbox metadata and cached message bodies remain in SQL.
 
-The per-user directory `data_dir/<user_hash>/` is not the active account database when MySQL/PostgreSQL is configured. It can contain filesystem-backed data:
+`data_dir` is persistent node state only when a selected local feature actually needs it: default SQLite, implicit filesystem storage, implicit local Tantivy, or local OpenPGP keys. In cluster mode it is not required when those features are replaced with cluster-safe backends. Cluster validation is capability-based:
 
-- `attachments/` — draft attachments; in a cluster `data_dir` must be mounted from shared storage at the same path on every node;
-- `tantivy/` — built-in full-text index; it is single-node only and must be disabled in cluster mode;
-- `pgp/` — OpenPGP private-key material; OpenPGP must be disabled in cluster mode;
-- legacy `db.sqlite` — migration input only. After a successful 0.1.0 import it is retained as `db.sqlite.pre-0.1.1.bak`.
+- SQL: shared MySQL/PostgreSQL; SQLite is rejected;
+- blobs: S3-compatible storage, or `filesystem` with `storage.shared=true` on a real NFS/CephFS/GlusterFS mount;
+- search: OpenSearch or `disabled`; local Tantivy is rejected;
+- OpenPGP: remains disabled until a cluster-safe key backend exists;
+- all nodes use the same `session_encryption_key`.
 
-For a multi-node deployment use a shared MySQL/PostgreSQL database, the same `session_encryption_key` on every node, and shared `data_dir` for draft attachments. RustBox 0.1.2 validates this explicitly with `cluster_shared_data_dir = true`. The built-in Tantivy backend has no cross-node writer coordination, so cluster mode requires `search_index_backend = "disabled"`. SQL-backed metadata/header search remains available; deep full-body Tantivy search is unavailable in cluster mode. An external search/index backend is not implemented in 0.1.2.
-
-`session_backend = "database"` stores sessions in the selected application database. `session_backend = "redis"` uses Redis instead; Redis Sentinel is supported through the Sentinel settings. Redis only replaces session persistence, not the application database or mailbox cache.
+Example stateless cluster:
 
 ```toml
-# Single node defaults.
-database_backend = "sqlite"
+database_backend = "pgsql"
+database_url = "postgres://rustbox:secret@db/rustbox"
 session_backend = "database"
-search_index_backend = "tantivy"
-pgp_enabled = true
+session_encryption_key = "<shared-key>"
+cluster_mode = true
+pgp_enabled = false
 
-# Multi-node PostgreSQL example:
-# database_backend = "pgsql"
-# database_url = "postgres://rustbox:secret@db/rustbox"
-# session_backend = "database" # or redis
-# session_encryption_key = "<base64-32-byte-key>"
-# cluster_mode = true
-# cluster_shared_data_dir = true
-# data_dir = "/srv/rustbox-shared" # same mounted path on every node
-# search_index_backend = "disabled"
-# pgp_enabled = false
+[storage]
+backend = "s3"
+bucket = "rustbox"
+region = "eu-central-1"
+# endpoint = "https://s3.example.com" # MinIO/Ceph RGW/etc.
 
-# Multi-node MySQL example (MySQL 8.0.13+):
-# database_backend = "mysql"
-# database_url = "mysql://rustbox:secret@db/rustbox"
-# session_backend = "database" # or redis
-# session_encryption_key = "<base64-32-byte-key>"
-# cluster_mode = true
-# cluster_shared_data_dir = true
-# data_dir = "/srv/rustbox-shared"
-# search_index_backend = "disabled"
-# pgp_enabled = false
-
-# Optional Redis session persistence; application data still uses database_backend.
-# session_backend = "redis"
-# redis_url = "redis://redis:6379/"
-
-# Sentinel alternative to redis_url:
-# redis_sentinel_nodes = ["sentinel-a:26379", "sentinel-b:26379", "sentinel-c:26379"]
-# redis_sentinel_master = "mymaster"
-# redis_sentinel_password = "sentinel-secret"
-# redis_password = "secret"
+[search]
+backend = "opensearch"
+urls = ["https://opensearch.example.com:9200"]
+required = true
 ```
 
-All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. Each engine has one complete baseline in `migrations/{sqlite,mysql,pgsql}/schema.sql`, including the session table. The squashed baseline is schema version 46 and the next schema migration number is V047 for all engines. Redis requires no SQL migration. See `migrations/README.md` and `docs/DB-AND-CLUSTER-AUDIT-0.1.2.md`.
+Shared POSIX is also supported without S3:
+
+```toml
+[storage]
+backend = "filesystem"
+path = "/mnt/rustbox-shared"
+shared = true
+```
+
+Existing attachment files can be migrated to the configured storage backend without deleting the source files:
+
+```bash
+rustbox storage migrate --from filesystem:/old/data/root --to configured --dry-run
+rustbox storage migrate --from filesystem:/old/data/root --to configured --verify
+```
+
+The SQL row is switched to the logical object key only after upload and size/checksum verification. Search state is rebuildable from SQL:
+
+```bash
+rustbox search reindex
+```
+
+Ready Docker examples are in `docker/`: standalone SQLite, PostgreSQL + SILO (S3-compatible) + OpenSearch, and MySQL + NFS + OpenSearch.
+
+All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. The squashed compatibility baseline remains V046; RustBox 0.2.0 adds runtime migration V047 for attachment checksums/logical blob keys and the durable search outbox. The next migration number is V048. See `migrations/README.md`.
 
 
 ## Development
@@ -197,7 +217,7 @@ Initialize Git after extracting the release archive:
 ```bash
 git init
 git add .
-git commit -m "RustBox 0.1.2 release"
+git commit -m "RustBox 0.2.0 release"
 ./scripts/setup-upstream.sh
 ```