v0.1.2
RustBox CI / rustbox (push) Canceled after 0s

This commit is contained in:
Mateusz Gruszczyński committed 2026-10-31 09:22:42 +01:00
1 parent 698e3c4a1b
commit e3e844ef4f
31 files changed
+287 -75

No files matched your search

+12
View File
@@ -1,5 +1,17 @@
# RustBox changelog
### 0.1.2
- Added `log_level` to TOML configuration (`trace`, `debug`, `info`, `warn`, `error`, `off`); `RUST_LOG` remains an explicit runtime override.
- Audited the MySQL and PostgreSQL query catalogs, schema baselines and backend-specific server SQL without adding application tests.
- Fixed a MySQL DDL incompatibility by changing indexed `pgp_keys.email` from `TEXT` to `VARCHAR(320)`; MySQL requires a prefix length when indexing `TEXT`/`BLOB` columns.
- Fixed destructive MySQL cache upserts: folder refresh and message-header refresh now use `INSERT ... ON DUPLICATE KEY UPDATE` instead of `REPLACE INTO`, preserving child rows, message tags, cached bodies and other columns not part of the refresh.
- Documented and enforced cluster storage behavior: mailbox metadata/body cache remains in the shared SQL backend, while draft attachments require shared `data_dir` mounted at the same path on all nodes.
- Added `search_index_backend = "tantivy" | "disabled"`; cluster mode requires `disabled` because the built-in Tantivy index has no multi-node writer coordination.
- Added `pgp_enabled`; cluster mode requires OpenPGP disabled because private-key material is filesystem-backed.
- Avoided provisioning per-user Tantivy directories when the search index backend is disabled and made diagnostics/deep indexing honor that setting.
- Added a release audit document with SQL compatibility and distributed-storage findings.
- Bumped the application/source release to 0.1.2.
### 0.1.1
- Fixed the multi-backend compatibility layer compile blockers reported by the Rust compiler: empty `[]` parameter inference no longer overlaps the generic array implementation, legacy SQLite probing imports `rusqlite::OptionalExtension` without colliding with RustBox's own `OptionalExtension`, and MySQL/PostgreSQL transaction batch execution now uses the typed SQLx transaction helpers instead of the lifetime-problematic `raw_sql` executor path.
- Unified the storage model: `database_backend = sqlite | mysql | pgsql` now selects the database engine for the whole application instead of only the SQL session store; `session_backend = database | redis` either keeps sessions in that same database or moves them to Redis/Redis Sentinel.
Generated
+1 -1
View File
@@ -3076,7 +3076,7 @@ dependencies = [
[[package]]
name = "rustbox"
version = "0.1.1"
version = "0.1.2"
dependencies = [
"aes-gcm",
"async-imap",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "rustbox"
version = "0.1.1"
version = "0.1.2"
edition = "2024"
rust-version = "1.88"
default-run = "rustbox"
+1 -1
View File
@@ -1,6 +1,6 @@
# RustBox attribution / notice
RustBox 0.1.1 is derived from the Oxi webmail project:
RustBox 0.1.2 is derived from the Oxi webmail project:
- Upstream: `c0h1b4/oxi`
- Base snapshot: `7f1507c486b78cf33e54beb701dd460d3bd2bd5d`
+34 -13
View File
@@ -1,4 +1,4 @@
# RustBox 0.1.1
# RustBox 0.1.2
RustBox is a webmail derived from `c0h1b4/oxi`. The consumed Oxi revision is recorded in `UPSTREAM_BASE`.
@@ -43,6 +43,7 @@ Edit `config.dev.toml` before the first run:
```toml
default_theme = "system" # system | dark | light
default_interface_skin = "rustbox" # rustbox | gmail | proton | zoho | outlook | fastmail | apple-mail | yahoo | hey | nord | github | gitlab | discord | slack | reddit | notion | ...
log_level = "info" # trace | debug | info | warn | error | off; RUST_LOG overrides it
# Mail hosts may be omitted: IMAP defaults to localhost:993 and SMTP to localhost:587.
# imap_host = "localhost"
# smtp_host = "localhost"
@@ -70,31 +71,51 @@ If the mail host settings are omitted, RustBox uses `localhost` with IMAP `993`
Mail-size limits are intentionally separate. `max_attachment_size_mb` applies to each raw uploaded file, while `max_message_size_mb` is checked against the final RFC822/MIME byte stream immediately before SMTP submission. The latter therefore includes message bodies, headers, MIME boundaries, base64 expansion, inline CID parts and OpenPGP transformations. The composer displays only an estimate; the server-side final-MIME check is authoritative.
## Database, sessions and cluster traffic
## Database, cache, sessions and cluster traffic
RustBox 0.1.1 uses one SQL backend for the whole application. `database_backend = "sqlite" | "mysql" | "pgsql"` selects where account data is stored and, by default, where persistent login sessions are stored. SQLite defaults to `data_dir/rustbox.sqlite`; MySQL and PostgreSQL require `database_url`.
RustBox 0.1.2 uses one SQL backend for the whole application. `database_backend = "sqlite" | "mysql" | "pgsql"` selects where account data is stored and, by default, where persistent login sessions are stored. SQLite defaults to `data_dir/rustbox.sqlite`; MySQL and PostgreSQL require `database_url`.
Account data remains isolated per account inside the selected database. Existing 0.1.0 per-user SQLite databases at `data_dir/<user_hash>/db.sqlite` are imported on first access when they are at schema version 46; the source file is retained as `db.sqlite.pre-0.1.1.bak` after a successful import.
Mailbox metadata and cached message bodies are stored in that SQL backend. With a shared MySQL/PostgreSQL database the persistent mailbox cache is therefore shared by all application nodes; Redis is not used for mailbox cache data.
`session_backend = "database"` stores sessions in the selected application database. `session_backend = "redis"` uses Redis instead; Redis Sentinel is supported through the Sentinel settings. For a multi-node deployment the application database itself must be shared MySQL/PostgreSQL, even when Redis stores sessions. Provide the same base64-encoded 32-byte `session_encryption_key` on every node that shares sessions.
The per-user directory `data_dir/<user_hash>/` is not the active account database when MySQL/PostgreSQL is configured. It can contain filesystem-backed data:
- `attachments/` — draft attachments; in a cluster `data_dir` must be mounted from shared storage at the same path on every node;
- `tantivy/` — built-in full-text index; it is single-node only and must be disabled in cluster mode;
- `pgp/` — OpenPGP private-key material; OpenPGP must be disabled in cluster mode;
- legacy `db.sqlite` — migration input only. After a successful 0.1.0 import it is retained as `db.sqlite.pre-0.1.1.bak`.
For a multi-node deployment use a shared MySQL/PostgreSQL database, the same `session_encryption_key` on every node, and shared `data_dir` for draft attachments. RustBox 0.1.2 validates this explicitly with `cluster_shared_data_dir = true`. The built-in Tantivy backend has no cross-node writer coordination, so cluster mode requires `search_index_backend = "disabled"`. SQL-backed metadata/header search remains available; deep full-body Tantivy search is unavailable in cluster mode. An external search/index backend is not implemented in 0.1.2.
`session_backend = "database"` stores sessions in the selected application database. `session_backend = "redis"` uses Redis instead; Redis Sentinel is supported through the Sentinel settings. Redis only replaces session persistence, not the application database or mailbox cache.
```toml
# Default: the whole application uses SQLite under data_dir.
# Single node defaults.
database_backend = "sqlite"
# database_url = "sqlite:///var/lib/rustbox/rustbox.sqlite?mode=rwc"
session_backend = "database"
search_index_backend = "tantivy"
pgp_enabled = true
# Full PostgreSQL backend:
# Multi-node PostgreSQL example:
# database_backend = "pgsql"
# database_url = "postgres://rustbox:secret@db/rustbox"
# session_backend = "database"
# session_backend = "database" # or redis
# session_encryption_key = "<base64-32-byte-key>"
# cluster_mode = true
# cluster_shared_data_dir = true
# data_dir = "/srv/rustbox-shared" # same mounted path on every node
# search_index_backend = "disabled"
# pgp_enabled = false
# Full MySQL backend (MySQL 8.0.13+):
# Multi-node MySQL example (MySQL 8.0.13+):
# database_backend = "mysql"
# database_url = "mysql://rustbox:secret@db/rustbox"
# session_backend = "database"
# session_backend = "database" # or redis
# session_encryption_key = "<base64-32-byte-key>"
# cluster_mode = true
# cluster_shared_data_dir = true
# data_dir = "/srv/rustbox-shared"
# search_index_backend = "disabled"
# pgp_enabled = false
# Optional Redis session persistence; application data still uses database_backend.
# session_backend = "redis"
@@ -107,7 +128,7 @@ session_backend = "database"
# redis_password = "secret"
```
All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. Each engine has one complete baseline in `migrations/{sqlite,mysql,pgsql}/schema.sql`, including the session table. The squashed baseline is schema version 46 and the next schema migration number is V047 for all engines. Redis requires no SQL migration. See `migrations/README.md`.
All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. Each engine has one complete baseline in `migrations/{sqlite,mysql,pgsql}/schema.sql`, including the session table. The squashed baseline is schema version 46 and the next schema migration number is V047 for all engines. Redis requires no SQL migration. See `migrations/README.md` and `docs/DB-AND-CLUSTER-AUDIT-0.1.2.md`.
## Development
@@ -176,7 +197,7 @@ Initialize Git after extracting the release archive:
```bash
git init
git add .
git commit -m "RustBox 0.1.1 release"
git commit -m "RustBox 0.1.2 release"
./scripts/setup-upstream.sh
```
+9 -1
View File
@@ -1,4 +1,4 @@
RustBox 0.1.1 release manifest
RustBox 0.1.2 release manifest
Storage model:
- database_backend = sqlite | mysql | pgsql selects the full application database
@@ -13,6 +13,14 @@ Database baselines:
- historical SQLite V001-V046 files are intentionally squashed out of this release
- next migration number for every SQL engine: V047
Cluster storage rules (0.1.2):
- SQL mailbox metadata/body cache is shared by the configured MySQL/PostgreSQL backend
- data_dir must be shared between nodes for filesystem-backed draft attachments
- cluster_shared_data_dir=true is required as an explicit deployment acknowledgement
- built-in Tantivy must be disabled in cluster mode (search_index_backend=disabled)
- OpenPGP must be disabled in cluster mode (pgp_enabled=false)
- legacy db.sqlite files remain migration-only; they are not the active database on MySQL/PostgreSQL
Compatibility:
- legacy RustBox 0.1.0 per-user SQLite databases at user_version 46 are imported on first access
- imported source DBs are retained as db.sqlite.pre-0.1.1.bak
+14 -22
View File
@@ -1,28 +1,20 @@
RustBox 0.1.1 - release validation
RustBox 0.1.2 - release validation
Completed in this environment:
- compiler-log remediation: all 47 reported compile errors map to and are covered by three root fixes (41 E0283 empty-parameter inference errors, 2 E0599 rusqlite OptionalExtension errors, 4 SQLx transaction Executor lifetime errors)
- version consistency: Cargo.toml / Cargo.lock / CHANGELOG.md / release metadata = 0.1.1
- migration layout: exactly one schema.sql for sqlite, mysql and pgsql; no session.sql or V001-V046 files
- query catalog parity: 27 modules / 197 public query constants present in sqlite, mysql and pgsql catalogs
- include_str! validation: all referenced files exist
- logical schema parity: sqlite/mysql/pgsql define the same 28 logical tables and column sets
- SQLite execution test: global schema + account tenant schema execute successfully with foreign keys enabled
- SQLite session smoke test: account and session rows insert successfully
- TOML parse: Cargo.toml, config.example.toml and config.dev.toml
- release metadata: Cargo.toml and the RustBox package entry in Cargo.lock = 0.1.2
- TOML parse: Cargo.toml, config.example.toml, config.dev.toml and packaging/rustbox.toml
- shell syntax: dev.sh and scripts/*.sh pass bash -n
- browser JavaScript: manifest resolves 13 files; concatenated production-order bundle passes node --check
- cleanup: no stale session.sql, V001-V046, migrations/user or migrations/session references outside historical changelog/manifest notes
- tests cleanup: only tests/fixtures/account-session-expired.json remains in tests/
- query catalog parity: SQLite/MySQL/PostgreSQL expose the same 27 modules / 197 public SQL constants, including 153 Q_* executable query constants
- logical schema parity: MySQL and PostgreSQL define the same 28 account/global tables
- static SQL dialect scan: no PostgreSQL-only constructs found in the MySQL catalog/baseline and no MySQL-only constructs found in the PostgreSQL catalog/baseline
- MySQL index audit: 49 indexes/constraints checked; no unprefixed TEXT/BLOB key remains and the largest utf8mb4 key estimate is 2568 bytes (<3072); pgp_keys.email is VARCHAR(320)
- MySQL upsert audit: no REPLACE INTO remains in the MySQL catalog; folder/message refresh uses non-destructive ON DUPLICATE KEY UPDATE
- SQLite baseline execution: schema executes successfully under Python sqlite3 with foreign keys enabled
- cluster storage review: SQL mailbox cache is shared; draft attachments are filesystem-backed; built-in Tantivy and OpenPGP are disabled by cluster-mode validation
- no application tests were added for the 0.1.2 audit
Not executable in this environment:
- cargo fmt/test/clippy/build: Rust/Cargo toolchain is not installed and outbound package/toolchain download is unavailable
- live MySQL/PostgreSQL integration tests: database servers/clients are not installed
- cargo fmt/test/clippy/build: Rust/Cargo toolchain is not installed
- live MySQL/PostgreSQL execution: database servers/clients are not installed
The repository contains native Rust tests for tenant scoping, placeholder adaptation and legacy SQLite import; run scripts/build-release.sh on a Rust 1.88+ build host before publishing a production binary.
Hotfix validation (release-gate lint failures):
- removed unusable null_db_value overrides for unsized str/[u8]
- removed redundant row_to_cached_message closure
- collapsed MySQL reordered-params conditional
- cargo/rustc unavailable in packaging environment; run scripts/build-release.sh on Rust host
Before publishing a production binary, run scripts/build-release.sh on a Rust 1.88+ build host and perform a live schema/query smoke test against the exact MySQL/PostgreSQL versions used in production.
+4
View File
@@ -9,6 +9,7 @@ default_theme = "system"
# apple-mail, yahoo, hey, nord, minimal and Roundcube variants.
default_interface_skin = "rustbox"
data_dir = ".dev/data"
log_level = "debug"
session_timeout_hours = 24
# Application database. The selected engine stores all RustBox account data.
@@ -18,6 +19,9 @@ database_backend = "sqlite" # sqlite | mysql | pgsql
# database_url = "postgres://rustbox:secret@db.example.com/rustbox"
session_backend = "database" # database | redis
cluster_mode = false
cluster_shared_data_dir = false
search_index_backend = "tantivy"
pgp_enabled = true
# For shared database sessions, Redis sessions, or cluster mode provide the same 32-byte key (base64) on all nodes.
# session_encryption_key = "..."
# Direct Redis session backend:
+10 -2
View File
@@ -1,4 +1,4 @@
# RustBox 0.1.1
# RustBox 0.1.2
host = "127.0.0.1"
port = 3001
environment = "production"
@@ -10,6 +10,8 @@ default_theme = "system"
default_interface_skin = "gmail"
data_dir = "/var/lib/rustbox"
log_dir = "/var/log/rustbox"
# trace | debug | info | warn | error | off. RUST_LOG still overrides this value.
log_level = "info"
session_timeout_hours = 24
# Application database. The selected engine stores all RustBox account data.
@@ -19,6 +21,11 @@ database_backend = "sqlite" # sqlite | mysql | pgsql
# database_url = "postgres://rustbox:secret@db.example.com/rustbox"
session_backend = "database" # database | redis
cluster_mode = false
# Required acknowledgement in cluster mode: data_dir must be shared by every node
# because draft attachments are stored there.
cluster_shared_data_dir = false
# Built-in Tantivy is single-node. Cluster mode requires disabled.
search_index_backend = "tantivy" # tantivy | disabled
# For shared database sessions, Redis sessions, or cluster mode provide the same 32-byte key (base64) on all nodes.
# session_encryption_key = "..."
# Direct Redis session backend:
@@ -75,7 +82,8 @@ trusted_proxies = "127.0.0.1,::1"
outgoing_headers = []
# OpenPGP. Private keys are stored per user below data_dir with 0700/0600
# permissions. Passphrases are never persisted. WKD is enabled by default;
# HKP lookup is opt-in and HTTPS-only.
# HKP lookup is opt-in and HTTPS-only. Cluster mode requires pgp_enabled=false.
pgp_enabled = true
pgp_wkd_enabled = true
pgp_hkp_enabled = false
pgp_hkp_url = "https://keys.openpgp.org"
+1 -1
View File
@@ -33,7 +33,7 @@ A release archive does not contain `.git`. Initialize the RustBox repository onc
```bash
git init
git add .
git commit -m "RustBox 0.1.1 baseline"
git commit -m "RustBox 0.1.2 baseline"
./scripts/setup-upstream.sh
```
+1 -2
View File
@@ -1,6 +1,6 @@
# Database schemas and migrations
RustBox 0.1.1 has one database backend for the whole application:
RustBox 0.1.2 has one database backend for the whole application:
```text
database_backend = sqlite | mysql | pgsql
@@ -28,7 +28,6 @@ Every `schema.sql` is the complete schema for that engine. It contains:
Account data is isolated inside the selected physical database with a deterministic, collision-checked table prefix derived from the account hash. The `{{p}}` marker in the tenant part of each schema is replaced by RustBox at provisioning time. Sessions are global and therefore are not prefixed.
The SQLite baseline is structurally equivalent to the legacy V001-V046 migration chain. On first access, RustBox 0.1.1 imports an existing legacy `data_dir/<user_hash>/db.sqlite` at `PRAGMA user_version = 46` into the configured backend. Only after a successful import is the legacy file renamed to `db.sqlite.pre-0.1.1.bak`. Older legacy schemas must first be upgraded with RustBox 0.1.0.
## Future migrations
+2 -2
View File
@@ -1,4 +1,4 @@
-- RustBox 0.1.1 MySQL schema.
-- RustBox 0.1.2 MySQL schema.
-- One physical database contains global sessions plus isolated per-account tables.
-- Application tables are materialized by replacing {{p}} with a collision-checked prefix.
-- Baseline application schema version: 46. Future migrations start at V047.
@@ -333,7 +333,7 @@ CREATE TABLE {{p}}notification_preferences (
CREATE TABLE {{p}}pgp_keys (
fingerprint VARCHAR(255) PRIMARY KEY,
key_id TEXT NOT NULL,
email TEXT NOT NULL DEFAULT (''),
email VARCHAR(320) NOT NULL DEFAULT '',
user_id TEXT NOT NULL DEFAULT (''),
public_key_armor LONGTEXT NOT NULL,
secret_key_path TEXT,
+1 -1
View File
@@ -1,4 +1,4 @@
-- RustBox 0.1.1 PostgreSQL schema.
-- RustBox 0.1.2 PostgreSQL schema.
-- One physical database contains global sessions plus isolated per-account tables.
-- Application tables are materialized by replacing {{p}} with a collision-checked prefix.
-- Baseline application schema version: 46. Future migrations start at V047.
+1 -1
View File
@@ -1,4 +1,4 @@
-- RustBox 0.1.1 SQLite schema.
-- RustBox 0.1.2 SQLite schema.
-- One physical database contains global sessions plus isolated per-account tables.
-- Application tables are materialized from the tenant section by replacing {{p}}.
-- Baseline application schema version: 46. Future migrations start at V047.
+17 -1
View File
@@ -1,4 +1,4 @@
# RustBox 0.1.1 - packaged production configuration (DEB/RPM)
# RustBox 0.1.2 - packaged production configuration (DEB/RPM)
#
# The active values below intentionally keep the existing package profile.
# Alternative storage/cluster examples are provided at the end of this file
@@ -12,10 +12,13 @@ port = 3001
environment = "production"
data_dir = "/var/lib/rustbox"
log_dir = "/var/log/rustbox"
log_level = "info"
# Database, sessions and "remember me".
# With no database/session backend keys configured, the whole application uses
# SQLite at data_dir/rustbox.sqlite; sessions are stored in that database.
cluster_shared_data_dir = false
search_index_backend = "tantivy"
session_timeout_hours = 72
remember_me_enabled = true
remember_me_days = 30
@@ -60,6 +63,7 @@ outgoing_headers = []
# -----------------------------------------------------------------------------
# OpenPGP
# -----------------------------------------------------------------------------
pgp_enabled = true
pgp_wkd_enabled = true
pgp_hkp_enabled = false
pgp_hkp_url = "https://keys.openpgp.org"
@@ -120,10 +124,14 @@ pgp_allow_private_export = true
# Profile E: Redis Sentinel sessions for multiple RustBox nodes
# -----------------------------------------------------------------------------
# Cluster nodes must also share the main application database.
# data_dir must be a shared filesystem mounted at the same absolute path on every node.
# database_backend = "pgsql"
# database_url = "postgres://rustbox:change-me@db.internal/rustbox"
# session_backend = "redis"
# cluster_mode = true
# cluster_shared_data_dir = true
# search_index_backend = "disabled"
# pgp_enabled = false
# redis_sentinel_nodes = ["sentinel-a:26379", "sentinel-b:26379", "sentinel-c:26379"]
# redis_sentinel_master = "mymaster"
# redis_sentinel_password = "sentinel-secret"
@@ -132,18 +140,26 @@ pgp_allow_private_export = true
# -----------------------------------------------------------------------------
# Profile F: PostgreSQL application database shared by a RustBox cluster
# data_dir must be shared and mounted at the same absolute path on every node.
# -----------------------------------------------------------------------------
# database_backend = "pgsql"
# database_url = "postgres://rustbox:change-me@db.internal/rustbox"
# session_backend = "database"
# cluster_mode = true
# cluster_shared_data_dir = true
# search_index_backend = "disabled"
# pgp_enabled = false
# session_encryption_key = "BASE64_32_BYTE_KEY"
# -----------------------------------------------------------------------------
# Profile G: MySQL application database shared by a RustBox cluster
# data_dir must be shared and mounted at the same absolute path on every node.
# -----------------------------------------------------------------------------
# database_backend = "mysql"
# database_url = "mysql://rustbox:change-me@db.internal/rustbox"
# session_backend = "database"
# cluster_mode = true
# cluster_shared_data_dir = true
# search_index_backend = "disabled"
# pgp_enabled = false
# session_encryption_key = "BASE64_32_BYTE_KEY"
+1 -1
View File
@@ -12,7 +12,7 @@ if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
echo "Initialize it first:" >&2
echo " git init" >&2
echo " git add ." >&2
echo " git commit -m 'RustBox 0.1.1 baseline'" >&2
echo " git commit -m 'RustBox 0.1.2 baseline'" >&2
exit 1
fi
+15 -3
View File
@@ -12,10 +12,16 @@ pub fn hash_email(email: &str) -> String {
}
pub fn provision_user_data(data_dir: &str, user_hash: &str) -> Result<PathBuf, String> {
let user_dir = PathBuf::from(data_dir).join(user_hash);
provision_user_data_with_search(data_dir, user_hash, true)
}
let tantivy_dir = user_dir.join("tantivy");
fs::create_dir_all(&tantivy_dir).map_err(|e| format!("failed to create directories: {e}"))?;
pub fn provision_user_data_with_search(
data_dir: &str,
user_hash: &str,
create_search_dir: bool,
) -> Result<PathBuf, String> {
let user_dir = PathBuf::from(data_dir).join(user_hash);
fs::create_dir_all(&user_dir).map_err(|e| format!("failed to create user directory: {e}"))?;
#[cfg(unix)]
{
@@ -26,6 +32,12 @@ pub fn provision_user_data(data_dir: &str, user_hash: &str) -> Result<PathBuf, S
}
}
if create_search_dir {
let tantivy_dir = user_dir.join("tantivy");
fs::create_dir_all(&tantivy_dir)
.map_err(|e| format!("failed to create search directory: {e}"))?;
}
// Ensure this account has its isolated tables in the configured RustBox database.
// The same database backend stores sessions unless session_backend=redis.
let conn = crate::db::pool::open_user_db(data_dir, user_hash)
+59
View File
@@ -46,6 +46,9 @@ pub struct AppConfig {
#[serde(default)]
pub log_dir: Option<String>,
#[serde(default = "default_log_level")]
pub log_level: String,
#[serde(default = "default_database_backend")]
pub database_backend: String,
@@ -76,6 +79,12 @@ pub struct AppConfig {
#[serde(default)]
pub cluster_mode: bool,
#[serde(default)]
pub cluster_shared_data_dir: bool,
#[serde(default = "default_search_index_backend")]
pub search_index_backend: String,
#[serde(default = "default_session_timeout_hours")]
pub session_timeout_hours: u64,
@@ -118,6 +127,9 @@ pub struct AppConfig {
#[serde(default)]
pub outgoing_headers: Vec<String>,
#[serde(default = "default_pgp_enabled")]
pub pgp_enabled: bool,
#[serde(default = "default_pgp_wkd_enabled")]
pub pgp_wkd_enabled: bool,
@@ -199,6 +211,10 @@ fn default_data_dir() -> String {
"/data".to_string()
}
fn default_log_level() -> String {
"info".to_string()
}
fn default_database_backend() -> String {
"sqlite".to_string()
}
@@ -207,6 +223,10 @@ fn default_session_backend() -> String {
"database".to_string()
}
fn default_search_index_backend() -> String {
"tantivy".to_string()
}
fn default_session_timeout_hours() -> u64 {
24
}
@@ -242,6 +262,10 @@ fn default_sieve_timeout_secs() -> u64 {
10
}
fn default_pgp_enabled() -> bool {
true
}
fn default_pgp_wkd_enabled() -> bool {
true
}
@@ -270,6 +294,7 @@ impl Default for AppConfig {
max_message_size_mb: default_max_message_size_mb(),
data_dir: default_data_dir(),
log_dir: None,
log_level: default_log_level(),
database_backend: default_database_backend(),
database_url: None,
session_backend: default_session_backend(),
@@ -280,6 +305,8 @@ impl Default for AppConfig {
redis_password: None,
redis_sentinel_password: None,
cluster_mode: false,
cluster_shared_data_dir: false,
search_index_backend: default_search_index_backend(),
session_timeout_hours: default_session_timeout_hours(),
remember_me_enabled: default_remember_me_enabled(),
remember_me_days: default_remember_me_days(),
@@ -294,6 +321,7 @@ impl Default for AppConfig {
sieve_timeout_secs: default_sieve_timeout_secs(),
trusted_proxies: None,
outgoing_headers: Vec::new(),
pgp_enabled: default_pgp_enabled(),
pgp_wkd_enabled: default_pgp_wkd_enabled(),
pgp_hkp_enabled: false,
pgp_hkp_url: default_pgp_hkp_url(),
@@ -446,6 +474,16 @@ impl AppConfig {
return Err("LOG_DIR / log_dir must be an absolute path".to_string());
}
}
let log_level = self.log_level.trim().to_ascii_lowercase();
if !matches!(
log_level.as_str(),
"trace" | "debug" | "info" | "warn" | "error" | "off"
) {
return Err(
"LOG_LEVEL / log_level must be one of: trace, debug, info, warn, error, off"
.to_string(),
);
}
if self.max_attachment_size_mb == 0 || self.max_attachment_size_mb > 512 {
return Err("MAX_ATTACHMENT_SIZE_MB must be between 1 and 512".to_string());
}
@@ -492,12 +530,33 @@ impl AppConfig {
return Err("Redis sessions require REDIS_URL or REDIS_SENTINEL_NODES + REDIS_SENTINEL_MASTER".to_string());
}
}
if !matches!(self.search_index_backend.as_str(), "tantivy" | "disabled") {
return Err("SEARCH_INDEX_BACKEND must be one of: tantivy, disabled".to_string());
}
if self.cluster_mode && self.database_backend == "sqlite" {
return Err(
"CLUSTER_MODE requires a shared mysql/pgsql application database; Redis only replaces session persistence"
.to_string(),
);
}
if self.cluster_mode && !self.cluster_shared_data_dir {
return Err(
"CLUSTER_MODE requires CLUSTER_SHARED_DATA_DIR=true after data_dir is mounted as shared storage on every node; draft attachments are stored below data_dir"
.to_string(),
);
}
if self.cluster_mode && self.search_index_backend != "disabled" {
return Err(
"CLUSTER_MODE requires SEARCH_INDEX_BACKEND=disabled; the built-in Tantivy index is local and does not coordinate writers across nodes"
.to_string(),
);
}
if self.cluster_mode && self.pgp_enabled {
return Err(
"CLUSTER_MODE requires PGP_ENABLED=false; OpenPGP private keys use node-local filesystem storage"
.to_string(),
);
}
if (self.cluster_mode
|| self.session_backend == "redis"
|| self.database_backend != "sqlite")
+3 -3
View File
@@ -1562,12 +1562,12 @@ fn read_legacy_sqlite(path: &Path) -> Result<Vec<LegacyTableData>, String> {
.map_err(|e| format!("failed to read legacy SQLite schema version: {e}"))?;
if version < BASE_SCHEMA_VERSION {
return Err(format!(
"legacy SQLite schema version {version} is older than the RustBox 0.1.1 baseline {BASE_SCHEMA_VERSION}; upgrade with RustBox 0.1.0 first"
"legacy SQLite schema version {version} is older than the RustBox 0.1.2 baseline {BASE_SCHEMA_VERSION}; upgrade with RustBox 0.1.0 first"
));
}
if version > BASE_SCHEMA_VERSION {
return Err(format!(
"legacy SQLite schema version {version} is newer than the RustBox 0.1.1 baseline {BASE_SCHEMA_VERSION}"
"legacy SQLite schema version {version} is newer than the RustBox 0.1.2 baseline {BASE_SCHEMA_VERSION}"
));
}
let mut output = Vec::new();
@@ -1922,7 +1922,7 @@ fn apply_sqlite_migrations(
) -> Result<(), String> {
if current < BASE_SCHEMA_VERSION {
return Err(format!(
"database schema version {current} is older than the RustBox 0.1.1 baseline {BASE_SCHEMA_VERSION}; upgrade with RustBox 0.1.0 first"
"database schema version {current} is older than the RustBox 0.1.2 baseline {BASE_SCHEMA_VERSION}; upgrade with RustBox 0.1.0 first"
));
}
let latest = latest_schema_version();
+12 -7
View File
@@ -47,7 +47,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
return Ok(());
}
init_tracing(loaded_config.log_dir.as_deref())?;
init_tracing(loaded_config.log_dir.as_deref(), &loaded_config.log_level)?;
loaded_config.validate().map_err(|message| {
std::io::Error::new(
std::io::ErrorKind::InvalidInput,
@@ -86,9 +86,12 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let imap_client: Arc<dyn imap::client::ImapClient> = Arc::new(RealImapClient::default());
let smtp_client: Arc<dyn smtp::client::SmtpClient> = Arc::new(RealSmtpClient);
let search_engine = Arc::new(search::engine::SearchEngine::new(std::path::PathBuf::from(
&config.data_dir,
)));
let search_base = std::path::PathBuf::from(&config.data_dir);
let search_engine = Arc::new(if config.search_index_backend == "disabled" {
search::engine::SearchEngine::disabled(search_base)
} else {
search::engine::SearchEngine::new(search_base)
});
let event_bus = Arc::new(realtime::events::EventBus::new());
let idle_manager = Arc::new(realtime::idle::IdleManager::new());
@@ -123,7 +126,9 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
Ok(())
}
fn init_tracing(log_dir: Option<&str>) -> Result<(), Box<dyn std::error::Error>> {
fn init_tracing(log_dir: Option<&str>, log_level: &str) -> Result<(), Box<dyn std::error::Error>> {
let env_filter = EnvFilter::try_from_default_env()
.unwrap_or_else(|_| EnvFilter::new(log_level.trim().to_ascii_lowercase()));
if let Some(log_dir) = log_dir.map(str::trim).filter(|value| !value.is_empty()) {
std::fs::create_dir_all(log_dir)?;
let log_path = std::path::Path::new(log_dir).join("rustbox.log");
@@ -136,7 +141,7 @@ fn init_tracing(log_dir: Option<&str>) -> Result<(), Box<dyn std::error::Error>>
}
let file = options.open(log_path)?;
tracing_subscriber::registry()
.with(EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info")))
.with(env_filter)
.with(
fmt::layer()
.json()
@@ -146,7 +151,7 @@ fn init_tracing(log_dir: Option<&str>) -> Result<(), Box<dyn std::error::Error>>
.init();
} else {
tracing_subscriber::registry()
.with(EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info")))
.with(env_filter)
.with(fmt::layer().json().with_ansi(false))
.init();
}
+1 -1
View File
@@ -126,7 +126,7 @@ async fn https_get_address(
format!("{host}:{port}")
};
let request = format!(
"GET {target} HTTP/1.1\r\nHost: {host_header}\r\nUser-Agent: RustBox/0.1.1\r\nAccept: application/octet-stream, application/pgp-keys, text/plain\r\nConnection: close\r\n\r\n"
"GET {target} HTTP/1.1\r\nHost: {host_header}\r\nUser-Agent: RustBox/0.1.2\r\nAccept: application/octet-stream, application/pgp-keys, text/plain\r\nConnection: close\r\n\r\n"
);
timeout(duration, stream.write_all(request.as_bytes()))
.await
+14 -4
View File
@@ -178,10 +178,14 @@ pub mod db_folder_preferences {
}
pub mod db_folders {
pub const Q_001: &str = "REPLACE INTO folders
pub const Q_001: &str = "INSERT INTO folders
(name, delimiter, parent, flags, is_subscribed,
total_count, unread_count, uid_validity, highest_modseq, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)";
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
delimiter=VALUES(delimiter), parent=VALUES(parent), flags=VALUES(flags),
is_subscribed=VALUES(is_subscribed), total_count=VALUES(total_count), unread_count=VALUES(unread_count),
uid_validity=VALUES(uid_validity), highest_modseq=VALUES(highest_modseq), updated_at=CURRENT_TIMESTAMP";
pub const Q_002: &str = "INSERT IGNORE INTO folders
(name, delimiter, flags, is_subscribed, updated_at)
VALUES (?, ?, ?, 1, CURRENT_TIMESTAMP)";
@@ -262,11 +266,17 @@ pub mod db_mail_preferences {
pub mod db_messages {
pub const MSG_SELECT_COLS: &str = "uid, folder, message_id, in_reply_to, references_header, subject, from_address, from_name, to_addresses, cc_addresses, date, flags, size, has_attachments, snippet, reaction";
pub const Q_001: &str = "REPLACE INTO messages
pub const Q_001: &str = "INSERT INTO messages
(uid, folder, message_id, in_reply_to, references_header,
subject, from_address, from_name, to_addresses, cc_addresses,
date, flags, size, has_attachments, snippet, date_epoch, reaction)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)";
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE
message_id=VALUES(message_id), in_reply_to=VALUES(in_reply_to), references_header=VALUES(references_header),
subject=VALUES(subject), from_address=VALUES(from_address), from_name=VALUES(from_name),
to_addresses=VALUES(to_addresses), cc_addresses=VALUES(cc_addresses), date=VALUES(date), flags=VALUES(flags),
size=VALUES(size), has_attachments=VALUES(has_attachments), snippet=VALUES(snippet),
date_epoch=VALUES(date_epoch), reaction=VALUES(reaction)";
pub const Q_002: &str = "DELETE FROM messages WHERE folder = ?";
pub const Q_003: &str = "SELECT COALESCE(MAX(uid), 0) FROM messages WHERE folder = ?";
pub const Q_004: &str = "SELECT uid, folder, subject, from_address, from_name, to_addresses, \
+3 -1
View File
@@ -154,7 +154,9 @@ async fn run_sync(
event_bus.publish(user_hash, MailEvent::FolderUpdated).await;
}
if let Err(e) = index_message_bodies(user_hash, creds, config, imap_client, search_engine).await
if search_engine.is_enabled()
&& let Err(e) =
index_message_bodies(user_hash, creds, config, imap_client, search_engine).await
{
tracing::debug!(error = %e, "Deep index phase skipped or failed");
}
+6 -1
View File
@@ -237,7 +237,12 @@ pub async fn login(
}
let user_hash = user_data::hash_email(&body.email);
if let Err(e) = user_data::provision_user_data(&config.data_dir, &user_hash) {
let provision_result = if config.search_index_backend == "tantivy" {
user_data::provision_user_data(&config.data_dir, &user_hash)
} else {
user_data::provision_user_data_with_search(&config.data_dir, &user_hash, false)
};
if let Err(e) = provision_result {
tracing::error!(error = %e, "failed to provision user data directory");
return (
StatusCode::INTERNAL_SERVER_ERROR,
+1 -1
View File
@@ -174,7 +174,7 @@ pub async fn export_ics(
.map_err(|e| AppError::InternalError(format!("Database error: {e}")))?;
let mut body = String::from(
"BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//RustBox//Calendar 0.1.1//EN\r\nCALSCALE:GREGORIAN\r\n",
"BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//RustBox//Calendar 0.1.2//EN\r\nCALSCALE:GREGORIAN\r\n",
);
for event in &events {
append_ics_event(&mut body, event);
+10 -1
View File
@@ -269,7 +269,11 @@ pub async fn search_diagnostics(
let cached_bodies: usize = conn
.query_row(crate::dbq!(routes_diagnostics::Q_003), [], |row| row.get(0))
.map_err(|e| format!("Failed to count cached bodies: {e}"))?;
let indexed_documents = engine.open_user_index(&user_hash)?.document_count();
let indexed_documents = if engine.is_enabled() {
engine.open_user_index(&user_hash)?.document_count()
} else {
0
};
Ok(SearchDiagnosticsResponse {
indexed_documents,
@@ -290,6 +294,11 @@ pub async fn rebuild_search_index(
Extension(config): Extension<Arc<AppConfig>>,
Extension(search_engine): Extension<Arc<SearchEngine>>,
) -> Result<Response, AppError> {
if !search_engine.is_enabled() {
return Err(AppError::ServiceUnavailable(
"Search index rebuild is unavailable because SEARCH_INDEX_BACKEND=disabled".to_string(),
));
}
let data_dir = config.data_dir.clone();
let user_hash = session.user_hash.clone();
let engine = search_engine.clone();
+25
View File
@@ -14,6 +14,16 @@ use crate::db;
use crate::db::pgp::{PgpKeyRecord, PgpPreferences};
use crate::error::AppError;
fn ensure_pgp_enabled(config: &AppConfig) -> Result<(), AppError> {
if config.pgp_enabled {
Ok(())
} else {
Err(AppError::ServiceUnavailable(
"OpenPGP is disabled by administrator policy".to_string(),
))
}
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct GenerateRequest {
@@ -137,6 +147,7 @@ pub async fn process_message(
Path((folder, uid)): Path<(String, u32)>,
Json(request): Json<ProcessMessageRequest>,
) -> Result<Json<crate::pgp::ProcessedMessage>, AppError> {
ensure_pgp_enabled(&config)?;
if request.passphrase.len() > 1024 {
return Err(AppError::BadRequest(
"OpenPGP passphrase is too long".to_string(),
@@ -240,6 +251,7 @@ pub async fn list_keys(
Extension(session): Extension<SessionState>,
Extension(config): Extension<Arc<AppConfig>>,
) -> Result<Json<Vec<PgpKeyRecord>>, AppError> {
ensure_pgp_enabled(&config)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let keys = database
.run(db::pgp::list_keys)
@@ -253,6 +265,7 @@ pub async fn generate_key(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<GenerateRequest>,
) -> Result<Json<PgpKeyRecord>, AppError> {
ensure_pgp_enabled(&config)?;
validate_email(&request.email)?;
if request.passphrase.len() > 1024 {
return Err(AppError::BadRequest(
@@ -275,6 +288,7 @@ pub async fn import_key(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<ImportRequest>,
) -> Result<Json<PgpKeyRecord>, AppError> {
ensure_pgp_enabled(&config)?;
validate_trust(&request.trust)?;
let input = match (request.key_data, request.key_data_base64) {
(Some(text), None) => {
@@ -330,6 +344,7 @@ pub async fn export_key(
Path(fingerprint): Path<String>,
Query(query): Query<ExportQuery>,
) -> Result<Response, AppError> {
ensure_pgp_enabled(&config)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let normalized = db::pgp::normalize_fingerprint(&fingerprint);
let key = database
@@ -381,6 +396,7 @@ pub async fn test_passphrase(
Path(fingerprint): Path<String>,
Json(request): Json<TestPassphraseRequest>,
) -> Result<Json<TestPassphraseResponse>, AppError> {
ensure_pgp_enabled(&config)?;
if request.passphrase.len() > 1024 {
return Err(AppError::BadRequest(
"OpenPGP passphrase is too long".to_string(),
@@ -417,6 +433,7 @@ pub async fn change_passphrase(
Path(fingerprint): Path<String>,
Json(request): Json<ChangePassphraseRequest>,
) -> Result<Json<TestPassphraseResponse>, AppError> {
ensure_pgp_enabled(&config)?;
if request.current_passphrase.len() > 1024 || request.new_passphrase.len() > 1024 {
return Err(AppError::BadRequest(
"OpenPGP passphrase is too long".to_string(),
@@ -466,6 +483,7 @@ pub async fn delete_key(
Extension(config): Extension<Arc<AppConfig>>,
Path(fingerprint): Path<String>,
) -> Result<Response, AppError> {
ensure_pgp_enabled(&config)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let normalized = db::pgp::normalize_fingerprint(&fingerprint);
let lookup = normalized.clone();
@@ -505,6 +523,7 @@ pub async fn set_trust(
Path(fingerprint): Path<String>,
Json(request): Json<TrustRequest>,
) -> Result<Json<PgpKeyRecord>, AppError> {
ensure_pgp_enabled(&config)?;
validate_trust(&request.trust)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let normalized = db::pgp::normalize_fingerprint(&fingerprint);
@@ -529,6 +548,7 @@ pub async fn get_preferences(
Extension(session): Extension<SessionState>,
Extension(config): Extension<Arc<AppConfig>>,
) -> Result<Json<PgpPreferences>, AppError> {
ensure_pgp_enabled(&config)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let prefs = database
.run(db::pgp::get_preferences)
@@ -542,6 +562,7 @@ pub async fn update_preferences(
Extension(config): Extension<Arc<AppConfig>>,
Json(mut prefs): Json<PgpPreferences>,
) -> Result<Json<PgpPreferences>, AppError> {
ensure_pgp_enabled(&config)?;
if !config.pgp_wkd_enabled {
prefs.wkd_enabled = false;
}
@@ -561,6 +582,7 @@ pub async fn discover_key(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<DiscoverRequest>,
) -> Result<Json<DiscoverResponse>, AppError> {
ensure_pgp_enabled(&config)?;
validate_email(&request.email)?;
let email = request.email.trim().to_ascii_lowercase();
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
@@ -635,6 +657,7 @@ pub async fn recipient_status(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<RecipientsRequest>,
) -> Result<Json<Vec<RecipientStatus>>, AppError> {
ensure_pgp_enabled(&config)?;
if request.emails.len() > 200 {
return Err(AppError::BadRequest(
"Too many OpenPGP recipients".to_string(),
@@ -682,6 +705,7 @@ pub async fn verify(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<VerifyRequest>,
) -> Result<Json<VerifyResponse>, AppError> {
ensure_pgp_enabled(&config)?;
let database = db::pool::UserDb::new(&config.data_dir, &session.user_hash);
let fingerprint = db::pgp::normalize_fingerprint(&request.fingerprint);
let lookup = fingerprint.clone();
@@ -719,6 +743,7 @@ pub async fn decrypt(
Extension(config): Extension<Arc<AppConfig>>,
Json(request): Json<DecryptRequest>,
) -> Result<Json<crate::pgp::DecryptResult>, AppError> {
ensure_pgp_enabled(&config)?;
if request.passphrase.len() > 1024 {
return Err(AppError::BadRequest(
"OpenPGP passphrase is too long".to_string(),
+7
View File
@@ -183,6 +183,13 @@ pub async fn send_message_handler(
}
let pgp_mode = resolve_pgp_mode(req.pgp_mode.as_deref(), selected_identity.as_ref())?;
if !config.pgp_enabled
&& (pgp_mode != PgpMode::None || req.pgp_attach_public_key || req.pgp_attach_recipient_keys)
{
return Err(AppError::ServiceUnavailable(
"OpenPGP is disabled by administrator policy".to_string(),
));
}
let pgp_inline = req.pgp_inline.unwrap_or_else(|| {
selected_identity
.as_ref()
+1 -1
View File
@@ -228,7 +228,7 @@ async fn post_one_click_to_address(
format!("{host}:{port}")
};
let request = format!(
"POST {path} HTTP/1.1\r\nHost: {host_header}\r\nUser-Agent: RustBox/0.1.1\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{ONE_CLICK_BODY}",
"POST {path} HTTP/1.1\r\nHost: {host_header}\r\nUser-Agent: RustBox/0.1.2\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{ONE_CLICK_BODY}",
ONE_CLICK_BODY.len()
);
+19 -1
View File
@@ -47,6 +47,7 @@ pub struct SearchQuery {
pub struct SearchEngine {
base_dir: PathBuf,
enabled: bool,
}
struct SchemaFields {
@@ -108,10 +109,27 @@ fn build_schema() -> (Schema, SchemaFields) {
impl SearchEngine {
pub fn new(base_dir: PathBuf) -> Self {
Self { base_dir }
Self {
base_dir,
enabled: true,
}
}
pub fn disabled(base_dir: PathBuf) -> Self {
Self {
base_dir,
enabled: false,
}
}
pub fn is_enabled(&self) -> bool {
self.enabled
}
pub fn open_user_index(&self, user_hash: &str) -> Result<UserIndex, String> {
if !self.enabled {
return Err("Tantivy search index is disabled by configuration".to_string());
}
let index_dir = self.base_dir.join(user_hash).join("tantivy");
std::fs::create_dir_all(&index_dir)
.map_err(|e| format!("Failed to create index directory: {e}"))?;
+1 -1
View File
@@ -29,7 +29,7 @@ impl fmt::Display for SieveError {
write!(f, "ManageSieve server rejected the command: {msg}")
}
Self::Timeout => write!(f, "ManageSieve operation timed out"),
Self::ScriptTooLarge => write!(f, "Sieve script exceeds the RustBox 0.1.1 size limit"),
Self::ScriptTooLarge => write!(f, "Sieve script exceeds the RustBox 0.1.2 size limit"),
}
}
}