Mateusz Gruszczyński committed
1 parent
0d9f3ff447
commit
95c0587c1e
26 files changed
+235
-50
No files matched your search
@@ -1,5 +1,15 @@
|
||||
# RustBox changelog
|
||||
|
||||
### 0.2.1
|
||||
- Fixed Docker development authentication by documenting and defaulting Compose to `ENVIRONMENT=development`, so session cookies work over direct HTTP while production keeps `Secure` cookies behind HTTPS.
|
||||
- Exposed `pgp_enabled` through the auth policy and hid OpenPGP settings, identity controls, composer controls and reader UI when OpenPGP is disabled by administrator policy.
|
||||
- Prevented disabled OpenPGP from producing repeated UI error toasts or unnecessary API requests.
|
||||
- Added OpenSearch outbox worker backoff for HTTP 429/flood-stage/read-only conditions and stopped the current batch after the first backpressure failure, preventing log storms while preserving durable retries without consuming dead-letter attempts.
|
||||
- Kept OpenSearch disk safety thresholds intact; Docker documentation explains that named volumes consume the Docker host filesystem and how to diagnose/recover flood-stage blocks.
|
||||
- Added a stable `rustbox` Compose project name, documented Docker environment variables, and made application port mapping configurable through `PORT`.
|
||||
- Reduced PostgreSQL startup log noise in the Docker example by suppressing repeated SQLx `NOTICE` messages at the default log filter.
|
||||
- Bumped the application/source release to 0.2.1.
|
||||
|
||||
### 0.2.0
|
||||
- Fixed nested database transactions during message move by reusing the caller-owned transaction for cache removal, header-sync invalidation, folder invalidation, and search outbox writes.
|
||||
- Kept `search_outbox` in sync between tenant and legacy-import table sets so schema/import parity tests pass.
|
||||
|
||||
Generated
+1
-1
@@ -3508,7 +3508,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustbox"
|
||||
version = "0.2.0"
|
||||
version = "0.2.1"
|
||||
dependencies = [
|
||||
"aes-gcm",
|
||||
"async-imap",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "rustbox"
|
||||
version = "0.2.0"
|
||||
version = "0.2.1"
|
||||
edition = "2024"
|
||||
rust-version = "1.88"
|
||||
default-run = "rustbox"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# RustBox attribution / notice
|
||||
|
||||
RustBox 0.2.0 is derived from the Oxi webmail project:
|
||||
RustBox 0.2.1 is derived from the Oxi webmail project:
|
||||
|
||||
- Upstream: `c0h1b4/oxi`
|
||||
- Base snapshot: `7f1507c486b78cf33e54beb701dd460d3bd2bd5d`
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RustBox 0.2.0
|
||||
# RustBox 0.2.1
|
||||
|
||||
RustBox is a webmail derived from `c0h1b4/oxi`. The consumed Oxi revision is recorded in `UPSTREAM_BASE`.
|
||||
|
||||
@@ -34,7 +34,7 @@ rustbox/
|
||||
|
||||
`build.rs` combines the CSS modules from `src/web/css/manifest.txt` and JavaScript modules from `src/web/js/manifest.txt`. Cargo embeds the generated CSS/JS bundles, HTML and SVG sprite in the RustBox executable. There is no standalone source `src/web/app.js`; `src/web/` is the single source of truth and Cargo generates the embedded bundles in `OUT_DIR`.
|
||||
|
||||
## Storage and search in 0.2.0
|
||||
## Storage and search in 0.2.1
|
||||
|
||||
RustBox separates three concerns: SQL metadata, blob storage and search. The SQL engine does **not** decide whether `data_dir` may be used. A single-node PostgreSQL/MySQL deployment may still use filesystem blobs and local Tantivy. In cluster mode, local node state is rejected unless the configured backend is explicitly shared.
|
||||
|
||||
@@ -148,7 +148,7 @@ rustbox search reindex
|
||||
|
||||
Ready Docker examples are in `docker/`: standalone SQLite, PostgreSQL + SILO (S3-compatible) + OpenSearch, and MySQL + NFS + OpenSearch.
|
||||
|
||||
All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. The squashed compatibility baseline remains V046; RustBox 0.2.0 adds runtime migration V047 for attachment checksums/logical blob keys and the durable search outbox. The next migration number is V048. See `migrations/README.md`.
|
||||
All executable application SQL is centralized by dialect under `src/queries/{sqlite,mysql,pgsql}/queries.rs`. The squashed compatibility baseline remains V046; RustBox 0.2.1 adds runtime migration V047 for attachment checksums/logical blob keys and the durable search outbox. The next migration number is V048. See `migrations/README.md`.
|
||||
|
||||
|
||||
## Development
|
||||
@@ -217,7 +217,7 @@ Initialize Git after extracting the release archive:
|
||||
```bash
|
||||
git init
|
||||
git add .
|
||||
git commit -m "RustBox 0.2.0 release"
|
||||
git commit -m "RustBox 0.2.1 release"
|
||||
./scripts/setup-upstream.sh
|
||||
```
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
RustBox 0.2.0 release manifest
|
||||
RustBox 0.2.1 release manifest
|
||||
|
||||
Architecture:
|
||||
- database_backend = sqlite | mysql | pgsql selects authoritative SQL only
|
||||
@@ -33,13 +33,13 @@ Docker examples:
|
||||
- docker/compose-mysql-nfs-opensearch.yml (MySQL + real NFS volume + OpenSearch)
|
||||
|
||||
Operations/docs:
|
||||
- README.md documents 0.2.0 storage/search configuration
|
||||
- README.md documents 0.2.1 storage/search configuration
|
||||
- docker/README.md documents Compose variants
|
||||
- docs/STORAGE_SEARCH_OPERATIONS.md documents failure modes, shared-storage requirements and migration behavior
|
||||
- TODO.md leaves live SILO/MinIO/AWS/Ceph and real multi-node acceptance validation explicitly pending
|
||||
|
||||
Build/release metadata:
|
||||
- Cargo.toml version = 0.2.0
|
||||
- Cargo.toml version = 0.2.1
|
||||
- rust-toolchain.toml pins Rust 1.88.0
|
||||
- Cargo.lock is intentionally not shipped because this environment could not regenerate it after dependency changes
|
||||
- CI and scripts/build-release.sh generate Cargo.lock, then run locked tests/Clippy/release build
|
||||
@@ -1,7 +1,7 @@
|
||||
RustBox 0.2.0 - final source validation
|
||||
RustBox 0.2.1 - final source validation
|
||||
|
||||
Completed in this environment:
|
||||
- offline release validator: scripts/validate-release-static.py (139/139 checks passed)
|
||||
- offline release validator: scripts/validate-release-static.py (144/144 checks passed)
|
||||
- all maintained TOML examples parse successfully
|
||||
- Docker Compose topology/config references are validated by the offline release validator
|
||||
- JSON files parse successfully
|
||||
@@ -11,9 +11,12 @@ Completed in this environment:
|
||||
- Rust source delimiter sanity check passes after removing partial-patch leftovers
|
||||
- git diff --check passes on the final source tree
|
||||
- no Cargo.lock, *.orig, *.rej, temporary FIX/APPLY scripts, or patch leftovers are shipped
|
||||
- regression source checks cover every Clippy failure reported during the 0.2.0 release attempt
|
||||
- regression source checks retain the 0.2.0 release gate and add 0.2.1 policy/backpressure checks
|
||||
|
||||
0.2.0 fixes included:
|
||||
0.2.1 fixes included:
|
||||
- OpenPGP policy visibility follows pgp_enabled and disabled OpenPGP is hidden from the UI
|
||||
- OpenSearch 429/flood-stage failures trigger worker backoff instead of per-event warning storms
|
||||
- Docker development defaults session cookies for direct HTTP and documents the production HTTPS requirement
|
||||
- cluster_mode controls the implicit data_dir default; PostgreSQL/MySQL remain valid with local data_dir backends in single-node deployments
|
||||
- storage/search configuration comes only from [storage] and [search]
|
||||
- removed pre-0.2 top-level storage/search switches and the tantivy backend alias
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# RustBox 0.2.0
|
||||
# RustBox 0.2.1
|
||||
host = "127.0.0.1"
|
||||
port = 3001
|
||||
environment = "production"
|
||||
|
||||
+31
-1
@@ -112,7 +112,7 @@ The bundled SILO topology is intended for local/integration validation of the S3
|
||||
|
||||
### `compose-mysql-nfs-opensearch.yml`
|
||||
|
||||
MySQL, shared POSIX/NFS blob storage and OpenSearch. The NFS volume driver is deliberate: a normal local Docker named volume is not shared between Docker hosts and therefore is not a valid blob backend for a multi-host RustBox deployment.
|
||||
MySQL, shared POSIX/NFS blob storage and OpenSearch. The NFS volume driver is deliberate: a normal local Docker named volume is not shared between Docker hosts and therefore is not a valid blob backend for a multi-host RustBox deployment. These examples intentionally run one RustBox application container; the shared backing services demonstrate storage/database topology, not multi-app load balancing.
|
||||
|
||||
## Production notes
|
||||
|
||||
@@ -135,3 +135,33 @@ ENVIRONMENT=production docker compose -p rustbox -f compose-postgres-s3-opensear
|
||||
In `production`, RustBox intentionally marks authentication cookies as `Secure`. Opening the application directly over plain HTTP in that mode causes the browser not to send the session cookie and protected requests return `Invalid or expired session`.
|
||||
|
||||
After changing from production/HTTP to development/HTTP, log in again. If the browser still has stale RustBox state, clear cookies/site data for the RustBox origin once and reload.
|
||||
|
||||
## OpenSearch disk watermark / 429 errors
|
||||
|
||||
The `opensearch-data` named volume still consumes space on the filesystem used by Docker. Running OpenSearch in a container does not give it an independent disk. If the Docker host filesystem reaches OpenSearch's flood-stage threshold, OpenSearch protects indices with `read_only_allow_delete` and RustBox indexing receives HTTP 429 responses.
|
||||
|
||||
Check the actual filesystem seen by the container:
|
||||
|
||||
```bash
|
||||
docker compose -p rustbox -f compose-postgres-s3-opensearch.yml exec opensearch \
|
||||
df -h /usr/share/opensearch/data
|
||||
```
|
||||
|
||||
Also inspect Docker usage on the host:
|
||||
|
||||
```bash
|
||||
df -h
|
||||
docker system df
|
||||
docker volume inspect rustbox_opensearch-data
|
||||
```
|
||||
|
||||
Free disk space first. Do not permanently lower the OpenSearch flood-stage safety threshold. If the index remains read-only after disk usage is back below the high watermark, clear the block explicitly:
|
||||
|
||||
```bash
|
||||
docker compose -p rustbox -f compose-postgres-s3-opensearch.yml exec opensearch \
|
||||
curl -fsS -X PUT http://localhost:9200/rustbox-messages-v1/_settings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"index.blocks.read_only_allow_delete":null}'
|
||||
```
|
||||
|
||||
RustBox 0.2.1 backs off its OpenSearch outbox worker after these 429/flood-stage responses, so a full disk no longer produces one warning per queued event.
|
||||
@@ -76,7 +76,7 @@ services:
|
||||
SMTP_HOST: ${SMTP_HOST:-mail.example.com}
|
||||
SMTP_PORT: ${SMTP_PORT:-587}
|
||||
TLS_ENABLED: ${TLS_ENABLED:-true}
|
||||
RUST_LOG: ${RUST_LOG:-info}
|
||||
RUST_LOG: ${RUST_LOG:-info,sqlx::postgres::notice=warn}
|
||||
DATABASE_URL: postgres://rustbox:${POSTGRES_PASSWORD:-rustbox-dev-password}@postgres:5432/rustbox
|
||||
SESSION_ENCRYPTION_KEY: ${SESSION_ENCRYPTION_KEY:-MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=}
|
||||
AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY:-rustbox}
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# RustBox 0.2.0 - implementation status
|
||||
# RustBox 0.2.1 - implementation status
|
||||
|
||||
This file tracks the `0.2.0` source snapshot. Storage/search scope and outstanding acceptance work are tracked in `TODO.md`. A feature is only listed as implemented when working code exists in this source tree; partial items are called out explicitly.
|
||||
This file tracks the `0.2.1` source snapshot. Storage/search scope and outstanding acceptance work are tracked in `TODO.md`. A feature is only listed as implemented when working code exists in this source tree; partial items are called out explicitly.
|
||||
|
||||
## 0.2.0 storage/search architecture
|
||||
## 0.2.1 storage/search architecture
|
||||
|
||||
- SQL backend selection (`sqlite`/`mysql`/`pgsql`) is independent from blob storage and search backend selection.
|
||||
- Cluster mode clears an implicit/default `data_dir`; PostgreSQL/MySQL single-node deployments may still intentionally use `data_dir`-backed filesystem storage and local Tantivy.
|
||||
|
||||
+1
-1
@@ -27,7 +27,7 @@ RUSTBOX_CONFIG=./config.dev.toml ./target/release/rustbox --check-config
|
||||
|
||||
Browser JavaScript source lives in functional modules under `src/web/js/`, ordered by `src/web/js/manifest.txt`. `build.rs` concatenates those modules into the embedded `rustbox-ui.js` asset during every Cargo build. The repository has no duplicate `ui/` snapshot; `src/web/` is the only browser UI source.
|
||||
|
||||
## Updating an existing checkout to 0.2.0
|
||||
## Updating an existing checkout to 0.2.1
|
||||
|
||||
Do not overlay the release with `cp *` or another command that skips dotfiles. That can retain stale `.github`, `.env.example`, or `Cargo.lock` files.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# RustBox 0.2.0 storage/search operations
|
||||
# RustBox 0.2.1 storage/search operations
|
||||
|
||||
RustBox 0.2.0 treats SQL, blob storage and search as independent capabilities. The database engine does not decide whether `data_dir` is allowed.
|
||||
RustBox 0.2.1 treats SQL, blob storage and search as independent capabilities. The database engine does not decide whether `data_dir` is allowed.
|
||||
|
||||
## Deployment matrix
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Database schemas and migrations
|
||||
|
||||
RustBox 0.2.0 keeps SQL, blob storage and search as independent concerns:
|
||||
RustBox 0.2.1 keeps SQL, blob storage and search as independent concerns:
|
||||
|
||||
```text
|
||||
database_backend = sqlite | mysql | pgsql
|
||||
@@ -26,7 +26,7 @@ migrations/
|
||||
└── V047__blob_keys_and_search_outbox.sql
|
||||
```
|
||||
|
||||
Each `schema.sql` is the current complete schema for fresh installations. The historical compatibility baseline remains **V046**. RustBox 0.2.0 adds **V047**, which adds attachment SHA-256 metadata and the durable search outbox used by external indexing.
|
||||
Each `schema.sql` is the current complete schema for fresh installations. The historical compatibility baseline remains **V046**. RustBox 0.2.1 adds **V047**, which adds attachment SHA-256 metadata and the durable search outbox used by external indexing.
|
||||
|
||||
Account data is isolated inside the selected physical database with a deterministic, collision-checked table prefix derived from the account hash. The `{{p}}` marker in tenant SQL is replaced by RustBox at provisioning/migration time. Sessions are global and therefore are not prefixed.
|
||||
|
||||
@@ -34,7 +34,7 @@ Account data is isolated inside the selected physical database with a determinis
|
||||
|
||||
Every schema-changing release must add the same logical migration for all supported SQL engines and register it in `src/db/pool.rs`. Do not update only `schema.sql`: fresh installs use the baseline while existing installations use the numbered migration path.
|
||||
|
||||
The next migration number after RustBox 0.2.0 is **V048**.
|
||||
The next migration number after RustBox 0.2.1 is **V048**.
|
||||
|
||||
Redis and Redis Sentinel do not use SQL migrations.
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RustBox 0.2.0 - packaged production configuration (DEB/RPM)
|
||||
# RustBox 0.2.1 - packaged production configuration (DEB/RPM)
|
||||
#
|
||||
# This file contains all supported RustBox configuration options.
|
||||
# Commented options are optional and do not change the packaged defaults.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Static/offline release checks for RustBox 0.2.0.
|
||||
"""Static/offline release checks for RustBox 0.2.1.
|
||||
|
||||
These checks intentionally avoid pretending to replace `cargo test`: they validate
|
||||
configuration, SQL migrations, Docker examples, query-catalog parity and release
|
||||
@@ -49,7 +49,7 @@ def extract_section(text: str, begin: str, end: str) -> str:
|
||||
|
||||
# Release/toolchain metadata.
|
||||
cargo = tomllib.loads(read("Cargo.toml"))
|
||||
require(cargo["package"]["version"] == "0.2.0", "Cargo package version is 0.2.0")
|
||||
require(cargo["package"]["version"] == "0.2.1", "Cargo package version is 0.2.1")
|
||||
require(cargo["package"]["rust-version"] == "1.88", "Cargo rust-version is 1.88")
|
||||
toolchain = tomllib.loads(read("rust-toolchain.toml"))
|
||||
require(toolchain["toolchain"]["channel"] == "1.88.0", "rust-toolchain pins Rust 1.88.0")
|
||||
@@ -58,13 +58,18 @@ require("cargo generate-lockfile" in read("scripts/build-release.sh"), "release
|
||||
require("cargo generate-lockfile" in read(".github/workflows/ci.yml"), "CI regenerates Cargo.lock before --locked checks")
|
||||
require("validate-release-static.py" in read("scripts/build-release.sh"), "release script runs offline/static release gate")
|
||||
require("validate-release-static.py" in read(".github/workflows/ci.yml"), "CI runs offline/static release gate")
|
||||
require(read("README.md").startswith("# RustBox 0.2.0"), "README identifies RustBox 0.2.0")
|
||||
require(read("RELEASE-MANIFEST.txt").startswith("RustBox 0.2.0"), "release manifest identifies RustBox 0.2.0")
|
||||
require(read("RELEASE-VALIDATION.txt").startswith("RustBox 0.2.0"), "release validation identifies RustBox 0.2.0")
|
||||
require(read("docs/IMPLEMENTATION_STATUS.md").startswith("# RustBox 0.2.0"), "implementation status identifies RustBox 0.2.0")
|
||||
require(read("README.md").startswith("# RustBox 0.2.1"), "README identifies RustBox 0.2.1")
|
||||
require(read("RELEASE-MANIFEST.txt").startswith("RustBox 0.2.1"), "release manifest identifies RustBox 0.2.1")
|
||||
require(read("RELEASE-VALIDATION.txt").startswith("RustBox 0.2.1"), "release validation identifies RustBox 0.2.1")
|
||||
require(read("docs/IMPLEMENTATION_STATUS.md").startswith("# RustBox 0.2.1"), "implementation status identifies RustBox 0.2.1")
|
||||
require((ROOT / "docs/STORAGE_SEARCH_OPERATIONS.md").is_file(), "storage/search operations guide exists")
|
||||
require('imap_host = ""' not in read("config.dev.toml"), "development config does not override IMAP host with an invalid empty value")
|
||||
require(not re.search(r"(?m)^IMAP_HOST=\s*$", read(".env.example")), "environment example does not set an invalid empty IMAP host")
|
||||
require("pgp_enabled: bool" in read("src/routes/auth.rs"), "auth policy exposes OpenPGP availability")
|
||||
require("setOpenPgpAvailability" in read("src/web/js/72-openpgp-identities.js"), "UI hides administratively disabled OpenPGP")
|
||||
require("is_opensearch_backpressure_error" in read("src/search/engine.rs"), "OpenSearch worker classifies backpressure failures")
|
||||
require("worker pass failed; backing off" in read("src/search/engine.rs"), "OpenSearch worker backs off after failures")
|
||||
require("ENVIRONMENT: ${ENVIRONMENT:-development}" in read("docker/compose-postgres-s3-opensearch.yml"), "PostgreSQL Compose defaults direct HTTP to development cookie mode")
|
||||
|
||||
# Parse all maintained TOML examples.
|
||||
toml_files = [
|
||||
|
||||
@@ -164,6 +164,45 @@ pub fn mark_failure(
|
||||
Ok(changed > 0)
|
||||
}
|
||||
|
||||
pub fn defer_transient_failure(
|
||||
conn: &Connection,
|
||||
event: &OutboxEvent,
|
||||
error: &str,
|
||||
delay_secs: u64,
|
||||
) -> Result<bool, String> {
|
||||
let available_at = now_epoch().saturating_add(delay_secs.max(1) as i64);
|
||||
let now = now_epoch();
|
||||
let truncated: String = error.chars().take(2048).collect();
|
||||
let changed = match crate::db::pool::configured_backend() {
|
||||
crate::db::pool::DatabaseBackend::Mysql => conn.execute(
|
||||
crate::dbq!(db_search_outbox::Q_004),
|
||||
params![
|
||||
event.attempts,
|
||||
available_at,
|
||||
false,
|
||||
&truncated,
|
||||
now,
|
||||
&event.id,
|
||||
&event.generation
|
||||
],
|
||||
),
|
||||
_ => conn.execute(
|
||||
crate::dbq!(db_search_outbox::Q_004),
|
||||
params![
|
||||
&event.id,
|
||||
&event.generation,
|
||||
event.attempts,
|
||||
available_at,
|
||||
false,
|
||||
&truncated,
|
||||
now
|
||||
],
|
||||
),
|
||||
}
|
||||
.map_err(|e| format!("failed to defer transient search event: {e}"))?;
|
||||
Ok(changed > 0)
|
||||
}
|
||||
|
||||
pub fn clear_before(conn: &Connection, updated_before: i64) -> Result<usize, String> {
|
||||
conn.execute(
|
||||
crate::dbq!(db_search_outbox::Q_007),
|
||||
|
||||
@@ -20,6 +20,7 @@ pub struct AuthPolicyResponse {
|
||||
remember_me_enabled: bool,
|
||||
remember_me_days: u64,
|
||||
session_timeout_hours: u64,
|
||||
pgp_enabled: bool,
|
||||
}
|
||||
|
||||
pub async fn auth_policy(Extension(config): Extension<Arc<AppConfig>>) -> Json<AuthPolicyResponse> {
|
||||
@@ -27,6 +28,7 @@ pub async fn auth_policy(Extension(config): Extension<Arc<AppConfig>>) -> Json<A
|
||||
remember_me_enabled: config.remember_me_enabled,
|
||||
remember_me_days: config.remember_me_days,
|
||||
session_timeout_hours: config.session_timeout_hours,
|
||||
pgp_enabled: config.pgp_enabled,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
+60
-6
@@ -275,16 +275,38 @@ impl SearchEngine {
|
||||
let batch_size = *worker_batch_size;
|
||||
let max_attempts = *worker_max_attempts;
|
||||
Some(tokio::spawn(async move {
|
||||
let mut interval = tokio::time::interval(Duration::from_secs(interval_secs));
|
||||
let mut failure_streak = 0u32;
|
||||
loop {
|
||||
interval.tick().await;
|
||||
match process_search_outbox(&backend, &data_dir, batch_size, max_attempts).await {
|
||||
Ok(processed) if processed > 0 => {
|
||||
let delay_secs = match process_search_outbox(
|
||||
&backend,
|
||||
&data_dir,
|
||||
batch_size,
|
||||
max_attempts,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(processed) => {
|
||||
failure_streak = 0;
|
||||
if processed > 0 {
|
||||
tracing::debug!(processed, "processed OpenSearch outbox events");
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(error) => tracing::warn!(%error, "OpenSearch outbox worker pass failed"),
|
||||
interval_secs
|
||||
}
|
||||
Err(error) => {
|
||||
failure_streak = failure_streak.saturating_add(1);
|
||||
let multiplier = 2u64.saturating_pow(failure_streak.min(5));
|
||||
let retry_after_secs = interval_secs
|
||||
.saturating_mul(multiplier)
|
||||
.clamp(interval_secs, 60);
|
||||
tracing::warn!(
|
||||
%error,
|
||||
retry_after_secs,
|
||||
"OpenSearch outbox worker pass failed; backing off"
|
||||
);
|
||||
retry_after_secs
|
||||
}
|
||||
};
|
||||
tokio::time::sleep(Duration::from_secs(delay_secs)).await;
|
||||
}
|
||||
}))
|
||||
}
|
||||
@@ -1033,6 +1055,14 @@ impl OpenSearchUserIndex {
|
||||
}
|
||||
}
|
||||
|
||||
fn is_opensearch_backpressure_error(error: &str) -> bool {
|
||||
let lower = error.to_ascii_lowercase();
|
||||
lower.contains("429 too many requests")
|
||||
|| lower.contains("status 429")
|
||||
|| lower.contains("flood-stage watermark")
|
||||
|| lower.contains("read-only-allow-delete")
|
||||
}
|
||||
|
||||
async fn process_search_outbox(
|
||||
backend: &OpenSearchBackend,
|
||||
data_dir: &str,
|
||||
@@ -1082,6 +1112,17 @@ async fn process_search_outbox(
|
||||
processed += 1;
|
||||
}
|
||||
Err(error) => {
|
||||
if is_opensearch_backpressure_error(&error) {
|
||||
crate::db::search_outbox::defer_transient_failure(
|
||||
&conn,
|
||||
&event,
|
||||
&error,
|
||||
30,
|
||||
)?;
|
||||
return Err(format!(
|
||||
"OpenSearch temporarily rejected indexing for tenant {user_hash}: {error}"
|
||||
));
|
||||
}
|
||||
crate::db::search_outbox::mark_failure(&conn, &event, &error, max_attempts)?;
|
||||
tracing::warn!(%error, event_id = %event.id, %user_hash, "OpenSearch outbox event failed");
|
||||
}
|
||||
@@ -1105,6 +1146,19 @@ mod tests {
|
||||
(tmp, user_index)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn opensearch_backpressure_classifier_detects_flood_stage_and_429() {
|
||||
assert!(is_opensearch_backpressure_error(
|
||||
"OpenSearch document upsert failed (429 Too Many Requests): flood-stage watermark"
|
||||
));
|
||||
assert!(is_opensearch_backpressure_error(
|
||||
"index blocked by read-only-allow-delete block"
|
||||
));
|
||||
assert!(!is_opensearch_backpressure_error(
|
||||
"OpenSearch document upsert failed (500 Internal Server Error)"
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn opensearch_event_lease_covers_timeout_and_failover_budget() {
|
||||
let backend = OpenSearchBackend {
|
||||
|
||||
+2
-2
@@ -943,7 +943,7 @@
|
||||
</div><button id="identity-new" class="button primary" type="button">New identity</button>
|
||||
</div>
|
||||
<div class="identity-overview">
|
||||
<div class="identity-overview-hint"><strong>Configured identities</strong><span class="muted">Select an
|
||||
<div class="identity-overview-hint"><strong>Configured identities</strong><span id="identity-overview-help" class="muted">Select an
|
||||
identity to edit its sender data, text/HTML footer, image and OpenPGP defaults.</span></div>
|
||||
<div id="identity-list" class="identity-list"></div>
|
||||
</div>
|
||||
@@ -1249,7 +1249,7 @@
|
||||
</div>
|
||||
</section>
|
||||
<section class="identity-modal-section identity-modal-options">
|
||||
<div class="identity-modal-heading"><strong>Defaults</strong><span class="muted">Choose where this signature
|
||||
<div class="identity-modal-heading"><strong>Defaults</strong><span id="identity-defaults-help" class="muted">Choose where this signature
|
||||
is inserted and, when OpenPGP is configured, its default behavior.</span></div>
|
||||
<div class="identity-toggle-grid">
|
||||
<label class="checkbox-row"><input id="identity-signature-new" type="checkbox"><span>Signature on new
|
||||
|
||||
@@ -33,7 +33,7 @@
|
||||
pgpKeys: [],
|
||||
pgpPrefs: { wkdEnabled: true, hkpEnabled: false, requireTrustedEncryption: false },
|
||||
pgpRecipientStatus: [],
|
||||
authPolicy: { rememberMeEnabled: true, rememberMeDays: 30, sessionTimeoutHours: 24 },
|
||||
authPolicy: { rememberMeEnabled: true, rememberMeDays: 30, sessionTimeoutHours: 24, pgpEnabled: true },
|
||||
contacts: [],
|
||||
trustedSenders: new Set(),
|
||||
contactGroups: [],
|
||||
|
||||
@@ -462,8 +462,9 @@ async function loadAuthPolicy() {
|
||||
try {
|
||||
state.authPolicy = await request("/auth/policy", { method: "GET" });
|
||||
} catch (_) {
|
||||
state.authPolicy = { rememberMeEnabled: true, rememberMeDays: 30, sessionTimeoutHours: 24 };
|
||||
state.authPolicy = { rememberMeEnabled: true, rememberMeDays: 30, sessionTimeoutHours: 24, pgpEnabled: true };
|
||||
}
|
||||
setOpenPgpAvailability(state.authPolicy.pgpEnabled !== false);
|
||||
const enabled = state.authPolicy.rememberMeEnabled !== false;
|
||||
["#login-remember-row", "#add-account-remember-row"].forEach((selector) => $(selector)?.classList.toggle("hidden", !enabled));
|
||||
["#login-remember", "#add-account-remember"].forEach((selector) => { const input = $(selector); if (input && !enabled) { input.checked = false; input.disabled = true; } });
|
||||
|
||||
@@ -470,7 +470,7 @@ function renderPgpReaderState(detail) {
|
||||
const title = $("#pgp-reader-title");
|
||||
const find = $("#pgp-reader-discover");
|
||||
const kind = pgpMessageKind(detail);
|
||||
const configured = state.pgpKeys.length > 0;
|
||||
const configured = state.authPolicy.pgpEnabled !== false && state.pgpKeys.length > 0;
|
||||
bar.classList.toggle("hidden", !kind.pgp || !configured);
|
||||
find.classList.add("hidden");
|
||||
if (!kind.pgp || !configured) return;
|
||||
|
||||
@@ -1,5 +1,41 @@
|
||||
|
||||
|
||||
function setOpenPgpAvailability(enabled) {
|
||||
const available = Boolean(enabled);
|
||||
state.authPolicy.pgpEnabled = available;
|
||||
document.querySelector('.settings-tab[data-tab="openpgp"]')?.classList.toggle("hidden", !available);
|
||||
const settingsPanel = document.querySelector('.settings-panel[data-panel="openpgp"]');
|
||||
if (settingsPanel && !available) settingsPanel.classList.add("hidden");
|
||||
$("#compose-pgp-option")?.classList.toggle("hidden", !available || state.pgpKeys.length === 0);
|
||||
$("#compose-pgp-panel")?.classList.toggle("hidden", !available || state.pgpKeys.length === 0);
|
||||
if (!available) $("#pgp-reader-bar")?.classList.add("hidden");
|
||||
$("#identity-pgp-key-row")?.classList.toggle("hidden", !available);
|
||||
$$(".identity-pgp-default").forEach((row) => row.classList.toggle("hidden", !available));
|
||||
const overview = $("#identity-overview-help");
|
||||
if (overview) overview.textContent = available
|
||||
? "Select an identity to edit its sender data, text/HTML footer, image and OpenPGP defaults."
|
||||
: "Select an identity to edit its sender data, text/HTML footer and image.";
|
||||
const defaultsHelp = $("#identity-defaults-help");
|
||||
if (defaultsHelp) defaultsHelp.textContent = available
|
||||
? "Choose where this signature is inserted and, when OpenPGP is configured, its default behavior."
|
||||
: "Choose where this signature is inserted.";
|
||||
if (!available) {
|
||||
state.pgpKeys = [];
|
||||
state.pgpRecipientStatus = [];
|
||||
const mode = $("#compose-pgp-mode");
|
||||
if (mode) mode.value = "none";
|
||||
const panel = $("#compose-pgp-panel");
|
||||
if (panel) panel.open = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function loadOpenPgp() {
|
||||
if (state.authPolicy.pgpEnabled === false) {
|
||||
setOpenPgpAvailability(false);
|
||||
updateComposePgpUi();
|
||||
if (state.currentDetail) renderPgpReaderState(state.currentDetail);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const [keys, prefs] = await Promise.all([
|
||||
request("/pgp/keys", { method: "GET" }),
|
||||
@@ -13,9 +49,11 @@ async function loadOpenPgp() {
|
||||
if (state.currentDetail) renderPgpReaderState(state.currentDetail);
|
||||
} catch (error) {
|
||||
state.pgpKeys = [];
|
||||
const disabled = String(error?.message || "").toLowerCase().includes("openpgp is disabled");
|
||||
if (disabled) setOpenPgpAvailability(false);
|
||||
updateComposePgpUi();
|
||||
if (state.currentDetail) renderPgpReaderState(state.currentDetail);
|
||||
toast(error.message, "error");
|
||||
if (!disabled) toast(error.message, "error");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -177,7 +215,8 @@ function updateComposePgpUi(applyDefaults = false) {
|
||||
const panel = $("#compose-pgp-panel");
|
||||
const option = $("#compose-pgp-option");
|
||||
if (!modeSelect || !panel) return;
|
||||
const configured = state.pgpKeys.length > 0;
|
||||
const available = state.authPolicy.pgpEnabled !== false;
|
||||
const configured = available && state.pgpKeys.length > 0;
|
||||
option?.classList.toggle("hidden", !configured);
|
||||
panel.classList.toggle("hidden", !configured);
|
||||
if (!configured) {
|
||||
|
||||
@@ -45,11 +45,12 @@ async function loadQuota() {
|
||||
}
|
||||
|
||||
function openSettings(tab = "appearance") {
|
||||
if (tab === "openpgp" && state.authPolicy.pgpEnabled === false) tab = "appearance";
|
||||
selectSettingsTab(tab);
|
||||
$("#settings-dialog").showModal();
|
||||
if (tab === "sieve") loadSieve();
|
||||
if (tab === "identities") { loadIdentities(); loadOpenPgp(); }
|
||||
if (tab === "openpgp") loadOpenPgp();
|
||||
if (tab === "identities") { loadIdentities(); if (state.authPolicy.pgpEnabled !== false) loadOpenPgp(); }
|
||||
if (tab === "openpgp" && state.authPolicy.pgpEnabled !== false) loadOpenPgp();
|
||||
if (tab === "notifications") loadNotificationPreferences();
|
||||
if (tab === "mailbox") loadMailPreferences();
|
||||
if (tab === "diagnostics") loadDiagnostics();
|
||||
@@ -58,6 +59,7 @@ function openSettings(tab = "appearance") {
|
||||
}
|
||||
|
||||
function selectSettingsTab(name) {
|
||||
if (name === "openpgp" && state.authPolicy.pgpEnabled === false) name = "appearance";
|
||||
$$(".settings-tab").forEach((button) => button.classList.toggle("active", button.dataset.tab === name));
|
||||
$$(".settings-panel").forEach((panel) => panel.classList.toggle("hidden", panel.dataset.panel !== name));
|
||||
}
|
||||
|
||||
@@ -292,7 +292,7 @@ function bindEvents() {
|
||||
$("#pgp-import-file").addEventListener("click", () => $("#pgp-import-file-input").click());
|
||||
$("#pgp-import-file-input").addEventListener("change", async (event) => { const file = event.target.files?.[0]; if (file) await importPgpFile(file); event.target.value = ""; });
|
||||
$("#pgp-discover").addEventListener("click", () => discoverPgpEmail());
|
||||
$$(".settings-tab").forEach((button) => button.addEventListener("click", () => { selectSettingsTab(button.dataset.tab); if (button.dataset.tab === "sieve") loadSieve(); if (button.dataset.tab === "identities") { loadIdentities(); loadOpenPgp(); } if (button.dataset.tab === "openpgp") loadOpenPgp(); if (button.dataset.tab === "notifications") loadNotificationPreferences(); if (button.dataset.tab === "mailbox") loadMailPreferences(); if (button.dataset.tab === "composing") loadResponseTemplates(); if (button.dataset.tab === "diagnostics") loadDiagnostics(); if (button.dataset.tab === "account") { renderAccountSettings(); loadActiveSessions(); } if (button.dataset.tab === "folders") { loadSpecialFolders(); loadQuota(); } }));
|
||||
$$(".settings-tab").forEach((button) => button.addEventListener("click", () => { selectSettingsTab(button.dataset.tab); if (button.dataset.tab === "sieve") loadSieve(); if (button.dataset.tab === "identities") { loadIdentities(); if (state.authPolicy.pgpEnabled !== false) loadOpenPgp(); } if (button.dataset.tab === "openpgp" && state.authPolicy.pgpEnabled !== false) loadOpenPgp(); if (button.dataset.tab === "notifications") loadNotificationPreferences(); if (button.dataset.tab === "mailbox") loadMailPreferences(); if (button.dataset.tab === "composing") loadResponseTemplates(); if (button.dataset.tab === "diagnostics") loadDiagnostics(); if (button.dataset.tab === "account") { renderAccountSettings(); loadActiveSessions(); } if (button.dataset.tab === "folders") { loadSpecialFolders(); loadQuota(); } }));
|
||||
$("#pref-theme").addEventListener("change", (event) => savePreference("theme", event.target.value));
|
||||
$("#pref-interface-skin").addEventListener("change", (event) => savePreference("interfaceSkin", event.target.value));
|
||||
$("#pref-preset-scale-unlocked").addEventListener("change", (event) => savePreference("presetScaleUnlocked", event.target.checked));
|
||||
@@ -515,7 +515,7 @@ async function boot() {
|
||||
await loadAuthPolicy();
|
||||
const authenticated = await loadAccounts();
|
||||
if (authenticated) {
|
||||
await loadPreferences(); await loadMailPreferences(); await loadOpenPgp(); await loadIdentities(); await loadResponseTemplates(); await loadFolders(true); await loadSpecialFolders(); await loadQuota(); await loadTags(); await loadNotificationPreferences(); await loadTrustedSenders(); startCalendarReminderWatcher(); connectRealtime();
|
||||
await loadPreferences(); await loadMailPreferences(); if (state.authPolicy.pgpEnabled !== false) await loadOpenPgp(); else setOpenPgpAvailability(false); await loadIdentities(); await loadResponseTemplates(); await loadFolders(true); await loadSpecialFolders(); await loadQuota(); await loadTags(); await loadNotificationPreferences(); await loadTrustedSenders(); startCalendarReminderWatcher(); connectRealtime();
|
||||
const params = new URLSearchParams(location.search);
|
||||
const folder = params.get("mail_folder");
|
||||
const uid = Number(params.get("mail_uid"));
|
||||
|
||||
Reference in new issue
Block a user