v0.2.3

2026-10-31 19:40:19 +01:00
1 parent 5b22add1f3
commit 2c98b8f925
@@ -28,6 +28,15 @@ POSTGRES_PASSWORD=rustbox-dev-password
 S3_ACCESS_KEY=rustbox
 S3_SECRET_KEY=rustbox-s3-dev-password

+# HA/Garage variant. Development credentials only; replace outside local testing.
+GARAGE_ACCESS_KEY_ID=GKb69252a3b0643e8bd08d4cd4
+GARAGE_SECRET_ACCESS_KEY=9115be9c9e4994306a4176543b0db461f4eb04c5c8a388676d1b57392d0f4e93
+GARAGE_CAPACITY=10G
+SIEVE_ENABLED=false
+DIAGNOSTICS_ENABLED=true
+ABOUT_ENABLED=true
+SECURITY_HEADERS_ENABLED=false
+
 # MySQL variant.
 MYSQL_PASSWORD=rustbox-dev-password
 MYSQL_ROOT_PASSWORD=rustbox-root-password
@@ -2,10 +2,26 @@

 Run all commands from the `docker/` directory.

-The Compose files use the project name `rustbox` and the examples below additionally pass `-p rustbox`. Using `-p` is intentional: a globally exported `COMPOSE_PROJECT_NAME` has higher priority than the `name: rustbox` field in a Compose file and could otherwise attach RustBox to an unrelated Compose project/network.
+Most Compose files use the project name `rustbox`; the HA/Garage example uses `rustbox-ha`. The examples pass `-p` explicitly so an exported `COMPOSE_PROJECT_NAME` cannot attach RustBox to an unrelated Compose project/network.

 ## Quick start

+Two RustBox nodes + Envoy + Valkey + PostgreSQL + OpenSearch + Garage S3:
+
+```bash
+PORT=3009 \
+IMAP_HOST=mail.linuxiarz.pl \
+SMTP_HOST=mail.linuxiarz.pl \
+docker compose -p rustbox-ha -f compose-ha-garage.yml up --build
+```
+
+Verify load balancing and the selected backend header:
+
+```bash
+curl -sSI http://localhost:3009/ | grep -i '^x-rustbox-backend:'
+curl -sSI http://localhost:3009/ | grep -i '^x-rustbox-backend:'
+```
+
 PostgreSQL + S3/SILO + OpenSearch:

 ```bash
@@ -58,7 +74,7 @@ cp .env.example .env
 | Variable | Default | Used by | Description |
 |---|---|---|---|
 | `HOST` | `0.0.0.0` | all | Address RustBox listens on inside the container. |
-| `PORT` | `3001` | all | RustBox HTTP port. The same value is used for the container listener and host port mapping. Example: `PORT=3009` publishes `3009:3009`. |
+| `PORT` | `3001` | all | Published HTTP port. Single-node examples also use it as the RustBox listener; HA keeps RustBox on `3001` internally and maps `${PORT}` to Envoy `8080`. |
 | `IMAP_HOST` | `mail.example.com` | all | IMAP server hostname used for mailbox access/authentication. Set this for a real mail system. |
 | `IMAP_PORT` | `993` | all | IMAP server port. |
 | `SMTP_HOST` | `mail.example.com` | all | SMTP server hostname used for sending mail. |
@@ -73,6 +89,13 @@ cp .env.example .env
 | `MYSQL_ROOT_PASSWORD` | `rustbox-root-password` | MySQL/NFS | MySQL root password used by the container and health check. |
 | `NFS_SERVER` | none | MySQL/NFS | NFS server address. Required for `compose-mysql-nfs-opensearch.yml`. |
 | `NFS_EXPORT` | `/rustbox` | MySQL/NFS | Export path on the NFS server. |
+| `GARAGE_ACCESS_KEY_ID` | development key | HA/Garage | Garage S3 access key imported by the one-shot initializer. Replace outside local testing. |
+| `GARAGE_SECRET_ACCESS_KEY` | development secret | HA/Garage | Garage S3 secret key. Replace outside local testing. |
+| `GARAGE_CAPACITY` | `10G` | HA/Garage | Capacity assigned to the single Garage node for layout balancing. |
+| `SIEVE_ENABLED` | `false` | HA/Garage | Enables ManageSieve routes/UI on both RustBox nodes. |
+| `DIAGNOSTICS_ENABLED` | `true` | HA/Garage | Enables diagnostics routes/UI. |
+| `ABOUT_ENABLED` | `true` | HA/Garage | Enables the About panel; keyboard shortcuts stay available independently. |
+| `SECURITY_HEADERS_ENABLED` | `false` | HA/Garage | Enables RustBox browser security headers. |
 | `COMPOSE_PROJECT_NAME` | `rustbox` in `.env.example` | Compose | Compose project name. A value exported in the shell overrides the file's `name:`. The documented commands use `-p rustbox`, which has the highest priority. |

 `RUSTBOX_CONFIG` and `DATABASE_URL` are set by the Compose files themselves and normally should not be supplied from the shell for these bundled topologies.
@@ -100,6 +123,14 @@ docker compose -p rustbox -f compose-postgres-s3-opensearch.yml \

 ## Topologies

+### `compose-ha-garage.yml`
+
+Runs two RustBox application containers behind Envoy round-robin load balancing. PostgreSQL stores account/application state, Valkey stores shared sessions, OpenSearch provides the shared search index, and Garage provides S3-compatible blob storage. Only Envoy publishes a host port; the application nodes stay internal.
+
+Envoy adds `X-RustBox-Backend` with the selected upstream address to every proxied response. WebSocket upgrades are enabled and Envoy health-checks `/api/health` on both nodes. The example uses a fixed Docker subnet because Garage advertises its RPC address; change `172.28.80.0/24` in both the Compose file and `configs/ha-garage.toml` if it conflicts with your network.
+
+Garage is a single storage node in this local example. The application tier is redundant, but the backing services are not an HA production cluster by themselves.
+
 ### `compose-single-sqlite.yml`

 Single RustBox node with SQLite, filesystem blob storage and local Tantivy search. Persistent application data is stored in the `rustbox-data` Docker volume.
@@ -120,7 +151,7 @@ Replace all example passwords, S3 credentials and `SESSION_ENCRYPTION_KEY` befor

 The OpenSearch examples disable its security plugin to keep the local Compose topology self-contained. Production should use TLS/authentication and HTTPS URLs in `search.urls`.

-The two cluster-ready examples intentionally start one RustBox application container so they can run directly on a single Docker host. For multiple RustBox application nodes, put them behind an external load balancer and do not publish the same host port from every node; SQL, blob storage and OpenSearch must remain shared between the nodes.
+The PostgreSQL/SILO and MySQL/NFS examples intentionally start one RustBox application container. `compose-ha-garage.yml` demonstrates two application nodes sharing SQL, sessions, blob storage and search behind Envoy. For production, make the backing PostgreSQL/Valkey/OpenSearch/Garage services appropriately redundant and terminate TLS before or at the load balancer.

 ## Sessions and HTTP/HTTPS

@@ -132,6 +163,12 @@ For a production deployment behind HTTPS set:
 ENVIRONMENT=production docker compose -p rustbox -f compose-postgres-s3-opensearch.yml up -d --build
 ```

+For the HA example, expose Envoy only behind HTTPS/TLS termination and use:
+
+```bash
+ENVIRONMENT=production docker compose -p rustbox-ha -f compose-ha-garage.yml up -d --build
+```
+
 In `production`, RustBox intentionally marks authentication cookies as `Secure`. Opening the application directly over plain HTTP in that mode causes the browser not to send the session cookie and protected requests return `Invalid or expired session`.

 After changing from production/HTTP to development/HTTP, log in again. If the browser still has stale RustBox state, clear cookies/site data for the RustBox origin once and reload.
@@ -164,4 +201,4 @@ docker compose -p rustbox -f compose-postgres-s3-opensearch.yml exec opensearch
  -d '{"index.blocks.read_only_allow_delete":null}'
 ```

-RustBox 0.2.2 backs off its OpenSearch outbox worker after these 429/flood-stage responses, so a full disk no longer produces one warning per queued event.
+RustBox 0.2.3 retains the OpenSearch outbox backoff introduced in 0.2.2, so these 429/flood-stage responses no longer produce one warning per queued event on a full disk.
@@ -0,0 +1,157 @@
+name: rustbox-ha
+
+x-rustbox-common: &rustbox-common
+  build:
+    context: ..
+    dockerfile: docker/Dockerfile
+  depends_on:
+    postgres:
+      condition: service_healthy
+    valkey:
+      condition: service_healthy
+    garage-init:
+      condition: service_completed_successfully
+    opensearch:
+      condition: service_healthy
+  environment: &rustbox-environment
+    RUSTBOX_CONFIG: /etc/rustbox/rustbox.toml
+    ENVIRONMENT: ${ENVIRONMENT:-development}
+    HOST: 0.0.0.0
+    PORT: 3001
+    IMAP_HOST: ${IMAP_HOST:-mail.example.com}
+    IMAP_PORT: ${IMAP_PORT:-993}
+    SMTP_HOST: ${SMTP_HOST:-mail.example.com}
+    SMTP_PORT: ${SMTP_PORT:-587}
+    TLS_ENABLED: ${TLS_ENABLED:-true}
+    RUST_LOG: ${RUST_LOG:-info}
+    DATABASE_URL: postgres://rustbox:${POSTGRES_PASSWORD:-rustbox-dev-password}@postgres:5432/rustbox
+    SESSION_ENCRYPTION_KEY: ${SESSION_ENCRYPTION_KEY:-MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=}
+    AWS_ACCESS_KEY_ID: ${GARAGE_ACCESS_KEY_ID:-GKb69252a3b0643e8bd08d4cd4}
+    AWS_SECRET_ACCESS_KEY: ${GARAGE_SECRET_ACCESS_KEY:-9115be9c9e4994306a4176543b0db461f4eb04c5c8a388676d1b57392d0f4e93}
+    SIEVE_ENABLED: ${SIEVE_ENABLED:-false}
+    DIAGNOSTICS_ENABLED: ${DIAGNOSTICS_ENABLED:-true}
+    ABOUT_ENABLED: ${ABOUT_ENABLED:-true}
+    SECURITY_HEADERS_ENABLED: ${SECURITY_HEADERS_ENABLED:-false}
+  volumes:
+    - ./configs/ha-garage.toml:/etc/rustbox/rustbox.toml:ro
+  expose:
+    - "3001"
+  networks:
+    - rustbox-ha
+  restart: unless-stopped
+
+services:
+  postgres:
+    image: postgres:17-bookworm
+    environment:
+      POSTGRES_DB: rustbox
+      POSTGRES_USER: rustbox
+      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-rustbox-dev-password}
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U rustbox -d rustbox"]
+      interval: 5s
+      timeout: 5s
+      retries: 20
+    volumes:
+      - postgres-ha-data:/var/lib/postgresql/data
+    networks:
+      - rustbox-ha
+
+  valkey:
+    image: valkey/valkey:9.1.3-alpine
+    command: ["valkey-server", "--appendonly", "yes"]
+    healthcheck:
+      test: ["CMD", "valkey-cli", "ping"]
+      interval: 5s
+      timeout: 3s
+      retries: 20
+    volumes:
+      - valkey-data:/data
+    networks:
+      - rustbox-ha
+
+  opensearch:
+    image: opensearchproject/opensearch:3.9.0
+    environment:
+      discovery.type: single-node
+      DISABLE_INSTALL_DEMO_CONFIG: "true"
+      DISABLE_SECURITY_PLUGIN: "true"
+      OPENSEARCH_JAVA_OPTS: -Xms512m -Xmx512m
+    healthcheck:
+      test: ["CMD-SHELL", "curl -fsS http://localhost:9200/_cluster/health >/dev/null || exit 1"]
+      interval: 10s
+      timeout: 5s
+      retries: 30
+    volumes:
+      - opensearch-ha-data:/usr/share/opensearch/data
+    networks:
+      - rustbox-ha
+
+  garage:
+    image: dxflrs/garage:v2.5.0
+    environment:
+      RUST_LOG: ${GARAGE_RUST_LOG:-info}
+    command: ["/garage", "-c", "/etc/garage.toml", "server"]
+    healthcheck:
+      test: ["CMD", "/garage", "-c", "/etc/garage.toml", "status"]
+      interval: 5s
+      timeout: 5s
+      retries: 30
+    volumes:
+      - ./garage/garage.toml:/etc/garage.toml:ro
+      - garage-meta:/var/lib/garage/meta
+      - garage-data:/var/lib/garage/data
+    networks:
+      rustbox-ha:
+        ipv4_address: 172.28.80.10
+
+  garage-init:
+    build:
+      context: ..
+      dockerfile: docker/garage/init/Dockerfile
+    depends_on:
+      garage:
+        condition: service_healthy
+    environment:
+      GARAGE_ACCESS_KEY_ID: ${GARAGE_ACCESS_KEY_ID:-GKb69252a3b0643e8bd08d4cd4}
+      GARAGE_SECRET_ACCESS_KEY: ${GARAGE_SECRET_ACCESS_KEY:-9115be9c9e4994306a4176543b0db461f4eb04c5c8a388676d1b57392d0f4e93}
+      GARAGE_BUCKET: rustbox
+      GARAGE_CAPACITY: ${GARAGE_CAPACITY:-10G}
+    volumes:
+      - ./garage/garage.toml:/etc/garage.toml:ro
+      - garage-meta:/var/lib/garage/meta
+      - garage-data:/var/lib/garage/data
+    networks:
+      - rustbox-ha
+
+  rustbox-1:
+    <<: *rustbox-common
+
+  rustbox-2:
+    <<: *rustbox-common
+
+  envoy:
+    image: envoyproxy/envoy:v1.40.2
+    depends_on:
+      - rustbox-1
+      - rustbox-2
+    ports:
+      - "${PORT:-3001}:8080"
+    volumes:
+      - ./envoy/envoy.yaml:/etc/envoy/envoy.yaml:ro
+    networks:
+      - rustbox-ha
+    restart: unless-stopped
+
+volumes:
+  postgres-ha-data:
+  valkey-data:
+  opensearch-ha-data:
+  garage-meta:
+  garage-data:
+
+networks:
+  rustbox-ha:
+    ipam:
+      config:
+        - subnet: 172.28.80.0/24
@@ -0,0 +1,32 @@
+host = "0.0.0.0"
+port = 3001
+environment = "development"
+database_backend = "pgsql"
+session_backend = "redis"
+redis_url = "redis://valkey:6379/"
+cluster_mode = true
+pgp_enabled = false
+sieve_enabled = false
+diagnostics_enabled = true
+about_enabled = true
+security_headers_enabled = false
+trusted_proxies = "172.28.80.0/24"
+
+[storage]
+backend = "s3"
+bucket = "rustbox"
+region = "garage"
+endpoint = "http://garage:3900"
+path_style = true
+allow_http = true
+prefix = "mail"
+
+[search]
+backend = "opensearch"
+urls = ["http://opensearch:9200"]
+index_prefix = "rustbox"
+allow_insecure_http = true
+required = true
+worker_interval_secs = 2
+worker_batch_size = 100
+worker_max_attempts = 8
@@ -0,0 +1,63 @@
+static_resources:
+  listeners:
+    - name: rustbox_http
+      address:
+        socket_address:
+          address: 0.0.0.0
+          port_value: 8080
+      filter_chains:
+        - filters:
+            - name: envoy.filters.network.http_connection_manager
+              typed_config:
+                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
+                stat_prefix: rustbox_ingress
+                use_remote_address: true
+                upgrade_configs:
+                  - upgrade_type: websocket
+                route_config:
+                  name: rustbox_routes
+                  virtual_hosts:
+                    - name: rustbox
+                      domains: ["*"]
+                      response_headers_to_add:
+                        - header:
+                            key: X-RustBox-Backend
+                            value: "%UPSTREAM_REMOTE_ADDRESS%"
+                          append_action: OVERWRITE_IF_EXISTS_OR_ADD
+                      routes:
+                        - match:
+                            prefix: "/"
+                          route:
+                            cluster: rustbox
+                            timeout: 0s
+                http_filters:
+                  - name: envoy.filters.http.router
+                    typed_config:
+                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
+
+  clusters:
+    - name: rustbox
+      type: STRICT_DNS
+      connect_timeout: 2s
+      lb_policy: ROUND_ROBIN
+      health_checks:
+        - timeout: 2s
+          interval: 5s
+          unhealthy_threshold: 2
+          healthy_threshold: 1
+          http_health_check:
+            path: /api/health
+      load_assignment:
+        cluster_name: rustbox
+        endpoints:
+          - lb_endpoints:
+              - endpoint:
+                  address:
+                    socket_address:
+                      address: rustbox-1
+                      port_value: 3001
+              - endpoint:
+                  address:
+                    socket_address:
+                      address: rustbox-2
+                      port_value: 3001
@@ -0,0 +1,12 @@
+metadata_dir = "/var/lib/garage/meta"
+data_dir = "/var/lib/garage/data"
+db_engine = "sqlite"
+replication_factor = 1
+rpc_bind_addr = "0.0.0.0:3901"
+rpc_public_addr = "172.28.80.10:3901"
+rpc_secret = "8f2ec1b94f2ff5fc39391d8e1666929f8d6adc4f2270d5c950866a0a65e5f4b3"
+
+[s3_api]
+s3_region = "garage"
+api_bind_addr = "0.0.0.0:3900"
+root_domain = ".s3.garage.localhost"
@@ -0,0 +1,6 @@
+FROM dxflrs/garage:v2.5.0 AS garage
+FROM alpine:3.22
+COPY --from=garage /garage /usr/local/bin/garage
+COPY docker/garage/init/init.sh /usr/local/bin/init-garage
+RUN chmod +x /usr/local/bin/init-garage
+ENTRYPOINT ["/usr/local/bin/init-garage"]
@@ -0,0 +1,22 @@
+#!/bin/sh
+set -eu
+
+CONFIG=/etc/garage.toml
+STATUS=/tmp/garage-status
+
+until garage -c "$CONFIG" status >"$STATUS" 2>/dev/null; do
+  sleep 1
+done
+
+node_id="$(awk '/NO ROLE ASSIGNED/ { print $1; exit }' "$STATUS")"
+if [ -n "$node_id" ]; then
+  garage -c "$CONFIG" layout assign -z local -c "${GARAGE_CAPACITY:-10G}" "$node_id"
+  garage -c "$CONFIG" layout apply --version 1
+fi
+
+garage -c "$CONFIG" key import -n rustbox --yes \
+  "$GARAGE_ACCESS_KEY_ID" "$GARAGE_SECRET_ACCESS_KEY" >/dev/null 2>&1 || true
+
+garage -c "$CONFIG" bucket create "$GARAGE_BUCKET" >/dev/null 2>&1 || true
+garage -c "$CONFIG" bucket allow --read --write \
+  "$GARAGE_BUCKET" --key "$GARAGE_ACCESS_KEY_ID"