poc2_worked
This commit is contained in:
Mateusz Gruszczyński
@@ -1,10 +1,95 @@
|
||||
# RouterOS TZSP + Suricata IDS
|
||||
# MikroSuricata
|
||||
|
||||
Project version: `0.5.3`
|
||||
Project version: `0.9.1`
|
||||
|
||||
|
||||
## What changed in 0.9.1
|
||||
|
||||
- Fixed a startup regression where `main.py` passed `backup_manager` to `EVEWatcher` even though the watcher has no such constructor argument. Backup management remains correctly attached to the web/admin layer.
|
||||
- Added a regression test that statically validates keyword arguments used to construct `EVEWatcher` against its Python constructor signature.
|
||||
- Made the local cleartext-FTP SYN policy rule explicitly `flow:to_server,stateless` and bumped SID `1000113` to revision 2, eliminating the Suricata 8 direction warning seen during validation/startup.
|
||||
|
||||
## What changed in 0.9.0
|
||||
|
||||
- Added conservative **MITRE ATT&CK** annotations to NDR evidence and correlated incidents. Mapping is limited to techniques supported by network-visible evidence and is backfilled for existing NDR event history during the schema migration.
|
||||
- Expanded autonomous behavioral analytics with established-asset domain/fingerprint baselines, NXDOMAIN bursts, repeated high-entropy DNS tunnel candidates and outbound transfer-volume detection.
|
||||
- Added **Adaptive Rule Intelligence** based on observed alert volume, duplication/concentration and incident correlation. It never disables signatures automatically; operators can explicitly apply a standard global Suricata `limit` threshold to high-noise candidates.
|
||||
- Added persistent **ruleset snapshots and rollback**. Local rules, threshold configuration, merged vendor rules and enabled `suricata-update` source state are captured before rule changes/vendor updates, with bounded retention.
|
||||
- Added persistent **IDS-state backups** under `/data/backups` using SQLite online backup. Backups include the database, custom Suricata state, enabled source definitions and merged rules while excluding Redis runtime data, EVE logs and the forensic PCAP ring.
|
||||
- Added an SQLite **administrative audit trail** for login attempts and IDS-management operations including IOC changes, incident state changes, threshold/rule operations, backups and maintenance actions.
|
||||
- Extended the Intelligence, Rules and System views with ATT&CK evidence, noisy-rule analysis, one-click threshold proposals, ruleset rollback, backup download/delete and audit history.
|
||||
- The new 0.9.0 functionality is self-contained in the IDS container and persistent `/data`; it does **not add any new RouterOS configuration/firewall modification path**. Existing optional RouterOS integrations are unchanged.
|
||||
|
||||
## What changed in 0.8.1
|
||||
|
||||
- Fixed RouterOS/container rebuild persistence for signature-feed selections. All `suricata-update` source-management commands (`list-sources`, `update-sources`, `enable-source`, `disable-source`) now use the same persistent `-D /data/lib/suricata` directory as rule downloads, so enabled source definitions survive image replacement together with the single `/data` mount.
|
||||
- Added bulk signature-source selection and a sequential download queue in **Signature Feeds**. Operators can select visible sources or all parameter-free free sources, queue them together, and the IDS enables each source before performing one merged download, `suricata -T` validation and live reload.
|
||||
- Queue status and per-source progress/errors are visible in the feed table. A failed source does not prevent the remaining selected sources from being processed; the merged ruleset still keeps the previous known-good file if final download or validation fails.
|
||||
|
||||
A lightweight IDS stack designed to run as a **single container on MikroTik RouterOS**.
|
||||
|
||||
## What changed in 0.8.0
|
||||
|
||||
- Added a persistent **MikroSuricata NDR correlation engine** on top of Suricata: multi-stage incidents, risk score 0-100, bounded evidence timelines and incident triage.
|
||||
- Added **asset intelligence** persisted in SQLite and enriched from RouterOS ARP/DHCP through the RouterOS v7 REST API. Baselines track applications, outbound ports, identities, domains and fingerprints.
|
||||
- Added behavioral detections for new services on established assets, periodic beaconing, DGA/high-entropy DNS bursts, internal lateral fan-out, outbound scans of sensitive services, unusually large outbound transfers and repeated IP/MAC identity changes consistent with ARP spoofing or address conflicts.
|
||||
- Added persistent local **threat intelligence** for IP, domain, SHA-256, JA3, JA4 and HASSH. IP/domain/JA3/JA4/HASSH are materialized as Suricata datasets; malicious SHA-256 lists are matched natively on supported file protocols.
|
||||
- Added Suricata 8 `xbits` correlation for scan -> administrative access and internal probe -> SMB/RDP/SSH/WinRM/WinBox sequences.
|
||||
- Added bounded forensic PCAP capture for alert-related flows (8 x 64 MiB) with authenticated listing/download in the Intelligence view.
|
||||
- Added MikroTik-specific detection for repeated RouterOS API/API-SSL access on TCP 8728/8729, in addition to WinBox/SSH/RDP and existing edge rules.
|
||||
- Added sensor-quality health monitoring for capture drops, Suricata alert-queue overflow and TZSP/TAP injection errors.
|
||||
- Managed Redis now uses **AOF everysec + RDB** persistence under the same `/data` volume.
|
||||
- Added optional asynchronous high-risk incident webhooks (`NOTIFY_WEBHOOK_URL`, default disabled) with risk threshold and anti-spam escalation logic.
|
||||
- Expanded the Intelligence UI with incident status actions, evidence, asset inventory, IOC management and the forensic PCAP ring.
|
||||
- RouterOS deployment now forwards the NDR, Redis AOF, inventory and optional notification settings from `deploy-routeros.env` into `IDS_ENV`, so the same controls work without manual container edits.
|
||||
|
||||
## What changed in 0.7.2
|
||||
|
||||
- Fixed the Suricata 8.0.6 EVE profile: removed unsupported `llmnr` and `ftp-data` logger entries. `FTP_DATA` events remain supported through the `ftp` EVE logger.
|
||||
- Normalized the IKE EVE entry to the Suricata 8.0.6 logger syntax.
|
||||
- Removed the visual `M` logo mark from both the sidebar header and authentication modal.
|
||||
- Removed the non-existent LLMNR EVE event filter from Live Sessions.
|
||||
|
||||
## What changed in 0.7.1
|
||||
|
||||
- All mutable state now uses a **single persistent `/data` mount**. SQLite, Redis, EVE logs, Suricata-update state/vendor rules, custom rules, sessions and analytics snapshots are kept below `/data`.
|
||||
- Docker Compose now creates only `routeros-suricata-data`; the migration helper can merge the old 0.7.0 log/rule volumes into `/data/logs/suricata` and `/data/lib/suricata` without overwriting initialized targets.
|
||||
- RouterOS deploy and upgrade helpers normalize `IDS_MOUNTS` to one mapping: `<disk>/containers/suricata-data -> /data`.
|
||||
- `suricata-update` uses its `-D /data/lib/suricata` data directory and runtime Suricata uses that persistent rule path directly.
|
||||
|
||||
## What changed in 0.7.0
|
||||
|
||||
- Dashboard authentication now uses a normal username/password modal. The browser keeps only an `HttpOnly` session cookie; sessions are stored in SQLite and survive container restarts.
|
||||
- Traffic analytics are materialized into SQLite for **15m / 1h / 6h / 24h** and refreshed periodically, so charts can render immediately while Redis is starting or reconnecting.
|
||||
- Redis, SQLite, sessions and chart snapshots live under persistent `/data`; Docker Compose uses named volumes and RouterOS keeps the existing `IDS_MOUNTS` across image-only upgrades.
|
||||
- Chart rendering is visibility-aware and re-runs on tab/view changes, resize, visibility changes and layout observation.
|
||||
- Mobile mode now uses an off-canvas navigation drawer, responsive metrics/panels/forms and horizontally scrollable investigation tables.
|
||||
- The System icon no longer depends on a font glyph; it is an inline SVG.
|
||||
- The image now installs Suricata 8 from Debian trixie-backports and loads a project-owned EVE profile so package updates do not silently remove required telemetry.
|
||||
- EVE telemetry explicitly enables DNS v3, `community_id`, Ethernet metadata, SHA-256 file hashes, JA3/JA4, SSH HASSH, ARP/DHCP, plus Windows/AD and application protocols such as SMB, RDP, Kerberos, DCERPC, LDAP, QUIC and HTTP/2/DoH2.
|
||||
- Security analytics add anomalies, NXDOMAIN counts, encrypted/cleartext session counts, local/remote endpoint inventory, passive IP/MAC asset observations, top signatures/severities and JA4/JA3/HASSH fingerprint inventory.
|
||||
- The built-in local ruleset adds NXDOMAIN/DNS-rate signals, outbound SMB/SMTP/FTP policy detections, exposed database-service probes and an internal administrative/lateral-movement burst detector.
|
||||
|
||||
RouterOS mirrors selected traffic with TZSP, the container decodes the frames into a TAP interface, Suricata analyzes them, and the Python service stores EVE alerts in SQLite and exposes a small web dashboard.
|
||||
|
||||
## NDR / MikroTik-specific options
|
||||
|
||||
The defaults are conservative: NDR analysis is enabled, but automatic NDR blocking and outbound webhooks are disabled until explicitly configured. Useful environment variables:
|
||||
|
||||
```text
|
||||
NDR_ENABLED=true
|
||||
NDR_CORRELATION_WINDOW_SECONDS=1800
|
||||
BEHAVIOR_MIN_OBSERVATIONS=50
|
||||
NDR_AUTO_BLOCK=false
|
||||
NDR_AUTO_BLOCK_RISK=92
|
||||
ROUTEROS_INVENTORY_INTERVAL_SECONDS=300
|
||||
NOTIFY_WEBHOOK_URL=
|
||||
NOTIFY_MIN_RISK=80
|
||||
REDIS_AOF=true
|
||||
```
|
||||
|
||||
All NDR state, IOC data, Redis persistence, Suricata logs/rules and forensic PCAP rotation remain below the single persistent `/data` mount.
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
@@ -21,7 +106,11 @@ single RouterOS container
|
||||
+ TAP suritap0
|
||||
+ Suricata IDS
|
||||
+ EVE JSON watcher
|
||||
+ SQLite
|
||||
+ SQLite alerts / assets / NDR incidents / sessions
|
||||
+ MikroSuricata behavior + correlation engine
|
||||
+ local IOC datasets (IP/domain/SHA256/JA3/JA4/HASSH)
|
||||
+ Redis/RAM bounded traffic history
|
||||
+ WebSocket live stream
|
||||
+ Web UI :8080
|
||||
+ optional RouterOS REST blocking
|
||||
```
|
||||
@@ -93,10 +182,13 @@ For a first test the defaults can be used. Before monitoring a real network, rev
|
||||
|
||||
```dotenv
|
||||
SURICATA_HOME_NET=[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]
|
||||
MONITORED_NETWORKS=192.168.100.0/24
|
||||
MONITORED_NETWORKS=192.168.0.0/16,10.0.0.0/8,172.16.0.0/12
|
||||
ALERT_MAX_SEVERITY=2
|
||||
ALERT_DEDUP_WINDOW_SECONDS=300
|
||||
ADMIN_TOKEN=<long-random-token>
|
||||
ADMIN_USERNAME=admin
|
||||
ADMIN_PASSWORD=<long-unique-password>
|
||||
SESSION_HOURS=168
|
||||
ANALYTICS_SNAPSHOT_INTERVAL_SECONDS=60
|
||||
AUTO_BLOCK=false
|
||||
```
|
||||
|
||||
@@ -108,6 +200,18 @@ Keep `AUTO_BLOCK=false` until alerts are verified.
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Compose uses one named volume, `routeros-suricata-data`, mounted at `/data`. SQLite, Redis, chart snapshots, sessions, Suricata logs, local configuration and downloaded vendor rules all live below that mount. RouterOS uses the same single-mount layout: `disk1/containers/suricata-data -> /data` through `IDS_MOUNTS`. Rebuilding or replacing the container therefore leaves all mutable IDS data outside the image root.
|
||||
|
||||
When upgrading a Docker installation from **0.6.1 or older**, migrate the old bind-mounted `./data`, `./logs` and `./data/vendor-rules` before the first 0.8.1 start:
|
||||
|
||||
```bash
|
||||
docker compose build
|
||||
./scripts/migrate-docker-volumes.sh
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
The migration keeps the existing `routeros-suricata-data` volume and folds legacy log/rule volumes into `/data/logs/suricata` and `/data/lib/suricata` only when those target directories are empty. `./scripts/first-run.sh` performs this step automatically. If you start Compose manually, set a strong `ADMIN_PASSWORD` in `.env`; `first-run.sh` generates one automatically for a new `.env`.
|
||||
|
||||
Check container status:
|
||||
|
||||
```bash
|
||||
@@ -227,11 +331,11 @@ Example local rule:
|
||||
alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"LOCAL suspicious URI marker"; flow:established,to_server; http.uri; content:"/admin/export"; nocase; classtype:web-application-activity; priority:2; sid:1000100; rev:1;)
|
||||
```
|
||||
|
||||
Use unique local SIDs. SID `1000001` is reserved for the marked pipeline self-test, built-in production detections use `1000101-1000108`, and site-specific rules should use `1001000+`.
|
||||
Use unique local SIDs. SID `1000001` is reserved for the marked pipeline self-test, built-in production detections use `1000101-1000115`, and site-specific rules should use `1001000+`.
|
||||
|
||||
Vendor rules are managed with `suricata-update`. A baseline ET/Open ruleset and a current OISF source index are baked into the image. Docker Compose and RouterOS deployments persist `/var/lib/suricata`, including enabled source definitions, downloaded feeds and the source index. An empty first-run mount is seeded from the image baseline. `scripts/update-rules.sh` applies persisted `/data/suricata/disable.conf`, `enable.conf`, and `modify.conf`.
|
||||
Vendor rules are managed with `suricata-update`. A baseline ET/Open ruleset and a current OISF source index are baked into the image. Runtime rule state is written with `suricata-update -D /data/lib/suricata`, so downloaded feeds, source definitions and caches are inside the single persistent `/data` mount. An empty first-run data directory is seeded from the image baseline. `scripts/update-rules.sh` applies persisted `/data/suricata/disable.conf`, `enable.conf`, and `modify.conf`.
|
||||
|
||||
The **Rules** page now has a **Signature sources** table backed by the official OISF `suricata-update` catalog. The UI lists free sources, shows vendor/license/tags/status, refreshes the OISF index, enables or disables parameter-free feeds, and downloads all active feeds on demand. ET/Open remains the default source and cannot be accidentally disabled from the panel. Feeds that require credentials or parameters are displayed but must be configured manually instead of prompting through the web UI.
|
||||
The dedicated **Signature Feeds** page has a provider table backed by the official OISF `suricata-update` catalog. The UI lists free sources, shows vendor/license/tags/status, refreshes the OISF index, enables or disables parameter-free feeds, and downloads all active feeds on demand. Multiple parameter-free sources can be selected and queued together; they are enabled sequentially and then rebuilt/validated once. All source-management commands and rule downloads use `-D /data/lib/suricata`, so the enabled-source definitions survive RouterOS container rebuilds with the same `/data` mount. ET/Open remains the default source and cannot be accidentally disabled from the panel. Feeds that require credentials or parameters are displayed but must be configured manually instead of prompting through the web UI.
|
||||
|
||||
Every feed update is transactional at the merged-rules level: the existing `suricata.rules` is backed up, new signatures are downloaded, the complete Suricata configuration is tested with `suricata -T`, and only a validated ruleset is kept. If download or validation fails, the previous known-good rules are restored. The periodic updater uses the same active-source set and runs every `RULE_UPDATE_INTERVAL_HOURS` when the interval is greater than zero.
|
||||
|
||||
@@ -244,11 +348,11 @@ RULE_UPDATE_INTERVAL_HOURS=24
|
||||
|
||||
## Built-in production detections
|
||||
|
||||
The image now ships with a conservative local baseline in addition to the ET/Open snapshot baked by `suricata-update`. The local baseline is intentionally rate-limited so one packet does not create an incident. It covers repeated SSH, RDP and WinBox connection attempts, high-rate SYN scanning, ICMP sweeps, external SMB access, unusually long DNS labels and outbound Telnet.
|
||||
The image ships with a conservative local baseline in addition to the ET/Open snapshot baked by `suricata-update`. Local rules are rate-limited so ordinary single packets do not become incidents. The baseline covers repeated SSH/RDP/WinBox attempts, SYN scans, ICMP sweeps, inbound and outbound SMB policy violations, unusually long or high-rate DNS activity, NXDOMAIN bursts, outbound Telnet/FTP/direct SMTP, external database-service probes and an internal RDP/SMB lateral-movement burst signal.
|
||||
|
||||
The baseline uses SIDs `1000101-1000108`. The deterministic pipeline self-test remains SID `1000001`, but it only matches the exact payload marker generated by `scripts/send_test_tzsp.py` and is ignored by the incident database. This keeps the end-to-end test available without turning ordinary ping traffic into alerts.
|
||||
The production baseline uses SIDs `1000101-1000116`, with NDR state rules `1000120-1000123` and managed threat-intelligence rules starting at `1000201`. The deterministic pipeline self-test remains SID `1000001`, but it only matches the exact payload marker generated by `scripts/send_test_tzsp.py` and is ignored by the incident database. Environment-specific exceptions should be handled with `threshold.config` or suppression rather than weakening the entire sensor.
|
||||
|
||||
ET/Open is still the main vendor signature source. `suricata-update` is the supported manager for refreshing it; the image seeds the persistent `/var/lib/suricata` volume on first start.
|
||||
ET/Open is still the main vendor signature source. `suricata-update` is the supported manager for refreshing it; the image seeds `/data/lib/suricata` inside the persistent data volume on first start.
|
||||
|
||||
---
|
||||
|
||||
@@ -269,29 +373,24 @@ It performs SCP upload plus a read-only file-list verification. It does **not**
|
||||
|
||||
The dashboard detects SQLite and persistent storage separately. It shows DB path, schema version, row count, DB/WAL size, filesystem usage and Suricata log size. SQLite uses WAL mode and performs a small schema migration automatically when upgrading from older project versions.
|
||||
|
||||
Administrative actions are disabled until `ADMIN_TOKEN` is set. The dashboard then provides:
|
||||
Administrative actions require a dashboard session. Configure `ADMIN_USERNAME` and a long unique `ADMIN_PASSWORD`; the server issues an `HttpOnly`, `SameSite=Strict` cookie and stores only a hash of the random session token in SQLite. Set `SESSION_COOKIE_SECURE=true` when the dashboard itself is served over HTTPS. `ADMIN_TOKEN` is accepted only as a migration fallback and is no longer stored by the browser.
|
||||
|
||||
- clear stored alert incidents,
|
||||
- truncate active Suricata `eve.json`, `fast.log`, `stats.log`, and `suricata.log`,
|
||||
- compact SQLite with `VACUUM`,
|
||||
- reset runtime counters,
|
||||
- validate/save/reload custom rules and `threshold.config`,
|
||||
- refresh the official OISF source catalog, enable/disable supported free feeds, and download/update the active vendor rulesets.
|
||||
Authenticated maintenance includes clearing incident/history data, SQLite `VACUUM`, runtime counter reset, RouterOS block-list actions, validated custom-rule/threshold edits, live Suricata rule reloads and managed signature-feed updates. Keep port `8080` on a trusted management network or place the dashboard behind HTTPS.
|
||||
|
||||
Set a long random admin token and keep port `8080` on a management-only network. The UI does not provide TLS termination.
|
||||
SQLite also stores the four rolling chart summaries (`900`, `3600`, `21600`, `86400` seconds). They refresh every `ANALYTICS_SNAPSHOT_INTERVAL_SECONDS` and are served immediately after UI entry/restart when newer live history is temporarily unavailable.
|
||||
|
||||
---
|
||||
|
||||
## Extended statistics
|
||||
|
||||
The dashboard reports alert hits vs deduplicated incidents, 1h/24h activity, top signatures, top sources, severity distribution data, filter/dedup/error counters, block attempts/results and the latest Suricata EVE `stats` counters such as decoder/capture/drop values when emitted by the installed Suricata configuration.
|
||||
The dashboard reports alert hits vs deduplicated incidents, selected-window activity, top signatures, severity distribution, protocol anomalies, NXDOMAINs, encrypted vs cleartext sessions, local clients vs remote peers, JA4/JA3/HASSH fingerprints, passive IP/MAC assets, filter/dedup/error counters, block attempts/results and Suricata EVE `stats` counters such as decoder/capture/drop values when emitted by the installed configuration. Live-history records remain searchable by IP/port, signature, `flow_id`, `community_id` and transaction ID.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Dashboard sections
|
||||
|
||||
The web UI is split into top-menu sections: **Overview**, **Incidents**, **Statistics**, **System**, **Rules**, and **Maintenance**. Incident timestamps are stored in UTC and rendered in the browser's local timezone. Repeated events are aggregated by SID, source, destination, protocol and destination port within the configured deduplication window.
|
||||
The web UI sections are **Overview**, **Live Sessions**, **Security**, **Blocks**, **Reports**, **Signature Feeds**, **Rules** and **System**. Incident timestamps are stored in UTC and rendered in the browser's local timezone. Repeated alerts are aggregated by SID, source, destination, protocol and destination port within the configured deduplication window.
|
||||
|
||||
---
|
||||
|
||||
@@ -480,15 +579,15 @@ Then deploy by giving the **RouterOS-side TAR path** directly:
|
||||
./scripts/deploy-routeros.sh routeros-suricata-tzsp-arm64.tar
|
||||
```
|
||||
|
||||
The deployer no longer builds, detects image architecture, renames, or re-uploads the image. For project version `0.5.3` it creates:
|
||||
The deployer no longer builds, detects image architecture, renames, or re-uploads the image. For project version `0.9.0` it creates:
|
||||
|
||||
```text
|
||||
name=suricata_0.5.3
|
||||
name=suricata_0.9.0
|
||||
file=routeros-suricata-tzsp-arm64.tar
|
||||
root-dir=/containers/suricata_0.5.3/root
|
||||
root-dir=/containers/suricata_0.9.0/root
|
||||
```
|
||||
|
||||
The remaining deployment work is unchanged: bridge/VETH/NAT, environment, persistent mounts, optional RouterOS REST user/firewall integration, TZSP sniffer configuration, image extraction wait, container start, and final status. Existing containers are not removed. Re-running deployment for the same version stops with `Container suricata_0.5.3 already exists`.
|
||||
The remaining deployment work is unchanged: bridge/VETH/NAT, environment, persistent mounts, optional RouterOS REST user/firewall integration, TZSP sniffer configuration, image extraction wait, container start, and final status. Existing containers are not removed. Re-running deployment for the same version stops with `Container suricata_0.9.0 already exists`.
|
||||
|
||||
For SSH key authentication set:
|
||||
|
||||
@@ -560,15 +659,23 @@ After extraction, inspect and start the container:
|
||||
|
||||
## Persistent data on RouterOS
|
||||
|
||||
The deployment keeps application state outside the image root directory:
|
||||
The deployment uses one persistent directory outside the image root:
|
||||
|
||||
```text
|
||||
<disk>/containers/suricata-ids-data -> /data
|
||||
<disk>/containers/suricata-ids-logs -> /var/log/suricata
|
||||
<disk>/containers/suricata-ids-rules -> /var/lib/suricata
|
||||
<disk>/containers/suricata-data -> /data
|
||||
```
|
||||
|
||||
This preserves SQLite data, custom signatures, threshold/suppression configuration, Suricata-update filters, raw logs, and downloaded vendor rules when the application image is replaced.
|
||||
Inside it the application keeps:
|
||||
|
||||
```text
|
||||
/data/ids.db SQLite, sessions and analytics snapshots
|
||||
/data/redis/ Redis persistence
|
||||
/data/logs/suricata/ EVE/raw Suricata logs
|
||||
/data/lib/suricata/ suricata-update feeds, cache and vendor rules
|
||||
/data/suricata/ custom rules, thresholds and update filters
|
||||
```
|
||||
|
||||
Replacing or restarting the application container does not remove any of these files.
|
||||
|
||||
---
|
||||
|
||||
@@ -675,6 +782,9 @@ TAP interface handling
|
||||
app/eve.py
|
||||
Suricata EVE JSON watcher
|
||||
|
||||
suricata/ids-output.yaml
|
||||
Project-owned Suricata 8 EVE telemetry profile
|
||||
|
||||
app/policy.py
|
||||
Alert/blocking policy
|
||||
|
||||
@@ -717,10 +827,12 @@ AUTO_BLOCK=false
|
||||
ALERT_MAX_SEVERITY=2
|
||||
UPDATE_RULES_ON_START=false
|
||||
ROUTEROS_PASSWORD=CHANGE_ME
|
||||
ADMIN_TOKEN=
|
||||
ADMIN_USERNAME=admin
|
||||
ADMIN_PASSWORD=
|
||||
SESSION_COOKIE_SECURE=false
|
||||
```
|
||||
|
||||
Keep automatic firewall actions disabled until the capture path and alert quality are validated on the real network. Set `ADMIN_TOKEN` before enabling dashboard maintenance/rule-management actions, and restrict the Web UI to a trusted management network.
|
||||
Keep automatic firewall actions disabled until the capture path and alert quality are validated on the real network. Set a strong `ADMIN_PASSWORD` before enabling dashboard maintenance/rule-management actions, and restrict the Web UI to a trusted management network.
|
||||
|
||||
---
|
||||
|
||||
@@ -740,17 +852,44 @@ After the first deployment, when `veth-ids`, bridge/NAT, TZSP sniffer, `IDS_ENV`
|
||||
./scripts/upgrade-routeros-container.sh routeros-suricata-tzsp-arm64.tar
|
||||
```
|
||||
|
||||
For version `0.5.3` the second command creates:
|
||||
For version `0.9.0` the second command creates:
|
||||
|
||||
```text
|
||||
name=suricata_0.5.3
|
||||
name=suricata_0.9.0
|
||||
file=routeros-suricata-tzsp-arm64.tar
|
||||
root-dir=/containers/suricata_0.5.3/root
|
||||
root-dir=/containers/suricata_0.9.0/root
|
||||
interface=veth-ids
|
||||
envlist=IDS_ENV
|
||||
mountlists=IDS_MOUNTS
|
||||
```
|
||||
|
||||
The upgrade helper does not create or modify the bridge, IP addresses, NAT, veth, TZSP/sniffer, firewall, REST user, envlist definitions, or mount definitions. It disables `start-on-boot` on older `suricata_*` containers, stops the running old Suricata container, creates the new versioned container, waits for image extraction, and starts it. Older containers are kept stopped for rollback.
|
||||
The upgrade helper does not modify the bridge, IP addresses, NAT, veth, TZSP/sniffer, firewall, REST user or envlist definitions. It stops older `suricata_*` containers, normalizes `IDS_MOUNTS` to the single `<disk>/containers/suricata-data -> /data` mapping, creates the new versioned container, waits for image extraction and starts it. Older containers are kept stopped for rollback.
|
||||
|
||||
Persistent `/data`, Suricata logs and vendor rules continue to use the existing mount list, so they survive the version change.
|
||||
All new mutable state is written below `/data`, so subsequent image upgrades need only that one persistent mount.
|
||||
|
||||
### 0.9.3 HTTP disconnect hardening
|
||||
|
||||
- Treat browser/client disconnects during JSON responses and file downloads as normal connection termination.
|
||||
- Suppress `BrokenPipeError`, `ConnectionResetError`, and equivalent socket disconnect tracebacks from the threaded HTTP server.
|
||||
- No change to Redis-only traffic history semantics from 0.9.2.
|
||||
|
||||
|
||||
### 0.9.4 Throughput visibility
|
||||
|
||||
- Traffic throughput now always renders the raw TZSP **Total** series, even when inbound/outbound classification is unavailable.
|
||||
- The dashboard exposes unclassified throughput as `OTHER` instead of silently drawing an empty IN/OUT chart.
|
||||
- Default `MONITORED_NETWORKS` now covers RFC1918 private LAN ranges (`192.168/16`, `10/8`, `172.16/12`) so common RouterOS LANs classify correctly without editing the image.
|
||||
- Throughput remains persisted in Redis; no RAM history fallback is reintroduced.
|
||||
|
||||
### 0.9.5 Traffic accuracy and CPU reduction
|
||||
|
||||
- **Observed traffic** is calculated from raw TZSP packet-byte samples for the selected time range. EVE transaction records are no longer summed as traffic volume, avoiding repeated cumulative flow counters (for example impossible hundreds of GB in a 15-minute view).
|
||||
- Traffic throughput has a lightweight `/api/traffic/throughput` path, so the speed graph can render without waiting for full EVE analytics.
|
||||
- `failed`, `unknown`, `none` and similar Suricata application classifications are excluded from Top applications. Application counts are deduplicated by flow.
|
||||
- `SURICATA IPv4 truncated packet` / IPv6 equivalents are hidden from dashboard history and analytics. Alerts rejected by the configured alert tuner are no longer persisted into dashboard Redis history; raw EVE remains on disk.
|
||||
- Analytics snapshots are **demand-driven**: only time windows used by a browser/API are refreshed. The old unconditional full 24h Redis scan every minute was removed. Default refresh cadence is 60s (15m), 120s (1h), 300s (6h), 900s (24h), with current throughput read separately in constant time.
|
||||
- The TZSP session tracker keeps packet throughput active but skips per-flow OrderedDict/hash work while Live Sessions streaming is off.
|
||||
- LAN membership lookups are cached, reducing repeated `ipaddress` parsing for packet direction classification.
|
||||
- Redis event persistence batches up to 128 EVE records into one multi-member `ZADD`, reducing Python socket and Redis command overhead while retaining every accepted dashboard event.
|
||||
- Duplicate `fast.log` and standalone `stats.log` outputs are disabled because the application consumes EVE alerts/stats already.
|
||||
- Analytics cache namespace is bumped to v3 so incorrect pre-0.9.5 traffic-volume snapshots are not reused after upgrade.
|
||||
|
||||
Reference in New Issue
Block a user