RouterOS TZSP + Suricata IDS

Project version: 0.5.3

A lightweight IDS stack designed to run as a single container on MikroTik RouterOS. RouterOS mirrors selected traffic with TZSP, the container decodes the frames into a TAP interface, Suricata analyzes them, and the Python service stores EVE alerts in SQLite and exposes a small web dashboard.

Architecture

VLAN / RouterOS traffic
        |
        v
RouterOS Packet Sniffer
        |
        | TZSP UDP/37008
        v
single RouterOS container
  Debian slim
  + Python TZSP receiver
  + TAP suritap0
  + Suricata IDS
  + EVE JSON watcher
  + SQLite
  + Web UI :8080
  + optional RouterOS REST blocking

The RouterOS deployment workflow is:

Docker/Podman build host
        |
        | build for RouterOS CPU architecture
        v
container image
        |
        | docker save / podman save
        v
*.tar
        |
        | SCP
        v
RouterOS external disk
        |
        | /container/add file=...
        v
running IDS container

Docker Compose is only provided for Linux integration testing. RouterOS receives one saved container image as a TAR archive.

---

Quick start - Docker Compose on Linux

Use these steps for the local full-stack test with TZSP, TAP and Suricata.

1. Check prerequisites

You need:

• Linux,
• Docker Engine,
• Docker Compose v2 (docker compose),
• /dev/net/tun available on the host.

Check them:

docker --version
docker compose version
test -c /dev/net/tun && echo "TUN/TAP: OK" || echo "TUN/TAP: MISSING"

If /dev/net/tun is missing on Linux, try:

sudo modprobe tun

Then check /dev/net/tun again.

2. Create the runtime configuration

From the project directory:

cp .env.example .env

For a first test the defaults can be used. Before monitoring a real network, review at least:

SURICATA_HOME_NET=[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]
MONITORED_NETWORKS=192.168.100.0/24
ALERT_MAX_SEVERITY=2
ALERT_DEDUP_WINDOW_SECONDS=300
ADMIN_TOKEN=<long-random-token>
AUTO_BLOCK=false

Keep AUTO_BLOCK=false until alerts are verified.

3. Build and start

docker compose up -d --build

Check container status:

docker compose ps

Follow logs:

docker compose logs -f ids

You can also use the container name directly:

docker logs -f routeros-suricata-tzsp

Do not run plain docker logs -f - Docker requires a container name.

4. Run the end-to-end self-test

./scripts/selftest.sh

Expected result:

SELFTEST OK: Suricata emitted marked TZSP pipeline test alert(s); UI filtering remains enabled

The self-test uses reserved SID 1000001 and a unique payload marker. Normal ICMP/ping traffic cannot match it, and SID 1000001 is filtered from SQLite/UI by default.

5. Open the dashboard

http://127.0.0.1:8080

Status API:

curl http://127.0.0.1:8080/api/status

6. Stop the stack

docker compose down

One-command first start

The helper script checks Docker, Compose and /dev/net/tun, creates .env if needed, starts the stack and runs the self-test:

./scripts/first-run.sh

If scripts are not executable after unpacking an archive:

chmod +x dev.sh scripts/*.sh
./scripts/first-run.sh

Rebuild after updating older images

Version 0.3.2 could restart continuously with:

chown: invalid user: 'suricata:suricata'

Version 0.3.3 added the suricata system account. Version 0.3.4 also fixes a second startup issue where the suricata -T configuration check could create root-owned eve.json, fast.log, and stats.log, causing the real Suricata process to fail with Permission denied. It also relocates the Unix command socket into /run/suricata/. Rebuild the image completely:

docker compose down
docker compose build --no-cache
docker compose up -d
docker compose ps
docker compose logs -f ids

---

Alert tuning and false-positive control

Version 0.5.2 uses two tuning layers. Raw Suricata EVE stays on disk, while the incident database/UI defaults to severity 1-2 and collapses repeated identical SID/source/destination tuples for five minutes. This prevents informational events from flooding the dashboard without changing the raw sensor log.

ALERT_MAX_SEVERITY=2
ALERT_DEDUP_WINDOW_SECONDS=300
ALERT_IGNORE_SIDS=
ALERT_IGNORE_CATEGORIES=

For a known false positive, prefer sensor-level tuning in /data/suricata/threshold.config. The dashboard can add a full SID suppression with Suppress SID, or the file can be edited directly from the rule-management panel. Examples:

suppress gen_id 1, sig_id 1234567
threshold gen_id 1, sig_id 1234567, type limit, track by_src, count 1, seconds 300

Do not globally suppress a rule just because it fired once. First verify the SID, endpoint, direction and expected application behavior.

---

Adding new Suricata detections

Suricata is signature/rule driven; it does not need model training to learn a new network detection. Add environment-specific signatures to /data/suricata/custom.rules, or use Custom Suricata signatures in the dashboard. Save performs a suricata -T validation first and only then writes the file and requests a live rule reload. Runtime loads persisted /data/suricata/*.rules, so additional rule files can be placed beside custom.rules without rebuilding the image.

Example local rule:

alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"LOCAL suspicious URI marker"; flow:established,to_server; http.uri; content:"/admin/export"; nocase; classtype:web-application-activity; priority:2; sid:1000100; rev:1;)

Use unique local SIDs. SID 1000001 is reserved for the marked pipeline self-test, built-in production detections use 1000101-1000108, and site-specific rules should use 1001000+.

Vendor rules are managed with suricata-update. A baseline ET/Open ruleset and a current OISF source index are baked into the image. Docker Compose and RouterOS deployments persist /var/lib/suricata, including enabled source definitions, downloaded feeds and the source index. An empty first-run mount is seeded from the image baseline. scripts/update-rules.sh applies persisted /data/suricata/disable.conf, enable.conf, and modify.conf.

The Rules page now has a Signature sources table backed by the official OISF suricata-update catalog. The UI lists free sources, shows vendor/license/tags/status, refreshes the OISF index, enables or disables parameter-free feeds, and downloads all active feeds on demand. ET/Open remains the default source and cannot be accidentally disabled from the panel. Feeds that require credentials or parameters are displayed but must be configured manually instead of prompting through the web UI.

Every feed update is transactional at the merged-rules level: the existing suricata.rules is backed up, new signatures are downloaded, the complete Suricata configuration is tested with suricata -T, and only a validated ruleset is kept. If download or validation fails, the previous known-good rules are restored. The periodic updater uses the same active-source set and runs every RULE_UPDATE_INTERVAL_HOURS when the interval is greater than zero.

UPDATE_RULES_ON_START=false
RULE_UPDATE_INTERVAL_HOURS=24

---

Built-in production detections

The image now ships with a conservative local baseline in addition to the ET/Open snapshot baked by suricata-update. The local baseline is intentionally rate-limited so one packet does not create an incident. It covers repeated SSH, RDP and WinBox connection attempts, high-rate SYN scanning, ICMP sweeps, external SMB access, unusually long DNS labels and outbound Telnet.

The baseline uses SIDs 1000101-1000108. The deterministic pipeline self-test remains SID 1000001, but it only matches the exact payload marker generated by scripts/send_test_tzsp.py and is ignored by the incident database. This keeps the end-to-end test available without turning ordinary ping traffic into alerts.

ET/Open is still the main vendor signature source. suricata-update is the supported manager for refreshing it; the image seeds the persistent /var/lib/suricata volume on first start.

---

Upload a ready RouterOS image without deploying it

The upload helper requires an already-built TAR and does exactly one job:

cp deploy-routeros.env.example deploy-routeros.env
./scripts/upload-routeros-image.sh build/routeros-suricata-tzsp-arm64.tar

It performs SCP upload plus a read-only file-list verification. It does not detect architecture, build an image, run /container/add, import an .rsc, change RouterOS configuration, or start a container.

---

Database, storage and maintenance

The dashboard detects SQLite and persistent storage separately. It shows DB path, schema version, row count, DB/WAL size, filesystem usage and Suricata log size. SQLite uses WAL mode and performs a small schema migration automatically when upgrading from older project versions.

Administrative actions are disabled until ADMIN_TOKEN is set. The dashboard then provides:

• clear stored alert incidents,
• truncate active Suricata eve.json, fast.log, stats.log, and suricata.log,
• compact SQLite with VACUUM,
• reset runtime counters,
• validate/save/reload custom rules and threshold.config,
• refresh the official OISF source catalog, enable/disable supported free feeds, and download/update the active vendor rulesets.

Set a long random admin token and keep port 8080 on a management-only network. The UI does not provide TLS termination.

---

Extended statistics

The dashboard reports alert hits vs deduplicated incidents, 1h/24h activity, top signatures, top sources, severity distribution data, filter/dedup/error counters, block attempts/results and the latest Suricata EVE stats counters such as decoder/capture/drop values when emitted by the installed Suricata configuration.

---

Dashboard sections

The web UI is split into top-menu sections: Overview, Incidents, Statistics, System, Rules, and Maintenance. Incident timestamps are stored in UTC and rendered in the browser's local timezone. Repeated events are aggregated by SID, source, destination, protocol and destination port within the configured deduplication window.

---

Local Web UI development without Docker

The web dashboard can be started locally without Docker, Suricata, TAP, /dev/net/tun, or root privileges.

Requirements:

• Linux or macOS
• Python 3
• Python venv support

Start the development dashboard:

./dev.sh

The script will:

1. create .venv/ if needed,
2. install dependencies from requirements.txt,
3. create data/dev/,
4. start the web-only application.

Open:

http://127.0.0.1:8080

This mode intentionally does not start Suricata, TZSP capture, TAP, or RouterOS integration.

Status API

The dashboard and external monitoring systems can use:

GET /api/status

Example:

curl http://127.0.0.1:8080/api/status

The response includes:

• overall application status and uptime,
• Web UI/API, TZSP, TAP, Suricata and EVE watcher state,
• SQLite existence/path/size/WAL/schema/row count,
• persistent filesystem usage and Suricata log size,
• managed custom-rule and threshold/suppression status,
• RouterOS REST integration status and ports,
• runtime packet/error/filter/dedup/block counters,
• the latest numeric Suricata EVE stats counters.

GET /api/health is kept as a compatibility alias and returns the same status payload.

Add a sample alert

To start the dashboard with one demo alert in an empty development database:

DEV_SEED_DATA=true ./dev.sh

Development database:

data/dev/ids.db

To use another port:

WEB_PORT=8090 ./dev.sh

To listen on all local interfaces:

WEB_BIND=0.0.0.0 ./dev.sh

Do not expose the development server directly to an untrusted network.

---

Local tests

Install dependencies and run tests:

./dev.sh --test

Or run tests before starting the dashboard:

RUN_TESTS=true ./dev.sh

The project currently uses only the Python standard library, including unittest for tests. requirements.txt is kept as the canonical dependency file so future packages can be installed automatically by dev.sh.

---

Docker integration test

Prepare local environment configuration:

cp .env.example .env

Build and start:

docker compose up -d --build

Dashboard:

http://127.0.0.1:8080

Send the built-in TZSP test packet:

./scripts/selftest.sh

The test rule should generate raw EVE alert SID 1000001 with signature LOCAL TEST TZSP PIPELINE MARKER. It is intentionally filtered from the incident database/UI.

Stop the stack:

docker compose down

---

RouterOS requirements

Before deployment, verify that the router has:

• a RouterOS version and package set that supports containers,
• the container package installed,
• container and sniffer functionality enabled in device mode,
• SSH access from the build machine,
• enough storage for the image, container root directory, Suricata rules, logs, and SQLite database,
• preferably an external disk instead of small internal flash storage.

Useful RouterOS checks:

/system/resource/print
/system/package/print
/system/device-mode/print
/disk/print
/container/print

---

Automated RouterOS deployment

Create the deployment configuration:

cp deploy-routeros.env.example deploy-routeros.env

Upload a ready image first:

./scripts/upload-routeros-image.sh build/routeros-suricata-tzsp-arm64.tar

Then deploy by giving the RouterOS-side TAR path directly:

./scripts/deploy-routeros.sh routeros-suricata-tzsp-arm64.tar

The deployer no longer builds, detects image architecture, renames, or re-uploads the image. For project version 0.5.3 it creates:

name=suricata_0.5.3
file=routeros-suricata-tzsp-arm64.tar
root-dir=/containers/suricata_0.5.3/root

The remaining deployment work is unchanged: bridge/VETH/NAT, environment, persistent mounts, optional RouterOS REST user/firewall integration, TZSP sniffer configuration, image extraction wait, container start, and final status. Existing containers are not removed. Re-running deployment for the same version stops with Container suricata_0.5.3 already exists.

For SSH key authentication set:

ROUTER_IDENTITY_FILE=/home/user/.ssh/id_ed25519

---

Build RouterOS image without deployment

ARM64:

./scripts/build-routeros.sh arm64

AMD64/x86_64:

./scripts/build-routeros.sh amd64

ARMv7/armhf:

./scripts/build-routeros.sh arm

Generated files are written to build/, for example:

build/routeros-suricata-tzsp-arm64.tar
build/routeros-suricata-tzsp-arm64.tar.sha256

---

Manual SCP and RouterOS import

Example:

./scripts/build-routeros.sh arm64
scp build/routeros-suricata-tzsp-arm64.tar admin@192.168.88.1:/

RouterOS templates are located in routeros/:

01-container-network.rsc
02-sniffer-vlan100.rsc
03-rest-and-firewall.rsc
04-container-import-amd64.rsc
04-container-import-arm64.rsc
04-container-import-arm.rsc
rollback.rsc

After extraction, inspect and start the container:

/container/print detail
/container/start suricata-ids
/log/print where message~"suricata|TZSP|IDS"

---

Persistent data on RouterOS

The deployment keeps application state outside the image root directory:

<disk>/containers/suricata-ids-data  -> /data
<disk>/containers/suricata-ids-logs  -> /var/log/suricata
<disk>/containers/suricata-ids-rules -> /var/lib/suricata

This preserves SQLite data, custom signatures, threshold/suppression configuration, Suricata-update filters, raw logs, and downloaded vendor rules when the application image is replaced.

---

TZSP capture

The default deployment configuration can configure RouterOS Packet Sniffer to stream VLAN traffic to the container address on UDP port 37008.

Relevant settings in deploy-routeros.env:

CONFIGURE_SNIFFER=true
START_SNIFFER=true
VLAN_ID=100

If the router already uses Packet Sniffer for another purpose, disable automatic sniffer configuration:

CONFIGURE_SNIFFER=false
START_SNIFFER=false

Then configure the capture manually.

Hardware-offloaded bridge traffic may require additional verification on the specific RouterOS device because some switched traffic can bypass software capture paths.

---

TAP and Suricata

Inside the container the application creates:

suritap0

The pipeline is:

TZSP datagram
  -> Python decoder
  -> Ethernet frame
  -> TAP suritap0
  -> Suricata
  -> eve.json
  -> Python EVE watcher
  -> SQLite / Web UI

If the RouterOS container cannot create the TAP interface, the application will fail early with an error related to /dev/net/tun, TUNSETIFF, or permissions. This is the main platform-specific capability to validate on the target router.

---

RouterOS REST reaction

Automatic blocking is intentionally disabled by default:

AUTO_BLOCK=false

Observation mode should be used first. After validating alerts and false positives, the optional reaction engine can add selected addresses to a RouterOS firewall address list through REST.

Relevant settings:

AUTO_BLOCK=true
AUTO_BLOCK_MAX_SEVERITY=1
ROUTEROS_URL=https://172.31.255.1
ROUTEROS_USER=suricata-api
ROUTEROS_PASSWORD=CHANGE_ME
ROUTEROS_ADDRESS_LIST=IDS-BLOCK
BLOCK_TIMEOUT=1h

Do not enable automatic blocking until the monitored networks, exclusion list, REST credentials, firewall rule placement, and alert policy have been reviewed.

---

Important files

Dockerfile
Dockerfile / Debian slim runtime image

dev.sh
Local web-only development launcher

requirements.txt
Local Python development/test dependencies

app/main.py
Full RouterOS/container application entry point

app/dev_web.py
Local web-only entry point

app/tzsp.py
TZSP receiver and decoder

app/tap.py
TAP interface handling

app/eve.py
Suricata EVE JSON watcher

app/policy.py
Alert/blocking policy

app/tuning.py
Second-stage alert severity/category/SID filter

app/rules.py
Validated custom rules, suppressions and vendor-rule updates

app/maintenance.py
Storage detection and safe maintenance helpers

app/routeros.py
RouterOS REST client

app/store.py
SQLite alert storage

app/webui.py
Dashboard and JSON API

scripts/build-routeros.sh
Build and save image to TAR

scripts/deploy-routeros.sh
Build, SCP, import, and start on RouterOS

routeros/
RouterOS configuration templates

---

Safety defaults

The default configuration is observation-oriented:

AUTO_BLOCK=false
ALERT_MAX_SEVERITY=2
UPDATE_RULES_ON_START=false
ROUTEROS_PASSWORD=CHANGE_ME
ADMIN_TOKEN=

Keep automatic firewall actions disabled until the capture path and alert quality are validated on the real network. Set ADMIN_TOKEN before enabling dashboard maintenance/rule-management actions, and restrict the Web UI to a trusted management network.

---

Image-only upgrade on an already configured RouterOS

After the first deployment, when veth-ids, bridge/NAT, TZSP sniffer, IDS_ENV and IDS_MOUNTS already exist, do not run the full deploy just to change the image.

1. Upload the ready TAR only:

./scripts/upload-routeros-image.sh build/routeros-suricata-tzsp-arm64.tar

2. Swap the Suricata container only:

./scripts/upgrade-routeros-container.sh routeros-suricata-tzsp-arm64.tar

For version 0.5.3 the second command creates:

name=suricata_0.5.3
file=routeros-suricata-tzsp-arm64.tar
root-dir=/containers/suricata_0.5.3/root
interface=veth-ids
envlist=IDS_ENV
mountlists=IDS_MOUNTS

The upgrade helper does not create or modify the bridge, IP addresses, NAT, veth, TZSP/sniffer, firewall, REST user, envlist definitions, or mount definitions. It disables start-on-boot on older suricata_* containers, stops the running old Suricata container, creates the new versioned container, waits for image extraction, and starts it. Older containers are kept stopped for rollback.

Persistent /data, Suricata logs and vendor rules continue to use the existing mount list, so they survive the version change.