routeros-suricata-tzsp/docs/routeros-tzsp-capture.md

RouterOS TZSP capture architecture

MikroSuricata 0.11.4 uses a hybrid RouterOS capture path. The goal is to keep high-volume routed IPv4 off /tool/sniffer while still preserving visibility into ARP and other non-IPv4 Ethernet traffic.

Default topology

                         routed IPv4
RouterOS forwarding  --------------------> /ip firewall mangle
                                            action=sniff-tzsp
                                                   |
                                                   | TZSP UDP/37008
                                                   v
                                            MikroSuricata Rust
                                                   |
                                                   v
                                              TAP -> Suricata

                         non-IPv4 Ethernet
RouterOS interfaces  --------------------> /tool/sniffer
                                            filter-mac-protocol=!ip
                                            filter-stream=yes
                                                   |
                                                   +---- TZSP UDP/37008 ----+

Both paths use the same TZSP receiver in the container.

Why hybrid capture

A full /tool/sniffer TZSP stream can become the limiting stage on some RouterOS systems long before the Rust receiver, TAP or Suricata are saturated. In testing on the system that motivated this change, full Packet Sniffer streaming plateaued around 150-270 Mbit/s while an IPv4 mangle action=sniff-tzsp rule delivered roughly 1.2 Gbit/s of encapsulated traffic to the container VETH.

This is not treated as a universal RouterOS speed guarantee. The deployer simply chooses the path that proved suitable for high-volume routed IPv4 and keeps Packet Sniffer for the much smaller non-IPv4 stream.

Routed IPv4

The deployer manages one rule:

/ip/firewall/mangle
add chain=forward \
    action=sniff-tzsp \
    sniff-target=172.31.255.2 \
    sniff-target-port=37008 \
    comment="MikroSuricata TZSP IPv4"

The target IP and port come from CONTAINER_IP and TZSP_PORT; the values above are examples.

sniff-tzsp clones the matching packet to the TZSP receiver and then continues processing the next mangle rule. MikroSuricata places its rule at the beginning of the IPv4 forward mangle chain so later existing mangle rules still run normally.

The deployer removes/replaces only the rule whose comment is exactly:

MikroSuricata TZSP IPv4

It does not delete, disable or reorder unrelated mangle rules.

What this IPv4 rule covers

chain=forward covers IPv4 routed through RouterOS. This is the main IDS path for client-to-Internet and inter-subnet traffic.

It intentionally does not capture:

• IPv4 traffic terminating on RouterOS itself (input),
• IPv4 traffic generated by RouterOS itself (output),
• pure L2 IPv4 switching that never enters the routed IPv4 forward chain.

The first two exclusions also avoid any risk of recursively observing RouterOS-generated TZSP traffic. If router-local IPv4 inspection is required later, it should be added as a separate, explicitly designed path rather than by broadening the default rule blindly.

ARP, IPv6 and other non-IPv4 Ethernet traffic

Packet Sniffer is configured as the complement:

/tool/sniffer/set \
    streaming-enabled=yes \
    filter-stream=yes \
    filter-interface=all \
    filter-mac-protocol=!ip \
    filter-direction=any

RouterOS names EtherType 0x0800 as ip and EtherType 0x86DD as ipv6. Therefore filter-mac-protocol=!ip means everything except IPv4 EtherType, not "everything except all IP".

With the default filter the stream can include, among other protocols:

• ARP (0x0806),
• IPv6 (0x86DD),
• LLDP,
• 802.1X/EAPoL,
• LACP,
• PPPoE session/discovery,
• MPLS EtherTypes,
• RARP,
• VLAN/service-VLAN frames when visible to the sniffer.

filter-stream=yes is kept enabled so packets destined for the configured sniffer server are ignored by Packet Sniffer rather than captured again.

IPv6 performance note

IPv6 is deliberately left in the Packet Sniffer complement in 0.11.4 because the requested design is "mangle for IPv4, stream everything except IPv4". If the network later carries sustained high-volume IPv6, the Packet Sniffer performance ceiling can reappear for that traffic. At that point IPv6 should be split into its own high-throughput capture path and the L2 sniffer narrowed to explicit non-IP EtherTypes.

No bridge or interface name is assumed

The deployer does not assume names such as bridge, bridge-trunk, br0, LAN, or any site-specific VLAN interface.

Default:

TZSP_L2_INTERFACE=

An empty value is translated to RouterOS:

filter-interface=all

If an operator intentionally wants to constrain non-IPv4 capture, set the exact RouterOS interface name:

TZSP_L2_INTERFACE=bridge-core

The deployer verifies that a non-empty interface name exists before applying the sniffer configuration.

CONTAINER_BRIDGE=br-ids is different: that is the new private bridge created for the IDS container, not the bridge from which production traffic is assumed to arrive. It remains configurable through deploy-routeros.env.

RouterOS sniffer endpoint syntax

Newer RouterOS CLI builds can expose the TZSP destination as a composite value:

streaming-server=172.31.255.2:37008

Some older documentation/builds expose the destination as separate properties:

streaming-server=172.31.255.2
streaming-port=37008

The deploy and migration helper try the composite IP:PORT form first and automatically fall back to the split form if RouterOS rejects it.

Deployment variables

Default capture settings:

TZSP_PORT=37008
CONFIGURE_TZSP_CAPTURE=true
CONFIGURE_IPV4_MANGLE=true
CONFIGURE_L2_SNIFFER=true
START_L2_SNIFFER=true
TZSP_L2_INTERFACE=
TZSP_L2_MAC_PROTOCOL=!ip

Meaning:

Variable	Meaning
CONFIGURE_TZSP_CAPTURE	Master switch for capture changes made by full deploy
CONFIGURE_IPV4_MANGLE	Manage the project-owned IPv4 forward sniff-tzsp rule
CONFIGURE_L2_SNIFFER	Reconfigure global Packet Sniffer as the non-IPv4 complement
START_L2_SNIFFER	Start Packet Sniffer after configuring it
TZSP_L2_INTERFACE	Empty = all interfaces; otherwise exact interface/bridge name
TZSP_L2_MAC_PROTOCOL	Default !ip; normally should not be changed
TZSP_PORT	UDP port used by Rust TZSP receiver

For backward compatibility, an old deploy-routeros.env containing CONFIGURE_SNIFFER and START_SNIFFER is still accepted as an alias for the new master/start settings. VLAN_ID is no longer used by the capture deploy.

Full deployment

Prepare configuration:

cp deploy-routeros.env.example deploy-routeros.env

Build/upload the image as usual, then deploy:

./scripts/deploy-routeros.sh routeros-suricata-tzsp-arm64.tar

The deployment:

1. creates/updates the IDS container infrastructure,
2. starts the new container,
3. replaces only the MikroSuricata IPv4 mangle capture rule,
4. stops the global Packet Sniffer,
5. clears stale Packet Sniffer filters,
6. configures Packet Sniffer for !ip,
7. uses all RouterOS interfaces unless TZSP_L2_INTERFACE is explicitly set,
8. starts the L2 sniffer when START_L2_SNIFFER=true.

Switch an existing installation without rebuilding the container

The capture migration is independent from the container image:

./scripts/configure-routeros-tzsp-hybrid.sh

It reads the same deploy-routeros.env file.

Preview the exact generated RouterOS script without making changes:

DRY_RUN=true ./scripts/configure-routeros-tzsp-hybrid.sh

The helper only manages the project-owned IPv4 mangle rule. Packet Sniffer itself is a global RouterOS facility, so enabling CONFIGURE_L2_SNIFFER=true necessarily replaces its current global filters/settings. If the router already uses /tool/sniffer for another administrative purpose, either coordinate that use or set:

CONFIGURE_L2_SNIFFER=false

and manage non-IPv4 capture manually.

Manual RouterOS template

A standalone template is included:

routeros/02-tzsp-hybrid.rsc

Edit the local variables at the top before importing if the default container address differs.

Verification

Check the RouterOS capture configuration:

/ip/firewall/mangle/print stats where comment="MikroSuricata TZSP IPv4"
/tool/sniffer/print
/interface/monitor-traffic veth-ids

Or from the project host:

./scripts/routeros-status.sh

During a large IPv4 download, expect the mangle rule packet/byte counters and the container VETH TX counters to increase rapidly. In the MikroSuricata UI compare:

TZSP RX
Inspected throughput
capture %
queue %
loss/s

If TZSP RX is close to the expected traffic and capture remains close to 100%, RouterOS -> Rust and Rust -> TAP are keeping up.

Coverage and caveats

• Packet Sniffer cannot necessarily see traffic switched entirely in hardware by a hardware-offloaded bridge. Broadcast/multicast behavior can differ by platform.
• The default mangle rule covers routed IPv4 only, not bridge-only IPv4 switching.
• IPv6 remains on Packet Sniffer in 0.11.4 and can therefore inherit Packet Sniffer throughput limits under sustained high-rate IPv6 traffic.
• filter-interface=all can expose the same low-level broadcast/L2 frame on more than one logical/physical observation point on some topologies. If this is noisy, set TZSP_L2_INTERFACE explicitly.
• The global Packet Sniffer has no per-consumer instance. MikroSuricata therefore cannot preserve a second independent sniffer configuration while also owning the non-IPv4 TZSP stream.
• Do not run an additional full-traffic Packet Sniffer stream to the same TZSP destination at the same time; duplicate packets will inflate traffic and Suricata processing.

Why existing mangle rules continue to work

RouterOS defines sniff-tzsp as an action that sends a copy to a TZSP target and then passes the matched packet to the next mangle rule, similar to passthrough. The MikroSuricata rule therefore observes traffic without accepting, dropping, marking or rerouting the original packet.

This is why the rule can safely sit at the beginning of chain=forward: later QoS, marking, policy-routing and other mangle rules remain in the processing path.