diff --git a/.research/findings/tasks/sur-001-c1.md b/.research/findings/tasks/sur-001-c1.md new file mode 100644 index 0000000..522a93b --- /dev/null +++ b/.research/findings/tasks/sur-001-c1.md @@ -0,0 +1,138 @@ +# sur-001: Suricata Integration Stabilization and Config Unification +**Cycle**: 1 | **Theme**: backend-detection | **Kind**: investigation + design + fix | **Status**: done +**Date**: 2026-05-22 + +--- + +## Summary + +Diagnosed and resolved a series of bugs in the Suricata daemon integration, then refactored +the configuration system so users interact with a single `config.toml` instead of maintaining +a separate `suricata.yaml`. Config and suppress content are now generated in-memory and +delivered to Suricata via `memfd_create`, writing nothing to disk. + +--- + +## Findings + +### Q: Why did Suricata fail with MirrorSetupFailed "No such file or directory (os error 2)"? + +A: Two independent causes, both present simultaneously. + +1. `iproute2` was not installed in the container. `Command::new("ip").output()` returns + `io::Error(ENOENT)` when the binary is not found, which maps to `MirrorSetupFailed`. + +2. `MIRROR_PEER = "mantis-mirror-peer"` is 18 characters, exceeding the Linux kernel limit + of `IFNAMSIZ - 1 = 15`. The kernel rejects the interface name; `ip link add` fails, so + `/sys/class/net/mantis-mirror/ifindex` never appears, and `get_ifindex` returns ENOENT. + +**Fix**: Installed `iproute2`; renamed `MIRROR_PEER` to `"mantis-peer"` (11 chars). + +**Confidence**: high — confirmed by testing `ip link add` with both names. + +--- + +### Q: Why did Suricata log output not appear when stderr was piped? + +A: Suricata's console logger calls `isatty(2)` at startup. When stderr is not a TTY +(i.e., `Stdio::piped()` or `Stdio::null()`), Suricata automatically disables console output — +this is documented behavior matching daemon mode. `stdbuf` has no effect because Suricata's +log system does not use libc stdio buffering. + +**Fix**: Configure Suricata to write operational logs to `/tmp/suricata.log` via the yaml +`logging: file:` section. A dedicated `suricata-log` thread tails the file and forwards lines +into Mantis's tracing system, routing by prefix (Error/Warn/Notice/Info). + +**Confidence**: high — confirmed via web search (Suricata forum + OISF docs). + +--- + +### Q: Why did Suricata fail to start after switching to generated config via memfd? + +A: The yaml template used Rust's `"\n\` + source newline` continuation syntax, which is +designed to strip leading whitespace from the next source line. This silently removed all +YAML indentation, producing a structurally invalid document that Suricata rejected. + +**Fix**: Switched template to `r#"..."#` raw string, which preserves whitespace exactly as +written in source. No external YAML library required. + +**Confidence**: high — confirmed by inspecting `/tmp/suricata-mantis.yaml` before and after. + +--- + +### Q: Did Suricata support /proc/self/fd/N as a config path? + +A: Yes. `memfd_create` without `MFD_CLOEXEC` produces a file descriptor that survives +`fork`+`exec` into the Suricata child process. Suricata can open `/proc/self/fd/N` to read +the in-memory config. The approach works for both the main config and the suppress/threshold +file. The earlier failure was entirely due to malformed YAML, not the memfd mechanism. + +**Confidence**: high — confirmed working after YAML fix. + +--- + +### Q: What was wrong with the af-packet interface name in suricata.yaml? + +A: The static `suricata.yaml` still referenced `mantis-mirror-peer` (the old peer name) +after the rename to `mantis-peer`. Suricata was listening on a non-existent interface and +capturing no traffic. This was silently ignored — Suricata started but processed zero packets. + +**Fix**: Interface name is now derived from the `MIRROR_PEER` constant in `engine.rs` and +injected into the generated yaml, making divergence impossible. + +**Confidence**: high. + +--- + +## Design Decisions + +### Single config entry point +`suricata.yaml` was promoted from a user-edited file to a generated internal artifact. +All user-facing Suricata settings live in `config.toml` under `[Config.suricata]`: + +| Field | Type | Notes | +|---|---|---| +| `home_net` | `String` | Required. Protected network CIDR. | +| `worker_cpu_set` | `Option<[u32; 2]>` | Same semantics as `xsk_cpu_set`. | +| `management_cpu` | `Option` | Management thread CPU pin. | +| `af_packet_threads` | `String` | Default `"auto"`. | +| `af_packet_ring_size` | `u32` | Default 2048. | +| `af_packet_block_size` | `u32` | Default 131072. | +| `suppress` | `Vec` | Raw Suricata suppress/threshold lines. | + +Removing `[Config.suricata]` entirely disables the rule engine. + +### suppress as raw strings +Rather than defining a structured `SuppressEntry` with parsed fields, suppress entries are +stored as raw Suricata syntax strings. This is more flexible (supports `threshold`, +`rate_filter`, etc.) and lets users copy directly from Suricata documentation. + +### memfd_create for config delivery +Both the generated yaml and suppress content are written to anonymous in-memory files via +`memfd_create(0)` (no `MFD_CLOEXEC`), inherited by the Suricata child process, and passed +as `/proc/self/fd/N` paths. Parent closes its copies immediately after `spawn()`. Nothing +is written to the filesystem. + +--- + +## Unexpected Discoveries + +- `suricata.yaml` referenced the old peer interface name (`mantis-mirror-peer`) even after + the veth rename, causing Suricata to silently capture zero traffic. The bug was masked + because Suricata started without error. +- Rust's `"\n\` continuation eats leading whitespace — a non-obvious footgun when building + indentation-sensitive file formats inline. + +--- + +## Open Questions + +- Should `af_packet_ring_size` and `af_packet_block_size` be exposed to users, or are the + defaults sufficient for the research prototype? + +## Impact on Downstream Tasks + +- **active-response**: XDP blocking now has a working rule engine to corroborate with. + Fusion alerts from Suricata + ML are available as the trigger signal. +- **tls-analysis**: The `app-layer: tls: enabled: yes` and EVE tls event output can be + enabled in the generated yaml without user-visible config changes.