DaLaw2 fbaef94082 feat: architecture, detection, security, SOAR, operations (#18)
* feat: Phase 2-5 — architecture, detection, security, SOAR, operations

Architecture:
- Hexagonal port traits (10 modules migrated from Arc<Database>)
- Domain model types moved to model/ directory
- Constants centralized + 7 made runtime-configurable via DB
- Dead Error/Log variants cleaned up, SystemLog split

Detection (Phase 5):
- Detection orchestrator with dedup + enrichment + source attribution
- Cross-flow correlation engine: botnet, scan, lateral movement (T9)
- Temporal beaconing detector: CV-based C2 periodicity (T10)
- LRU flow eviction replacing O(n) min_by_key scan (T12)

Security hardening:
- 7 fixes: alg:none, config secret leak, HTTPS open redirect,
  log traversal, HKDF salt, SOAR whitelist+cooldown, operator validation
- 4 memory safety fixes: LRU dedup, frequency cleanup, drift cap, clock
- Envelope encryption for secrets (AES-256-GCM + HKDF)
- 17 new tests (SecretStore + SOAR conditions)

SOAR (Phase 3):
- Multi-condition playbooks (5 condition types, AND logic)
- Playbook update API (PUT + toggle endpoints)

Operations (Phase 4):
- Dynamic log level, system control APIs (shutdown/restart)
- HTTP config hot reload, spawn_blocking for CPU-bound work
- CLI encrypt-db / decrypt-db commands
- Audit log API

Log level audit:
- 16 variants adjusted (noisy hot-path → TRACE/DEBUG)
- 5 dead variants removed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot review — 6 issues from PR #18

1. Botnet detector source_ip was set to victim dst_ip, causing SOAR
   to block the victim instead of the attacker
2. HTTPS redirect host header injection: validate host is private IP,
   localhost, or .local hostname before constructing redirect URL
3. smtp_password plaintext residue: clear settings table after writing
   to SecretStore to prevent pre-migration plaintext from persisting
4. install.sh: add apt-get update before install on Debian/Ubuntu
5. download_log OOM risk: add 50MB file size limit before reading
6. update_config restart trigger: check return value, report if
   shutdown already in progress instead of claiming success

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Agent Team review — security, perf, correctness

Security:
- S1: Add RBAC permission check for /api/logs/ and /api/audit/ endpoints
  (previously any authenticated user could access)
- S2/S3: Remove report_dir and log_dir from configurable settings to
  prevent arbitrary directory write via config API
- A2: Pin DNS-resolved IPs in webhook reqwest client to prevent DNS
  rebinding TOCTOU attack (resolve() instead of re-resolving)

Performance:
- P7: Add 50K key cap to FrequencyTracker to prevent unbounded growth
  under DDoS (was unbounded, worst case 1.6GB)
- P9: Increase ML alert broadcast capacity 100 → 1024 to prevent lost
  alerts during DDoS spikes (3 subscribers contend on 100-slot buffer)
- P2: Reduce FLOW_MAX_PERIODS 10000 → 1000 (saves 144KB/flow, feature
  extraction only uses aggregate stats)
- P1: Remove unnecessary FlowKey clone on hot path (~1.9MB/s saved)
- P5: Beaconing detector: split analyze_and_alert into read-lock scan
  + selective write-lock update (reduces DashMap contention)

Correctness:
- A4: Capture correlation counts inside DashMap guard before dropping,
  eliminating TOCTOU in logged values (botnet, scan, lateral)
- A6: Log warning when SOAR playbook action params JSON is malformed
  instead of silently replacing with empty object

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add trainer submodule, update frontend submodule

- Add net-guardia-trainer submodule (ParrotXray/NetGuardia-Trainer@dalaw2-dev)
- Update frontend submodule with code quality fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 15:56:33 +08:00

179 lines
6.4 KiB
Bash

#!/bin/bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
DEPLOY_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
COMPOSE_FILE="$DEPLOY_DIR/compose/podman-compose.yml"
if command -v podman-compose &>/dev/null; then
COMPOSE="podman-compose -f $COMPOSE_FILE"
RT="podman"
elif command -v docker &>/dev/null && docker compose version &>/dev/null 2>&1; then
COMPOSE="docker compose -f $COMPOSE_FILE"
RT="docker"
else
echo "ERROR: No container runtime found"
exit 1
fi
echo "=== Runtime: $RT ==="
echo "=== Kernel: $(uname -r) ==="
echo ""
echo "=== Building containers ==="
$COMPOSE build
echo "=== Starting containers ==="
$COMPOSE up -d
echo ""
echo "=== Containers running ==="
$RT ps --format "table {{.Names}}\t{{.Status}}" 2>/dev/null || $RT ps
get_pid() {
$RT inspect --format '{{.State.Pid}}' "$1"
}
mkdir -p /var/run/netns
EXT_PID=$(get_pid external)
INT_PID=$(get_pid internal)
RTR_PID=$(get_pid router)
NG_PID=$(get_pid netguardia)
ln -sf /proc/$EXT_PID/ns/net /var/run/netns/external
ln -sf /proc/$INT_PID/ns/net /var/run/netns/internal
ln -sf /proc/$RTR_PID/ns/net /var/run/netns/router
ln -sf /proc/$NG_PID/ns/net /var/run/netns/netguardia
# ============================================================
# Segment 1: external <-> router (10.10.1.0/24)
# Direct connection, no inspection needed
# ============================================================
echo ""
echo "=== Segment 1: external <-> router (10.10.1.0/24) ==="
ip link add ext-eth0 type veth peer name rtr-ext
ip link set ext-eth0 netns external
ip link set rtr-ext netns router
ip netns exec external ip link set lo up
ip netns exec external ip link set ext-eth0 up
ip netns exec external ip addr add 10.10.1.2/24 dev ext-eth0
for i in 3 4 5 6 7; do
ip netns exec external ip addr add 10.10.1.${i}/24 dev ext-eth0
done
ip netns exec external ip route add default via 10.10.1.1
ip netns exec router ip link set lo up
ip netns exec router ip link set rtr-ext up
ip netns exec router ip addr add 10.10.1.1/24 dev rtr-ext
echo " external: ext-eth0 10.10.1.{2-7}/24, gw 10.10.1.1"
echo " router: rtr-ext 10.10.1.1/24"
# ============================================================
# Segment 2: router <-> netguardia <-> internal (10.10.2.0/24)
# NetGuardia inline: XDP on ng-ext (router side) and ng-int (internal side)
# No bridges, no inline veth pair — direct XSK forwarding
# ============================================================
echo ""
echo "=== Segment 2: router <-> [NetGuardia] <-> internal (10.10.2.0/24) ==="
# router <-> netguardia: ng-ext is the netguardia side
ip link add rtr-int type veth peer name ng-ext
ip link set rtr-int netns router
ip link set ng-ext netns netguardia
# netguardia <-> internal: ng-int is the netguardia side
ip link add int-eth0 type veth peer name ng-int
ip link set int-eth0 netns internal
ip link set ng-int netns netguardia
# Router internal side
ip netns exec router ip link set rtr-int up
ip netns exec router ip addr add 10.10.2.1/24 dev rtr-int
ip netns exec router sh -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
# Internal container
ip netns exec internal ip link set lo up
ip netns exec internal ip link set int-eth0 up
ip netns exec internal ip addr add 10.10.2.2/24 dev int-eth0
for i in 3 4 5 6; do
ip netns exec internal ip addr add 10.10.2.${i}/24 dev int-eth0
done
ip netns exec internal ip route add default via 10.10.2.1
# NetGuardia interfaces (no IP, transparent)
ip netns exec netguardia ip link set ng-ext up
ip netns exec netguardia ip link set ng-int up
# Disable checksum offload on ALL veth endpoints.
# AF_XDP TX bypasses the kernel stack, so checksums are not computed.
# Without this, TCP packets forwarded through XSK have bad checksums and get dropped.
ip netns exec router ethtool -K rtr-int tx off rx off 2>/dev/null || true
ip netns exec router ethtool -K rtr-ext tx off rx off 2>/dev/null || true
ip netns exec internal ethtool -K int-eth0 tx off rx off 2>/dev/null || true
ip netns exec external ethtool -K ext-eth0 tx off rx off 2>/dev/null || true
ip netns exec netguardia ethtool -K ng-ext tx off rx off 2>/dev/null || true
ip netns exec netguardia ethtool -K ng-int tx off rx off 2>/dev/null || true
echo " router: rtr-int (10.10.2.1) <-> ng-ext (XDP ingress)"
echo " netguardia: ng-ext <-> [XSK forwarding] <-> ng-int"
echo " internal: int-eth0 (10.10.2.{2-6}) <-> ng-int (XDP egress)"
echo " checksum offload disabled on all veth endpoints"
# ============================================================
# Verify
# ============================================================
echo ""
echo "=== Interfaces inside netguardia ==="
ip netns exec netguardia ip -br link show
echo ""
echo "=== Testing connectivity ==="
echo -n " external -> router: "
ip netns exec external ping -c 1 -W 2 10.10.1.1 >/dev/null 2>&1 && echo "OK" || echo "FAIL"
# Without net-guardia, traffic between router and internal won't pass
# because ng-ext/ng-int are just veth endpoints with no forwarding
echo -n " router -> internal: "
ip netns exec router ping -c 1 -W 2 10.10.2.2 >/dev/null 2>&1 && echo "OK" || echo "FAIL (expected - needs net-guardia)"
cat > /tmp/netguardia_interfaces.txt << IEOF
# NetGuardia interface mapping - realistic inline deployment
# Router handles L3 (10.10.1.0/24 <-> 10.10.2.0/24)
# NetGuardia inline on 10.10.2.0/24 (no IP, no bridge)
# ng-ext - XDP ingress (router side, attached to rtr-int peer)
# ng-int - XDP egress (internal side, attached to int-eth0 peer)
# XSK forwards packets: ng-ext RX -> ng-int TX and ng-int RX -> ng-ext TX
# Management: eth0 (10.10.3.10)
IEOF
$RT cp /tmp/netguardia_interfaces.txt netguardia:/root/NetGuardia/interfaces.txt 2>/dev/null || true
rm -f /var/run/netns/external /var/run/netns/internal /var/run/netns/router /var/run/netns/netguardia
echo ""
echo "=========================================="
echo " NetGuardia realistic inline deployment!"
echo ""
echo " external (10.10.1.{2-7})"
echo " |"
echo " [router] 10.10.1.1 <-> 10.10.2.1"
echo " | rtr-int"
echo " |"
echo " ng-ext (no IP) <- XDP ingress"
echo " |"
echo " [net-guardia XSK]"
echo " |"
echo " ng-int (no IP) <- XDP egress"
echo " |"
echo " | int-eth0"
echo " internal (10.10.2.{2-6})"
echo ""
echo " All 10.10.2.0/24 traffic requires net-guardia!"
echo " Mgmt: 10.10.3.10"
echo " SSH: ssh -p 2222 root@<host-ip>"
echo " Web: http://<host-ip>:8080"
echo "=========================================="