Network Flow Monitoring with Threat Intelligence
go kafka p4 linux prometheus grafana networking security
Overview
Built as part of my Research Assistant role at BelWü. The system enriches live network flows with threat intelligence data in real time, flagging suspicious traffic at line rate.
Pipeline
- Language: Go — chosen for its concurrency model and low-latency characteristics
- Message broker: Apache Kafka as the backbone for high-throughput event streaming
- Throughput: 896K+ Kafka messages processed; sustained ~732 msg/s
- IP matching: O(1) hash-based lookup against threat feed blocklists — no per-packet linear scans
Infrastructure
- Switches: Deployed P4-programmable Intel Tofino 2 and Asterfusion X732Q-T switches
- Bring-up: BSP compilation, kernel module loading, low-level debugging with bf-sde-9.13.3
- Observability: Prometheus metrics + Grafana dashboards for pipeline health and flow statistics
Status
Completed — deployed in production at BelWü (2025–2026).