johanels/akvorado
1
0
Fork 0
mirror of https://github.com/akvorado/akvorado.git synced 2025-12-11 22:14:02 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
Files
0d5d07dec62b11519f57d6d0d08af2a386147b4b
akvorado/console/data/docs/00-intro.md
Tchadel Icard 0d5d07dec6 inet/flow: add sflow support (#23)
2022-07-19 18:18:06 +02:00

2.7 KiB
Raw Blame History

Introduction

Akvorado1 receives flows (currently Netflow/IPFIX and sFlow), hydrates them with interface names (using SNMP), geo information (using MaxMind), and exports them to Kafka, then ClickHouse. It also exposes a web interface to browse the result.

Quick start

A docker-compose.yml file is provided to quickly get started. Once running, Akvorado web interface should be running on port 80 and an inlet accepting both NetFlow (port 2055) and sFlow (port 6343). You need to configure SNMP on your exporters to accept requests from Akvorado.

# docker-compose up

A few synthetic flows are generated in the background. They can be disabled by removing the akvorado-exporter* services from docker-compose.yml (or you can just stop them with docker-compose stop akvorado-exporter{1,2,3,4}).

Take a look at the docker-compose.yml file if you want to setup the GeoIP database. It requires two environment variables to fetch them from MaxMind.

Big picture

General design

Akvorado is split into three components:

  • The inlet service receives flows from exporters. It poll each exporter using SNMP to get the system name, the interface names, descriptions and speeds. It query GeoIP databases to get the country and the AS number. It applies rules to classify exporters into groups. Interface rules attach to each interface a boundary (external or internal), a network provider and a connectivity type (PNI, IX, transit). The flow is exported to Kafka, serialized using Protobuf.

  • The configuration service configures the external components. It creates the Kafka topic and configures ClickHouse to receive the flows from Kafka.

  • The console service exposes a web interface to look and manipulate the flows stored inside the ClickHouse database.

Serialized flow schemas

Flows sent to Kafka are encoded with a versioned schema, described in the flow-*.proto files. For each version of the schema, a different Kafka topic is used. For example, the flows-v2 topic receive serialized flows using the first version of the schema. The inlet service exports the schemas as well as the current version with its HTTP service, via the /api/v0/inlet/schemas.json endpoint.

ClickHouse database schemas

Flows are stored in a ClickHouse database using a table flows (and a few consolidated versions). The orchestrator service keeps the table schema up-to-date. You can check the schema using SHOW CREATE TABLE flows.

  1. Akvorado means "water wheel" in Esperanto. ↩︎