johanels/akvorado
1
0
Fork 0
mirror of https://github.com/akvorado/akvorado.git synced 2025-12-11 22:14:02 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
Files
42c794cd975ef380ad35d05e816479e39563b036
akvorado/console/data/docs/00-intro.md
Vincent Bernat 42c794cd97 doc: various updates
2022-07-20 07:49:00 +02:00

3.3 KiB
Raw Blame History

Introduction

Akvorado1 receives flows (currently Netflow/IPFIX and sFlow), hydrates them with interface names (using SNMP), geo information (using MaxMind), and exports them to Kafka, then ClickHouse. It also exposes a web interface to browse the result.

Quick start

A docker-compose.yml file is provided to quickly get started. Once running, Akvorado web interface should be running on port 8080.

# docker-compose up

A few synthetic flows are generated in the background. They can be disabled by removing the akvorado-exporter* services from docker-compose.yml (or you can just stop them with docker-compose stop akvorado-exporter{1,2,3,4}).

If you want to send you own flows, the inlet is accepting both NetFlow (port 2055) and sFlow (port 6343). You should also customize some settings in akvorado.yaml. They are described in details in the “configuration” section section of the documentation.

  • clickhouse → asns to give names to your internal AS numbers
  • clickhouse → networks to attach attributes to your networks
  • inlet → core → exporter-classifiers to define rules to attach attributes to your exporters
  • inlet → core → interface-classifiers to define rules to attach attributes to your interfaces (including the "boundary" attribute which is used by default by the web interface)

Take a look at the docker-compose.yml file if you want to setup the GeoIP database. It requires two environment variables to fetch them from MaxMind.

Big picture

General design

Akvorado is split into three components:

  • The inlet service receives flows from exporters. It poll each exporter using SNMP to get the system name, the interface names, descriptions and speeds. It query GeoIP databases to get the country and the AS number. It applies rules to add attributes to exporters. Interface rules attach to each interface a boundary (external or internal), a network provider and a connectivity type (PNI, IX, transit). The flow is exported to Kafka, serialized using Protobuf.

  • The orchestrator service configures the internal and external components. It creates the Kafka topic and configures ClickHouse to receive the flows from Kafka. It exposes configuration settings for the other services to use.

  • The console service exposes a web interface to look and manipulate the flows stored inside the ClickHouse database.

Serialized flow schemas

Flows sent to Kafka are encoded with a versioned schema, described in the flow-*.proto files. For each version of the schema, a different Kafka topic is used. For example, the flows-v2 topic receive serialized flows using the first version of the schema. The inlet service exports the schemas as well as the current version with its HTTP service, via the /api/v0/inlet/schemas.json endpoint.

ClickHouse database schemas

Flows are stored in a ClickHouse database using a table flows (and a few consolidated versions). The orchestrator service keeps the table schema up-to-date. You can check the schema using SHOW CREATE TABLE flows.

  1. Akvorado means "water wheel" in Esperanto. ↩︎