docs: mention that max-workers is capped by the number of partitions

This commit is contained in:
Vincent Bernat
2025-10-15 12:11:30 +02:00
parent 98d1028850
commit bd4a349fba
2 changed files with 10 additions and 5 deletions

View File

@@ -157,7 +157,9 @@ keys are accepted:
- `fetch-max-wait-time` defines the maximum time to wait for the minimum
number of bytes to become available.
- `min-workers` defines the minimum number of Kafka workers to use.
- `max-workers` defines the maximum number of Kafka workers to use.
- `max-workers` defines the maximum number of Kafka workers to use (it should
not be more than the number of partitions for the topic, as defined in
`kafka``num-partitions`)
- `worker-increase-rate-limit` defines the duration before increasing the
number of workers.
- `worker-decrease-rate-limit` defines the duration before decreasing the

View File

@@ -443,10 +443,13 @@ akvorado_inlet_flow_input_udp_buffer_size_bytes{listener=":2055",worker="2"} 212
### Outlet
The outlet is expected to automatically scale the number of workers to ensure that the
data is delivered efficiently to ClickHouse. Increasing the maximum number of
Kafka workers (`max-workers`) past the default value of 8 may put more pressure
on ClickHouse. However, you can increase `maximum-batch-size`.
The outlet is expected to automatically scale the number of workers to ensure
that the data is delivered efficiently to ClickHouse. Increasing the maximum
number of Kafka workers (`max-workers`) past the default value of 8 may put more
pressure on ClickHouse. However, you can increase `maximum-batch-size`.
Moreover, there cannot be more workers than the number of partitions for the
topic. This part is configurable on the orchestrator
(`kafka``topic-configuration``num-partitions`).
The BMP routing component may have some challenges with scaling, especially when peers
disappear, as it requires cleaning up many entries in the routing tables. BMP