outlet/kafka: prevent discarding flows on shutdown

When shutting down the outlet, the core component is first shutdown, and
only after that Kafka and ClickHouse. This means we were still consuming
messages from Kafka and throwing them away. Instead, let the core
component stop the Kafka workers before shutting down itself.

Fix #2100
This commit is contained in:
Vincent Bernat
2025-11-15 16:14:24 +01:00
parent 6b27794dd2
commit 1a7fbf66cb
8 changed files with 143 additions and 18 deletions

View File

@@ -16,6 +16,7 @@ identified with a specific icon:
ClickHouse, Kafka and remote data sources (previously, `verify` was set to
false by default)
- 🩹 *outlet*: provide additional gracetime for a worker to send to ClickHouse
- 🩹 *outlet*: prevent discarding flows on shutdown
- 🩹 *outlet*: enhance scaling up and down workers to avoid hysteresis
- 🩹 *outlet*: accept flows where interface names or descriptions are missing
- 🩹 *docker*: update Traefik to 3.6.1 (for compatibility with Docker Engine 29)