8 Commits

Author	SHA1	Message	Date
Vincent Bernat	c782312c2d	cmd: automatic restart of orchestrator on configuration change	2025-09-19 20:58:10 +02:00
Vincent Bernat	cc5d60239d	orchestrator/geoip: do not notify subscribers on start	2024-03-18 07:54:15 +01:00
Vincent Bernat	b1fe97765a	orchestrator/geoip: don't loose notifications when subscriber is busy ... Use a buffered channel to ensure they receive at least one notification.	2024-03-17 08:04:06 +01:00
Vincent Bernat	d2a9f2c9c2	orchestrator/geoip: add tests for notifications	2024-03-17 00:00:32 +01:00
Vincent Bernat	520e396ad6	orchestrator/geoip: fix refresh of ASN database ... And add a test.	2024-03-17 00:00:32 +01:00
Vincent Bernat	41412d430d	orchestrator/clickhouse: buffer networks.csv to disk ... And build it on changes, not on the fly.	2024-03-17 00:00:32 +01:00
Vincent Bernat	f8e306e45a	orchestrator/clickhouse: build networks.csv on the fly ... With the IPinfo GeoIP database, we have 1.3GB of memory used by ClickHouse and 3.3GB of memory used inside the orchestrator, plus 3.3GB of memory used during updates. This is quite a step up compared to almost no memory used, except caching when GeoIP was done in the inlet. Instead, we build the data on the fly. It is requested by ClickHouse on each database update (configurable for network sources, more seldomly for GeoIP database) and every hour with the default configuration. During this update, it will use 1.5GB of memory, so the max usage could be 1.3+1.5GB = 3.8GB against 1.3+3.3+3.3GB = 7.9GB (the double). It takes about 30s to build the dictionary on the fly (on Intel(R) Xeon(R) W-2295 CPU @ 3.00GHz).	2024-03-17 00:00:32 +01:00
Vincent Bernat	318c6de17c	orchestrator: move geoip as a top-level component ... It is easier if we have a flat model for components.	2024-03-13 11:43:22 +01:00