johanels/akvorado
1
0
Fork 0
mirror of https://github.com/akvorado/akvorado.git synced 2025-12-12 06:24:10 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,719 Commits 83 Branches 70 Tags
b5db24d9424c5bebf71c52bfb541a430e8f3e92c
Commit Graph

8 Commits

Author SHA1 Message Date
Vincent Bernat
f7cc5e3dbc orchestrator/clickhouse: add a benchmark for networks.csv 
```
goos: linux
goarch: amd64
pkg: akvorado/orchestrator/clickhouse
cpu: AMD Ryzen 5 5600X 6-Core Processor
BenchmarkNetworks-12                 482                 2.447 ms/op
```
 2025-08-17 11:05:58 +02:00
Vincent Bernat
cce61cb0d6 common/remotedatasource: rename from remotedatasourcefetcher 
Also rename RemoteDataSource to Source.
 2025-07-28 18:41:50 +02:00
Vincent Bernat
41412d430d orchestrator/clickhouse: buffer networks.csv to disk 
And build it on changes, not on the fly.
 2024-03-17 00:00:32 +01:00
Vincent Bernat
f8e306e45a orchestrator/clickhouse: build networks.csv on the fly 
With the IPinfo GeoIP database, we have 1.3GB of memory used by
ClickHouse and 3.3GB of memory used inside the orchestrator, plus 3.3GB
of memory used during updates. This is quite a step up compared to
almost no memory used, except caching when GeoIP was done in the inlet.

Instead, we build the data on the fly. It is requested by ClickHouse on
each database update (configurable for network sources, more seldomly
for GeoIP database) and every hour with the default configuration.
During this update, it will use 1.5GB of memory, so the max usage could
be 1.3+1.5GB = 3.8GB against 1.3+3.3+3.3GB = 7.9GB (the double).

It takes about 30s to build the dictionary on the fly (on Intel(R)
Xeon(R) W-2295 CPU @ 3.00GHz).
 2024-03-17 00:00:32 +01:00
Francois Espinet
87a57bf82e Do geoip enrich in clickhouse instead of inlet 
One solution to https://github.com/akvorado/akvorado/issues/62
 2024-03-11 15:29:09 +01:00
Itah
705e56cac4 inlet/metadata/static: allow exporters configuration refresh from http data source using common/remotedatasourcefetcher, refactored with orchestrator/clickhouse network-sources. 2024-01-17 11:11:34 +01:00
Vincent Bernat
85561c44f7 orchestrator/clickhouse: add method and headers for source HTTP request 
See discussion #640. This can be used to use POST or specify
authentication tokens.
 2023-04-23 12:53:41 +02:00
Vincent Bernat
95482c9201 orchestrator/clickhouse: ability to fetch network attributes with HTTP 2022-10-14 19:50:04 +02:00