Commit Graph

6 Commits

Author SHA1 Message Date
Vincent Bernat
e20645c92e outlet/metadata: synchronous fetching of metadata
As we are not constrained by time that much in the outlet, we can
simplify the fetching of metadata by doing it synchronously. We still
keep the breaker design to avoid continously polling a source that is
not responsive, so we still can loose some data if we are not able to
poll metadata. We also keep the background cache refresh. We also
introduce a grace time of 1 minute to avoid loosing data during start.

For the static provider, we wait for the remote data sources to be
ready. For the gNMI provider, there are target windows of availability
during which the cached data can be polled. The SNMP provider is loosing
its ability to coalesce requests.
2025-07-27 21:44:28 +02:00
Vincent Bernat
4c0b15e1cd inlet/outlet: rename a few metrics
For example:

```
 17:35 ❱ curl -s 127.0.0.1:8080/api/v0/outlet/metrics | promtool check metrics
akvorado_outlet_core_classifier_exporter_cache_size_items counter metrics should have "_total" suffix
akvorado_outlet_core_classifier_interface_cache_size_items counter metrics should have "_total" suffix
akvorado_outlet_flow_decoder_netflow_flowset_records_sum counter metrics should have "_total" suffix
akvorado_outlet_flow_decoder_netflow_flowset_records_sum non-histogram and non-summary metrics should not have "_sum" suffix
akvorado_outlet_flow_decoder_netflow_flowset_sum counter metrics should have "_total" suffix
akvorado_outlet_flow_decoder_netflow_flowset_sum non-histogram and non-summary metrics should not have "_sum" suffix
akvorado_outlet_kafka_buffered_fetch_records_total non-counter metrics should not have "_total" suffix
akvorado_outlet_kafka_buffered_produce_records_total non-counter metrics should not have "_total" suffix
akvorado_outlet_metadata_cache_refreshs counter metrics should have "_total" suffix
akvorado_outlet_routing_provider_bmp_peers_total non-counter metrics should not have "_total" suffix
akvorado_outlet_routing_provider_bmp_routes_total non-counter metrics should not have "_total" suffix
```

Also ensure metrics using errors as label don't have a too great
cardinality by using constants for error messages used.
2025-07-27 21:44:28 +02:00
Vincent Bernat
e08331a286 common/helpers: switch to a blessed fork of mapstructure 2025-02-15 14:51:17 +01:00
Vincent Bernat
e3c8f13562 build: enable loopvar experiment
This is enabled by default once we switch to Go 1.22 (in go.mod). See
https://tip.golang.org/wiki/LoopvarExperiment
2024-02-18 09:46:49 +01:00
Vincent Bernat
194019535c chore: add missing SPDX header to some source files 2024-02-01 00:17:32 +01:00
Itah
705e56cac4 inlet/metadata/static: allow exporters configuration refresh from http data source using common/remotedatasourcefetcher, refactored with orchestrator/clickhouse network-sources. 2024-01-17 11:11:34 +01:00