By default, the 5-tuple is used to load balance flows. Exporters with
many flows are bound to a specific worker. Use eBPF to do a per-packet
load-balancing.
Currently, this is done randomly, but we will use a percpu counter in
the next commit. This will make the test easier too, maybe?
This should also enable graceful restart but not with the current
Docker Compose setup, we would need to use mode host or spawn a new one
in the same network namespace than the old one. This does not look like
very complex:
- spawn a new inlet in the same network namespace, but listening to a
different HTTP port
- stop the previous inlet
- spawn a new inlet in the same network namespace
- stop the previous inlet
Alternatively, we could use SO_REUSEPORT for the HTTP socket too!
This is not really tested. The initial plan was to provide a demo with
Authelia, but the hard requirement on TLS makes it a bit difficult to
provide something that works out-of-the-box.
See also #1969, which is using a proxy instead.
This is used for demo, it makes sense to also do that for the other
components. This helps test one of the component temporarily, for
example during the e2e tests.
Some of the files were quite big:
- asns.csv ~ 3 MB
- index.js ~ 1.5 MB
- *.svg ~ 2 MB
Use a ZIP archive to put them all and embed it. This reduce the binary
size from 89 MB to 82 MB. 🤯
This also pulls some code modernization (use of http.ServeFileFS).
Add ability to enable the demo flows just with a profile instead of
modifying .env. Add more instructions on how to use Docker Compose and
how to hack on the console.
While I was relunctant to let Go download the right toolchain if we
didn't have one, this makes everything simpler. The Go version is now
fully controlled by `go.mod`. It also a nice for people wanting to build
on older distributions.
For Nix, GOTOOLCHAIN is set to local, so we rely on `go_latest` being
up-to-date enough. But they are usually quite fast to update, so it
should be OK.
Alloy does not allow to turn the parsed metadata into actual metadata,
without enumerating each of them. Also, Vector is far more versatile.
And you can put unittests!
Also, parse more logs. Everything should be there, except ClickHouse.
Fix#1907
This is a bit like Traefik. We set metrics.port on each container we
want to scrape metrics from (and optionally metrics.path).
Semi-related, but we also rely on exposed port for Traefik and we override
it for all containers to be sure we select the right one. This is less
error prone as we need at least one exposed port and some containers may
or may not have one. Just always set an exposed port if we have metrics
or traefik rules.
The idea is that alloy can also be used for more. For example, we could
introduce Loki (with a `docker-compose-loki.yml`) and it would use alloy
too. Alloy configuration needs to be split into several parts and both
`docker-compose-prometheus.yml` and `docker-compose-loki.yml` would
define it but with an additional volume for their specific part of the
configuration (using the `extend` mechanism).
However, we don't use the bundled Node Exporter, nor the bundled
cAdvisor. It is better to have individual components to avoid reduce the
amount of code with elevated privileges (both Node Exporter and cAdvisor
need specific privileges). Also, we keep Prometheus instead of switching
to the full Grafana stack with Mimir as it is a more common setup and
this is not a goal to provide something universally scalable.
Also, Prometheus is now behind the private endpoint as it is possible to
send metrics.
Docker can easily break the firewall rules such that masquerading
happens internally.
```
ip saddr 247.16.12.0/24 oifname != "br-65eaa81ed142" counter packets 812 bytes 132030 masquerade
ip saddr 247.16.12.0/24 oifname != "br-fa3db0ecc1de" counter packets 0 bytes 0 masquerade
ip saddr 247.16.12.0/24 oifname != "br-c7a7788478c5" counter packets 0 bytes 0 masquerade
```
When the "current" bridge is the second one, inter-container
communication gets masqueraded. I didn't find an associated issue.
