johanels/akvorado
1
0
Fork 0
mirror of https://github.com/akvorado/akvorado.git synced 2025-12-12 06:24:10 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,263 Commits 83 Branches 70 Tags
2d091617d3f0e23f6bf0487f5b6f11b7f872fc1c
Commit Graph

3263 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vincent Bernat
2d091617d3 docs: prepare for another beta v2.0.0-beta.3 2025-08-11 07:35:37 +02:00
Vincent Bernat
a423ec44d6 docker: move TLS configuration into its own docker-compose file 
This makes it easier to use.
 2025-08-10 23:01:18 +02:00
Vincent Bernat
1a27bb1bc2 docker: add examples to enable authentication and TLS 2025-08-10 22:33:04 +02:00
Vincent Bernat
84b6f4584e docker: explain how to not expose Kafka-UI and Traefik dashboard 2025-08-10 15:58:37 +02:00
Vincent Bernat
dbadbf3adf docker: expose Traefik dashboard on the public endpoint 
It is also read-only.
 2025-08-10 15:55:04 +02:00
Vincent Bernat
1070e5b4f0 docker: document how to properly bind on port 80 
Add more documentation around merging in Docker. The previous
documentation was incorrect.
 2025-08-10 15:43:10 +02:00
Vincent Bernat
ed4691d169 docker: always use a mapping for "environment" 2025-08-10 15:16:51 +02:00
Vincent Bernat
c254b405d2 console/frontend: add test for linter 2025-08-10 11:20:04 +02:00
Vincent Bernat
ad10a2a783 build: add DOCKER_BUILD_OPTIONS variable to arguments to docker build 
Notably, we can do:

```
make docker DOCKER_BUILD_OPTIONS=--platform=linux/amd64/v3
```
 2025-08-10 10:42:22 +02:00
Vincent Bernat
f976d66bd4 outlet/flow: decode IPFIX ingress/egressPhysicalInterface 
Also, don't decode IPv4/IPv6 addresses when they are 0 (some templates
will include both). Also decode dot1VlanId and postDot1qVlanId but
prefer vlanId and postVlanId if they are present.

Fix #1621
 2025-08-10 10:14:48 +02:00
Vincent Bernat
79d0c019bb outlet: remove deprecated kafka configuration for outlet 2025-08-09 23:39:05 +02:00
Vincent Bernat
12c1315696 github: fix schedule of Nix hashes updates 2025-08-09 23:35:38 +02:00
Vincent Bernat
322d2382ed github: update Nix hashes less often 
But fail on release if they are not up-to-date.
 2025-08-09 23:33:58 +02:00
Vincent Bernat
09a5a32375 docs: make the minimum configuration more prominent 2025-08-09 16:59:01 +02:00
Vincent Bernat
c2616974c1 github: remove scheduled monthly action 
Not really useful, everything is pinned.
 2025-08-09 15:58:58 +02:00
Vincent Bernat
5f7e8db3e8 github: don't run scheduled actions on forks 
This is useless and a waste of resources. I wonder why actions are
enabled by default on forks! Well, I think I know.
 2025-08-09 15:58:30 +02:00
Vincent Bernat
b71ed907e8 outlet/kafka: collect metrics have creating new client 
We cannot do that before, otherwise, we run into a race condition as the
list of metrics is not protected inside kprom franz-go plugin.

Also do that for inlet/kafka, even if it is less important there, since
there is only one client.
 2025-08-09 15:58:25 +02:00
Vincent Bernat
e5a625aecf outlet: make the number of Kafka workers dynamic 
Inserting into ClickHouse should be done in large batches to minimize
the number of parts created. This would require the user to tune the
number of Kafka workers to match a target of around 50k-100k rows. Instead,
we dynamically tune the number of workers depending on the load to reach
this target.

We keep using async if we are too low in number of flows.

It is still possible to do better by consolidating batches from various
workers, but that's something I wanted to avoid.

Also, increase the maximum wait time to 5 seconds. It should be good
enough for most people.

Fix #1885
 2025-08-09 15:58:25 +02:00
Vincent Bernat
e8bce09aec docs: ensure people don't run docker system prune -a 
Maybe it would even be better to remove it?
 2025-08-09 15:26:01 +02:00
Vincent Bernat
4f68acee48 docker: don't use enable_ipv4 
It requires a fairly recent version of Docker Compose. Not needed since
this is the default value.
 2025-08-08 13:30:05 +02:00
Vincent Bernat
a74a41a6a0 docker: enable IPv6 networking, use a specific IPv4 subnet 
And also add documentation on how to use IPv6. The proposed setup relies
on NAT66, which is not good, but it works on any host with IPv6
connectivity. The documentation explains how to configure routed IPv6.

By using an IPv4 subnet in class E, we ensure that it is very unlikely
users will have overlap between their Docker setup and their production
network. This way, no need to change the Docker daemon configuration.
 2025-08-08 12:08:00 +02:00
Vincent Bernat
3e68a41f57 docker: for dev, separate standalone ClickHouse setup from cluster 
This way, there is no need to start a whole cluster just to work on a
single ClickHouse. Also add some hints in CONTRIBUTING.md.
 2025-08-08 08:55:29 +02:00
Vincent Bernat
fd9dc0dbf3 docs: add more tips for incorrect metadata 2025-08-08 08:21:31 +02:00
Vincent Bernat
0bbe62b1d4 docs: remove advice on the active parts 
The advice was not true. An active part is not one that should be
actively merged, it's one that is used (and not to be deleted).
ClickHouse is good with more than 10k parts.
 2025-08-06 19:05:35 +02:00
Vincent Bernat
8aca3e4406 outlet/flow: improve performance of unumber decoding 
Just unroll the 8 possible cases.
 2025-08-06 08:38:18 +02:00
Vincent Bernat
a862f302f2 docs: also mention tuning maximum-wait-time for ClickHouse 2025-08-06 07:44:03 +02:00
Vincent Bernat
abb9125502 outlet/clickhouse: use async insert when flow count is too low 
This should help to resolve the issue behind #1885.
 2025-08-06 06:58:12 +02:00
Vincent Bernat
222deb80b5 outlet/clickhouse: do not flush if no row to send 2025-08-06 06:58:12 +02:00
Vincent Bernat
25fce25b94 docs: add documentation for ClickHouse component in outlet 2025-08-06 06:58:12 +02:00
github-actions[bot]
2ffba21953 build: update Nix dependency hashes 2025-08-05 13:57:01 +00:00
dependabot[bot]
fa05caeea3 build: bump the typescript group in /console/frontend with 3 updates 
Bumps the typescript group in /console/frontend with 3 updates: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node), [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) and [typescript](https://github.com/microsoft/TypeScript).


Updates `@types/node` from 24.1.0 to 24.2.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `@typescript-eslint/eslint-plugin` from 8.35.1 to 8.39.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.39.0/packages/eslint-plugin)

Updates `typescript` from 5.8.3 to 5.9.2
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.8.3...v5.9.2)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.39.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
- dependency-name: typescript
  dependency-version: 5.9.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 14:27:21 +02:00
dependabot[bot]
e1e480d425 build: bump github.com/ClickHouse/clickhouse-go/v2 from 2.37.2 to 2.40.1 
Bumps [github.com/ClickHouse/clickhouse-go/v2](https://github.com/ClickHouse/clickhouse-go) from 2.37.2 to 2.40.1.
- [Release notes](https://github.com/ClickHouse/clickhouse-go/releases)
- [Changelog](https://github.com/ClickHouse/clickhouse-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ClickHouse/clickhouse-go/compare/v2.37.2...v2.40.1)

---
updated-dependencies:
- dependency-name: github.com/ClickHouse/clickhouse-go/v2
  dependency-version: 2.40.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 14:27:09 +02:00
dependabot[bot]
31e978b9c7 build: bump github.com/twmb/franz-go/plugin/kprom from 1.2.1 to 1.3.0 
Bumps [github.com/twmb/franz-go/plugin/kprom](https://github.com/twmb/franz-go) from 1.2.1 to 1.3.0.
- [Changelog](https://github.com/twmb/franz-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/twmb/franz-go/compare/v1.2.1...v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/twmb/franz-go/plugin/kprom
  dependency-version: 1.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 14:27:00 +02:00
Vincent Bernat
f181171e9a docs: mention the problem with slow inserts because of too many parts 
And as a temporary workaround, reduce the number of Kafka workers to 1.
This should be more dynamic to keep the number of inserted rows around
100k. Another strategy would be buffer tables or async inserts.
 2025-08-05 14:24:57 +02:00
Vincent Bernat
60f0ff29c7 outlet/clickhouse: better buckets for metrics 
For wait time, we know this should be around the configured max wait
time. For insert time, we are interested if this is very long.
 2025-08-05 13:58:18 +02:00
Vincent Bernat
2058403ca0 outlet/clickhouse: fix wait time to not include insert time 2025-08-05 13:50:35 +02:00
Vincent Bernat
94cade91ac outlet/clickhouse: fix spelling for flow_per_batch metric 2025-08-05 13:50:35 +02:00
Vincent Bernat
e65301fc6a outlet/clickhouse: use a summary for flow per batch count 
This should provide a better view on how many flows we put in a single
batch.
 2025-08-05 13:26:44 +02:00
github-actions[bot]
d2d534f1ac build: update Nix dependency hashes
Some checks failed
CI / 🤖 Check dependabot status (push) Has been cancelled
Details
CI / ⚖️ Check licenses (push) Has been cancelled
Details
Update Nix dependency hashes / Update dependency hashes (push) Has been cancelled
Details
CI / 🐧 Build and test on Linux (push) Has been cancelled
Details
CI / ❄️ Build on Nix (push) Has been cancelled
Details
CI / 🍏 Build and test on macOS (push) Has been cancelled
Details
CI / 🔍 Upload code coverage (push) Has been cancelled
Details
CI / 🔭 Build Go backend (1.24) (push) Has been cancelled
Details
CI / 🔭 Build JS frontend (20) (push) Has been cancelled
Details
CI / 🔭 Build JS frontend (22) (push) Has been cancelled
Details
CI / 🔭 Build JS frontend (24) (push) Has been cancelled
Details
CI / 🐋 Build Docker images (push) Has been cancelled
Details
CI / 🚀 Publish release (push) Has been cancelled
Details
Update Nix flake.lock / Update Nix lockfile (asn2org) (push) Has been cancelled
Details
Update Nix flake.lock / Update Nix lockfile (nixpkgs) (push) Has been cancelled
Details
2025-08-05 09:47:52 +00:00
dependabot[bot]
76136af6a8 build: bump github.com/bits-and-blooms/bitset from 1.22.0 to 1.24.0 
Bumps [github.com/bits-and-blooms/bitset](https://github.com/bits-and-blooms/bitset) from 1.22.0 to 1.24.0.
- [Release notes](https://github.com/bits-and-blooms/bitset/releases)
- [Commits](https://github.com/bits-and-blooms/bitset/compare/v1.22.0...v1.24.0)

---
updated-dependencies:
- dependency-name: github.com/bits-and-blooms/bitset
  dependency-version: 1.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 10:25:42 +02:00
Vincent Bernat
03d69fa06a build: fix syntax error 2025-08-05 08:36:22 +02:00
Vincent Bernat
dccce086ca build: update external Nix hashes more often 
Let's see how this works...
 2025-08-05 08:28:32 +02:00
Vincent Bernat
c9f7af245e build: split Nix-related options in two parts 
And don't schedule update of Nix hashes after push action. This is too
noisy. A daily run should be enough.
 2025-08-05 08:11:38 +02:00
Vincent Bernat
82822a393c build: only use "fetch-depth" when we need the final version 
This means only for Docker and Linux build.
 2025-08-05 08:05:48 +02:00
github-actions[bot]
f2ef135345 build: update Nix dependency hashes 2025-08-05 05:54:42 +00:00
dependabot[bot]
4c3262dc06 build: bump github.com/twmb/franz-go/pkg/kadm from 1.16.0 to 1.16.1 
Bumps [github.com/twmb/franz-go/pkg/kadm](https://github.com/twmb/franz-go) from 1.16.0 to 1.16.1.
- [Changelog](https://github.com/twmb/franz-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/twmb/franz-go/compare/v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: github.com/twmb/franz-go/pkg/kadm
  dependency-version: 1.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:43:07 +02:00
dependabot[bot]
5c182cb987 build: bump typescript from 5.8.3 to 5.9.2 in /console/frontend 
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.8.3 to 5.9.2.
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.8.3...v5.9.2)

---
updated-dependencies:
- dependency-name: typescript
  dependency-version: 5.9.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:41:22 +02:00
Vincent Bernat
85b5b68fbf github: sleep even more before updating Nix hashes 2025-08-05 07:40:13 +02:00
Vincent Bernat
a45931236f github: move typescript stuff to their own group 
There is a bit of dependencies on the dev group. Let's see if it works
better this way.
 2025-08-05 07:38:26 +02:00
dependabot[bot]
f6c43b3faa build: bump the vue group in /console/frontend with 3 updates 
Bumps the vue group in /console/frontend with 3 updates: [vue](https://github.com/vuejs/core), [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc) and [@vue/language-server](https://github.com/vuejs/language-tools/tree/HEAD/packages/language-server).


Updates `vue` from 3.5.17 to 3.5.18
- [Release notes](https://github.com/vuejs/core/releases)
- [Changelog](https://github.com/vuejs/core/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vuejs/core/compare/v3.5.17...v3.5.18)

Updates `vue-tsc` from 3.0.1 to 3.0.5
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.0.5/packages/tsc)

Updates `@vue/language-server` from 3.0.1 to 3.0.5
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.0.5/packages/language-server)

---
updated-dependencies:
- dependency-name: vue
  dependency-version: 3.5.18
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: vue
- dependency-name: vue-tsc
  dependency-version: 3.0.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: vue
- dependency-name: "@vue/language-server"
  dependency-version: 3.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: vue
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:37:27 +02:00