johanels/akvorado
1
0
Fork 0
mirror of https://github.com/akvorado/akvorado.git synced 2025-12-11 22:14:02 +01:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
3,263 Commits 83 Branches 70 Tags
2d091617d3f0e23f6bf0487f5b6f11b7f872fc1c
Commit Graph

1477 Commits

Author SHA1 Message Date
Vincent Bernat
2d091617d3 docs: prepare for another beta 2025-08-11 07:35:37 +02:00
Vincent Bernat
a423ec44d6 docker: move TLS configuration into its own docker-compose file 
This makes it easier to use.
 2025-08-10 23:01:18 +02:00
Vincent Bernat
1a27bb1bc2 docker: add examples to enable authentication and TLS 2025-08-10 22:33:04 +02:00
Vincent Bernat
84b6f4584e docker: explain how to not expose Kafka-UI and Traefik dashboard 2025-08-10 15:58:37 +02:00
Vincent Bernat
dbadbf3adf docker: expose Traefik dashboard on the public endpoint 
It is also read-only.
 2025-08-10 15:55:04 +02:00
Vincent Bernat
1070e5b4f0 docker: document how to properly bind on port 80 
Add more documentation around merging in Docker. The previous
documentation was incorrect.
 2025-08-10 15:43:10 +02:00
Vincent Bernat
c254b405d2 console/frontend: add test for linter 2025-08-10 11:20:04 +02:00
Vincent Bernat
f976d66bd4 outlet/flow: decode IPFIX ingress/egressPhysicalInterface 
Also, don't decode IPv4/IPv6 addresses when they are 0 (some templates
will include both). Also decode dot1VlanId and postDot1qVlanId but
prefer vlanId and postVlanId if they are present.

Fix #1621
 2025-08-10 10:14:48 +02:00
Vincent Bernat
09a5a32375 docs: make the minimum configuration more prominent 2025-08-09 16:59:01 +02:00
Vincent Bernat
e5a625aecf outlet: make the number of Kafka workers dynamic 
Inserting into ClickHouse should be done in large batches to minimize
the number of parts created. This would require the user to tune the
number of Kafka workers to match a target of around 50k-100k rows. Instead,
we dynamically tune the number of workers depending on the load to reach
this target.

We keep using async if we are too low in number of flows.

It is still possible to do better by consolidating batches from various
workers, but that's something I wanted to avoid.

Also, increase the maximum wait time to 5 seconds. It should be good
enough for most people.

Fix #1885
 2025-08-09 15:58:25 +02:00
Vincent Bernat
e8bce09aec docs: ensure people don't run docker system prune -a 
Maybe it would even be better to remove it?
 2025-08-09 15:26:01 +02:00
Vincent Bernat
a74a41a6a0 docker: enable IPv6 networking, use a specific IPv4 subnet 
And also add documentation on how to use IPv6. The proposed setup relies
on NAT66, which is not good, but it works on any host with IPv6
connectivity. The documentation explains how to configure routed IPv6.

By using an IPv4 subnet in class E, we ensure that it is very unlikely
users will have overlap between their Docker setup and their production
network. This way, no need to change the Docker daemon configuration.
 2025-08-08 12:08:00 +02:00
Vincent Bernat
fd9dc0dbf3 docs: add more tips for incorrect metadata 2025-08-08 08:21:31 +02:00
Vincent Bernat
0bbe62b1d4 docs: remove advice on the active parts 
The advice was not true. An active part is not one that should be
actively merged, it's one that is used (and not to be deleted).
ClickHouse is good with more than 10k parts.
 2025-08-06 19:05:35 +02:00
Vincent Bernat
a862f302f2 docs: also mention tuning maximum-wait-time for ClickHouse 2025-08-06 07:44:03 +02:00
Vincent Bernat
abb9125502 outlet/clickhouse: use async insert when flow count is too low 
This should help to resolve the issue behind #1885.
 2025-08-06 06:58:12 +02:00
Vincent Bernat
25fce25b94 docs: add documentation for ClickHouse component in outlet 2025-08-06 06:58:12 +02:00
dependabot[bot]
fa05caeea3 build: bump the typescript group in /console/frontend with 3 updates 
Bumps the typescript group in /console/frontend with 3 updates: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node), [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) and [typescript](https://github.com/microsoft/TypeScript).


Updates `@types/node` from 24.1.0 to 24.2.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `@typescript-eslint/eslint-plugin` from 8.35.1 to 8.39.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.39.0/packages/eslint-plugin)

Updates `typescript` from 5.8.3 to 5.9.2
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.8.3...v5.9.2)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.39.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
- dependency-name: typescript
  dependency-version: 5.9.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 14:27:21 +02:00
Vincent Bernat
f181171e9a docs: mention the problem with slow inserts because of too many parts 
And as a temporary workaround, reduce the number of Kafka workers to 1.
This should be more dynamic to keep the number of inserted rows around
100k. Another strategy would be buffer tables or async inserts.
 2025-08-05 14:24:57 +02:00
Vincent Bernat
e65301fc6a outlet/clickhouse: use a summary for flow per batch count 
This should provide a better view on how many flows we put in a single
batch.
 2025-08-05 13:26:44 +02:00
dependabot[bot]
5c182cb987 build: bump typescript from 5.8.3 to 5.9.2 in /console/frontend 
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.8.3 to 5.9.2.
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.8.3...v5.9.2)

---
updated-dependencies:
- dependency-name: typescript
  dependency-version: 5.9.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:41:22 +02:00
dependabot[bot]
f6c43b3faa build: bump the vue group in /console/frontend with 3 updates 
Bumps the vue group in /console/frontend with 3 updates: [vue](https://github.com/vuejs/core), [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc) and [@vue/language-server](https://github.com/vuejs/language-tools/tree/HEAD/packages/language-server).


Updates `vue` from 3.5.17 to 3.5.18
- [Release notes](https://github.com/vuejs/core/releases)
- [Changelog](https://github.com/vuejs/core/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vuejs/core/compare/v3.5.17...v3.5.18)

Updates `vue-tsc` from 3.0.1 to 3.0.5
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.0.5/packages/tsc)

Updates `@vue/language-server` from 3.0.1 to 3.0.5
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.0.5/packages/language-server)

---
updated-dependencies:
- dependency-name: vue
  dependency-version: 3.5.18
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: vue
- dependency-name: vue-tsc
  dependency-version: 3.0.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: vue
- dependency-name: "@vue/language-server"
  dependency-version: 3.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: vue
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:37:27 +02:00
dependabot[bot]
ae38e380c9 build: bump the vueuse group in /console/frontend with 2 updates 
Bumps the vueuse group in /console/frontend with 2 updates: [@vueuse/core](https://github.com/vueuse/vueuse/tree/HEAD/packages/core) and [@vueuse/router](https://github.com/vueuse/vueuse/tree/HEAD/packages/router).


Updates `@vueuse/core` from 13.5.0 to 13.6.0
- [Release notes](https://github.com/vueuse/vueuse/releases)
- [Commits](https://github.com/vueuse/vueuse/commits/v13.6.0/packages/core)

Updates `@vueuse/router` from 13.5.0 to 13.6.0
- [Release notes](https://github.com/vueuse/vueuse/releases)
- [Commits](https://github.com/vueuse/vueuse/commits/v13.6.0/packages/router)

---
updated-dependencies:
- dependency-name: "@vueuse/core"
  dependency-version: 13.6.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: vueuse
- dependency-name: "@vueuse/router"
  dependency-version: 13.6.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: vueuse
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:26:02 +02:00
dependabot[bot]
2f07aede56 build: bump @codemirror/view 
Bumps the codemirror group in /console/frontend with 1 update: [@codemirror/view](https://github.com/codemirror/view).


Updates `@codemirror/view` from 6.38.0 to 6.38.1
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/6.38.0...6.38.1)

---
updated-dependencies:
- dependency-name: "@codemirror/view"
  dependency-version: 6.38.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: codemirror
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:25:32 +02:00
dependabot[bot]
ec3e1a6e96 build: bump @types/node in /console/frontend in the typing group 
Bumps the typing group in /console/frontend with 1 update: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node).


Updates `@types/node` from 24.0.10 to 24.1.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 24.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typing
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:25:17 +02:00
dependabot[bot]
b064e2f735 build: bump the dev-dependencies group 
Bumps the dev-dependencies group in /console/frontend with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [@eslint/js](https://github.com/eslint/eslint/tree/HEAD/packages/js) | `9.30.1` | `9.32.0` |
| [@vitejs/plugin-vue](https://github.com/vitejs/vite-plugin-vue/tree/HEAD/packages/plugin-vue) | `6.0.0` | `6.0.1` |
| [eslint](https://github.com/eslint/eslint) | `9.30.1` | `9.32.0` |
| [eslint-plugin-prettier](https://github.com/prettier/eslint-plugin-prettier) | `5.5.1` | `5.5.3` |
| [eslint-plugin-vue](https://github.com/vuejs/eslint-plugin-vue) | `10.3.0` | `10.4.0` |
| [prettier-plugin-tailwindcss](https://github.com/tailwindlabs/prettier-plugin-tailwindcss) | `0.6.13` | `0.6.14` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `7.0.0` | `7.0.6` |


Updates `@eslint/js` from 9.30.1 to 9.32.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/commits/v9.32.0/packages/js)

Updates `@vitejs/plugin-vue` from 6.0.0 to 6.0.1
- [Release notes](https://github.com/vitejs/vite-plugin-vue/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-vue/blob/main/packages/plugin-vue/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-vue/commits/plugin-vue@6.0.1/packages/plugin-vue)

Updates `eslint` from 9.30.1 to 9.32.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v9.30.1...v9.32.0)

Updates `eslint-plugin-prettier` from 5.5.1 to 5.5.3
- [Release notes](https://github.com/prettier/eslint-plugin-prettier/releases)
- [Changelog](https://github.com/prettier/eslint-plugin-prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/eslint-plugin-prettier/compare/v5.5.1...v5.5.3)

Updates `eslint-plugin-vue` from 10.3.0 to 10.4.0
- [Release notes](https://github.com/vuejs/eslint-plugin-vue/releases)
- [Changelog](https://github.com/vuejs/eslint-plugin-vue/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/eslint-plugin-vue/compare/v10.3.0...v10.4.0)

Updates `prettier-plugin-tailwindcss` from 0.6.13 to 0.6.14
- [Release notes](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/prettier-plugin-tailwindcss/compare/v0.6.13...v0.6.14)

Updates `vite` from 7.0.0 to 7.0.6
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.0.6/packages/vite)

---
updated-dependencies:
- dependency-name: "@eslint/js"
  dependency-version: 9.32.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: "@vitejs/plugin-vue"
  dependency-version: 6.0.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: eslint
  dependency-version: 9.32.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: eslint-plugin-prettier
  dependency-version: 5.5.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: eslint-plugin-vue
  dependency-version: 10.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: prettier-plugin-tailwindcss
  dependency-version: 0.6.14
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: vite
  dependency-version: 7.0.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-08-05 07:24:54 +02:00
Vincent Bernat
98eb1bdba5 chore: make a run of gofumpt 2025-08-05 06:21:34 +02:00
Vincent Bernat
4ba396da42 docker: really expose Kafka UI interface 2025-08-04 21:54:36 +02:00
Vincent Bernat
6323fed23b docs: prepare for another beta 2025-08-04 21:29:55 +02:00
Vincent Bernat
704376f316 docs: IPFIX 3215 is IPFIX 315 2025-08-04 16:06:20 +02:00
Vincent Bernat
268e95ee18 docker: switch away from Nix to build Docker images 
This should be faster to build multiarch images, as Nix requires
emulation, while we can build everything natively using builtin Go
cross-compilation support. Moreover, caching should be better.

Also, add Nix build to CI.

Also, build more architectures (linux/arm/v7).
 2025-08-04 12:30:22 +02:00
Vincent Bernat
a1d9d3fd6e docs: remove "removed feature" icon 
Not using it. When we did that, it was a breaking change.
 2025-08-03 12:18:53 +02:00
Vincent Bernat
b8825611a1 docker: expose metrics and Kafka UI to public endpoint 
Kafka UI is in read-only mode. This makes following the troubleshooting
documentation easier.
 2025-08-03 12:18:49 +02:00
Vincent Bernat
5607f6962d docker: switch from Provectus Kafka UI to Kafbat UI 
The Provectus one is unmaintained. See
https://github.com/provectus/kafka-ui/discussions/4255.
 2025-08-03 12:14:45 +02:00
Vincent Bernat
6abce2a0b3 docs: minor spelling corrections 2025-08-03 12:03:14 +02:00
Vincent Bernat
890052815b docs: add more tips for space-related problems 
Even put them first.
 2025-08-03 11:55:10 +02:00
Vincent Bernat
d701292edb docker: change default log level for ClickHouse 
On small setup, the default trace level is too big. See https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose
 2025-08-03 11:54:46 +02:00
Vincent Bernat
6a24e15a08 docker: build a linux/amd64/v3 image 
In addition to the regular linux/amd64. Both "docker image pull" and
"docker compose" will use linux/amd64/v3 if the architecture allows it.
This enables some optimizations at Go level.

I would have wished that Go would have been able to automatically
support fat binaries. This is in discussion, but mostly for different
architectures, not for different microarchitectures. It exists some
naive approach, like https://github.com/CAFxX/mgo/, but it does not play
well with embeds (the ASN database would make the binary image too big).

The Docker approach should be good enough for most people.
 2025-08-03 10:18:28 +02:00
Vincent Bernat
3ab97c6218 docs: add a word on how to run Go tests 2025-08-02 19:47:55 +02:00
Vincent Bernat
1234e788e9 docs: add an up-to-date example for SNMP configuration 2025-08-01 21:00:30 +02:00
Vincent Bernat
332435d650 docs: Netflow → NetFlow 2025-07-31 09:13:37 +02:00
Vincent Bernat
f43685b40b docs: add an example of tcpdump usage for troubleshooting 2025-07-31 09:12:00 +02:00
Vincent Bernat
74343c7c3e outlet/kafka: commit records queued to ClickHouse 
Instead of autocommit a whole fetch, commit only records queued to
ClickHouse. This should improve durability. Hopefully, the performance
should be good enough.
 2025-07-29 21:42:08 +02:00
Vincent Bernat
18beb310ee chore: replace interface{} with any 2025-07-29 07:42:49 +02:00
Vincent Bernat
fa7e4745b8 common/remotedatasource: be stricter on results from remote sources 
Also:
 - don't return partial results (not used)
 - fix tests
 - add more tests
 2025-07-29 07:25:42 +02:00
Vincent Bernat
32080f21c5 docs: rewrite a bit the custom dictionary example 2025-07-29 07:25:42 +02:00
Vincent Bernat
864e2ed4bf docs: add a release date 2025-07-28 07:11:45 +02:00
Vincent Bernat
d204d5b417 docs: prepare for a beta release 2025-07-27 23:04:53 +02:00
Vincent Bernat
17a272d0ba docs: update troubleshooting documentation 2025-07-27 21:44:28 +02:00
Vincent Bernat
ca160f648e docker: rename docker-compose-monitoring.yml to -prometheus.yml 
And move back kafka-ui into the common setup. kafka-ui is useful to
debug stuff and I want to be able to refer to it when troubleshooting
setup.
 2025-07-27 21:44:28 +02:00