Commit Graph

1370 Commits

Author SHA1 Message Date
Vincent Bernat
874d52f05f orchestrator/clickhouse: set TTL for system logs tables 2023-01-03 14:26:58 +01:00
Vincent Bernat
7d1ba478a1 orchestrator/clickhouse: rework migrations to use an abstract schema
We introduce an leaky abstraction for flows schema and use it for
migrations as a first step.

For views and dictionaries, we stop relying on a hash to know if they
need to be recreated, but we compare the select statements with our
target statement. This is a bit fragile, but strictly better than the
hash.

For data tables, we add the missing columns.

We give up on the abstraction of a migration step and just rely on
helper functions to get the same result. The migration code is now
shorter and we don't need to update it when adding new columns.

This is a preparatory work for #211 to allow a user to specify
additional fields to collect.
2023-01-02 23:42:05 +01:00
Vincent Bernat
86810beb6e docs: mention other system tables that can be dropped 2023-01-02 23:42:05 +01:00
Vincent Bernat
8a6b6614d4 build: update Shopify/sarama
This should reduce the memory usage when using zstd.
2023-01-02 23:42:05 +01:00
dependabot[bot]
cc4dc3d67e build: bump github.com/osrg/gobgp/v3 from 3.9.0 to 3.10.0 (#377) 2023-01-02 21:38:36 +00:00
dependabot[bot]
f9762ce3a4 build: bump gorm.io/gorm from 1.24.2 to 1.24.3 (#375) 2023-01-02 21:36:21 +00:00
dependabot[bot]
31d38431ea build: bump vue-echarts from 6.3.3 to 6.5.0 in /console/frontend (#382) 2023-01-02 21:32:34 +00:00
dependabot[bot]
44e2ed2228 build: bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17 (#378) 2023-01-02 21:15:29 +00:00
dependabot[bot]
315ab8728a build: bump typescript-language-server in /console/frontend (#380) 2023-01-02 21:15:09 +00:00
dependabot[bot]
10cb28b37d build: bump vue-tsc from 1.0.18 to 1.0.19 in /console/frontend (#381) 2023-01-02 21:14:34 +00:00
dependabot[bot]
ca07bfa9cf build: bump @vitest/coverage-c8 in /console/frontend (#383) 2023-01-02 21:14:24 +00:00
dependabot[bot]
9c327afb2a build: bump eslint from 8.30.0 to 8.31.0 in /console/frontend (#376) 2023-01-02 21:14:09 +00:00
dependabot[bot]
698bd3f0ae build: bump @volar/vue-language-server in /console/frontend (#374) 2023-01-02 21:13:13 +00:00
vincentbernat
72957553ea build: flake.lock automated update 2023-01-02 09:29:31 +01:00
Vincent Bernat
8e51084e5f docs: move build instructions for the Docker image 2022-12-31 16:05:26 +01:00
Vincent Bernat
b1397c4452 build: update gin-cache 2022-12-30 11:28:10 +01:00
Vincent Bernat
7418bd0517 inlet/flow: handle correctly interfaces with high index for sFlow
There are two kind of flow samples in sFlow: regular ones and expanded
ones. For the regular ones, the first two bits of the interface are used
for meta data. For the expanded ones, there is a dedicated 32-bit
integer for that (however, no documentation of what it means).
Unfortunately, the information is lost after the decoding abstraction
done by GoFlow2. Therefore, we take note of the kind of samples we get
to do the decoding.

However, we take a shortcut and assume all flows in a packet are either
the regular ones or the expanded ones. This is less error-prone than
trying to do more complex stuff. Let's see if this is really the case or
not.

Fix #370
2022-12-29 16:32:43 +01:00
Vincent Bernat
eeeec3f58c docker: fix Kafka healthcheck
Fix #368
2022-12-27 15:50:51 +01:00
dependabot[bot]
9a82c99f1f build: bump @volar/vue-language-server in /console/frontend (#364) 2022-12-26 22:04:12 +00:00
dependabot[bot]
4663e89741 build: bump @types/node from 18.11.17 to 18.11.18 in /console/frontend
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 18.11.17 to 18.11.18.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-26 23:00:32 +01:00
dependabot[bot]
98f2cb4ff6 build: bump vue-tsc from 1.0.17 to 1.0.18 in /console/frontend
Bumps [vue-tsc](https://github.com/johnsoncodehk/volar/tree/HEAD/vue-language-tools/vue-tsc) from 1.0.17 to 1.0.18.
- [Release notes](https://github.com/johnsoncodehk/volar/releases)
- [Changelog](https://github.com/johnsoncodehk/volar/blob/master/CHANGELOG.md)
- [Commits](https://github.com/johnsoncodehk/volar/commits/v1.0.18/vue-language-tools/vue-tsc)

---
updated-dependencies:
- dependency-name: vue-tsc
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-26 23:00:16 +01:00
dependabot[bot]
1a8a9e3798 build: bump @codemirror/commands in /console/frontend
Bumps [@codemirror/commands](https://github.com/codemirror/commands) from 6.1.2 to 6.1.3.
- [Release notes](https://github.com/codemirror/commands/releases)
- [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/commands/compare/6.1.2...6.1.3)

---
updated-dependencies:
- dependency-name: "@codemirror/commands"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-26 23:00:00 +01:00
dependabot[bot]
356d0b4517 build: bump @vueuse/router from 9.7.0 to 9.9.0 in /console/frontend (#362) 2022-12-26 07:17:01 +00:00
dependabot[bot]
a41afc9cff build: bump @vitest/coverage-c8 in /console/frontend (#357) 2022-12-26 07:06:28 +00:00
Vincent Bernat
ed2915970b build: update dependabot PR limit 2022-12-26 08:04:54 +01:00
dependabot[bot]
975183e62e build: bump @vueuse/core from 9.7.0 to 9.9.0 in /console/frontend (#358) 2022-12-26 07:03:01 +00:00
dependabot[bot]
2fef4ac230 build: bump @volar/vue-language-server in /console/frontend (#359) 2022-12-26 07:02:24 +00:00
dependabot[bot]
0943628188 build: bump vue-tsc from 1.0.16 to 1.0.17 in /console/frontend (#360) 2022-12-26 07:02:08 +00:00
dependabot[bot]
034a2b3d68 build: bump vitest from 0.26.1 to 0.26.2 in /console/frontend (#361) 2022-12-26 07:01:40 +00:00
dependabot[bot]
514ea9bb4f build: bump github.com/go-redis/redis/v8 from 8.11.4 to 8.11.5 (#356) 2022-12-26 06:42:50 +00:00
dependabot[bot]
2f6e3011b6 build: bump github.com/itchyny/gojq from 0.12.10 to 0.12.11 (#354) 2022-12-26 06:40:52 +00:00
dependabot[bot]
8beebce00c build: bump github.com/gin-gonic/gin from 1.8.1 to 1.8.2 (#355) 2022-12-26 06:32:48 +00:00
vincentbernat
62a16d6588 build: flake.lock automated update 2022-12-26 07:24:10 +01:00
Vincent Bernat
3055f7e3e3 build: use GitHub token when updating Nix 2022-12-26 07:12:28 +01:00
Vincent Bernat
c610a738a1 common/http: use SHA256 to compare POST requests
In our case, this shouldn't matter. However, performance hit should be
low and maybe at some point this middleware could be used for more
sensitive stuff.
2022-12-24 11:33:58 +01:00
Vincent Bernat
68654cc898 docker: update healthcheck for Kafka v1.6.4 2022-12-22 18:42:05 +01:00
Vincent Bernat
4db5ac7544 http: fix cache configuration marshal/unmarshal
Also, enable Redis by default, as stated in changelog.
2022-12-22 18:32:52 +01:00
Vincent Bernat
07a7524f8d docs: prepare for release 2022-12-22 18:12:35 +01:00
Vincent Bernat
788e12dc26 console: make cache duration configurable 2022-12-22 16:32:26 +01:00
Vincent Bernat
775dc08db2 console: implement an HTTP cache for Clickhouse-backed requests
Fix #328
2022-12-22 16:23:49 +01:00
Vincent Bernat
fe4295614e build: only run race tests for Go in GitLab 2022-12-22 15:30:34 +01:00
Vincent Bernat
f1f83a2ba8 common/helpers: move parametrized configuration to helpers package 2022-12-21 16:34:22 +01:00
Vincent Bernat
86ef33fa7d docs: update badge in README 2022-12-20 14:08:34 +01:00
dependabot[bot]
29dd22d68c build: bump postcss from 8.4.19 to 8.4.20 in /console/frontend
Bumps [postcss](https://github.com/postcss/postcss) from 8.4.19 to 8.4.20.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.4.19...8.4.20)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-20 12:03:08 +01:00
dependabot[bot]
0a9d61742a build: bump vue-tsc from 1.0.14 to 1.0.16 in /console/frontend (#350) 2022-12-20 11:00:05 +00:00
dependabot[bot]
91edab0080 build: bump echarts from 5.4.0 to 5.4.1 in /console/frontend (#349) 2022-12-20 10:58:20 +00:00
dependabot[bot]
0197614563 build: bump vitest from 0.25.7 to 0.26.1 in /console/frontend (#348) 2022-12-20 10:56:51 +00:00
Vincent Bernat
40968abed9 inlet/bmp: make racy tests more robust 2022-12-20 11:53:19 +01:00
Vincent Bernat
f57835b015 console/frontend: remove hack to make charts responsive
Not needed anymore since vue-echarts 6.3.3.
2022-12-20 11:53:19 +01:00
Vincent Bernat
0be0a173de console/frontend: rename computedStart → start
And rename start to humanStart. Less code.
2022-12-20 11:53:19 +01:00