30 Commits

Author SHA1 Message Date
Vincent Bernat
2df8b5785b console: use avg/max for LimitType (instead of Avg/Max) 2024-11-23 23:48:02 +01:00
dimbdr
6c0e8e1791 feat: configure limit type (#1482)
* Add limit type field selection

* Take into account LimitType to generate SQL request for Line graph

Also, add LimitType in default configuration for console

* Take into account LimitType to generate SQL request for Sankey graph

* Refactor on SQL query used by line and sankey

* Add limitType description in doc

* Order by max in graphLine when limitType max is used

* Fix query when using top max

Revert some modifications, as they were no longer relevant with the query fixed.

* Rework way to sort by max in line graph type

* Add configuration validation on LimitType

---------

Co-authored-by: Dimitri Baudrier <github.52grm@simplelogin.com>
2024-11-23 18:39:28 +01:00
Vincent Bernat
41510619ff console: fix top row selection when multiple sampling rates 2024-10-31 07:52:46 +01:00
Vincent Bernat
0239cd0a9f common: remove MarshalJSON helpers for mapstructure
They are not needed anymore since we don't exchange configuration files
using JSON, since baac495b9c.
2024-07-20 14:51:40 +02:00
netixx
a312bc307e Use unit for sorting topX instead of always using bytes 2024-07-16 21:58:54 +02:00
Vincent Bernat
eba3af5183 schema/common: rename MainOnly to ClickHouseMainOnly 2023-01-18 18:09:45 +01:00
Vincent Bernat
c6a9319b57 common/schema: turns into a component
This is a first step to make it accept configuration. Most of the
changes are quite trivial, but I also ran into some difficulties with
query columns and filters. They need the schema for parsing, but parsing
happens before dependencies are instantiated (and even if it was not the
case, parsing is stateless). Therefore, I have added a `Validate()`
method that must be called after instantiation. Various bits `panic()`
if not validated to ensure we catch all cases.

The alternative to make the component manages a global state would have
been simpler but it would break once we add the ability to add or
disable columns.
2023-01-18 12:22:10 +01:00
Vincent Bernat
8a779fb905 common/schema: make schema fields private
This is useful later to bundle cached field and ensure they stay
up-to-date.
2023-01-17 20:53:00 +01:00
Vincent Bernat
727807b937 common/schema: use a symbol to identify columns 2023-01-17 20:53:00 +01:00
Vincent Bernat
e88d2a2974 common/schema: namespace column members by subsystem
Otherwise, it will be a mess once we introduce IPFIX/sFlow specific fields
2023-01-17 20:53:00 +01:00
Vincent Bernat
a30024cfa1 console: use common/schema for dimensions
This is a bit less type-safe. We could keep type safety by redefining
all the consts in `query_consts.go` in `common/schema`, but this is
pointless as the goal is to have arbitrary dimensions at some point.
2023-01-03 18:40:19 +01:00
Vincent Bernat
689497aa13 console: add SrcNetPrefix and DstNetPrefix as dimensions
This is not added to filtering as I fail to see how it would be useful.
One can still filter on SrcAddr and DstAddr.

Fix #218
2022-11-26 15:49:26 +01:00
Vincent Bernat
7cf38703ae console: extend DstCommunities to handle large communities 2022-09-28 14:42:11 +02:00
Vincent Bernat
9b821419c3 console: use DstCommunities 2022-09-27 00:34:41 +02:00
Vincent Bernat
9c90ee89f6 console: use Dst1stAS/Dst2ndAS/Dst3rdAS/DstASPath
DstASPath is not available in filters.
2022-09-27 00:34:41 +02:00
Vincent Bernat
78fb01c223 chore: fix some small issues detected by golangci-lint
But not using it as some linters are either plain incorrect (the one
suggesting to not use nil for `c.t.Context()`) or just
debatable (checking for err value is a good practice, but there are
good reasons to opt out in some cases).
2022-08-10 17:44:32 +02:00
Vincent Bernat
b5e48e8cfd console: also use dimensions to tell if we need to use main table 2022-08-08 11:35:18 +02:00
Vincent Bernat
b745cfd356 console: fix test about reverse filter direction 2022-08-08 09:42:27 +02:00
Vincent Bernat
50614cef5b console: add a bidirectional mode for graphs
It allows one to also display flows in the opposite direction.
2022-08-07 23:15:18 +02:00
Vincent Bernat
1aa260bae2 console/filter: let parser tells us if we need the main table or not
This is more robust this way. We also introduce the ability to reverse
the direction of a filter.
2022-08-04 18:54:41 +02:00
Vincent Bernat
085cb93e4b console: hide ::ffff: prefix from IPv6-mapped IPv4 addresses 2022-07-25 10:35:45 +02:00
Vincent Bernat
02dc9401e2 orchestrator: add more attributes to classify networks
Like for exporters, we add role, site, region, and tenant. This time,
this is done in ClickHouse.
2022-07-18 11:34:56 +02:00
Vincent Bernat
1e147704c7 inlet: classify exporters to group, role, site, region, and tenant
Previously, this was done only for groups. Encoding everything into
groups is a bit restrictive. The same should be done for IP networks.
2022-07-18 11:01:30 +02:00
Benjamin Collet
ef3b6dfc6b Convert InIfBoundary/OutIfBoundary from Enum8 to String to use as a dimension 2022-07-16 18:28:10 +02:00
Vincent Bernat
dff8773c7f console: move ethertypes for IPv4/IPv6 to helpers package 2022-07-07 02:43:22 +02:00
Vincent Bernat
8be1bca4fd license: AGPL-3.0-only
```
git ls-files \*.js \*.go \
  | xargs sed -i '1i // SPDX-FileCopyrightText: 2022 Free Mobile\n// SPDX-License-Identifier: AGPL-3.0-only\n'
git ls-files \*.vue \
  | xargs sed -i '1i <!-- SPDX-FileCopyrightText: 2022 Free Mobile -->\n<!-- SPDX-License-Identifier: AGPL-3.0-only -->\n'
```
2022-06-29 11:42:28 +02:00
Vincent Bernat
9cd90e6a14 console/filter: enable use of SrcNetName and DstNetName 2022-06-03 16:39:46 +02:00
Vincent Bernat
36a101fd90 console/filter: accept empty filters 2022-05-27 08:59:51 +02:00
Vincent Bernat
9d3b74d305 console: add PacketSizeBucket dimensions 2022-05-22 23:48:40 +02:00
Vincent Bernat
375a0fe88f console: split graph.go into query.go and graph.go
We will introduce more query types.
2022-05-21 19:53:08 +02:00