Commit Graph

2 Commits

Author SHA1 Message Date
Vincent Bernat
aae92e7fe0 common/schema: use an ordered map for columns
For ClickHouse, we can iterate, but for other aspects, we need a direct
access.
2023-01-03 17:45:50 +01:00
Vincent Bernat
7d1ba478a1 orchestrator/clickhouse: rework migrations to use an abstract schema
We introduce an leaky abstraction for flows schema and use it for
migrations as a first step.

For views and dictionaries, we stop relying on a hash to know if they
need to be recreated, but we compare the select statements with our
target statement. This is a bit fragile, but strictly better than the
hash.

For data tables, we add the missing columns.

We give up on the abstraction of a migration step and just rely on
helper functions to get the same result. The migration code is now
shorter and we don't need to update it when adding new columns.

This is a preparatory work for #211 to allow a user to specify
additional fields to collect.
2023-01-02 23:42:05 +01:00