Adapter Extensions for Clickhouse

[Pipboyguy]

Feature description

The Clickhouse adapter is limited, and should be extended to include, among others settings:

• Storage policy
• Compression codec
• Sorting key
• Clustering key
• Partitioning key

Are you a dlt user?

Yes, I'm already a dlt user.

Use case

Suppose the user wants to specify details around the example_table resource, in order to compress message column, partition on timestamp:

CREATE TABLE example_table
(
    timestamp DateTime,
    user_id UInt32,
    message String CODEC(ZSTD),
)
ENGINE = ReplicatedMergeTree
ORDER BY (user_id, timestamp)
PARTITION BY toYYYYMM(timestamp)

Proposed solution

clickhouse_adapter(
	data, 
	compression_codec={"message": "ZSTD"}, 
	partition="col2"
)

Related issues

See this community comment: #1055 (comment)