现象:

自助ETL抽数到clickhouse中时,抽取失败,报错:
ClickHouse exception, code: 252, host: 10.28.3.101, port: 8123; Code: 252, e.displayText() = DB::Exception: Too many partitions for single INSERT block (more than 100).
The limit is controlled by 'max_partitions_per_insert_block' setting. Large number of partitions is a common misconception. It will lead to severe negative performance impact, including slow server startup,
slow INSERT queries and slow SELECT queries. Recommended total number of partitions for a table is under 1000..10000. Please note, that partitioning is not intended to speed up SELECT queries (ORDER BY key is sufficient to make range queries fast). Partitions are intended for data manipulation (DROP PARTITION, etc). (version 21.6.6.51 (official build))

原因:

同时存在太多的insert操作导致异常,clickhouse 要求每次写入的数据不能跨越特别多的 partitions,具体受参数 max_partitions_per_insert_block 控制,调整该参数即可。

解决方案:

修改调整max_partitions_per_insert_block参数,如配置文件中没有该配置,则新增一条;

相关参数配置项说明可参考如下,类似参数报错同时可采用此方法:
https://blog.csdn.net/chy555chy/article/details/121664246