@sdairs

Product @ ClickHouse Thinking about agentic experience & identity

alasdairb.com
Joined April 2015
  • Tweets43
  • Following43
  • Followers374
1/5 ClickHouse 26.10 can reject a string during deserialization, before copying its bytes into a column. A selective URL LIKE predicate can save materialization work as well as filtering rows.
1
32
more replies
4/5 The filter disables itself when the checks aren't paying for themselves, including when too many values match or skipped strings are too short. apply_string_filters_during_scan is enabled by default in 26.10.
1
4
5/5 Inspect StringValueFilterBytesSkipped and peak memory as well as time. Avoiding copies can produce a substantial memory improvement even when the query takes roughly the same time.
3
1/5 ClickHouse 26.10 adds written_rows to ingestion quotas. A public event endpoint can set a budget in rows per minute alongside query_inserts and written_bytes, instead of using payload size as a proxy for event count.
1
1
66
more replies
4/5 A chunk that crosses the quota is still charged, then raises QUOTA_EXCEEDED, so usage can show 120,000 against a 100,000 limit after a rejected insert. It counts attempted chunks, not just committed rows.
1
5
5/5 Once the quota is exceeded, later inserts can fail at the pre-check. A multi-chunk insert still needs its usual failure handling; this is an ingestion allowance, not an all-or-nothing batch transaction.
4
1/5 ClickHouse 26.10 can reduce hash-table memory for large GROUP BY and uniqExact queries by leaving fewer slots empty. Load factor means the fraction of allocated slots that hold keys.
2
2
6
312
more replies
4/5 The trade-off is extra CPU work for inserts and lookups. Highly parallel queries waiting on random memory access can benefit. With fewer threads, that extra work can make queries slower.
1
12
5/5 This covers large two-level GROUP BY tables, uniqExact sets and parallel hash joins. StringHashMap is unchanged. Buffers grow in steps, so savings depend on key count. Other query allocations still need memory.
10
1/5 ClickHouse 26.10 adds friendlier functional syntax for array transformations. With lower | reverse, the order you read matches the order the functions run, with less nesting and less counting brackets.
1
1
59
more replies
3/5 Placeholders make short lambdas smaller too. arrayMap(plus(_, 1) | multiply(_, 2), [1, 2, 3]) gives [4, 6, 8], adding 1 then multiplying by 2. _1 and _2 can refer to separate arguments.
1
14
4/5 This is syntax sugar over ordinary lambdas, rewritten by the analyser during query analysis, so normal typing, captures and constant folding still apply. Generated argument names avoid accidental capture.
6
1/5 ClickHouse 26.10 has an experimental columns cache. It keeps columns after decompression and deserialization, so another query can reuse the in-memory column objects.
1
1
46
more replies
4/5 For MergeTree, it currently covers wide parts in databases with table UUIDs. Compact parts do not use it. use_columns_cache = 1 enables it explicitly; the feature is experimental and off by default.
1
5
5/5 Inspect system.columns_cache and ColumnsCacheHits / ColumnsCacheMisses. Compare warm queries with the cache on and off. Cache lookups and memory traffic can outweigh the decoding work they save.
6
Load more