feat(clickhouse): P0 + P1 of the 4-month coverage-gap audit (items 0a, 0b, 1-18)

[fm4v]

Implements the P0 and P1 tiers of docs/plans/2026-08-15-001-feat-clickhouse-4month-coverage-gap-plan.md, the coverage-gap audit of everything ClickHouse merged between 2026-04-15 and 2026-08-15. P0 (items 0a, 0b, 1-6) landed in the first three commits and was validated on head 26.8.1.1470; the fourth commit adds all of P1 (items 7-18).

Why

The audit was grounded in a measured miss: on 2026-08-13 a 10-minute run of upstream SQLancer found a silent wrong result that this fork's 98 oracles had never surfaced, because the gap was in the predicate grammar rather than in the oracle set. Three structural patterns explain the rest of the gaps: sound oracles starved by a narrow grammar, deliberate float avoidance that has become a blind spot, and whole young subsystems (plan-based parallel replicas, the new join-order enumerators, pipe operators, IEJoin, GROUPS frames, the second text-index wave) that the fork could not generate at all.

What is in P1 (items 7-18)

Three new oracles, all wired into ALL_ORACLES:

Oracle	Invariant
PipeEquivalence	one single-relation read rendered in classic SQL and in pipe-operator syntax (PR #111151) must return the same rows
IEJoin	a join whose ON carries two inequalities (PR #109920) must equal the equivalent CROSS JOIN ... WHERE
TupleFinalAggregation	per-element Tuple aggregation in SummingMergeTree and CoalescingMergeTree (PR #98039) == a Java element-wise ground truth, and query-time FINAL == a physical OPTIMIZE ... FINAL

Generator work. GROUPS window frames plus explicit ROWS/RANGE frames (the window generator previously emitted no frame clause at all); negative LIMIT / LIMIT BY / WITH TIES; long LIKE-OR and !=-AND predicate chains so optimize_or_like_chain and optimize_and_compare_chain are reachable at all; uniq_v2 and basic statistics everywhere plus an ADD STATISTICS form and auto_statistics_types table settings; descending and mixed-direction sorting keys; sparse serialization that actually engages (low ratio_of_defaults_for_sparse_serialization plus default-biased inserts on plain-MergeTree tables, verified through system.parts_columns); the icu tokenizer.

Oracle work. A GROUPS peer-group ground truth and the two degenerate GROUPS identities; negative-limit reversal identities; an indexHint containment arm; query-condition-cache ORDER BY ... LIMIT coverage, the two open poisoning shapes as triggers, and a re-verify-after-DROP step that separates a genuine cache bug from a merge artifact; hasPhrase with a token-position ground truth, a trivial-count-from-text-index arm, and text index parameters supplied as table settings; parallel_full_sorting_merge in the join-algorithm sweep; multi-key GROUP BY and mixed-direction ORDER BY in the read-in-order sweep.

Every new surface is behind a default-on flag (--groups-window-frame-emission, --negative-limit-emission, --comparison-chain-emission, --index-hint-emission, --sparse-column-emission, --mixed-direction-sorting-key, --text-index-second-wave, --pipe-equivalence-oracle, --ie-join-oracle, --tuple-final-aggregation-oracle) plus one deliberately default-off arm, --summing-subset-projection-arm.

Validation

Dev VM, fresh clickhouse/clickhouse-server:head.

• 30 minutes, all 97 oracles except TextIndexDirectRead, head 26.8.1.1473: 162,814 queries, 0 reproducers, 0 threads shut down.
• Every new arm was confirmed to actually execute by counting it in system.query_log for that run, not just by the absence of failures: 7322 GROUPS frames, 8982 trivial-count-from-text-index reads, 4005 hasPhrase, 2092 ie_join joins, 1046 tuple-aggregation reads, 922 pipe queries, 685 negative LIMITs, 623 uniq_v2 statistics statements, 562 chain-rewrite flips, 414 indexHint reads, 246 icu tokenizer statements, 90 parallel_full_sorting_merge joins, 6092 columns in Sparse serialization.
• A preceding 15-minute run of only the new and changed oracles did 80,962 queries and produced exactly two reproducers, both fixed here: a DROP STATISTICS versus in-flight-mutation DDL race (now tolerated) and the indexHint constant-truncation bug below.
• 337 unit tests pass. Three test expectations left stale by the P0 batch's comma-join rendering are fixed in the same commit.

Two unfiled ClickHouse wrong results found on the way

1. optimize_aggregation_in_order collapses every GROUP BY group over a DESC sorting key. Exactly what item 18 was built to find. Needs a single part; type-independent. Same family as the open #111901, which is filed for the two-column case, but this single-column form is strictly smaller and is not on that issue.

CREATE TABLE t (c Int64) ENGINE = MergeTree ORDER BY c DESC;
INSERT INTO t VALUES (1),(2),(3);
SELECT c, count() FROM t GROUP BY c SETTINGS optimize_aggregation_in_order = 1;  -- 1 row: (3,3) WRONG
SELECT c, count() FROM t GROUP BY c SETTINGS optimize_aggregation_in_order = 0;  -- 3 rows, correct

2. An integer constant inside indexHint is narrowed to UInt8 during index analysis, so any multiple of 256 reads as false, the key condition becomes unsatisfiable and every granule is pruned. Same family as #112236.

CREATE TABLE t (c0 Int64) ENGINE = MergeTree ORDER BY c0;
INSERT INTO t VALUES (1),(2),(3);
SELECT count() FROM t WHERE 256 AND c0 > 0;             -- 3
SELECT count() FROM t WHERE indexHint(256) AND c0 > 0;  -- 0, must be 3
SELECT count() FROM t WHERE indexHint(1) AND c0 > 0;    -- 3, so it is the value, not indexHint itself

Five plan assumptions turned out to be wrong

Each was caught by a five-minute probe against a fresh head rather than by reading a PR description, and each changed the shipped design:

• indexHint is not result-neutral. It does not evaluate its argument as a filter, but it does restrict the read to the granules index analysis selects, so rows outside them are legitimately dropped (5 rows versus 6 on head). The shipped invariant is containment, rows(P AND Q) ⊆ rows(indexHint(P) AND Q) ⊆ rows(Q), whose lower bound is the pruning-soundness assertion the item actually wanted. For the same reason indexHint is deliberately not emitted into the general fleet: inside a TLP partition the three branches would read different granule sets and their union would no longer be the whole table.
• ASC-to-DESC is not an order reversal. ClickHouse sorts NULLs last in both directions, so the negative-limit identities pin ASC NULLS LAST against DESC NULLS FIRST.
• Pipe operators wrap every stage in a subquery. A table-qualified column reference stops resolving after stage 1, and SELECT * does not carry MATERIALIZED or ALIAS columns. Ignoring either rule produced 1184 Code-47 reproducers in one run.
• IEJoin has nothing to sweep against. The value is ie_join, and every other algorithm rejects a two-inequality ON outright, so the reference arm is the equivalent cross join.
• null_count statistics do not exist on head (accepted: basic, countmin, minmax, tdigest, uniq, uniq_v2), and two item-13 sub-items are unreachable: the Japanese tokenizer needs a server-side dictionary, and no posting-list apply-mode setting exists.

All of this is written up in the plan's new "P1 implementation status" section and in the ## P1 coverage batch, 2026-08-16 section of .claude/CLAUDE.md, so a future run can triage against it.

Still open

P2 (items 19-25) of the same plan.