Atnalya 26.6 EXPORT PARTITION with position matching + extra columns in source

[k-morozov]

From 26.3 #2111

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Added the ignore_extra_source_columns_by_name mode for the export_merge_tree_part_schema_mismatch_mode setting. EXPORT PART/EXPORT PARTITION matches destination columns to source columns by name instead of position, so the source table may have extra columns in any position and the columns may be reordered.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [c162ded]

[mkmkme]

@blau-ai

[blau-ai]

CI triage for #2229

Verdict: 5 failing checks, 0 caused by this PR. Everything red is pre-existing on antalya-26.6, flaky, or infra. The two Stateless tests failures are even marked non-blocking (do_not_block_pipeline_on_failure). Net-net this PR is healthy — and it actually improves the iceberg regression suite (see the health check at the bottom).

Analyzed against head c162dede (run 32128125581).

---

1. Stateless tests (amd_debug, parallel) — 2 failed — not PR-caused (randomized-settings flakiness)

• 00071_merge_tree_optimize_aio and 03595_extract_url_parameters, both timeouts.
• praktika's own diagnosis for both: "Passes without randomization. Confirmed: the failure is caused by randomized settings." (52/52 and 71/71 clean reruns without the random settings), labeled setting.
• Unrelated to EXPORT (merge-tree AIO read + URL-function parsing). Safe to re-run.

2. Stateless tests (amd_debug, distributed plan, s3 storage, parallel) — 1 failed — not PR-caused (flaky)

• 01509_check_parallel_quorum_inserts_long: Code: 319 ... UNKNOWN_STATUS_OF_INSERT ... Timeout while waiting for quorum.
• praktika diagnosis: "All reruns passed (19/19). The failure is not reproducible.", labeled flaky. Well-known flaky replicated-quorum test; PR doesn't touch ReplicatedMergeTreeSink. Safe to re-run.

3. SQLLogic test — not PR-caused (infra / memory)

• Gate message: total tests 5,655,727 < minimum 5,939,581; ... 21,041 new failures detected (the failed-ratio gate itself passed: 173,883 <= maximum 174,004).
• All 21k "new failures" are one file, random/select/slt_good_103.test, failing with Code: 241 ... memory limit exceeded ... maximum: 26.00 GiB — i.e. the runner hit its memory cap and that file aborted, dragging the total-tests count below the minimum threshold.
• Environmental, nothing to do with EXPORT/schema. Re-run; if it recurs it's a runner-memory issue for that random file, not this PR.

4. Regression release iceberg_2 — 8 scenarios — not PR-caused (pre-existing on antalya-26.6)

All 8 leaf failures are:

/iceberg/export partition/no catalog/replicated merge tree/datatypes/export partition id/accepted/accepted﹕ {Int8, UInt8, Bool, Decimal(10,2)}
/iceberg/export partition/no catalog/replicated merge tree/datatypes/export part/accepted/accepted﹕        {Int8, UInt8, Bool, Decimal(10,2)}

Underlying error: Code: 36. DB::Exception: Unsupported type for iceberg Int8 (BAD_ARGUMENTS). That comes from Iceberg type-mapping code, which this PR does not touch — the diff is purely the column-count-mismatch setting (default strict = no behavior change), plus docs/gtest.

Proof it's pre-existing: unrelated PR #2239 ("Fix iceberg partition pruning by nanoseconds", which touches no export-partition code) shows the identical 8 failures and identical totals — 1902 scenarios (1845 ok, 8 failed, 26 skipped, 23 xfail) — with the same Unsupported type for iceberg error (its iceberg_2 run).

These are a pre-existing product gap: the regression suite marks Int8/UInt8/Bool/Decimal(10,2) partition columns as expected-to-be-accepted for Iceberg export, but ClickHouse rejects them as Iceberg partition types. Worth a separate issue for the export/iceberg team (either support these types or mark the cases xfail); it's out of scope for this column-count PR.

5. Regression release swarms — 1 scenario — not PR-caused (flaky/infra)

• /swarms/feature/node failure/check restart swarm node — swarm node-restart resilience, unrelated to EXPORT. Also failed on the predecessor PR Antalya 26.6 Export partition - check name and pos of columns in partition key #2218. Flaky node-restart timing. Safe to re-run.

The top-level PR check is just the aggregate of the above.

---

Health check

The diff is tight and low-risk. It adds export_merge_tree_part_schema_mismatch_mode (default strict, so no behavior change unless the user opts in), gates the trailing-source-column trim behind the non-default mode in both ExportPartTask::addExportConvertingActions and ExportPartitionUtils, threads the value through ExportReplicatedMergeTreePartitionManifest with sensible back-compat (absent field ⇒ strict), and covers it with settings-history + docs + a gtest.

Strong positive signal: on the predecessor #2218 the iceberg_2 suite had 239 failing scenarios including all the settings/schema mismatch mode/* cases (strict rejects extra source columns, ignore extra source columns by position drops trailing columns, ... rejects extra destination columns, ... still rejects type mismatch) — because the feature didn't exist yet. On this PR those all pass, and iceberg_2 is down to the 8 unrelated pre-existing datatype failures. That's direct end-to-end evidence the feature works.

Recommendation: no code changes needed for the red CI. Re-run the flaky stateless/SQLLogic/swarms jobs; the iceberg_2 datatype failures can be tracked separately and don't block this PR.

(Note: I can't build or run ClickHouse in this environment — this is entirely from the praktika S3 reports and CI logs. Final correctness is validated by CI.)