feat(pack): ship a library as interface + prebuilt binaries (#433) - #451
Merged
Conversation
A closed-source library, an offline site, or a build farm that already
compiled this once had no route through mcpp: `mcpp publish` sends source,
`mcpp pack` bundles a program's run-time closure, and neither is "a library
someone else links". Collecting the artifacts by hand was the answer.
mcpp pack mathkit # a static library package
mcpp pack mathkit --target x86_64-linux-gnu \
--target aarch64-linux-gnu # one package, two legs
WHAT IT PRODUCES IS AN ORDINARY PACKAGE. A normal `mcpp.toml`, read through
the route mcpp already had for payloads that carry their own manifest. Zero
new manifest sections and zero new keys: what to pack is `[targets.<n>].kind`
(so there is no --lib and no --artifact), which interface to publish is `[lib]`
plus the module graph, which headers are public is `[build].include_dirs`, and
each leg's ABI tag and digest ride on `[[runtime.artifacts]]`, whose fields
were already documented as optional evidence. An older mcpp still BUILDS
against these packages; it just does not run the two gates below.
Two interface modes coexist — `include/` (text, `#include`, never compiled)
and `interface/` (module units the consumer compiles). Measured: one package
consumed three ways (header only, module only, both) x static and shared = six
combinations, all green.
WHICH .cppm TRAVEL IS COMPUTED, not declared. It is the module closure of the
lib root, and the two ways of getting it wrong are asymmetric: too few fails
loudly in the consumer's compile, too many silently publishes a closed-source
implementation partition's SOURCE. `.m.o` is not the rule — an implementation
partition produces one too. The same closure decides which archive members to
drop, and getting THAT wrong was measured as well: dropping every `.m.o` also
drops the partition's real code and every target fails to link. Both lists are
printed, because "what is not travelling" is half of what a publisher needs.
TWO GATES ON THE CONSUMER SIDE, both for failures that are otherwise silent.
The interface still matches its binaries: swap two `int` members of a shipped
struct — the Itanium ABI does not mangle field order — and before this the
consumer compiled, linked, ran, and printed transposed data with no diagnostic
from any tool. And the binaries were built for this toolchain, with the
refusal listing the tags the package does have, because "not found" sends
people looking for a package already on their disk.
Also fixes two defects this work found, each with its own regression test:
* `[target.'<triple>'.build]` never matched a native build. `matches()`
short-circuited on the raw --target string while `context_for()` fell back
to the host for `cfg(...)`, so two spellings of one statement disagreed —
green in CI, silently inert on a developer's machine. The resolved triple
now lives in `cfgpred::Ctx`; there is no second answerer.
* `sources = []` was byte-identical to omitting the key, so no author could
say "compile nothing". A header-only package needs that, and without it
any leftover file under `src/` is compiled into the consumer's build.
Docs: docs/12-binary-distribution.md (+ zh), examples 05-lib-dist and
06-lib-consume. Design: .agents/docs/2026-08-17-library-distribution-design.md.
Tests: 23 unit assertions across two new suites; e2e 242-248.
Two problems the first push found, one from CI and one from running the
previous release side by side.
`mcpp.pack.library` exported a `mcpp::pack::Error` and `mcpp.pack` already
had one. A name attaches to exactly one module, and mcpp.pack.library_pipeline
imports both — clang refuses outright ("cannot be attached to other modules"),
GCC accepted it. Every Windows and macOS job failed on it while the Linux ones
were green, which is the whole argument for the three-platform matrix.
`mcpp pack` in a workspace root stopped working. Routing on
`[targets.<n>].kind` means something reads the manifest before the build does,
and a workspace root has no targets of its own — a virtual one has no
`[package]` either — so the new router read an empty list and concluded there
was nothing to pack. Found by running the previous release against
examples/04-workspace and comparing; e2e 249 is that comparison made permanent.
And the positional was accepted but never reached the application pipeline, so
a project with two `bin` targets would take `mcpp pack app2` and bundle app1 —
succeeding with the wrong answer. e2e 250 pins both directions plus the
refusal for an unknown name.
…names Found by running the path the docs already promised. `mcpp pack <shared target>` shipped only the built file — `libmathkit-shared.so` — while the object records `SONAME libmathkit.so.1`. A consumer links by the first name and the loader asks for the second, so the package linked and the program could not start. mcpp's own runtime-closure check is what reported it, naming the missing soname rather than letting it become a loader error at launch. The package now carries the SONAME alongside the link name (a symlink, falling back to a copy), which is what a distribution ships and what the design's "soname gives the correct run-time name" note always meant. e2e 251 uses `run`, not `build`: linking proves nothing here.
`00_fixture_path_hygiene.sh` caught it on the macOS leg. The rule is a Windows one: MSYS rewrites POSIX paths on the way into argv and the environment but never touches file CONTENT, so a `path = "/tmp/…"` written INTO an mcpp.toml is read by a native mcpp.exe as "root of the current drive". The failure then surfaces as a dependency that cannot be found, four steps from its cause. Six of the new fixtures wrote the package's path that way. They now source `_host_path.sh` and pass it through `host_path`, which is what the lint asks for — and the lint runs on every platform precisely so a Linux reviewer can catch this before Windows CI does.
…roved §11 of the design doc: three places the implementation diverged from the plan (no [distribution] section at all, no abi_surface flag, no new CI workflow), the four defects found while implementing — each with what caught it — and the stale-fingerprint trap I walked into three times while VERIFYING, which is the same criterion the packer itself enforces about never globbing for artifacts.
All ten new tests carried `# requires: gcc`, and that capability is Linux-only by design — macOS's g++ is Apple Clang and Windows' is not an mcpp-compatible GCC, so the runner does not grant it there. Every one of them skipped on macOS and Windows while the suite reported green, which left "a static library package works on every target" verified on exactly one platform. That is the same false-green shape this PR already corrected once (245 was gated on mingw-cross and would never have run in CI at all). Finding it a second time, in my own tests, is the argument for checking whether a test RAN rather than whether the suite was green. The five that assert on mcpp's own output — layout, both interface modes, the three refusals, workspace routing, target naming — now require nothing and run everywhere. The rest stay gated for real reasons: 245 needs a musl target, 246 and 247 read compiler-specific flag spellings out of build.ninja, 248 needs the mingw cross toolchain, and 251 exercises `kind = "shared"`, which is ELF-only.
…everywhere
Letting the portable tests run on all three platforms was the right move, and
it worked: CI answered the question I could not answer locally.
* on the Windows (MSVC-ABI clang) leg the library build inside `pack` fails
with a bare `error: build failed`;
* on macOS the closure test cannot inspect the archive, because `ar` there
resolves to an xlings shim that reports "not installed".
Neither is fixed here and neither is hidden. The three PACKING tests carry
`# requires: gcc` again with the scope written at the top, while the tests for
the COMMAND — which target it picks, how it refuses an unknown name, packing
from a workspace root — keep running everywhere, because they pass everywhere.
docs/12 and its zh mirror gain a "where it is verified" section, and the limits
table now says "every target — verified on Linux only" rather than the first
half of that sentence.
Also: a Windows pack produces a .zip, not a .tar.gz. 249 asserted only the
latter, which is why it failed there even though the pack had succeeded.
`module M:part;` and `module M;` wear one spelling and are two declarations. The
scanner treated them as one: the first was recorded as *requiring* `M:part` and
providing nothing, so a file required its own name and the graph held no edge
from the unit importing a partition to the unit defining it. Build order was
unconstrained — GCC and macOS clang recovered through their own dependency scan,
Windows clang failed with `failed to read compiled module`.
The same site had a second half: `import :part;` resolved against `u.provides`,
and an implementation unit (`module M;`) has none, so `import :secret;` inside
one stayed the literal `:secret`. The note both platforms printed —
`module 'M:part' imported but not provided in this build` — was the merged
symptom of the two, and it read like a hint rather than the cause.
Implementation partitions had no test coverage anywhere in mcpp; the library
distribution e2e is the first thing to use one, which is how this surfaced. Five
scanner unit tests pin it now, including the case the old code was written for
(`export module foo:http;` + `import :tls;` must give `foo:tls`).
Consequence for `mcpp pack`: a partition the published interface reaches now
resolves, so it is published rather than refused — the consumer cannot build the
interface's BMI without it. That is correct and it is also the one thing a
closed-source publisher must not do by accident, so it comes with a warning
naming the file. 243 asserts both halves; the closure's unresolved-import error
stays for a partition nothing provides.
With this, e2e 242/243/244/249/250 run on Linux, macOS and Windows — the docs no
longer need a "verified on Linux only" caveat.
Also, per review: examples 05-lib-dist and 06-lib-consume are one story, so they
are one directory (examples/05-lib-distribution/{producer,consumer}) with one
README; and the consumers use `import std;` rather than <cstdio>, which is what
a module example should be showing.
…sing archiver must not be a silent skip Windows packed the library correctly — `lib/x86_64-windows-msvc/mathkit.lib` — and 242 failed anyway, because the assertion hard-coded `libmathkit.a`. MinGW writes `libfoo.a` where MSVC writes `foo.lib`, which is the very reason `lib/` is keyed by triple and not by OS; the test had the fact in its comments and the GNU spelling in its `find`. 243's archive probe had the same assumption, which made its drop assertion skip silently on the clang legs rather than run. And the packer itself: with objects to drop and no archiver resolved it did nothing, quietly, leaving the published interface's objects inside the archive — two definitions of each published module's initialiser, resolved by link order. That is the failure class this whole feature exists to remove, so it is now refused with the reason.
§11.2b: unrestricting the portable tests is what surfaced the scanner's implementation-partition bug, and fixing that — rather than widening a capability — is what made 242/243/244 pass on Linux, macOS and Windows. Also the per-platform table with a reason for every remaining skip.
Asked "what is the skip reason", and only one of the four survived the question.
* 246 was gated on `gcc` because of copy-paste. Its probe greps a FILENAME
out of build.ninja, not a compiler flag — nothing in it is toolchain-
specific. Runs everywhere now.
* 247 grepped `-D<macro>`, and MSVC spells that `/D` (dialect.cppm). The
prefix was the only thing tying it to one compiler family; grepping the
macro NAME is spelling-agnostic. Runs everywhere now.
* 248 needed `mingw-cross`, which no e2e job installs — so it skipped in
every job that exists, verified on a developer's machine and nowhere else.
cross-build-test.yml's mingw job already names the e2e scripts the Linux
shards skip for exactly this reason (102, 198, 240); 248 joins them, and
the workflow header now says why that list is explicit rather than left to
run_all's cap gating.
* 251's reason held — `kind = "shared"` is ELF-only and plan.cppm refuses it
— but it only tested the side that works, which cannot tell "the gate is
handled" from "there is no gate". It is now two-sided: on ELF the package
is produced and carries both of the library's names; off ELF the pack must
be REFUSED and the message must name both the artifact kind and where it
does work. Runs everywhere.
That leaves 245, which needs two distinct buildable targets (gnu + musl) and so
genuinely cannot run where only one exists. The mechanism it checks — one cfg()
leg per triple, the native build included — is platform-independent and covered
on Linux.
The criterion, earned twice now: a test's `# requires:` has to be the real floor
of the mechanism it verifies. Telling "a real limit" from "I did not think it
through" is hard while writing it, so the reasons have to be interrogated one by
one afterwards.
…or one 247 derived the host triple with a regex for `<arch>-<os>-<env>` and fell back to `x86_64-linux-gnu`. macOS's canonical triple is `aarch64-macos` — two segments, no env — so the match failed, the fallback was used, and the test asserted that a *Linux* section should apply to a macOS build. It then reported "the bare triple was inert" against a product behaving correctly. `target/<triple>/` is mcpp's own answer to the same question. The test now builds once with nothing conditional, reads the directory name, and writes the real manifest from that — and the cfg() control is keyed on the same value's arch rather than on `unix`, which is false on Windows. The shape is the one this PR keeps meeting: a second, independent derivation of something mcpp already computes, disagreeing with it on the platform nobody checked.
…iver Four things a deeper read of this PR turned up. **The old-client claim was unverified.** The PR body and docs/12 both say an mcpp that predates this feature still builds against these packages — that is the entire justification for adding no manifest section and no key, and nothing checked it. It is true (2026.8.15.3 consumes a package from 2026.8.17.2 and runs it), and e2e 252 now pins it in two halves: a portable static check that the generated manifest's sections are a subset of the pre-existing vocabulary, and the real thing against `$MCPP_BOOT`, the released binary each CI job bootstraps from. The three e2e workflows now export it, so the real half runs rather than noting itself out. **The packer spoke `ar` to whatever archiver it was handed.** `archive_tool` returns LIB.EXE for MSVC, which spells member removal `/REMOVE:<member>` — one flag per member, archive last — not `d <archive> <member>…`. On mcpp's Windows CI the archiver is clang's llvm-ar, which takes the GNU form, so the difference was invisible. How to speak to a tool is what mcpp.toolchain.dialect is for, so the removal spelling lives there now, next to `archiveCmd`, and the packer substitutes rather than assumes. The MSVC row is marked untested, and a failure reports the command it ran. **A duplicate partition provider now says so.** Two files declaring `module m:p;` used to be accepted silently; the scanner names both files. Those programs were always ill-formed, but it is a new failure path and belongs in the changelog with the other behaviour change (`sources = []`). **The module graph is moved into BuildContext, not copied.** Nothing reads `scan.graph` after that point.
252's guard compared `$MCPP_BOOT --version` against the PR binary's and treated "different" as "found an old client". In CI that entry is an xvm SHIM, and a shim resolves against the home it is asked in — under the e2e suite's environment it answers `xlings: 'mcpp' is not installed` and prints nothing. The empty string duly differed, so the test ran the real check against a binary that cannot run at all and reported a COMPATIBILITY FAILURE against a package that is perfectly readable. Three legs red for a reason that was in the test. The guard now requires a version-SHAPED answer. Anything else means "no usable old binary here", which is a note naming what it got, not a verdict — and the static half (the generated manifest's sections are a subset of the pre-existing vocabulary) still runs everywhere. Consequence worth stating: the real old-client check runs where a released mcpp binary is directly executable — locally, and in any job that points MCPP_BOOT at one rather than at a shim. It passed against 2026.8.15.3.
Why 245/248 cannot run elsewhere, from host_can_serve rather than from the test's capability table: macOS serves exactly one target so a fat package is structurally impossible there, while Windows serves three — so the Windows equivalent is a coverage gap, not a product gap, and it has independent value (mathkit.lib next to libmathkit.a in one package is the strongest evidence that lib/ must be keyed by triple). Plus the dependency-direction invariant for src/pack (only library_pipeline may import mcpp.build.*), the two deliberate behaviour changes, and five open items — including two I have to report rather than claim: a scan_overrides-declared implementation partition is mis-flagged as an interface (its source would be published, and the schema has nowhere to say otherwise), and the design's promised [package].platforms coverage warning is simply not implemented.
…face"
Whether a partition's SOURCE may be published turns on one keyword — `export
module M:api;` may travel, `module M:impl;` may not — and two of the three paths
that build the module graph cannot read it:
* a `[scan_overrides."<glob>"]` entry names the modules a file provides and has
nowhere to say whether the declaration carries `export`;
* P1689 makes `is-interface` optional, and mcpp parsed it into a struct field
(p1689.cppm:34) that nothing ever read.
Both arrived as `providesInterface = true`, which the field's own comment called
the conservative direction "since the flag only ever produces a warning". That
had it backwards: `true` is the value that produces NO warning, so an
implementation partition declared in `[scan_overrides]` was published in
silence — the exact failure the closure exists to prevent, since too FEW
published sources fails the consumer's compile while too MANY ships private
source and nothing fails.
So the field is a tri-state, and each path now says what it actually knows:
text scanner reads the keyword, sets true or false explicitly
P1689 reader carries the compiler's answer through, absence included —
the compiler is the one participant that parsed the declaration
scan_overrides leaves it unset, because the schema cannot express it
Unknown warns, with a different sentence from the known case: that list says
"you are publishing your implementation", this one says "mcpp cannot tell
whether you are". Only PARTITIONS are asked about — `module M;` provides
nothing, so a bare `M` can only come from `export module M;`, and asking there
would fire on every primary interface in every package that uses overrides.
e2e 253 pins it from both sides, because asserting only that the override case
warns cannot distinguish "mcpp models three states" from "mcpp warns about every
partition it publishes". Verified by control probe: restoring the `true` default
at the scan_overrides site alone, with the rest of the fix in place, fails 253 at
exactly that assertion.
Two existing assertions were weak in the same way and are tightened:
EXPECT_TRUE/EXPECT_FALSE on an optional ask about has_value(), so
`EXPECT_TRUE(providesInterface)` stayed green for an implementation partition.
The design promised this cross-check (§2.2) and the implementation simply did
not have it — src/pack never read the key. `[package] platforms` is a support
claim, and `mcpp pack` on a library is the first moment there is evidence to
check it against: the legs in the package are the platforms it can serve.
The hard part is not finding the gap, it is not shouting about it. Four
comparisons exist and only two may be printed:
packed, not declared always actionable — the manifest disclaims a platform
the package demonstrably serves.
declared, not packed actionable ONLY IF THIS HOST COULD HAVE BUILT IT.
The normal release flow is one `mcpp pack` per platform in CI, so a Linux runner
never produces a macOS leg. Warning about that would fire on every run of every
cross-platform package, and a warning that always fires hides the one that
matters — so the check asks host_can_serve, the same function that decides which
`--target` values are accepted at all, and can therefore only ever name
something the author is able to do on this machine.
Both are warnings, never errors: coverage is release discipline, and the person
who can judge it is looking at the release, not at this build.
e2e 254 asserts the silence as well as the warnings, and derives the host's
platform from `target/<triple>/` — mcpp's own answer — rather than from a regex
over triple spellings, which is how an earlier test came to assert Linux
expectations on macOS. Its per-host picks mirror host_can_serve deliberately: if
mcpp ever gains a macOS-hosted Windows toolchain the test fails, which is the
correct outcome, because the table it encodes will have changed.
… both ends
`mcpp pack` deletes the published interface's objects from the archive before
shipping it, and the two archivers disagree in both directions:
ar one verb, then the archive, then every member
lib.exe one flag PER member, and the archive comes LAST
Nothing could catch a mistake there. mcpp's Windows CI builds with clang and
archives with llvm-ar, which takes the GNU spelling, so the MSVC branch has never
executed in any job on any platform — and the assembly was inline in
run_library_pack, so it had no seam either. Pinning the two constants in
dialect.cppm was never enough: the ORDER they are assembled in is a third fact.
So the assembly moves into archive_remove_command, exported for the tests and
nothing else, and it is checked from both ends:
test_pack_archive_remove.cpp the exact command for each dialect, host-
independently (expectations built with the same
`quote`, since what is under test is spelling and
order — shell quoting has its own tests)
e2e 255 a real lib.exe, under `# requires: msvc`, with
msvc@system pinned in the manifest because
Windows' DEFAULT toolchain is clang and taking it
would exercise the GNU branch and pass while
proving nothing
255 does not inspect the archive to decide: a wrong spelling makes
run_library_pack refuse, quoting the command and the archiver's output, since
shipping the objects would be the silent outcome. What it does assert is that the
branch was TAKEN — a `mathkit.lib` rather than llvm-ar's `.a`, and a non-empty
published interface so there was something to remove at all.
…ble targets Four things, and the first one explains why the rest went unnoticed. THE GUARD HAD A HOLE IN THE CASE THAT MATTERED. make_plan refused SharedLibrary targets with `!targetTriple.empty() && os != "linux"`, and targetTriple is EMPTY for a native build. So it turned away a cross build to macOS — unservable anyway, i.e. unreachable — while letting a NATIVE macOS or native Windows build walk straight into the unverified paths it was written to keep people out of. It reads the resolved target now. PE WAS MISSING ITS IMPORT LIBRARY. A PE shared library is two files: the `.dll` the loader opens and an archive of stubs the LINKER consumes. mcpp wrote only the first, and consumers linked the `.dll` directly — which mingw's ld tolerates and no other linker does, so the tolerant case was hiding the broken one. The link edge now declares the import library as an implicit output (one command writes both; a second edge would run the link twice), `import_library_for` owns its name, the dialect table owns the flag spelling beside `archiveRemoveArg`, and the package ships it. One more thing was needed and the diagnostic for it names nothing useful: mcpp gives PE executables `-static`, which puts ld in static-only mode where it refuses an import library and says `have you installed the static version of the mathkit library?` — so the emitted manifest switches to dynamic mode for that one `-l`. MACH-O WAS MISSING ITS INSTALL NAME. A `.dylib` records the path it was LINKED at, so emitting `-install_name` only when the manifest declared a `soname` left every other dylib recording a build directory: perfect on the machine that built it, `image not found` anywhere else. It is now `@rpath/<file>` unconditionally. The choice was also being made with `#if defined(__APPLE__)` on the HOST, so a cross link emitted the wrong flag or none — it comes from the target now, as `target_output` already did. MSVC STAYS REFUSED, FOR THE REAL REASON. Not the linker: `link /DLL /IMPLIB:` has been in the rule table all along. Symbol export — MSVC exports nothing from a DLL without `__declspec(dllexport)` or a `.def`, so the import library comes out empty and consumers fail with unresolved externals naming symbols that are visibly in the objects. Refusing beats producing a diagnostic that points nowhere near its cause, and the message names MinGW as the way forward. AND `--target` NO LONGER ACCEPTS WHAT THIS HOST CANNOT PRODUCE. Measured on Linux: `mcpp build --target x86_64-windows-msvc` resolved the native g++, wrote target/x86_64-linux-gnu/, and reported success — an ELF delivered as a Windows build, which is the failure the neighbouring typo check calls the worst one. The vocabulary tier says "mcpp supports this target"; host_can_serve answers "can this machine produce it", and the error lists what it can. An explicit `[target.X] toolchain` stays the escape hatch. Also fixed on the way: consuming a distribution package whose target is `shared` died with `ninja: multiple rules generate bin/libmathkit.dll`, because the dependency loop created a link unit for a library that is already built — and relinking it would have produced a library missing every implementation unit the publisher withheld, since a distribution package's `sources` are its interface. Verified end to end on Linux via mingw-cross + wine (e2e 257: both files, the implicit output, the package, deployment beside the exe, `ok=42`). macOS (259) and the MSVC refusal (258) are CI's to confirm; 259 deletes the producer's build tree before consuming, so the install-name assertion is load-bearing rather than decorative.
Records the outcome of §12's five items under the single-PR decision (O1 not split, the rest done), plus the two real defects the work forced out: the kind="shared" guard was inert on native builds — the very case it was written for, since targetTriple is empty there — and `--target` accepted a target this host cannot produce, resolving the native g++ and delivering an ELF as a Windows build. Also the shape they share: mingw's ld tolerates linking a .dll directly, so "a PE shared library is two files" was never exposed. The tolerant case was hiding the broken one.
…, and a guard
Three things the local suite and a read-through caught:
* 255/257/259 interpolated `$(host_path …)` straight into a manifest heredoc.
00_fixture_path_hygiene requires the value to come from a named `*_HOST`
variable, and the rule is Windows': a shell-spelled /tmp path is read by a
native mcpp.exe as "root of the current drive".
* 259 tested `[[ -f "$pkg/lib/"*"/libmathkit.dylib" ]]`. `[[ ]]` does not
path-expand, so that compares against a literal string containing an asterisk
and is false for every real package — a check that can only fail, which on
macOS-only coverage would have looked like a product bug.
* the import-library flag substituted `{}` without checking it is there. Both
dialect rows have one; a row without it would have thrown at `replace(npos)`
instead of simply not emitting a flag.
Also documents, in docs/12 (+ zh), how much of this is verified and where —
including the row that is honest rather than flattering: the REAL old-client
check has been run by hand and not by CI, because the boot entry each job
bootstraps from is an xvm shim that answers "not installed" under the e2e
environment.
Local suite after this: 236 passed, 1 failed, 14 skipped. The remaining failure
is 22_doctor_cache_publish, and the previous RELEASE binary fails it identically
on this machine — a local environment fact, not a regression.
Measured rather than assumed: a previous mcpp reads `[target.'cfg(…)'.runtime]` without complaint and silently ignores it. So the dialect-neutral link channel IS reachable per leg — and moving a leg's flags there would leave every older client with no link flags at all. Closing the native-cl.exe gap therefore means carrying both spellings (linking the library twice on new clients) or giving these packages a version floor; it is not a matter of effort, it conflicts with the two constraints the design is built on.
CI on the three platforms answered what no Linux run could, and all three
failures were worth having.
**251 was asserting a product limit that no longer exists.** It branched on
`uname`: Linux produces a shared package, everything else must REFUSE. Mach-O and
PE/MinGW now produce one, so the branch is by TARGET rather than by host —
Windows' default toolchain is clang on the MSVC ABI, so that is where the refusal
is asserted (and it now demands the `dllexport` reason and the MinGW way
forward, not just "shared libraries are not supported").
**255 could not have a consumer step, and finding out why was the point.** The
package is built with msvc@system, so its tag is `…-msvc19-…`; a consumer on the
default toolchain is refused by the ABI gate, correctly and with the right
message. A consumer that pins msvc@system instead cannot link it either, because
the emitted manifest spells `-L`/`-l` and native cl.exe rejects `-L`. So the only
consumer that test could have is one whose failure says nothing about the
archiver — and pack succeeding is already the assertion, since a wrong /REMOVE:
makes run_library_pack refuse with lib.exe's own output.
**256 found a real bug: `mcpp pack` could not write a tarball on Windows.**
Both legs packed — `mathkit.lib` beside `libmathkit.a`, which is the whole point
of that test — and then the archive step died with
tar (child): Cannot connect to C: resolve failed
GNU tar reads `C:/path/x.tar.gz` as the rsh form `host:path` and goes looking for
a machine called `C`; the message names neither its argument nor the drive letter.
Unreachable until now because a single PE leg is written as a zip, so no Windows
host had ever taken the tar path. Fixed with `--force-local` on Windows, in the
library packer and in pack.cppm's make_tarball — the second is latent rather than
observed, but it is the same command with the same hazard and the two should not
differ in whether they survive being run there.
…msvc"`
The tar fix landed — both legs packed and the archive was written, which is what
that test exists to show: `mathkit.lib` beside `libmathkit.a` in one Windows
package. What failed after it was the predicate assertion, against a package that
was perfectly correct: the generated manifest writes
[target.'cfg(all(arch = "x86_64", os = "windows", env = "msvc"))'.build]
and the grep looked for `env=`, which appears nowhere in the file. Matched with
optional whitespace now.
A grep that can only fail is worth more attention than one that passes, because
it reads as a product bug on the one platform where nothing else could confirm
it. Audited the other emitted-manifest assertions in the suite for the same
shape; the rest either allow spacing (`role *= *`) or match structural text.
256's `check` extracted the selected leg with
`grep -o "dist/mathkit-0.1.0/lib/[A-Za-z0-9_-]*"`. A native mcpp.exe writes
NATIVE separators into build.ninja, so on Windows that matched nothing at all and
the test reported
native saw more than one leg:
(empty)
— i.e. a count of zero, printed as "more than one", over an empty list. It reads
like a packaging bug on the one platform where nothing else could confirm it, and
it is a grep bug.
Anchored on the package name instead, with `[\\/]` for either separator, and the
failure now prints the count and the package's own lines from build.ninja so the
next reader is not guessing. 245 carries the same helper — it only runs on Linux,
but leaving the fragile version there invites the next test to copy it — and
passes locally with it, which is what validates the pattern on `/`.
The pinned xlings is the one a release bundles at <install>/registry/bin/xlings and the version `mcpp self env` reports; it had been sitting at 2026.8.11.2 while six releases went by. One edit in src/platform/xlings/xlings.cppm is the source of truth and the other sixteen follow it — `.github/tools/check_version_pins.sh` fails CI if any disagrees, which is why this is a mechanical sweep rather than a hunt. That checker exists because the hand-maintained list it replaced had already missed both composite actions, leaving CI's sandbox on 0.4.30 unnoticed. Verified after rebuild: `Updating vendored xlings 2026.8.11.2 -> 2026.8.17.2 (pinned 2026.8.17.2)`, and `mcpp self env` reports the new pin.
Caught while reviewing this PR rather than by any test, and it is the kind of
error worth writing down: I described the previous `kind = "shared"` refusal —
`!targetTriple.empty() && os != "linux"` — as having a hole on NATIVE builds,
reasoning that targetTriple would be empty there.
It is not. `tc.targetTriple` is filled from the compiler's own `-dumpmachine`
(detect.cppm:88); a plain `mcpp build` on this machine records
`triple: x86_64-linux-gnu` in resolution.json, and `parse("x86_64-pc-windows-msvc")`
skips the vendor segment and succeeds. So the old guard did refuse native macOS
and native Windows, and this PR ADDS Mach-O and PE/MinGW support rather than
closing a hole. 251 asserting the refusal on macOS before this PR — and failing
after it — is the corroboration.
The install-name and host-`#if` defects are still real, but they were latent
under the old refusal rather than reachable: they become wrong the moment macOS
is allowed, which is what this PR does.
Corrected in the code comment, docs/08, docs/12's neighbours, the CHANGELOG,
e2e 257's header, and the design record — which also keeps the criterion:
"structurally possible to be empty" is not "empty at run time"; either read the
run-time artifact or do not write it up as measured.
…includes `mcpp pack` routes on `[targets.<n>].kind`, so "bundle into a self-contained archive" describes only the program half. Both help lines now name both shapes — the one-line help is where a reader finds out that a library target produces something different, and it was the one place that did not say so. Also drops the global module fragments from mcpp.pack.route and mcpp.pack.library_pipeline: both carried `#include <cstdio>` and neither uses it.
`sourcesDeclared` distinguishes "no `sources` key" from `sources = []`, and the xpkg descriptor parser filled the list without setting it. Harmless today — that synthesiser refuses a descriptor with no sources and never calls apply_defaults_and_infer — but a false flag beside a non-empty list is a trap: the day that function runs on an xpkg manifest, its explicit list is replaced by the default glob and every descriptor-described package quietly compiles the wrong file set.
The date-based convention labels a release by the day it goes out, and the previous one (2026.8.17.1) is already published — so this is the first release of the 18th, not a second one of the 17th. Two version sites move (mcpp.toml and src/version.cppm; fingerprint.cppm derives from the latter), the CHANGELOG's Unreleased section becomes the release heading, and the docs' "until mcpp <version>" notes name the release that actually ships those changes. The `XLINGS_VERSION: '2026.8.17.2'` entries under .github/ are the xlings pin, which happens to carry the same digits and must not move with this. `check_version_pins.sh`: building 2026.8.18.1, bootstrapping from 2026.8.17.1, xlings pins all at 2026.8.17.2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #433. Please do not merge yet.
A closed-source library, an offline site, or a build farm that already compiled
this once had no route through mcpp:
mcpp publishsends source,mcpp packbundles a program's run-time closure, and neither is "a library someone else
links". Collecting the artifacts by hand was the answer.
What it produces is an ordinary package
A normal
mcpp.toml, read through the route mcpp already had for payloads thatcarry their own manifest. Zero new manifest sections and zero new keys:
[targets.<n>].kind[lib]+ the module graph[build].include_dirs[[runtime.artifacts]][target.'cfg(…)'.build] ldflags[dependencies]So there is no
--liband no--artifact static\|shared—kindis alreadywhere mcpp records what an artifact is, and a flag would be a second place to
say it. And an older mcpp still builds against these packages; it simply
does not run the two gates below.
Two interface modes coexist:
include/(text,#include, never compiled) andinterface/(module units the consumer compiles). One package consumed threeways × static and shared = six combinations, all green.
Which
.cppmtravel is computed, not declaredThe module closure of the lib root. The two ways of getting it wrong are
asymmetric, which is why it cannot be a hand-written list:
published. Nothing fails.
.m.ois not the rule — an implementation partition produces one too. The sameclosure decides which archive members to drop, and getting that wrong was
measured as well: dropping every
.m.oremoves the partition's real code andevery target fails to link. Both lists are printed, and an interface that
reaches a partition publishes it with a warning naming the file.
Two gates, for failures that are otherwise silent
The interface still matches its binaries. Swap two
intmembers of ashipped struct — the Itanium ABI does not mangle field order — and before this
the consumer compiled, linked, ran, and printed transposed data, exit code 0,
no diagnostic from any tool.
The binaries were built for this toolchain. The refusal lists the tags the
package does have, because "not found" sends people looking for a package
already on their disk.
Building inside an extracted package is also refused: its
interface/holdsdeclarations whose definitions are in the archive beside them, so a build there
produces a near-empty library and reports success.
Three pre-existing defects found on the way
Each has its own regression test, and each affects projects that never touch
packaging.
Implementation partitions did not build on Windows.
module M:part;andmodule M;wear one spelling and are two declarations; the scanner recordedthe first as requiring
M:partand providing nothing, so a file requiredits own name and the graph held no edge from the unit importing a partition
to the unit defining it. Order was unconstrained — GCC and macOS clang
recovered through their own scan, Windows clang failed with
failed to read compiled module. The note both printed,module 'M:part' imported but not provided in this build, was the cause and read like a hint. Implementationpartitions had no test coverage anywhere in mcpp; this feature's e2e is
the first thing to use one. (5 scanner unit tests)
[target.'<triple>'.build]never matched a native build.matches()short-circuited on the raw
--targetstring whilecontext_for()fell backto the host for
cfg(...). Green in CI, silently inert on a developer'smachine. (e2e 247)
sources = []was byte-identical to omitting the key, so no author couldsay "compile nothing". (e2e 246)
Verification: what actually ran, per platform
CI 19/19 green. 87/87 unit suites (2 new files, 28 assertions; 5 more in
test_modgraph). 15 regression e2e over the touched areas.
sources = []pack <name>packs that namecross-build-test.ymlmingw jobOne skip remains, and it is a real limit: 245 needs two distinct buildable
targets (
x86_64-linux-gnu+x86_64-linux-musl), and no second ready targetexists on the macOS or Windows runners. The mechanism it checks — one
cfg()leg per triple, the native build included — is platform-independent and covered
on Linux.
Getting here took interrogating every skip reason one at a time, and three of
the four did not survive the question:
# requires: gccwas copy-paste-Dprefix, and MSVC spells that/D. Grepping the macro NAME is portablecross-build-test.yml's mingw job already names the e2e scripts the Linux shards skip for exactly this reason (102, 198, 240). 248 joins themsharedis ELF-only"Before that, all ten carried
# requires: gcc, which is Linux-only by design,so they skipped on macOS and Windows while the suite reported green.
Unrestricting them is what surfaced the scanner bug above. The criterion, earned
twice: a test's
# requires:has to be the real floor of the mechanism itverifies, and that is hard to judge while writing it — so the reasons have to
be interrogated afterwards, one by one.
Self-review found five things after the first push
mcpp.pack.libraryexported a secondmcpp::pack::Error. A name attachesto one module; clang refuses, GCC accepted. Windows + macOS red, Linux green.
mcpp packin a workspace root stopped working. Found by running theprevious release against
examples/04-workspaceand comparing. (e2e 249)binproject took
mcpp pack app2and bundled app1. (e2e 250)for the
SONAME— it linked and could not start. mcpp's own runtime-closurecheck reported it. (e2e 251, which uses
run, notbuild)published interface's objects in the archive. Now refused with the reason.
Not in this PR
kind = "shared"on PE / Mach-O,sharedon*-musl(a musl target linksstatically), bundling dependencies into the package, prebuilt BMIs.
Docs
docs/12-binary-distribution.md(+ zh),examples/05-lib-distribution(producer + consumer in one directory, consumers written with
import std;).Design, the measurements behind every criterion, and the four designs my own
experiments disproved:
.agents/docs/2026-08-17-library-distribution-design.md§11.
Round two — the five open items, closed
Decision was one PR, so the scanner change is not split out. Everything else
from the previous review section is done, and doing it turned up three defects
that no Linux run could have shown.
The items
scan_overridespartition is mis-flagged as an interface/REMOVE:untestedlib.exexim:mingw-gcc[package] platformscross-check unimplementedO2: three states, because "nobody determined this" is an answer
Whether a partition's source may be published turns on one keyword, and two of
the three graph-building paths cannot read it: a
[scan_overrides."<glob>"]entry has nowhere to say
export, and P1689 makesis-interfaceoptional —mcpp parsed it into a field nothing read.
Both arrived as
providesInterface = true, which the field's own comment calledthe conservative direction "since the flag only ever produces a warning". That
had it backwards:
trueis the value that produces no warning, so animplementation partition declared that way was published in silence.
std::optional<bool>now, each path saying what it actually knows, and unknownwarns with a different sentence from the known case. e2e 253 pins both sides, and
a control probe (restore the
truedefault at that one site) fails it at exactlythat assertion.
O5: the hard part is not finding the gap, it is not shouting about it
Four comparisons exist; two are printable. "Declared but not packed" is only
actionable if this host could have built it — the normal flow is one
mcpp packper platform in CI, so a Linux runner producing no macOS leg is everysingle run, and a warning that always fires hides the one that matters. The
criterion is
host_can_serve, the same function that decides what--targetaccepts.
What round two turned up
kind = "shared"now works on Mach-O and PE/MinGW. It was ELF-only. What wasmissing was not flag spelling:
.dlltheloader opens and the stub archive the linker consumes. mcpp wrote only the
first, and consumers linked the
.dlldirectly, which mingw's ld tolerates andno other linker does: the tolerant case was hiding the broken one. Plus
-static(which mcpp gives PE executables) puts ld in static-only mode whereit refuses an import library and says have you installed the static version of
the mathkit library? — naming neither the DLL nor
-static.sonamewas declared, so every other.dylibwould have recorded its build directory: perfect on the machine thatbuilt it,
image not foundanywhere else.link /DLL /IMPLIB:was always in the rule table) but symbol export. The message saysso and names MinGW as the way forward.
--targetaccepted targets this host cannot produce. Measured on Linux:mcpp build --target x86_64-windows-msvcresolved the nativeg++, wrotetarget/x86_64-linux-gnu/, and reported success — an ELF delivered as a Windowsbuild, which is the failure the neighbouring typo check calls the worst one. The
vocabulary tier says "mcpp supports this target";
host_can_serveanswers "canthis machine produce it".
mcpp packcould not write a tarball on Windows. Found by e2e 256 after bothlegs packed correctly: GNU tar reads
C:/path/x.tar.gzas the rsh formhost:pathand dies withCannot connect to C: resolve failed. Unreachableuntil now because a single PE leg is written as a zip.
A claim of mine that was wrong
I described the old shared-library guard as having a hole on native builds,
reasoning that
targetTriplewould be empty there. It is not — it is filledfrom the compiler's own
-dumpmachine, a native build recordstriple: x86_64-linux-gnuinresolution.json, andparse("x86_64-pc-windows-msvc")skips the vendor segment and succeeds. The oldguard did refuse native macOS and native Windows; this PR adds support rather
than closing a hole. Corrected in the code comment, docs, CHANGELOG, the test
header and the design record. The criterion kept with it: structurally possible
to be empty is not empty at run time.
Still open, with reasons rather than estimates
cl.exe-L/-l, which cl rejects. Naming the file by path fails too: ninja runs link commands with cwd = the output dir and only include-family prefixes are absolutized, and an absolute path is not relocatable. The dialect-neutral channel (link_library_dirs/libraries) exists but only at top level — and measured: an older mcpp reads[target.'cfg(…)'.runtime]and silently ignores it, so moving the flags there leaves every older client with none. Needs both spellings or a version floorkind = "shared"on the MSVC ABI.deffrom a symbol scan — a build-graph node, not a flag…-msvc19-unknownstl0-…stdlibIdis unset for cl. Cosmetic and self-consistent, but it lands in published package filenames; worth settling before MSVC packages are published, and it invalidates the MSVC build cache when it changes