Photograph one device per runner, and build the app once - #170
Merged
Conversation
The screenshot run failed on whichever device was behind. Nothing was wrong with it: two simulators on a runner with three cores and seven gigabytes took ten minutes over the six screens, and XCTest allows a test ten. Both devices were killed at the same second. So one device per xcodebuild instead of both at once, and a ceiling high enough that only a real hang reaches it. The job gets a bound as well, and a failed run now keeps whatever it managed to photograph. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E6tn8P25cBDmrA67zrhfwN
snapshot builds and tests on every run, and there are eighteen of them: two devices in nine languages. The build is the same every time and checking it costs about two minutes each on a runner, so it is built once up front and the runs only photograph. CI keeps the build between runs as well, which is why it goes somewhere with a name rather than the temp directory snapshot picks. The simulator is also told which language it is in. It was only ever the app that knew, so an iPad - which shows the date up there - said Mon Aug 17 above a Russian document. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E6tn8P25cBDmrA67zrhfwN
Two simulators never sat on one machine again by accident: they are on two now, a device to a runner, and the two halves photograph at the same time. What used to be an hour and a quarter is about half that. Neither runner can see the other's pictures, so the set is checked once they are back together rather than by the lane. A run started by hand is unchanged - both devices, every language, checked at the end. The edit screen also stopped waiting two seconds for a keyboard tutorial that the launch argument had already turned off. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E6tn8P25cBDmrA67zrhfwN
Both runners saved under the same key, so the second one found it taken and the two halves shared whichever build got there first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E6tn8P25cBDmrA67zrhfwN
andiwand
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The screenshot job of the 1.41 dry run failed, and nothing in the test or the
app was wrong with it. This makes the run correct, then quicker.
What was wrong
Four attempts, two devices:
Both devices pass and both fail, so it is neither device. Attempt 1 died on both
within forty milliseconds of each other at 660 seconds - a stopwatch, not a bug.
XCTest kills a test that runs longer than 600 seconds, and nothing here
raised it. Not one of our own assertion messages appears anywhere in that log.
The runner is
macos-26-arm64: three cores, seven gigabytes, photographing twosimulators at once. Six screens take under a minute on a desktop and were taking
ten there. Attempt 3's iPad never started at all -
Timed out while loading Accessibility, which is what a simulator does when it cannot get what it needs.The changes
One simulator at a time.
concurrent_simulators: falsegives each device itsown
xcodebuildinstead of handing one invocation two destinations.A runner per device. The matrix takes that further: the two never share a
machine, so the memory they were fighting over is not shared to begin with, and
the halves photograph at the same time.
Room to finish.
executionTimeAllowance = 1800- a ceiling, not a target.Built once. snapshot builds and tests on every run, and there are eighteen.
scan(build_for_testing:)builds once intobuild/screenshots, which CI keepsbetween runs, and the runs only photograph. That step takes 77 seconds against
the nine to eleven minutes the old first build did.
The simulator speaks the language too. Only the app knew before, so an iPad -
which shows the date up there - read
9:41 AM Mon Aug 17over a Russiandocument. Now
09:41 Пн 17 авг., in that locale's own clock.The lane cannot check half a set, so
screenshot-setputs the two together andchecks them once both are in.
timeout-minutes: 180bounds a job whose defaultwas six hours, and the raw captures are archived even when the lane failed.
Checked
A dry run of all nine languages on this branch: 108 screenshots in all 9
captured locales, 54 at 1320x2868 and 54 at 2064x2752, merged from
framed-iphoneandframed-ipadand validated as one set. 62 minutes against76 before.
Every change was run locally first, both ways round:
ODR_SCREENSHOT_DEVICE=iphoneas CI does it, and with nothing set as a person does - both devices, framed,
checked. A local run is unchanged.
Known, and not fixed here
The test itself got slower between dry runs - 85s, then 156s and 166s - which is
the wrong direction for a change that removed contention. The boot is not it
(26s against 27s), and the spread within one run is 124s to 241s, so runner noise
cannot be ruled out from two runs. It does not fail anything and the set is
right; it is worth a look when the next release gives another sample.
🤖 Generated with Claude Code
https://claude.ai/code/session_01E6tn8P25cBDmrA67zrhfwN