I have:
Bug description
quarto pandoc runs Pandoc and relays that process's stdout to its own. On some documents the relayed copy is missing a block of bytes from the middle. Nothing reports the loss: Pandoc exits 0, quarto pandoc exits 0, and stderr is empty. The only sign of a problem is that some of the content in the original Markdown is absent on the rendered page.
Asking Pandoc to write the file to disk with --output is unaffected. That copy has been complete in every case measured.
Our investigation indicates that the cause may be a discarded short-write return value in processOutput. See below in Potential cause of bytes lost, and proposed resolution for details and a change that we believe will fix it.
Steps to reproduce
The following script implements the steps to reproduce; the procedure is described in the comments.
# 1. Create a temporary file with a single heading line followed by one
# 300,000-character line of unbreakable text.
# (The bug reproduces reliably with this particular example, but we have
# not identified the exact properties of a general file that cause the
# bug to occur.)
cd "$(mktemp -d)"
printf '# Title\n\n' > doc.md
head -c 300000 /dev/zero | tr '\0' 'a' >> doc.md
printf '\n' >> doc.md
# 2. Run `quarto pandoc` on the file with output going to stdout.
# Observe that 234,524 bytes are returned.
quarto pandoc --to html doc.md | wc -c # 234524
# 3. Run `quarto pandoc` again, this time writing output to a file, and check
# the size of that file. Observe that it holds 300,034 bytes.
quarto pandoc --to html --output out.html doc.md
wc -c < out.html # 300034
No reader extensions, no project config, and no wrapping flags are involved.
Actual behavior
The discrepancy between the two byte counts demonstrates the problem: reading the result from stdout delivers 234,524 of the 300,034 bytes Pandoc produced, losing 65,510 of them.
Comparing the two copies byte by byte:
- The first 26 bytes —
<h1 id="title">Title</h1> and its newline — arrive.
- Then 65,510 bytes that appear in the written file are missing from stdout.
- Then the rest of the written file and the rest of stdout match exactly.
Running the bundled Pandoc binary directly on the same document — on this machine /Applications/quarto/bin/tools/aarch64/pandoc --to html doc.md | wc -c — gives the full 300,034 bytes, so the loss happens in the wrapper rather than in Pandoc.
More generally, we have observed real documents lose text in several places rather than one. The 6.7 KB Markdown file on which we first observed this behavior renders to 624,525 bytes through --output and to 533,854 bytes through stdout, with the 90,671 missing bytes falling in 18 separate gaps.
Expected behavior
quarto pandoc --to html doc.md should deliver on stdout exactly the bytes that quarto pandoc --to html --output out.html doc.md writes to the file.
Your environment
- Quarto 1.10.18, the current release, and the v1.11.1 pre-release. Both lose the same 65,510 bytes on the document above.
- No IDE — the commands above were run at a
zsh prompt.
- macOS 15.7.3, Darwin 24.6.0, arm64.
Not tried on Linux or Windows.
Quarto check output
Both runs exited 0. The Julia line at the end of each is the last thing it printed, not a truncation on my part.
quarto check on 1.10.18, the installed release
Quarto 1.10.18
[✓] Checking environment information...
Quarto cache location: /Users/brahn/Library/Caches/quarto
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.10.0: OK
Dart Sass version 1.101.0: OK
Deno version 2.7.14: OK
Typst version 0.15.1: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.10.18
Path: /Applications/quarto/bin
[✓] Checking tools....................OK
TinyTeX: (not installed)
Chrome Headless Shell: (not installed)
VeraPDF: (not installed)
[✓] Checking LaTeX....................OK
Tex: (not detected)
[✓] Checking Chrome Headless....................OK
Using: Chrome found on system
Path: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
Source: MacOS known location
[✓] Checking basic markdown render....OK
[✓] Checking R installation...........(None)
Unable to locate an installed version of R.
Install R from https://cloud.r-project.org/
[✓] Checking Python 3 installation....OK
Version: 3.14.7
Path: /opt/homebrew/opt/python@3.14/bin/python3.14
Jupyter: (None)
Jupyter is not available in this Python installation.
Install with python3 -m pip install jupyter
[✓] Checking Julia installation...
quarto check on the v1.11.1 pre-release, unpacked from the tarball and run in place
The install path below is shortened; it was a temporary directory, and the installed 1.10.18 was left alone.
Quarto 1.11.1
[✓] Checking environment information...
Quarto cache location: /Users/brahn/Library/Caches/quarto
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.10.0: OK
Dart Sass version 1.101.0: OK
Deno version 2.7.14: OK
Typst version 0.15.1: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.11.1
Path: /private/tmp/…/q1111/bin
[✓] Checking tools....................OK
TinyTeX: (not installed)
Chrome Headless Shell: (not installed)
VeraPDF: (not installed)
[✓] Checking LaTeX....................OK
Tex: (not detected)
[✓] Checking Chrome Headless....................OK
Using: Chrome found on system
Path: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
Source: MacOS known location
[✓] Checking basic markdown render....OK
[✓] Checking R installation...........(None)
Unable to locate an installed version of R.
Install R from https://cloud.r-project.org/
[✓] Checking Python 3 installation....OK
Version: 3.14.7
Path: /opt/homebrew/opt/python@3.14/bin/python3.14
Jupyter: (None)
Jupyter is not available in this Python installation.
Install with python3 -m pip install jupyter
[✓] Checking Julia installation...
Potential cause of bytes lost, and proposed resolution
processOutput in src/core/process.ts relays the child's output like this, on main as of today:
for await (const chunk of iterator) {
if (output === "inherit" || output === undefined) {
if (which === "stdout") {
Deno.stdout.writeSync(chunk);Deno.stdout.writeSync returns the number of bytes it actually wrote, which is not guaranteed to be the whole chunk, and that return value is discarded.
To find out whether that is what happens here, I had Claude write a Deno script that spawns the same bundled Pandoc on the same document and relays its stdout exactly as processOutput does, but keeps the number writeSync returns. Four runs gave identical output:
chunks: 5
bytes offered to writeSync: 300034
bytes writeSync reported writing: 234524
short writes:
chunk 1 at output offset 0: offered 65536, wrote 26
The first chunk is 65,536 bytes. writeSync accepts 26 of them — the heading and its newline — and reports 26. The remaining 65,510 bytes are never retried. Those are the same 26 and 65,510 that Actual behavior reports above.
Changing that same script to repeat writeSync until the chunk has gone out delivers all 300,034 bytes, and so does patching the real thing. Replacing the bare Deno.stdout.writeSync(chunk) in processOutput with that loop, in the quarto.js of a v1.11.1 build, makes quarto pandoc deliver all 300,034 bytes on the document above where the build as shipped delivers 234,524.
StdErrOutputHandler.log in src/core/log.ts already does exactly that, in a loop added by 6fbbc1f on 2025-04-01 under the description "Deno.stderr.writeSync() may not wright full error message buffer in one go for very long message". processOutput never got the same treatment.
Measurements taken before the short write was found — supporting detail, none of it needed to act on the above
These were the evidence when the cause was still unknown. Nothing in them contradicts the short-write explanation, and the last table is a direct consequence of it.
Nothing at or under 65,536 bytes of output ever failed. The rows measured closest to that threshold on either side:
| bytes in |
bytes Pandoc produced |
bytes sent to stdout |
bytes missing |
| 65,006 |
65,019 |
65,019 |
0 |
| 65,510 |
65,534 |
65,534 |
0 |
| 66,006 |
66,019 |
494 |
65,525 |
| 66,010 |
66,034 |
524 |
65,510 |
Neither line length nor total size explains it alone. Holding total output near 400 KB and varying only the length of each output line:
| line length |
lines |
bytes Pandoc produced |
bytes sent to stdout |
bytes missing |
| 57 |
7,017 |
406,986 |
406,986 |
0 |
| 207 |
1,932 |
401,856 |
401,856 |
0 |
| 1,007 |
397 |
400,176 |
400,176 |
0 |
| 1,507 |
265 |
399,620 |
395,840 |
3,780 |
| 5,007 |
79 |
395,632 |
387,856 |
7,776 |
| 20,007 |
19 |
380,152 |
337,488 |
42,664 |
| 40,007 |
9 |
360,072 |
257,208 |
102,864 |
| 80,007 |
4 |
320,032 |
210,256 |
109,776 |
| 200,007 |
1 |
200,008 |
200,008 |
0 |
A long line was never damaged when nothing preceded it. A document whose entire output was a single 1,648,902-byte line came through intact, as did the single-line 200,007-byte row above. Put a # Title in front of that same long paragraph and 65,510 bytes vanish; move the heading to the end and it is intact again. The short-write measurement explains this: with nothing before it, the first writeSync is not the one that gets truncated.
Bytes intact before the gap, plus bytes missing, always summed to 65,536:
| bytes intact before the gap |
bytes missing |
sum |
| 11 |
65,525 |
65,536 |
| 18 |
65,518 |
65,536 |
| 26 |
65,510 |
65,536 |
| 102 |
65,434 |
65,536 |
| 1,016 |
64,520 |
65,536 |
Three further things that were checked:
- Deterministic. Every repeat run gave byte-identical output — six consecutive runs on each of two failing documents, five on a third.
- Pipe or file, no difference. Piping to
wc -c and redirecting to a regular file produce identical bytes, and neither is complete.
- Not about
--wrap=none. That flag appears in the case that first showed this only because it is a convenient way to make output lines long. The reproduction above uses an unbreakable 300,000-character token and default wrapping instead.
Related issues
The closest existing issues are #2699 and #10661, both closed and both about quarto pandoc sending its output to stderr rather than stdout. That is a different failure: here the output does reach stdout, and part of it is missing. If this duplicates something I missed, please point me at it and I will close this one.
How this was investigated and verified (including LLM use notes)
I encountered this bug while testing code in a project that used quarto pandoc to render Markdown. I then had Claude Code do the investigation: the reproduction, the measurements above, and the Deno script that found the short write are its work.
I have run the reproduction myself on this machine (with the environment indicated above), on Quarto 1.10.18, and seen the two byte counts it reports. I have read the rest of the report before posting it, though the measurements in it are Claude's work rather than mine.
What I am asking for
Please make processOutput write each chunk in full before moving to the next one, the way StdErrOutputHandler.log already does. That is the change the measurement above points at, and it made the loss go away in the standalone script.
If that turns out not to be the whole story, then failing loudly on the path that loses bytes would still be a large improvement on its own, because the most expensive part of this was that nothing anywhere said anything had gone wrong.
I have:
Bug description
quarto pandocruns Pandoc and relays that process's stdout to its own. On some documents the relayed copy is missing a block of bytes from the middle. Nothing reports the loss: Pandoc exits 0,quarto pandocexits 0, and stderr is empty. The only sign of a problem is that some of the content in the original Markdown is absent on the rendered page.Asking Pandoc to write the file to disk with
--outputis unaffected. That copy has been complete in every case measured.Our investigation indicates that the cause may be a discarded short-write return value in
processOutput. See below in Potential cause of bytes lost, and proposed resolution for details and a change that we believe will fix it.Steps to reproduce
The following script implements the steps to reproduce; the procedure is described in the comments.
No reader extensions, no project config, and no wrapping flags are involved.
Actual behavior
The discrepancy between the two byte counts demonstrates the problem: reading the result from stdout delivers 234,524 of the 300,034 bytes Pandoc produced, losing 65,510 of them.
Comparing the two copies byte by byte:
<h1 id="title">Title</h1>and its newline — arrive.Running the bundled Pandoc binary directly on the same document — on this machine
/Applications/quarto/bin/tools/aarch64/pandoc --to html doc.md | wc -c— gives the full 300,034 bytes, so the loss happens in the wrapper rather than in Pandoc.More generally, we have observed real documents lose text in several places rather than one. The 6.7 KB Markdown file on which we first observed this behavior renders to 624,525 bytes through
--outputand to 533,854 bytes through stdout, with the 90,671 missing bytes falling in 18 separate gaps.Expected behavior
quarto pandoc --to html doc.mdshould deliver on stdout exactly the bytes thatquarto pandoc --to html --output out.html doc.mdwrites to the file.Your environment
zshprompt.Not tried on Linux or Windows.
Quarto check output
Both runs exited 0. The Julia line at the end of each is the last thing it printed, not a truncation on my part.
quarto checkon 1.10.18, the installed releasequarto checkon the v1.11.1 pre-release, unpacked from the tarball and run in placeThe install path below is shortened; it was a temporary directory, and the installed 1.10.18 was left alone.
Potential cause of bytes lost, and proposed resolution
processOutputinsrc/core/process.tsrelays the child's output like this, onmainas of today:Deno.stdout.writeSyncreturns the number of bytes it actually wrote, which is not guaranteed to be the whole chunk, and that return value is discarded.To find out whether that is what happens here, I had Claude write a Deno script that spawns the same bundled Pandoc on the same document and relays its stdout exactly as
processOutputdoes, but keeps the numberwriteSyncreturns. Four runs gave identical output:The first chunk is 65,536 bytes.
writeSyncaccepts 26 of them — the heading and its newline — and reports 26. The remaining 65,510 bytes are never retried. Those are the same 26 and 65,510 that Actual behavior reports above.Changing that same script to repeat
writeSyncuntil the chunk has gone out delivers all 300,034 bytes, and so does patching the real thing. Replacing the bareDeno.stdout.writeSync(chunk)inprocessOutputwith that loop, in thequarto.jsof a v1.11.1 build, makesquarto pandocdeliver all 300,034 bytes on the document above where the build as shipped delivers 234,524.StdErrOutputHandler.loginsrc/core/log.tsalready does exactly that, in a loop added by 6fbbc1f on 2025-04-01 under the description "Deno.stderr.writeSync() may not wright full error message buffer in one go for very long message".processOutputnever got the same treatment.Measurements taken before the short write was found — supporting detail, none of it needed to act on the above
These were the evidence when the cause was still unknown. Nothing in them contradicts the short-write explanation, and the last table is a direct consequence of it.
Nothing at or under 65,536 bytes of output ever failed. The rows measured closest to that threshold on either side:
Neither line length nor total size explains it alone. Holding total output near 400 KB and varying only the length of each output line:
A long line was never damaged when nothing preceded it. A document whose entire output was a single 1,648,902-byte line came through intact, as did the single-line 200,007-byte row above. Put a
# Titlein front of that same long paragraph and 65,510 bytes vanish; move the heading to the end and it is intact again. The short-write measurement explains this: with nothing before it, the firstwriteSyncis not the one that gets truncated.Bytes intact before the gap, plus bytes missing, always summed to 65,536:
Three further things that were checked:
wc -cand redirecting to a regular file produce identical bytes, and neither is complete.--wrap=none. That flag appears in the case that first showed this only because it is a convenient way to make output lines long. The reproduction above uses an unbreakable 300,000-character token and default wrapping instead.Related issues
The closest existing issues are #2699 and #10661, both closed and both about
quarto pandocsending its output to stderr rather than stdout. That is a different failure: here the output does reach stdout, and part of it is missing. If this duplicates something I missed, please point me at it and I will close this one.How this was investigated and verified (including LLM use notes)
I encountered this bug while testing code in a project that used
quarto pandocto render Markdown. I then had Claude Code do the investigation: the reproduction, the measurements above, and the Deno script that found the short write are its work.I have run the reproduction myself on this machine (with the environment indicated above), on Quarto 1.10.18, and seen the two byte counts it reports. I have read the rest of the report before posting it, though the measurements in it are Claude's work rather than mine.
What I am asking for
Please make
processOutputwrite each chunk in full before moving to the next one, the wayStdErrOutputHandler.logalready does. That is the change the measurement above points at, and it made the loss go away in the standalone script.If that turns out not to be the whole story, then failing loudly on the path that loses bytes would still be a large improvement on its own, because the most expensive part of this was that nothing anywhere said anything had gone wrong.