Skip to content

fix: include src/scanner.c in the source distribution - #345

Open
SidoJain wants to merge 1 commit into
tree-sitter:masterfrom
SidoJain:fix-sdist-missing-scanner-325
Open

fix: include src/scanner.c in the source distribution#345
SidoJain wants to merge 1 commit into
tree-sitter:masterfrom
SidoJain:fix-sdist-missing-scanner-325

Conversation

@SidoJain

Copy link
Copy Markdown

Fixes #325

Bug

setup.py's Extension(...) only lists src/parser.c as a source. src/scanner.c gets appended dynamically at build time (if path.exists("src/scanner.c"): ext.sources.append(...) in BuildExt), which works from a git checkout but never tells setuptools' sdist file-collection that scanner.c needs to be packaged. There's no MANIFEST.in either. So the sdist ships without it, the build succeeds silently (missing symbols only get caught at link time on import, not at compile time), and the only visible symptom is an ImportError for anyone installing from source.

Fix

One line in setup.py's EggInfo.find_sources, matching the existing conditional style already used in BuildExt right above it:

if path.exists("src/scanner.c"):
    self.filelist.include("src/scanner.c")

Note on CI

ci.yml's Python step runs pip install -e .[core], an editable install straight from the checkout - it never builds or installs from an actual sdist, so it can't and won't catch this bug either way. I verified this manually instead (below), since there's no automated check to rely on.

Verification (before)

uv build --sdist
tar tzf dist/tree_sitter_python-*.tar.gz | grep scanner.c   # nothing
uv venv /tmp/before-env
uv pip install --python /tmp/before-env/bin/python3 dist/tree_sitter_python-*.tar.gz
/tmp/before-env/bin/python3 -c "import tree_sitter_python"

ImportError: .../_binding.abi3.so: undefined symbol: tree_sitter_python_external_scanner_create

Verification (after)

Same steps, fresh venv, after the fix:

Import succeeded
<capsule object "tree_sitter.Language" at 0x...>

tree_sitter_python-0.25.0/src/scanner.c now present in the tarball

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Missing "src/scanner.c" in source distribution

1 participant