fix(inspect): handle 4-byte bounds in _readable_bound after int/float type promotion (#3744) - #3771
fix(inspect): handle 4-byte bounds in _readable_bound after int/float type promotion (#3744)#3771ArjunPakhan wants to merge 4 commits into
Conversation
|
Thanks for picking this up @ArjunPakhan! The diff here only contains Two notes for when it does:
Happy to test a revised branch against the S3 Tables table where I hit this. |
|
Hi @tasty0tomato! 👋 Thanks for the thorough review! I've updated the PR branch:
All 18 CI checks have passed! Ready for review when you have a moment. |
|
Hi @tasty0tomato! 👋 I've resolved the merge conflicts with upstream/main and verified that all tests in test_inspect.py pass cleanly locally. The branch is pushed and ready for review whenever you or the maintainers have a moment to take a final look! Thank you! 🚀 |
|
Hi @tasty0tomato! 👋 All 15 unit tests, docs, security, and lint checks are passing cleanly. The single failure in Python Integration / integration-test appears to be the DeprecationWarning in test_deletes.py on upstream/main. The PR is clean, conflict-free, and ready for review whenever you have time to take a look! Thank you! 🚀 |
|
@claude check again for this |
Fixes #3744
Summary
When a table undergoes spec-allowed type promotion (
int→longorfloat→double), pre-existing manifest files retain their 4-byte bounds (IntegerType/FloatType). Currently,_readable_boundpasses these bytes directly tofrom_bytes(field.field_type, bound). Becausefield.field_typeis updated toLongType()/DoubleType(),from_bytesattempts an 8-byte unpack (_LONG_STRUCT.unpack(b)), raisingstruct.error: unpack requires a buffer of 8 bytes.Changes Made
_readable_boundinpyiceberg/table/inspect.pyto inspect byte lengths before decoding.field_typeisLongType/DoubleTypeandlen(bound) == 4, decode usingIntegerType()/FloatType().tests/table/test_inspect.pycovering promoted bound deserialization forLongTypeandDoubleType.Testing
python -m pytest tests/table/test_inspect.py(5/5 tests passed).