Apply impersonation_chain to deferred BigQuery existence checks - #71648
Open
SEPURI-SAI-KRISHNA wants to merge 1 commit into
Open
Apply impersonation_chain to deferred BigQuery existence checks#71648SEPURI-SAI-KRISHNA wants to merge 1 commit into
SEPURI-SAI-KRISHNA wants to merge 1 commit into
Conversation
The sensors handed the impersonation chain to the trigger inside hook_params, which nothing reads, while the trigger authenticated from an attribute those call sites never set. A deferred existence check therefore ran as the connection's service account, and only in deferrable mode, so disabling deferral appeared to fix the resulting permission error.
| ], | ||
| ) | ||
| @mock.patch("airflow.providers.google.cloud.sensors.bigquery.BigQueryHook") | ||
| def test_deferred_trigger_receives_impersonation_chain(mock_hook, sensor_class, trigger_class, extra_kwargs): |
Contributor
There was a problem hiding this comment.
Could we move this test under the respective sensor/trigger test classes rather than having it as a standalone parametrized test?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BigQueryTableExistenceSensorandBigQueryTablePartitionExistenceSensorhand theirimpersonation chain to the trigger inside
hook_params:but the trigger authenticates from a different attribute entirely:
Nothing ever sets
self.impersonation_chainfrom those call sites, so it staysNoneandthe deferred existence check runs as the connection's service account.
hook_paramsisstored and serialized but never read anywhere in the module — it carries the value and
then drops it.
poke()builds its hook correctly, so the sensor honours impersonation whendeferrable=Falseand ignores it whendeferrable=True. That divergence is the awkwardpart: turning deferrable off appears to "fix" the permissions error, which points
investigation away from the real cause.
The trigger already grew a proper
impersonation_chainparameter in #36341; the sensorswere simply never moved onto it. This change passes it explicitly at both defer sites.
hook_paramsis left untouched here. It is a required argument on the trigger's public__init__, and now that nothing reads it, removing or deprecating it is a separatedecision — happy to follow up if maintainers would like it gone.
For reviewers:
GoogleBaseHook.__init__falls back to the connection'simpersonation_chainextra when the argument is absent, so deployments that configureimpersonation on the connection were never affected. This only bites when impersonation is
set on the sensor, which is the documented parameter.
No newsfragment: this is a provider change, and provider changelogs are regenerated from
git logby the release manager.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines