add support for running bugbug-selected tasks for autoland-based reviewbot pushes - #3578
add support for running bugbug-selected tasks for autoland-based reviewbot pushes#3578bhearsum wants to merge 1 commit into
Conversation
…ewbot pushes A large percentage of failures on autoland are caught by bugbug-selected tasks. I'd like to experiment with having bugbug-selected tasks run on reviewbot pushes to see if it helps developers notice and fix such problems earlier. Enabling this mostly requires tweaks to the parameters; it will also require a change on the Gecko side to ensure all of the right tasks are in target tasks (otherwise there's nothing bugbug to choose from).
| self.user_blacklist = {} | ||
|
|
||
| # bugbug test configuration | ||
| self.bugbug_enabled_repositories = ["firefox-autoland"] |
There was a problem hiding this comment.
I realize that a bunch of config for bugbug is in tc secrets. I wasn't sure if that was helpful or desired here; I can make that adjustment if needed.
| self.bugbug_enabled_repositories = ["firefox-autoland"] | ||
| # Because it's unclear how much load this will add, this is being rolled out gradually. | ||
| self.bugbug_enabled_percent = 0.1 | ||
| self.bugbug_optimize_strategy = "gecko_taskgraph.optimize:tryselect.bugbug_reduced_manifests_config_selection_medium" |
There was a problem hiding this comment.
In an ideal world, I'd want to use the exact same strategy as autoland. In reality that doesn't seem possible because it uses a long composite strategy that takes into account push id, backstops, and other things. I'm not sure if what I have here is the best one; I'm happy to be corrected on this.
There was a problem hiding this comment.
On autoland, the builds for various operating systems get scheduled. On try, the fewer builds the better, and they better are fast ones and without a small machine pool to run one. By default Linux opt should run, debug if the penalty for slower tests is acceptable. macOS and Windows shall get test tasks if the changes are expected to affect operating systems differently.
Archaeopteryx
changed the title
|
There are plans to meet with Bastien next week and have dkl set up a separate build plan for build + test which are not linting. For linting, reporting after each failed task should rarely report more than one failed task. For test tasks or even multiple builds, the results need to be bundled and if possible deduplicated. @marco-c will be able to guide about the scheduling strategy. Without builds scheduled by default, will |
| # bugbug test configuration | ||
| self.bugbug_enabled_repositories = ["firefox-autoland"] | ||
| # Because it's unclear how much load this will add, this is being rolled out gradually. | ||
| self.bugbug_enabled_percent = 0.1 |
There was a problem hiding this comment.
Incorrect variable name
| self.bugbug_enabled_repositories = ["firefox-autoland"] | ||
| # Because it's unclear how much load this will add, this is being rolled out gradually. | ||
| self.bugbug_enabled_percent = 0.1 | ||
| self.bugbug_optimize_strategy = "gecko_taskgraph.optimize:tryselect.bugbug_reduced_manifests_config_selection_medium" |
There was a problem hiding this comment.
On autoland, the builds for various operating systems get scheduled. On try, the fewer builds the better, and they better are fast ones and without a small machine pool to run one. By default Linux opt should run, debug if the penalty for slower tests is acceptable. macOS and Windows shall get test tasks if the changes are expected to affect operating systems differently.
|
This also needs to cancel the already running tasks if a new version of the patch is submitted. |
Thanks; I wasn't aware of these plans.
They'll get pulled in when tests are chosen, or when bugbug decides to schedule them explicitly. https://treeherder.mozilla.org/jobs?repo=try&revision=fa79adf8be1d73ff7848525b4cc96fb3f1f106ef is an example I hacked together that didn't need this (it inserts the necessary parameter changes directly) |
Does this mean a separate reviewbot push would happen? That would solve the reporting delay concerns for sure. |
2 participants
A large percentage of failures on autoland are caught by bugbug-selected tasks. I'd like to experiment with having bugbug-selected tasks run on reviewbot pushes to see if it helps developers notice and fix such problems earlier.
Enabling this mostly requires tweaks to the parameters; it will also require a change on the Gecko side to ensure all of the right tasks are in target tasks (otherwise there's nothing bugbug to choose from).
We shouldn't do this until reviewbot reports failures back to phab without waiting for the entire task group to be completed, otherwise reporting, eg: lint failures will be delayed.