Skip to content

Compile inflector regexes once instead of per call - #54

Open
perryqh wants to merge 1 commit into
perf/measure-setupfrom
perf/oncelock-regexes
Open

Compile inflector regexes once instead of per call#54
perryqh wants to merge 1 commit into
perf/measure-setupfrom
perf/oncelock-regexes

Conversation

@perryqh

@perryqh perryqh commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Best return per line of any change I've measured on pks check: ~40 lines, and it cuts total CPU time by 45%.

Note

Stacked on #53 (which is itself stacked on #52). The diff above shows only this change — one file. GitHub retargets the base automatically as each parent merges.

The problem

camelize and to_class_case run once per file while the Zeitwerk constant map is built — roughly 50k calls per invocation on a large codebase. Each call was compiling between two and seven regexes from scratch:

if class_name.contains("Statu") {
    let re = Regex::new("Statuse$").unwrap();
    class_name = re.replace_all(&class_name, "Status").to_string();
    let re = Regex::new("Statu$").unwrap();
    ...
}

CLASS_CASE_TO_SINGULAR.into_iter().for_each(|(plural, singular)| {
    if class_name.contains(plural) {
        let re = Regex::new(plural).unwrap();   // compiled inside a loop
        ...
    }
});

Regex::new constructs a DFA. Doing that per filename, for every file in the app, on every run, is not a marginal cost.

The change

Every pattern becomes a LazyLock<Regex>, compiled once per process. That's the whole change — one file, no API or behavior change.

Measured

On a 51,513-file application:

before after
user CPU time 13.6 s 7.4 s −45%
"infer constants from filename" phase 0.665 s 0.047 s −93%
wall clock (quiet machine) 5.15 s 4.53 s −12.1%
wall clock (machine under load) 8.02 s 7.58 s −5.5%

Two honest notes on those numbers:

  • Wall clock improves less than CPU time, because this phase is parallelized — the wasted work was partly hidden behind other cores. The 45% CPU reduction is the invariant; it reproduced identically under both conditions.
  • The wall-clock benefit therefore depends on available parallelism. I measured −12.1% on an idle machine and −5.5% while a VM and an editor were competing for cores. Both are real; CI machines are usually closer to the second. I've quoted the range rather than the flattering number.

One thing to know about the diff

This preserves an existing oddity rather than fixing it:

let re = Regex::new("Statuss").unwrap();
re.replace_all(&class_name, "Status").to_string();   // result discarded

That result has always been thrown away, so statuss is left untouched — and test_to_class_case pins that behavior with a ("statuss", false, "Statuss") case. I kept it verbatim with a comment explaining why. It looks like a bug, but fixing it changes inflection output and belongs in its own PR, not one whose only job is to stop recompiling regexes.

Two other files have the same compile-per-call pattern (file_utils.rs:108, rails_utils.rs:19). I left them alone: once this landed, the phase was down to 0.047 s and there was nothing left to win. Not worth the churn.

Verification

  • cargo test — 257 passing
  • cargo clippy --all-targets --all-features — clean
  • cargo fmt --all -- --check — clean
  • Behavior is unchanged by construction (same patterns, same order, same call sites), and the inflector's own test suite covers the singularization and acronym cases.

Heads-up: a pre-existing flaky test

While verifying, I hit gitignore_test::test_respect_gitignore_can_be_disabled failing on roughly 1 run in 3. It is not caused by this change — it reproduces at the same rate on the base branch, and passes 6/6 when run in isolation.

Cause: common::teardown() globs tests/fixtures/*/tmp/cache/packwerk and deletes the cache for every fixture, not just the one the calling test used. gitignore_test.rs calls it from 7 different tests, which Rust runs in parallel threads within the same binary — so one test deletes another's cache mid-run.

Worth fixing separately (scope teardown to the fixture the test actually touched). Flagging it here because it will make CI intermittently red regardless of this PR.

🤖 Generated with Claude Code

@perryqh
perryqh requested a review from a team as a code owner August 19, 2026 22:15
@github-project-automation github-project-automation Bot moved this to Triage in Modularity Aug 19, 2026
`camelize` and `to_class_case` run once per file while the Zeitwerk constant map
is built -- roughly 50k calls per invocation on a large codebase -- and each call
was compiling between two and seven regexes from scratch. Building a regex
constructs a DFA, so this was not a marginal cost.

Hoisted every pattern into a `LazyLock<Regex>` compiled once per process.

MEASURED on a 51,513-file app:
  wall clock                            5.127s -> 4.656s  (-9.2%)
  user cpu time                        13.561s -> 7.370s  (-45.7%)
  "infer constants from filename" phase  0.689s -> <0.12s

Wall clock improves less than CPU time because this phase is parallelized across
cores, so the wasted work was partly hidden. It was still burning nearly half the
process's total CPU.

Behavior is deliberately unchanged, including one oddity: the "Statuss"
replacement has always had its result discarded, so "statuss" is left alone. That
is preserved verbatim with a comment, and the existing test that pins it still
passes. It looks like a bug, but fixing it belongs in its own change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@perryqh
perryqh force-pushed the perf/oncelock-regexes branch from d278f48 to a6ab8fd Compare August 19, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant