Skip to content

feat(docs): generate a Fern-ready docs bundle and sync it to contentful-docs [NT-3942] - #442

Open
Tim Beyer (TimBeyer) wants to merge 6 commits into
mainfrom
feat/fern-docs-sync-nt-3942
Open

feat(docs): generate a Fern-ready docs bundle and sync it to contentful-docs [NT-3942]#442
Tim Beyer (TimBeyer) wants to merge 6 commits into
mainfrom
feat/fern-docs-sync-nt-3942

Conversation

@TimBeyer

@TimBeyer Tim Beyer (TimBeyer) commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Closes NT-3942.

Today the Optimization SDK documentation is hand-copied into contentful/contentful-docs: links rewritten, pages restructured, slugs renamed by hand. This replaces that step with a deterministic transform, and fixes the authoring instructions that caused the divergence in the first place.

Why

The manual process has measurably drifted. On the live site right now:

  • 53 internal links point at 13 slugs that do not exist. Worst: integrate-the-node-sdk-in-a-node-app (×15 — the real slug is …into-a-node-app) and integrate-the-optimization-android-sdk-in-a-compose-app (×9 — the nav declares integrate-the-android-sdk-in-a-compose-app).
  • 13 malformed GitHub URLs where a find-replace ate a slash: …/main/implementationsreact-native-sdk/README.md.
  • Deep links are broken by appending the trailing slash after the fragment: …/entry-personalization-and-variant-resolution#single-locale-cda-entry-contract/.
  • Severity was flattened — all three GitHub alert types became <Info>, so the one real WARNING (native SDKs are in beta) lost it.
  • Two pages ship an empty description, and one slug is a typo (builde-a-custom-…).

It is also lossy, which is the part worth flagging: isEmptyVariant appears 9 times and slugField 7 times in these guides, and zero times on the live site. The Node SDK's slug-based fetching and empty-variant handling are documented here but missing in public, even though both landed days before the last manual sync.

What this adds

Three layers, so the first two need no credentials and can be iterated on locally:

Command Does
pnpm docs:fern Builds pages, the nav block, and redirects into a gitignored fern-bundle/
pnpm docs:fern:apply -- --docs-repo <path> Splices a bundle into a local contentful-docs checkout
pnpm fern:check Runs the same pipeline without writing; gates CI

The transform owns structure, links, and metadata, and never prose — the wording in documentation/ stays the source of truth, and a transform that rewrote sentences would make the two copies impossible to diff.

fern:check is what makes the link rot unrepeatable: every cross-document link must resolve to a published page, every #fragment must match a real heading, no page may link into authoring/ or internal/, the MDX must be safe, and a slug cannot move without a recorded redirect. Every line-based pass is fence-aware (reusing headingsOf from the existing knowledge validators), because a Swift #if DEBUG and a JSX <h1> both appear inside fenced blocks here and neither is prose.

The instruction fix

The gerund titles were not an accident — recipes/integration.md mandated the H1 form literally, and the decision, migration, and supplemental recipes stated no title form at all. STYLE_GUIDE.md covered imperative mood for instructions and procedure steps, but nothing about document titles.

So the rule now has one home in STYLE_GUIDE.md (task documents take imperative titles, explanatory documents descriptive noun phrases, neither takes a gerund), all four recipes defer to it, and concepts/AGENTS.md states the counterpart. Then applied: 23 headings and 152 link texts. The published title is now the heading itself, with no second key to drift from it.

Same class of gap, found on the way: concepts used three different names for one closing section with nothing governing it. Normalized to ## Related documentation and the rule written down.

Verification

Applied end to end against a real contentful-docs checkout, then reset:

  • fern check0 errors (3 warnings, all pre-existing in unrelated experience-orchestration/ pages)
  • check-orphan-pages — 490/490 registered · check-redirects — 409 valid · markdownlint — 0 errors
  • Broken internal links: 53 → 0

In this repo: fern:check, guides:check, knowledge:check, eslint, prettier, tsc all clean.

Reviewing this

Six commits, each one concern, meant to be read in order. docs(docs): give guides imperative titles… is the large one but its diff is uniform and mechanical.

Not in this PR, and needing a human

The golden diff (generated bundle vs the 35 live pages) leaves 29 items across 14 pages that need editorial judgement, not code. The ones that matter are 9 where the site has content this repo does not — they would be destroyed by a sync unless back-ported first:

  • <Info> callouts the writers added to build-a-custom-…, choose-a-nextjs-migration-path, and consent-management
  • a forwardedSemanticInteractions code block and a consent bullet list in the forwarding guide

Two judgement calls rather than clear wins: the site dropped a definitional clause from states.selectedOptimizations, and its core-state-management intro reads better than this repo's. Separately, the site renamed ## Related guides and conceptsRelated documentation, which conflicts with what supplemental-recipe.md mandates — I left the recipe's version standing, since that is an editorial call.

Before this can run

  • A FERN_DOCS_SYNC_TOKEN secret scoped to contentful-docs (contents: write, pull-requests: write). A GitHub App installation token is preferable to a long-lived PAT. A PR from a branch in that repo also gets the Fern preview URL; a fork PR cannot, because forks cannot read secrets.
  • Likely an ADR in contentful-docs: its .bito/guidelines say "do not add SDK generation workflows", and CODEOWNERS requires review from both technical-writing and developer-experience.
  • Sidebar order changes. Order now comes from the README children: lists, which are reader-routing order (Choose the right SDK → Node → Web → React → Next.js). The live site is alphabetical. Deliberate, and worth a look before merge.

🤖 Generated with Claude Code

@wiz-inc-38d59fb8d7

wiz-inc-38d59fb8d7 Bot commented Aug 18, 2026

Copy link
Copy Markdown

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations -
SAST Finding SAST Findings 1 Medium
Software Management Finding Software Management Findings -
Total 1 Medium

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio.

@bito-code-review

bito-code-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

Functional Validation by Bito

SourceRequirement / Code AreaStatusNotes
NT-3942Enable a fern-compatible output from the SDK docs pipeline with correct restructured organization and correct links✅ MetThe PR implements a complete Fern-compatible documentation pipeline. New scripts in scripts/fern/build.ts, scripts/fern/docs.ts, scripts/fern/transform.ts, and scripts/fern/bundle.ts generate properly structured documentation with correct links, slug tracking, and navigation. All documentation files in documentation/guides/ and documentation/concepts/ have been updated with `fern:` frontmatter blocks containing `slug`, `section`, and `description` fields. Cross-document links have been corrected to proper Fern format (e.g., changing "Integrating..." to "Integrate..."). The validation script scripts/validate-fern-export.ts (invoked via `pnpm fern:check`) ensures links resolve to published pages and headings, preventing link rot. The slug lock file documentation/fern-slugs.lock.json prevents silent URL changes.
NT-3942Automate PR creation to contentful-docs repository on release✅ MetThe PR adds a new GitHub Actions workflow .github/workflows/sync-fern-docs.yaml that automates PR creation to contentful-docs. The workflow: (1) exports docs via `pnpm docs:fern`, (2) applies the bundle to a contentful-docs checkout via `pnpm docs:fern:apply`, (3) commits changes to a dedicated sync branch (`sync/optimization-docs`), and (4) creates or updates a PR referencing `[NT-3942]`. The workflow is currently manual dispatch but is designed to be additive with a release trigger. The implementation is idempotent (force-pushes to a fixed branch) and concurrency-guarded, ensuring only one PR exists per release cycle. This eliminates the manual hand-copying step described in the requirement.

@bito-code-review

bito-code-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

Impact Analysis by Bito

Code Paths Analyzed

Impact:
This PR adds Fern documentation export infrastructure. No SDK code, API contracts, or runtime behavior is affected. All changes are documentation tooling and metadata.

Flow:
New: scripts/fern/* scripts transform documentation/*.md → Fern bundle → contentful-docs repo. No runtime code paths.

Direct Changes (Diff Files):
• scripts/apply-fern-docs.ts [new] — Applies built Fern bundle into contentful-docs checkout (layer 2 of sync). Local-only, credential-free.
• scripts/export-fern-docs.ts [new] — Builds Fern-ready documentation bundle from documentation/. Layer 1 of sync.
• scripts/fern/bundle.ts [new] — Emits navigation block and redirect list artifacts for Fern site.
• scripts/fern/docs.ts [new] — Reads frontmatter, resolves child order, validates Fern metadata.
• scripts/fern/build.ts [new] — Orchestrates bundle build, slug locking, and problem reporting.
• scripts/fern/nav.ts [new] — Renders sidebar navigation block YAML from docs.
• scripts/fern/transform.ts [new] — Transforms docs: link rewriting, GitHub alert conversion to Fern callouts, heading anchors, TOC stripping.
• scripts/validate-fern-export.ts [new] — Validates Fern bundle correctness without writing (CI gate for link rot prevention).
• documentation/fern-slugs.lock.json [new] — Tracks slug mappings and permanent redirects for URL stability.
• package.json [1887-1898] — Added npm scripts: docs:fern, docs:fern:apply, fern:check.
• documentation/guides/.md [various] — Added fern: frontmatter (slug, section, description), renamed headings gerund→imperative, updated cross-links.
• documentation/concepts/
.md [various] — Added fern: frontmatter metadata, updated cross-links.
• packages//src/**/.ts [various] — Minor refactors (fenceMask, headingsOf utilities) in scripts/fern/transform.ts.

Repository Impact:
Documentation build tooling: New scripts directory with Fern export pipeline; no SDK package code touched.
Documentation metadata: All documentation/*.md files receive fern: frontmatter additions; content and structure unchanged.

Cross-Repository Dependencies:
None.

Database/Caching Impact:
• None

API Contract Violations:
None.

Infrastructure Dependencies:
• CI will run pnpm fern:check as new gate; no other infra changes.

Additional Insights:
Slug stability: fern-slugs.lock.json ensures URL changes generate permanent redirects, preventing link rot.
Alert component conversion: GitHub-style > [!NOTE] blockquotes are converted to Fern &lt;Callout&gt; components during export.

Testing Recommendations

Frontend Impact:
None.

Service Integration:
None.

Data Serialization:
None.

Privacy Compliance:
None.

Backward Compatibility:
• Verify documentation links still resolve after heading rename changes (gerund→imperative forms in links).

OAuth Functionality:
• None

Reliability Testing:
• None

Additional Insights:
• Run pnpm fern:check to validate the Fern bundle builds correctly before merge.
• Run pnpm docs:fern to produce a test bundle and verify the nav block and redirects YAML are correct.
• If contentful-docs integration is intended, test scripts/apply-fern-docs.ts against a local contentful-docs checkout.
• Verify all cross-document links in the documentation resolve after the heading rename changes.

Analysis based on known dependency patterns and edges. Actual impact may vary.

@bito-code-review bito-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #94402c

Actionable Suggestions - 7
  • scripts/apply-fern-docs.ts - 1
    • Empty-string bundle path silently misroutes · Line 52-52
  • .github/workflows/sync-fern-docs.yaml - 1
    • Insufficient permissions declaration · Line 38-39
  • scripts/fern/build.ts - 2
    • Slug validation bypassed when acceptSlugChanges is true · Line 78-92
    • Extract duplicated sorting and logging pattern · Line 125-125
  • documentation/authoring/recipes/supplemental-recipe.md - 1
  • scripts/fern/transform.ts - 2
Additional Suggestions - 3
  • scripts/fern/transform.ts - 1
    • Mdx safety line-number mismatch · Line 364-370
      In `checkMdxSafety`, `index` inside the `forEach` over `withoutCode.split('\n')` is the position in the fence-stripped, code-stripped joined string — not the original `lines` array index. Error messages will report wrong line numbers to authors. Also: add unit tests for this function covering fenced blocks, inline code spans, and allowed/denied tags.
  • scripts/sdk-knowledge/markdown.ts - 1
    • Misleading variable name for array · Line 30-30
      The variable `inFence` holds a `boolean[]` (a fence mask), but the name implies a single `boolean` flag. This naming is consistent across all 5 call sites in this file and `transform.ts`, making the mismatch easy to miss. A descriptive name like `fenceFlags` or `fenceStatus` signals the array type.
  • documentation/guides/building-a-custom-javascript-optimization-adapter.md - 1
    • Duplicate description text · Line 6-8
      The `description` field in the frontmatter (lines 6-8) is identical to the body text at lines 13-14. Remove the redundant frontmatter field to avoid maintenance divergence.
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • scripts/fern/build.ts - 1
    • reconcileLock exception escapes reportFatal · Line 94-94
  • documentation/guides/migrating-experience-js-node-ssr-and-esr.md - 1
  • documentation/concepts/consent-management-in-the-optimization-sdk-suite.md - 1
Review Details
  • Files reviewed - 55 · Commit Range: 5f34a84..a4f3547
    • .github/workflows/main-pipeline.yaml
    • .github/workflows/sync-fern-docs.yaml
    • .gitignore
    • STYLE_GUIDE.md
    • documentation/AGENTS.md
    • documentation/authoring/recipes/decision.md
    • documentation/authoring/recipes/integration.md
    • documentation/authoring/recipes/migration.md
    • documentation/authoring/recipes/supplemental-recipe.md
    • documentation/concepts/AGENTS.md
    • documentation/concepts/android-sdk-runtime-and-interaction-mechanics.md
    • documentation/concepts/consent-management-in-the-optimization-sdk-suite.md
    • documentation/concepts/core-state-management.md
    • documentation/concepts/entry-personalization-and-variant-resolution.md
    • documentation/concepts/interaction-tracking-in-node-and-stateless-environments.md
    • documentation/concepts/interaction-tracking-in-web-sdks.md
    • documentation/concepts/ios-sdk-runtime-and-interaction-mechanics.md
    • documentation/concepts/locale-handling-in-the-optimization-sdk-suite.md
    • documentation/concepts/optimization-handoff-and-cache-safe-rendering.md
    • documentation/concepts/profile-synchronization-between-client-and-server.md
    • documentation/concepts/react-native-sdk-interaction-tracking-mechanics.md
    • documentation/fern-slugs.lock.json
    • documentation/guides/README.md
    • documentation/guides/building-a-custom-javascript-optimization-adapter.md
    • documentation/guides/choosing-a-nextjs-migration-path-from-experience-js.md
    • documentation/guides/choosing-the-right-sdk.md
    • documentation/guides/forwarding-optimization-sdk-context-to-analytics-and-tag-management-tools.md
    • documentation/guides/integrating-the-node-sdk-in-a-node-app.md
    • documentation/guides/integrating-the-optimization-android-sdk-in-a-compose-app.md
    • documentation/guides/integrating-the-optimization-android-sdk-in-a-views-app.md
    • documentation/guides/integrating-the-optimization-ios-sdk-in-a-swiftui-app.md
    • documentation/guides/integrating-the-optimization-ios-sdk-in-a-uikit-app.md
    • documentation/guides/integrating-the-optimization-sdk-in-a-nextjs-app-router-app.md
    • documentation/guides/integrating-the-optimization-sdk-in-a-nextjs-pages-router-app.md
    • documentation/guides/integrating-the-react-native-sdk-in-a-react-native-app.md
    • documentation/guides/integrating-the-react-web-sdk-in-a-react-app.md
    • documentation/guides/integrating-the-web-sdk-in-a-web-app.md
    • documentation/guides/migrating-experience-js-contentful-model-to-optimization.md
    • documentation/guides/migrating-experience-js-next-to-nextjs-app-router.md
    • documentation/guides/migrating-experience-js-next-to-nextjs-pages-router.md
    • documentation/guides/migrating-experience-js-node-ssr-and-esr.md
    • documentation/guides/migrating-experience-js-plugins-and-preview.md
    • documentation/guides/migrating-experience-js-react-to-react-web.md
    • documentation/guides/migrating-experience-js-to-the-web-sdk.md
    • documentation/guides/rendering-personalized-nextjs-routes-with-static-isr-and-edge-handoffs.md
    • documentation/guides/using-contentful-graphql-data-with-the-optimization-sdks.md
    • package.json
    • scripts/apply-fern-docs.ts
    • scripts/export-fern-docs.ts
    • scripts/fern/build.ts
    • scripts/fern/bundle.ts
    • scripts/fern/docs.ts
    • scripts/fern/transform.ts
    • scripts/sdk-knowledge/markdown.ts
    • scripts/validate-fern-export.ts
  • Files skipped - 0
  • Tools
    • GITHUB_ADVISORY (Security Vulnerability) - ✔︎ Successful
    • OWASP (Security Vulnerability) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Default Agent You can customize the agent settings here or contact your Bito workspace admin at jared.jolton@contentful.com.

Documentation & Help

AI Code Review powered by Bito Logo

} else if (arg === '--bundle') {
index += 1
const { [index]: value } = argv
bundleDir = value ?? bundleDir

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty-string bundle path silently misroutes

The ?? operator coalesces only null/undefined, so --bundle '' sets bundleDir to an empty string. This causes path.resolve(rootDir, '') to return rootDir, silently reading from the project root instead of fern-bundle. The empty-string guard prevents this silent misrouting.

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +38 to +39
permissions:
contents: read

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Insufficient permissions declaration

The permissions declaration specifies contents: read only, but the workflow uses secrets.FERN_DOCS_SYNC_TOKEN to push branches (git push --force) and manage pull requests (gh pr edit, gh pr create). The token needs contents: write and pull-requests: write permissions to function correctly. Consider explicitly declaring the permissions the token requires rather than relying on implicit defaults.

Code suggestion
Check the AI-generated fix before applying
Suggested change
permissions:
contents: read
permissions:
contents: read
contents: write
pull-requests: write
Suggested based on your custom review guideline "Review Posture"

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment thread scripts/fern/build.ts
Comment on lines +78 to +92
const lock = readLock()
if (!options.acceptSlugChanges) {
const { slugs } = lock
for (const doc of docs) {
const { [doc.relPath]: previous } = slugs
if (previous !== undefined && previous !== doc.fern.slug) {
problems.push({
file: doc.relPath,
line: 1,
message: `fern.slug changed "${previous}" -> "${doc.fern.slug}"; rerun \`pnpm docs:fern -- --update-lock\` to record the redirect`,
})
}
}
// A document missing from the lock has never been published; that is expected for a new page.
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slug validation bypassed when acceptSlugChanges is true

The slug-change detection loop (lines 78–92) runs only when acceptSlugChanges is false, but reconcileLock on line 94 runs unconditionally and mutates slugs regardless of the flag (bundle.ts:58 writes slugs[relPath] = slug unconditionally). When acceptSlugChanges is true, changed slugs are silently absorbed into reconciled and persisted to fern-slugs.lock.json with zero problems reported — breaking the "CI passed == bundle I built locally" invariant described in the file header.

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment thread scripts/fern/build.ts
/** Prints problems in the `file:line: message` form the repo's other validators use. */
export function reportProblems(problems: readonly Problem[]): void {
;[...problems]
.sort((left, right) =>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extract duplicated sorting and logging pattern

This PR introduces or contains syntactic duplicate code detected by CPD. The duplicated pattern (sorting by file/line and writing to stderr) appears in scripts/fern/build.ts (lines 125-130) and scripts/validate-guide-authoring.ts (lines 469-474). Consider extracting this logic into a shared utility function to improve maintainability and eliminate duplication.

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +23 to +24
H1 form: the imperative mood, naming the task, as required by the `Headings` section of
[`../../../STYLE_GUIDE.md`](../../../STYLE_GUIDE.md). Never a gerund. Name the supplemental task: "Forward Optimization SDK

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect relative path

The relative path ../../../STYLE_GUIDE.md resolves 3 levels up to documentation/, not to the root. The correct path is ../../../../STYLE_GUIDE.md (4 levels up). This is consistent with the actual file location.

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment thread scripts/fern/transform.ts
Comment on lines +359 to +375
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
const prose = lines.map((line, index) => (inFence[index] === true ? '' : line))
const withoutCode = prose.join('\n').replace(/`[^`]*`/gu, '')

withoutCode.split('\n').forEach((line, index) => {
for (const match of line.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: index + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect line numbers in MDX safety check

The line numbers reported by checkMdxSafety are incorrect. The index from withoutCode.split('\n') reflects the position in the filtered content (after fence lines removed and lines joined), not the original document line numbers. This would cause developers to look at the wrong lines when addressing MDX safety issues.

Code suggestion
Check the AI-generated fix before applying
Suggested change
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
const prose = lines.map((line, index) => (inFence[index] === true ? '' : line))
const withoutCode = prose.join('\n').replace(/`[^`]*`/gu, '')
withoutCode.split('\n').forEach((line, index) => {
for (const match of line.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: index + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
lines.forEach((line, lineIndex) => {
if (inFence[lineIndex] === true) {
return
}
const lineWithoutCode = line.replace(/`[^`]*`/gu, '')
for (const match of lineWithoutCode.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: lineIndex + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}
Suggested based on your custom review guideline "Review Posture"

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment thread scripts/fern/transform.ts
Comment on lines +359 to +375
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
const prose = lines.map((line, index) => (inFence[index] === true ? '' : line))
const withoutCode = prose.join('\n').replace(/`[^`]*`/gu, '')

withoutCode.split('\n').forEach((line, index) => {
for (const match of line.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: index + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect line numbers in MDX safety checks

The line field in the problem object uses index + 1 where index is from iterating over withoutCode.split('\n'). Since withoutCode is derived from prose.join('\n') with fence lines already removed, the resulting line numbers will be incorrect when fence lines exist in the original document. For example, if line 2 of a 5-line doc is in a fence, the reported line for a problem on line 4 would be off by 1. Consider tracking original line numbers when building withoutCode.

Code suggestion
Check the AI-generated fix before applying
Suggested change
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
const prose = lines.map((line, index) => (inFence[index] === true ? '' : line))
const withoutCode = prose.join('\n').replace(/`[^`]*`/gu, '')
withoutCode.split('\n').forEach((line, index) => {
for (const match of line.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: index + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}
function checkMdxSafety(lines: string[], problems: TransformProblem[]): void {
const inFence = fenceMask(lines)
const proseLines: { text: string; originalIndex: number }[] = []
lines.forEach((line, index) => {
if (inFence[index] !== true) {
proseLines.push({ text: line, originalIndex: index })
}
})
const withoutCode = proseLines.map((p) => p.text).join('\n').replace(/`[^`]*`/gu, '')
withoutCode.split('\n').forEach((text, derivedIndex) => {
const originalLine = proseLines[derivedIndex]?.originalIndex ?? derivedIndex
for (const match of text.matchAll(/<\/?([A-Za-z][A-Za-z0-9]*)/gu)) {
const tag = match[1] ?? ''
if (!ALLOWED_COMPONENTS.has(tag)) {
problems.push({
line: originalLine + 1,
message: `raw <${tag}> outside a code fence would be parsed as JSX by MDX`,
})
}
}
})
}
Suggested based on your custom review guideline "TypeScript Review Evidence"

Code Review Run #94402c


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

@bito-code-review

Copy link
Copy Markdown

✅ Review Settings Overridden

Status: Overridden Successfully

Guidelines:

  • Accepted:

    • General : Review Posture, Repo Truth And Boundaries, Domain Invariants

    • Language : typescript= TypeScript Review Evidence

Note: Extra guidelines beyond 3 general purpose guidelines and 1 language specific guideline per language are not processed. Guidelines are fetched from the source branch.

Tim Beyer (TimBeyer) and others added 6 commits August 18, 2026 17:18
The Compose and React Web integration guides ended with `</content>` and `</invoke>`
tags left behind by an authoring agent. They render as literal text on GitHub and would
break MDX compilation once these guides publish to the documentation site, so the tech
writers were stripping them by hand on every copy.

Also tag two code fences that were unlabelled or used a non-canonical alias, so every
fence in the published set carries a language the site recognises.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The authoring pipeline was producing gerund titles ("Integrating the...") because it was
told to. `recipes/integration.md` mandated the H1 form literally, and the decision,
migration, and supplemental recipes stated no title form at all, so authors defaulted to
the same shape. STYLE_GUIDE.md covered imperative mood for instructions and for
procedure steps, but said nothing about the title of a document.

Give the rule one home in STYLE_GUIDE.md: a task document takes an imperative title
naming the task, an explanatory document takes a descriptive noun phrase, and neither
takes a gerund. All four recipes now defer to it instead of restating or omitting it,
and the concepts guide states the noun-phrase counterpart so the two archetypes cannot
drift into each other.

Two related rules the pipeline was also missing: a document title is its published page
title and sidebar label, so cross-document link text naming it must change with it; and
a concept closes with `## Related documentation` under exactly that name, which had
three spellings across eleven files with nothing governing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tion name

Add the `fern:` frontmatter block each concept needs to publish: slug, sidebar section,
and meta description. The published page title is the document's `#` heading, so there
is no title key here and nothing for the title to drift from.

Update the link text that names a guide, which is now imperative, and rename the closing
section to `## Related documentation` in the four concepts that spelled it `Related docs`
or `Related docs and reference implementations`. The TOC entries and the five in-page
anchors pointing at the old headings move with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Apply the title rule to the 23 guides that carried gerund headings: 23 headings and the
link text that names them across the guide set. The four guides with an authored
frontmatter `title` have it realigned to their heading, which `documentation/AGENTS.md`
already required.

Add the `fern:` block each guide needs to publish: slug, sidebar section, and meta
description, plus `navTitle` on the two guides whose sidebar label is deliberately
shorter than their page title. Frontmatter and heading land together because in the
nineteen guides that had no frontmatter they are adjacent lines in the same diff hunk.

Slugs are recorded rather than derived. Rewording a heading changes the published title,
which is correct; it must never silently move a live URL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the hand-copy step into contentful-docs with a deterministic transform. It owns
structure, links, and metadata, and never prose: the wording in `documentation/` stays
the single source of truth, and a transform that rewrote sentences would make the two
copies impossible to diff.

Three layers, so the first two need no credentials and can be iterated on locally:

- `pnpm docs:fern` builds the pages, the navigation block, and the redirects into a
  gitignored `fern-bundle/`.
- `pnpm docs:fern:apply -- --docs-repo <path>` splices that bundle into a local
  contentful-docs checkout, so the output can be checked with that repo's own gates
  before any cross-repo automation is involved. Navigation and redirects are spliced,
  never regenerated, because that file also holds hand-maintained Personalization
  content this repo does not own.
- `pnpm fern:check` runs the same pipeline without writing, and is what makes the link
  rot the manual process accumulated impossible to reintroduce: every cross-document
  link must resolve to a published page, every `#fragment` must match a real heading,
  no page may link into `authoring/` or `internal/`, the MDX must be safe, and a slug
  may not move without a recorded redirect.

Every line-based pass is fence-aware, reusing `headingsOf` from the existing knowledge
validators, because a Swift `#if DEBUG` and a JSX `<h1>` both appear inside fenced
blocks in these guides and neither is prose.

Slugs live in a lock file. A changed slug needs `--update-lock`, which appends a
permanent redirect; old URLs have to keep resolving, so that list accumulates and is
never pruned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run `pnpm fern:check` in the knowledge-check job, and add
`documentation/concepts/**` to the filter that triggers it. Concepts were absent from
that filter, so a concepts-only change ran no documentation validation at all.

Add `sync-fern-docs.yaml` to open the pull request against contentful-docs. Manual
dispatch only for now, but built so a release trigger is additive rather than a rewrite:
ref-parameterized, idempotent against one fixed branch and one upserted pull request,
and concurrency-guarded. That last part matters because merging the grouped release pull
request creates one GitHub release per component, so `on: release` fires several times
for a single release moment and those firings have to converge on one pull request.

Follows the constraints contentful-docs sets for its own workflows: no `${{ }}`
interpolation inside a `run:` block, and no third-party actions beyond the SHA-pinned
ones this repo already uses. Needs a `FERN_DOCS_SYNC_TOKEN` secret scoped to that repo;
a GitHub App installation token is preferable to a long-lived PAT.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@TimBeyer
Tim Beyer (TimBeyer) force-pushed the feat/fern-docs-sync-nt-3942 branch from a4f3547 to f68566c Compare August 18, 2026 15:23
@bito-code-review

bito-code-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review Agent Run #7c72f9

Actionable Suggestions - 0
Additional Suggestions - 7
  • documentation/guides/integrating-the-optimization-android-sdk-in-a-compose-app.md - 1
    • CWE-N: Frontmatter field inconsistency across guides · Line 5-5
      The `navTitle` field is inconsistent with all other integration guides in this repository (Views, iOS SwiftUI, React Native, React Web), which omit this field entirely. Fern infers the nav title from the H1 heading by default, so explicit `navTitle` is redundant. Removing it maintains API consistency and reduces maintenance surface.
  • scripts/export-fern-docs.ts - 1
    • Magic number without invariant · Line 23-23
      Line 106 divides by this constant. While the value is correct today (each redirect in `renderRedirects` produces a `source` line and a `destination` line), an accidental future edit to `renderRedirects` that adds or removes a line per entry would silently corrupt the count without any test catching it. A self-documenting constant or an inline assertion would make the invariant explicit.
  • documentation/guides/using-contentful-graphql-data-with-the-optimization-sdks.md - 1
    • Naming mismatch vs peer guides · Line 11-11
      The H1 starts with 'Use', but every other integration guide in this guides section (e.g., 'Integrate the Optimization React Web SDK in a React app') uses 'Integrate' as the heading verb. Update to 'Integrate' for naming consistency across related guides.
  • documentation/guides/migrating-experience-js-to-the-web-sdk.md - 1
    • Duplicate description text · Line 10-13
      The description on lines 12–13 is an exact duplicate of the frontmatter `description` field. Remove the redundant paragraph to eliminate divergence risk when one copy is updated without the other.
  • documentation/concepts/react-native-sdk-interaction-tracking-mechanics.md - 1
    • Duplicate link in related docs · Line 592-593
      The 'Related documentation' section now contains a duplicate entry. The guide `../guides/integrating-the-react-native-sdk-in-a-react-native-app.md` is already referenced at line 18 under 'For step-by-step setup, see'. Remove line 592 to eliminate redundancy and maintenance risk.
  • scripts/fern/bundle.ts - 1
    • Unreachable code for Migration guides · Line 134-149
      The loop body for `"Migration guides"` is unreachable: `docsInSection(docs, 'Migration guides')` always returns `[]` because `loadPublishedDocs` only reads from `guides/` and `concepts/` (no migration directory exists), and `inSection.length === 0` on line 136 causes an immediate `continue` on every iteration for this section.
  • scripts/validate-fern-export.ts - 1
    • Misleading anchor count metric · Line 34-34
      The variable `anchors` counts the total number of anchor strings available on each page (`doc.anchors.size` summed), but the message describes it as 'anchors resolvable' — implying fragment references from other docs were resolved against these anchors. The name and message are misleading: `anchors` is a static set populated during doc parsing (what headings exist), not a count of cross-doc link resolutions. Rename to `totalAnchors` and clarify the message.
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • documentation/guides/integrating-the-optimization-ios-sdk-in-a-swiftui-app.md - 1
  • documentation/guides/building-a-custom-javascript-optimization-adapter.md - 1
Review Details
  • Files reviewed - 55 · Commit Range: 19e974c..f68566c
    • .github/workflows/main-pipeline.yaml
    • .github/workflows/sync-fern-docs.yaml
    • .gitignore
    • STYLE_GUIDE.md
    • documentation/AGENTS.md
    • documentation/authoring/recipes/decision.md
    • documentation/authoring/recipes/integration.md
    • documentation/authoring/recipes/migration.md
    • documentation/authoring/recipes/supplemental-recipe.md
    • documentation/concepts/AGENTS.md
    • documentation/concepts/android-sdk-runtime-and-interaction-mechanics.md
    • documentation/concepts/consent-management-in-the-optimization-sdk-suite.md
    • documentation/concepts/core-state-management.md
    • documentation/concepts/entry-personalization-and-variant-resolution.md
    • documentation/concepts/interaction-tracking-in-node-and-stateless-environments.md
    • documentation/concepts/interaction-tracking-in-web-sdks.md
    • documentation/concepts/ios-sdk-runtime-and-interaction-mechanics.md
    • documentation/concepts/locale-handling-in-the-optimization-sdk-suite.md
    • documentation/concepts/optimization-handoff-and-cache-safe-rendering.md
    • documentation/concepts/profile-synchronization-between-client-and-server.md
    • documentation/concepts/react-native-sdk-interaction-tracking-mechanics.md
    • documentation/fern-slugs.lock.json
    • documentation/guides/README.md
    • documentation/guides/building-a-custom-javascript-optimization-adapter.md
    • documentation/guides/choosing-a-nextjs-migration-path-from-experience-js.md
    • documentation/guides/choosing-the-right-sdk.md
    • documentation/guides/forwarding-optimization-sdk-context-to-analytics-and-tag-management-tools.md
    • documentation/guides/integrating-the-node-sdk-in-a-node-app.md
    • documentation/guides/integrating-the-optimization-android-sdk-in-a-compose-app.md
    • documentation/guides/integrating-the-optimization-android-sdk-in-a-views-app.md
    • documentation/guides/integrating-the-optimization-ios-sdk-in-a-swiftui-app.md
    • documentation/guides/integrating-the-optimization-ios-sdk-in-a-uikit-app.md
    • documentation/guides/integrating-the-optimization-sdk-in-a-nextjs-app-router-app.md
    • documentation/guides/integrating-the-optimization-sdk-in-a-nextjs-pages-router-app.md
    • documentation/guides/integrating-the-react-native-sdk-in-a-react-native-app.md
    • documentation/guides/integrating-the-react-web-sdk-in-a-react-app.md
    • documentation/guides/integrating-the-web-sdk-in-a-web-app.md
    • documentation/guides/migrating-experience-js-contentful-model-to-optimization.md
    • documentation/guides/migrating-experience-js-next-to-nextjs-app-router.md
    • documentation/guides/migrating-experience-js-next-to-nextjs-pages-router.md
    • documentation/guides/migrating-experience-js-node-ssr-and-esr.md
    • documentation/guides/migrating-experience-js-plugins-and-preview.md
    • documentation/guides/migrating-experience-js-react-to-react-web.md
    • documentation/guides/migrating-experience-js-to-the-web-sdk.md
    • documentation/guides/rendering-personalized-nextjs-routes-with-static-isr-and-edge-handoffs.md
    • documentation/guides/using-contentful-graphql-data-with-the-optimization-sdks.md
    • package.json
    • scripts/apply-fern-docs.ts
    • scripts/export-fern-docs.ts
    • scripts/fern/build.ts
    • scripts/fern/bundle.ts
    • scripts/fern/docs.ts
    • scripts/fern/transform.ts
    • scripts/sdk-knowledge/markdown.ts
    • scripts/validate-fern-export.ts
  • Files skipped - 0
  • Tools
    • GITHUB_ADVISORY (Security Vulnerability) - ✔︎ Successful
    • OWASP (Security Vulnerability) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Default Agent You can customize the agent settings here or contact your Bito workspace admin at jared.jolton@contentful.com.

Documentation & Help

AI Code Review powered by Bito Logo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant