feat(sdk): add extension auth for middleware E2E

[pimlock]

Summary

Establish the first Rust SDK surface for building OpenShell extension services and add authenticated supervisor middleware end-to-end coverage.

This PR intentionally starts with a narrow foundation: the opt-in openshell-sdk extension feature provides service-side caller verification, and one Docker E2E proves the complete gateway → sandbox supervisor → authenticated middleware → upstream HTTP path. The middleware replaces sensitive content and adds a header before the request reaches the upstream.

Related Issue

Related to #1733.

Follow-up coverage and SDK adoption are tracked in #2794.

SDK Direction

The extension module is intended to become the Rust SDK for implementing extension services. As it grows, the SDK should own common lifecycle concerns such as service interfaces, authentication before handler dispatch, key caching and refresh after trusted bootstrap, protocol lifecycle validation, consistent errors, and observability. Extension authors should primarily implement manifests, configuration validation, and request or message business logic.

Rust is the initial proving ground, not a Rust-only extension model. OpenShell's other language SDKs should provide equivalent extension support within their existing packages rather than introducing a separate family of extension packages. The cross-language API shape still needs dedicated design work so each SDK remains idiomatic while sharing the same lifecycle, trust, and interoperability guarantees.

The deployment must continue to provide cold-start trust configuration, including the trusted gateway identity, issuer, audience, and CA or initial verification keys. Extension-specific authorization also remains application-owned. In this initial PR, key refresh remains deployment-owned and middleware protobuf interfaces still come from the existing core surface.

Alternative Considered

We considered publishing a dedicated extension SDK, separate from the existing API SDK in each language. Keeping extension support in the regular SDKs gives OpenShell fewer packages, release paths, and dependency surfaces to maintain. It also reflects the expected usage model: extensions will likely call OpenShell APIs in addition to implementing service interfaces.

A unified SDK can eventually provide an extension-oriented API client wrapper with the appropriate authentication configured for extension-to-OpenShell calls. This avoids requiring extension authors to combine and configure two SDK families themselves. Cargo features and language-appropriate modules can preserve an opt-in boundary without introducing separate packages.

Changes

• Extension SDK: Add the opt-in openshell-sdk extension feature with self-contained caller authentication from trusted Ed25519 keys or JWKS, strict token and identity validation, and typed authenticated caller information without a direct openshell-extension-core dependency.
• Authenticated middleware E2E: Add a TLS fixture built against the new SDK feature and one Docker happy path covering gateway and supervisor authentication, request transformation, local upstream verification, ephemeral trust setup, a dedicated mise task, and a GitHub Actions E2E lane.

Follow-up

Issue #2794 tracks the next layer of work:

• migrate examples/supervisor-middleware-content-guard to the SDK and use it as the behavioral E2E fixture
• add HTTP and WebSocket redact/deny coverage, invalid configuration, failure modes, chaining, and payload-limit scenarios
• add direct negative verifier tests for issuer, audience, signing key, kid, algorithm, time bounds, claim shape, and JWKS validation
• require authentication even on unsupported fixture RPC stubs

Testing

• mise run pre-commit passes
• SDK extension authentication unit tests pass
• Standalone middleware fixture builds with its locked dependency graph
• mise run e2e:supervisor-middleware passes against a Docker-backed gateway

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2784.docs.buildwithfern.com/openshell

[pimlock]

/ok to test 3128169

[github-actions]

Label test:e2e applied for 3128169. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[pimlock]

gator-agent

PR Review Status

Validation: Project-valid maintainer-authored work implementing the authenticated supervisor-middleware SDK direction tracked by #1733, with follow-up scope tracked in #2794.
Head SHA: eb6130a1bfb4e8cc3407352ff20fc09b707df10c
Base SHA: 4dfeff59c39840ebaa076364e59f009b9829d20e
Merge base SHA: 44bf0df485f15c2992b04c1d597e6967b4b37bcd
Patch ID: d637a3d18fae55967333b59894ee8afd9719c7b5
Gator payload: 5
Review mode: initial
Previous reviewed SHA: none
Review budget exhausted: no
Maintainer decision required: no

Blocking findings:

• No blocking findings remain.

Carried findings:

• None.

Non-blocking suggestions:

• None.

Docs: The SDK README and the relevant Fern supervisor-middleware documentation are updated; no navigation change is needed for the existing page.

Checks: Branch Checks and Helm Lint passed. The required test:e2e workflow ran on this head, and the new supervisor-middleware E2E lane passed. The aggregate E2E gate is red because the unrelated rootless Podman sandbox_stop_start_preserves_workspace test failed while restarting a stopped sandbox.

Next state: gator:blocked

Gator is blocked pending an authorized rerun of the current-head E2E workflow. No rerun was performed by gator. Next action: a maintainer or the operator must authorize or perform the rerun of workflow run 32296038035.