Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/debug-openshell-cluster/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ journalctl -u <driver-service> --no-pager --lines=200
journalctl -u openshell-gateway --no-pager --lines=200
```

Custom names use `[openshell.drivers.<name>].socket_path`. A launch-time `--compute-driver-socket` override may also use `docker`, `podman`, `kubernetes`, or `vm`; the endpoint then takes precedence over built-in construction. The socket must be accessible only to the intended gateway identity. Check gateway logs for connection errors, `GetCapabilities` failures, or an unexpected advertised driver name. The advertised name is diagnostic metadata; negotiated features control optional behavior. The gateway does not create or supervise operator-supplied driver processes or sockets.
Custom names use `[openshell.drivers.<name>].socket_path`. A launch-time `--compute-driver-socket` override may also use `docker`, `podman`, `kubernetes`, or `vm`; the endpoint then takes precedence over built-in construction. First-party standalone drivers require the socket parent directory to be owned by the driver's effective UID, force its mode to `0700`, create the socket with mode `0600`, and accept only peers with that same UID. Check the parent and socket separately with `stat`; a gateway running under a different UID cannot connect even when filesystem permissions or group membership would otherwise allow it. Operator-supplied drivers must provide equivalent access control appropriate to their implementation. Check gateway logs for connection errors, `GetCapabilities` failures, or an unexpected advertised driver name. The advertised name is diagnostic metadata; negotiated features control optional behavior. The gateway does not create or supervise operator-supplied driver processes or sockets.

For configured gateway interceptors, inspect `[[openshell.gateway.interceptors]]`, their Unix or network endpoints, and gateway startup logs:

Expand Down
18 changes: 17 additions & 1 deletion .github/workflows/branch-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,20 @@ jobs:
e2e-task: e2e:kubernetes:workspace-managed
cli-artifact-prefix: rust-binary-cli

kubernetes-external-driver-e2e:
needs: [pr_metadata, build-gateway, build-supervisor, build-cli]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
permissions:
actions: read
contents: read
packages: read
uses: ./.github/workflows/e2e-kubernetes-test.yml
with:
image-tag: ${{ github.sha }}
job-name: Kubernetes E2E (external compute driver)
e2e-task: e2e:kubernetes:external-driver
cli-artifact-prefix: rust-binary-cli

kubernetes-workspace-operator-e2e:
needs: [pr_metadata, build-gateway, build-supervisor, build-cli]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
Expand Down Expand Up @@ -240,7 +254,7 @@ jobs:

core-e2e-result:
name: Core E2E result
needs: [pr_metadata, build-gateway, build-supervisor, build-cli, build-driver-vm-linux, e2e, kubernetes-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
needs: [pr_metadata, build-gateway, build-supervisor, build-cli, build-driver-vm-linux, e2e, kubernetes-e2e, kubernetes-external-driver-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
if: always() && needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
runs-on: ubuntu-latest
steps:
Expand All @@ -252,6 +266,7 @@ jobs:
BUILD_DRIVER_VM_RESULT: ${{ needs.build-driver-vm-linux.result }}
E2E_RESULT: ${{ needs.e2e.result }}
KUBERNETES_E2E_RESULT: ${{ needs.kubernetes-e2e.result }}
KUBERNETES_EXTERNAL_DRIVER_E2E_RESULT: ${{ needs.kubernetes-external-driver-e2e.result }}
KUBERNETES_WORKSPACE_MANAGED_E2E_RESULT: ${{ needs.kubernetes-workspace-managed-e2e.result }}
KUBERNETES_WORKSPACE_OPERATOR_E2E_RESULT: ${{ needs.kubernetes-workspace-operator-e2e.result }}
run: |
Expand All @@ -264,6 +279,7 @@ jobs:
"build-driver-vm-linux:$BUILD_DRIVER_VM_RESULT" \
"e2e:$E2E_RESULT" \
"kubernetes-e2e:$KUBERNETES_E2E_RESULT" \
"kubernetes-external-driver-e2e:$KUBERNETES_EXTERNAL_DRIVER_E2E_RESULT" \
"kubernetes-workspace-managed-e2e:$KUBERNETES_WORKSPACE_MANAGED_E2E_RESULT" \
"kubernetes-workspace-operator-e2e:$KUBERNETES_WORKSPACE_OPERATOR_E2E_RESULT"; do
name="${item%%:*}"
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/e2e-kubernetes-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,17 @@ jobs:
run: mise install --locked

# The openshell-policy crate transitively pulls in z3-sys, whose
# build script needs the z3 C/C++ headers and clang/bindgen to
# compile. The bare runner doesn't ship them; the CI container
# build script needs the z3 C/C++ headers, clang/bindgen, and CMake to
# compile both system-linked and bundled-Z3 builds. The bare runner
# doesn't ship them; the CI container
# image used by other Rust e2e jobs does, but we can't run this job
# there (the runner's container handler injects its own --network
# bridge, which conflicts with the --network host we need so kind's
# API server is reachable from the test process).
- name: Install z3 build deps
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends libz3-dev clang
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends libz3-dev clang cmake

- name: Log in to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
Expand Down
31 changes: 25 additions & 6 deletions .github/workflows/e2e-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ jobs:
- suite: rust-docker
cmd: "mise run --no-deps --skip-deps e2e:rust"
apt_packages: "openssh-client"
- suite: rust-docker-external-driver
cmd: "env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:docker:external-driver"
apt_packages: "openssh-client"
- suite: mcp
cmd: "mise run --no-deps --skip-deps e2e:mcp"
apt_packages: ""
Expand Down Expand Up @@ -127,7 +130,7 @@ jobs:
run: ${{ matrix.cmd }}

e2e-podman-rootless:
name: E2E (rust-podman-rootless, ${{ matrix.runner }})
name: E2E (rust-podman-${{ matrix.suite }}, ${{ matrix.runner }})
# Run directly on the Ubuntu host so the test observes the host's AppArmor
# and unprivileged-user-namespace policy. A privileged job container masks
# the restrictions that production rootless Podman installations enforce.
Expand All @@ -142,10 +145,18 @@ jobs:
include:
# Keep package versions explicit so hosted-runner tool overrides
# cannot silently change the supported test environment.
- runner: ubuntu-26.04
- suite: rootless
runner: ubuntu-26.04
podman_major: "5"
podman_package_version: "5.7.0+ds2-3build1"
conmon_package_version: "2.1.13+ds1-2"
cmd: "mise run --no-deps --skip-deps e2e:podman:rootless"
- suite: external-driver
runner: ubuntu-26.04
podman_major: "5"
podman_package_version: "5.7.0+ds2-3build1"
conmon_package_version: "2.1.13+ds1-2"
cmd: "env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:podman:external-driver"
env:
IMAGE_TAG: ${{ inputs.image-tag }}
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -261,20 +272,28 @@ jobs:
- name: Log in to GHCR with Podman
run: echo "${{ secrets.GITHUB_TOKEN }}" | podman login ghcr.io -u "${{ github.actor }}" --password-stdin

- name: Run rootless Podman E2E
run: mise run --no-deps --skip-deps e2e:podman:rootless
- name: Run Podman E2E
run: ${{ matrix.cmd }}

- name: Print AppArmor denials
if: always()
run: sudo dmesg | grep -E 'apparmor=.*DENIED|profile="unprivileged_userns"' | tail -100 || true

e2e-vm:
name: E2E (rust-vm)
name: E2E (rust-vm-${{ matrix.suite }})
# libkrun needs KVM, so this job must run directly on a GitHub-hosted
# Linux VM. GitHub-hosted macOS runners do not support nested
# virtualization, and a job container would hide the host KVM device.
runs-on: ubuntu-24.04
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
include:
- suite: managed
cmd: "mise run --no-deps --skip-deps e2e:vm"
- suite: external-driver
cmd: "env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:vm:external-driver"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -362,4 +381,4 @@ jobs:
cache-on-failure: "true"

- name: Run VM E2E
run: mise run --no-deps --skip-deps e2e:vm
run: ${{ matrix.cmd }}
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,15 @@ to `run_cli_with_compute_drivers`; factories receive merged driver config and
finish through the same in-process runtime adapter. A configured UDS endpoint
still takes precedence over a compiled registration with the same name.

The standard server crate groups first-party registrations behind the
`in-tree-compute-drivers` feature. Protocol-only gateway builds disable that
feature and link no compute-driver crates. E2E lanes compose that gateway with
Docker, Podman, Kubernetes, and VM driver executables over the public UDS gRPC
contract so an in-tree driver cannot silently depend on a server-only API.
External Kubernetes drivers support shared and managed workspace modes.
Operator mode requires an in-process dynamic namespace allowlist and is
rejected when Kubernetes is configured through an external endpoint.

## Stop and Start Lifecycle

The gateway persists lifecycle intent before mutating compute:
Expand Down
1 change: 1 addition & 0 deletions crates/openshell-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ tempfile = { version = "3", optional = true }

[target.'cfg(unix)'.dependencies]
nix = { workspace = true }
rustix = { workspace = true }

[features]
default = ["telemetry"]
Expand Down
Loading
Loading