Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .agents/skills/debug-openshell-cluster/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@ The target deployment flow is:
4. The CLI registers a reachable gateway endpoint with `openshell gateway add`.
5. The gateway creates sandboxes through the selected compute driver.

The standard gateway binary explicitly installs its compiled Docker, Podman,
Kubernetes, and VM registrations at startup. With no configured driver, the
gateway probes only installed registrations in priority order (Kubernetes,
Podman, then Docker); VM has no probe and remains opt-in. A custom gateway
binary may install a different set, so confirm the binary's registered drivers
when auto-detection reports that no suitable driver is available.

For local evaluation only, TLS may be disabled and the gateway can be reached through `http://127.0.0.1:<port>`.

## Prerequisites
Expand Down
22 changes: 22 additions & 0 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,28 @@ The gateway records driver identity and version from the startup capability
response. Elevated gateway info reports that initialized driver snapshot instead
of re-querying drivers on each request.

## Compiled Driver Selection

The gateway binary explicitly installs the compute drivers compiled into that
binary before entering server startup. The server selects a configured driver
by normalized registry name. When no driver is configured, it evaluates only
the installed drivers' probes in registered priority order, records every
available registration, and selects the first. Drivers without a probe,
including VM, remain opt-in.

Startup computes this selection once after merging configuration. The same
selection drives authentication defaults and runtime construction, so a probe
result cannot change which driver is constructed later in startup.

This follows the same composition model as SQLx's `Any` drivers: the binary
defines the available implementation set, while the runtime consumes a generic
registry. Adding or removing a compiled driver therefore changes registration
rather than the server's selection flow. Alternate gateway binaries can install
their own `ComputeDriverFactory` registrations and hand the completed registry
to `run_cli_with_compute_drivers`; factories receive merged driver config and
finish through the same in-process runtime adapter. A configured UDS endpoint
still takes precedence over a compiled registration with the same name.

## Stop and Start Lifecycle

The gateway persists lifecycle intent before mutating compute:
Expand Down
8 changes: 6 additions & 2 deletions crates/openshell-core/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,9 @@ pub fn detect_driver() -> Option<ComputeDriverKind> {
None
}

fn is_podman_available() -> bool {
/// Return whether a responsive local Podman API socket is available.
#[must_use]
pub fn is_podman_available() -> bool {
detect_podman_socket().is_some()
}

Expand Down Expand Up @@ -266,7 +268,9 @@ fn podman_socket_candidates_from_env(
candidates
}

fn is_docker_available() -> bool {
/// Return whether a responsive local Docker API socket is available.
#[must_use]
pub fn is_docker_available() -> bool {
detect_docker_socket().is_some()
}

Expand Down
196 changes: 139 additions & 57 deletions crates/openshell-server/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ use crate::compute::driver_config::GuestTlsPaths;
use crate::config_file::{self, ConfigFile, GatewayFileSection};
use crate::defaults::{self, LocalTlsPaths};
use crate::{
ServerStartupConfig, configured_compute_driver_for_startup, run_server,
tracing_bus::TracingLogBus,
ComputeDriverRegistry, ServerStartupConfig, configured_compute_driver_for_startup,
install_default_compute_drivers, run_server, tracing_bus::TracingLogBus,
};

/// `OpenShell` gateway process - gRPC and HTTP server with protocol multiplexing.
Expand Down Expand Up @@ -223,6 +223,11 @@ pub fn command() -> Command {
}

pub async fn run_cli() -> Result<()> {
run_cli_with_compute_drivers(install_default_compute_drivers()).await
}

/// Run the gateway CLI with the compute drivers linked by the binary.
pub async fn run_cli_with_compute_drivers(compute_drivers: ComputeDriverRegistry) -> Result<()> {
rustls::crypto::ring::default_provider()
.install_default()
.map_err(|e| miette::miette!("failed to install rustls crypto provider: {e:?}"))?;
Expand All @@ -232,11 +237,15 @@ pub async fn run_cli() -> Result<()> {

match cli.command {
Some(Commands::GenerateCerts(args)) => certgen::run(args).await,
None => Box::pin(run_from_args(cli.run, matches)).await,
None => Box::pin(run_from_args(cli.run, matches, compute_drivers)).await,
}
}

fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<ServerStartupConfig> {
fn prepare_server_config(
args: &mut RunArgs,
matches: &ArgMatches,
compute_drivers: &ComputeDriverRegistry,
) -> Result<ServerStartupConfig> {
// Load TOML when explicitly requested, or from the default XDG location
// when that file exists. Missing default config is not an error: runtime
// defaults and OPENSHELL_* env vars are enough for package-managed starts.
Expand All @@ -250,6 +259,10 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
merge_file_into_args(args, &file.openshell.gateway, matches);
}
normalize_compute_driver_socket_args(args, matches)?;
let compute_driver = compute_drivers
.select(&args.drivers)
.map_err(|error| miette::miette!("{error}"))?;
let compute_driver_kind = compute_driver.name().parse::<ComputeDriverKind>().ok();

let local_tls = apply_runtime_defaults(args)?;
let guest_tls = local_tls.as_ref().map(GuestTlsPaths::from);
Expand All @@ -259,7 +272,8 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser

let has_client_ca = args.tls_client_ca.is_some();
let has_oidc = args.oidc_issuer.is_some();
let mtls_auth_enabled = resolve_mtls_auth_enabled(args, matches, file.as_ref());
let mtls_auth_enabled =
resolve_mtls_auth_enabled(args, matches, file.as_ref(), compute_driver_kind);

if args.disable_tls && has_client_ca {
return Err(miette::miette!(
Expand All @@ -276,12 +290,7 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
"mTLS user authentication requires --tls-client-ca so client certificates can be verified."
));
}
if mtls_auth_enabled
&& matches!(
effective_single_driver(args),
Some(ComputeDriverKind::Kubernetes)
)
{
if mtls_auth_enabled && matches!(compute_driver_kind, Some(ComputeDriverKind::Kubernetes)) {
return Err(miette::miette!(
"mTLS user authentication is not supported with the Kubernetes compute driver. Configure OIDC or a trusted fronting proxy for user authentication."
));
Expand Down Expand Up @@ -469,12 +478,17 @@ fn prepare_server_config(args: &mut RunArgs, matches: &ArgMatches) -> Result<Ser
config,
config_file: file,
guest_tls,
compute_driver,
})
}

async fn run_from_args(mut args: RunArgs, matches: ArgMatches) -> Result<()> {
let prepared = prepare_server_config(&mut args, &matches)?;
let compute_driver = configured_compute_driver_for_startup(&prepared)?;
async fn run_from_args(
mut args: RunArgs,
matches: ArgMatches,
compute_drivers: ComputeDriverRegistry,
) -> Result<()> {
let prepared = prepare_server_config(&mut args, &matches, &compute_drivers)?;
let compute_driver = configured_compute_driver_for_startup(&compute_drivers, &prepared)?;

let tracing_log_bus = TracingLogBus::new();
let otlp_config = prepared
Expand Down Expand Up @@ -779,17 +793,9 @@ fn normalize_compute_driver_socket_args(args: &mut RunArgs, matches: &ArgMatches
}
}

fn effective_single_driver(args: &RunArgs) -> Option<ComputeDriverKind> {
match args.drivers.as_slice() {
[] => openshell_core::config::detect_driver(),
[driver] => driver.parse().ok(),
_ => None,
}
}

fn is_singleplayer_driver(args: &RunArgs) -> bool {
fn is_singleplayer_driver(driver: Option<ComputeDriverKind>) -> bool {
matches!(
effective_single_driver(args),
driver,
Some(ComputeDriverKind::Docker | ComputeDriverKind::Podman | ComputeDriverKind::Vm)
)
}
Expand All @@ -798,6 +804,7 @@ fn resolve_mtls_auth_enabled(
args: &RunArgs,
matches: &ArgMatches,
file: Option<&ConfigFile>,
compute_driver: Option<ComputeDriverKind>,
) -> bool {
let file_configured = file
.and_then(|f| f.openshell.gateway.mtls_auth.as_ref())
Expand All @@ -810,7 +817,7 @@ fn resolve_mtls_auth_enabled(
return false;
}

is_singleplayer_driver(args)
is_singleplayer_driver(compute_driver)
}

#[cfg(test)]
Expand All @@ -819,6 +826,43 @@ mod tests {
use crate::TEST_ENV_LOCK as ENV_LOCK;
use clap::Parser;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicUsize, Ordering};

static REGISTRY_DETECTION_CALLS: AtomicUsize = AtomicUsize::new(0);

fn detect_registered_docker() -> bool {
REGISTRY_DETECTION_CALLS.fetch_add(1, Ordering::SeqCst);
true
}

#[derive(Clone, Copy)]
struct TestComputeDriverFactory;

#[async_trait::async_trait]
impl crate::ComputeDriverFactory for TestComputeDriverFactory {
async fn build(
&self,
_context: crate::ComputeDriverBuildContext<'_>,
) -> openshell_core::Result<crate::ComputeDriverBuildOutput> {
unreachable!("configuration tests do not construct the driver")
}
}

fn detected_docker_registry() -> crate::ComputeDriverRegistry {
let mut registry = crate::ComputeDriverRegistry::new();
registry
.install(
crate::ComputeDriverRegistration::new(
"docker",
100,
Some(detect_registered_docker),
TestComputeDriverFactory,
)
.unwrap(),
)
.unwrap();
registry
}

struct EnvVarGuard {
key: &'static str,
Expand Down Expand Up @@ -1286,7 +1330,47 @@ mod tests {
"/tmp/ca.crt",
]);

assert!(super::resolve_mtls_auth_enabled(&args, &matches, None));
assert!(super::resolve_mtls_auth_enabled(
&args,
&matches,
None,
Some(openshell_core::ComputeDriverKind::Docker)
));
}

#[test]
fn registry_detection_drives_auth_defaults_once() {
let _lock = ENV_LOCK
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let state = tempfile::tempdir().unwrap();
let config = tempfile::tempdir().unwrap();
let _state = EnvVarGuard::set("XDG_STATE_HOME", state.path().to_str().unwrap());
let _config = EnvVarGuard::set("XDG_CONFIG_HOME", config.path().to_str().unwrap());
let _kubernetes = EnvVarGuard::set("KUBERNETES_SERVICE_HOST", "10.0.0.1");
let _mtls = EnvVarGuard::remove("OPENSHELL_ENABLE_MTLS_AUTH");
let _drivers = EnvVarGuard::remove("OPENSHELL_DRIVERS");
REGISTRY_DETECTION_CALLS.store(0, Ordering::SeqCst);

let (mut args, matches) = parse_with_args(&[
"openshell-gateway",
"--db-url",
"sqlite::memory:",
"--tls-cert",
"/tmp/server.crt",
"--tls-key",
"/tmp/server.key",
"--tls-client-ca",
"/tmp/ca.crt",
]);
let registry = detected_docker_registry();

let prepared = super::prepare_server_config(&mut args, &matches, &registry).unwrap();

assert_eq!(prepared.compute_driver.name(), "docker");
assert!(prepared.config.compute_drivers.is_empty());
assert!(prepared.config.mtls_auth.enabled);
assert_eq!(REGISTRY_DETECTION_CALLS.load(Ordering::SeqCst), 1);
}

#[test]
Expand All @@ -1310,7 +1394,12 @@ mod tests {
"/tmp/ca.crt",
]);

assert!(!super::resolve_mtls_auth_enabled(&args, &matches, None));
assert!(!super::resolve_mtls_auth_enabled(
&args,
&matches,
None,
Some(openshell_core::ComputeDriverKind::Kubernetes)
));
}

#[test]
Expand Down Expand Up @@ -1345,7 +1434,8 @@ enabled = false
assert!(!super::resolve_mtls_auth_enabled(
&args,
&matches,
Some(&file)
Some(&file),
Some(openshell_core::ComputeDriverKind::Docker)
));
}

Expand Down Expand Up @@ -1573,37 +1663,21 @@ ssh_session_ttl_secs = 1234

#[test]
fn singleplayer_driver_matches_only_one_local_driver() {
for driver in ["docker", "podman", "vm"] {
let (args, _) = parse_with_args(&[
"openshell-gateway",
"--db-url",
"sqlite::memory:",
"--drivers",
driver,
]);
for driver in [
openshell_core::ComputeDriverKind::Docker,
openshell_core::ComputeDriverKind::Podman,
openshell_core::ComputeDriverKind::Vm,
] {
assert!(
super::is_singleplayer_driver(&args),
super::is_singleplayer_driver(Some(driver)),
"{driver} should be singleplayer"
);
}

let (k8s, _) = parse_with_args(&[
"openshell-gateway",
"--db-url",
"sqlite::memory:",
"--drivers",
"kubernetes",
]);
assert!(!super::is_singleplayer_driver(&k8s));

let (multi, _) = parse_with_args(&[
"openshell-gateway",
"--db-url",
"sqlite::memory:",
"--drivers",
"docker,podman",
]);
assert!(!super::is_singleplayer_driver(&multi));
assert!(!super::is_singleplayer_driver(Some(
openshell_core::ComputeDriverKind::Kubernetes
)));
assert!(!super::is_singleplayer_driver(None));
}

#[test]
Expand All @@ -1629,7 +1703,11 @@ ssh_session_ttl_secs = 1234
Some(std::path::Path::new("/run/openshell/kyma.sock"))
);
assert_eq!(args.drivers, ["kyma"]);
assert!(super::effective_single_driver(&args).is_none());
assert!(
args.drivers[0]
.parse::<openshell_core::ComputeDriverKind>()
.is_err()
);
}

#[test]
Expand Down Expand Up @@ -1809,8 +1887,12 @@ mem_mib = "not-a-number"
"--disable-tls",
]);

let prepared =
super::prepare_server_config(&mut args, &matches).expect("server config is prepared");
let prepared = super::prepare_server_config(
&mut args,
&matches,
&crate::install_default_compute_drivers(),
)
.expect("server config is prepared");

assert_eq!(prepared.config.compute_drivers, vec!["podman".to_string()]);
assert_eq!(
Expand Down
Loading
Loading