SandboxClaim/warm_pool usage

[manstis]

Regarding support for SandboxClaim / warm_pool:

• feat: add warm-pool provisioning for Kubernetes sandboxes #2157
• feat(kubernetes): create SandboxClaims for matching warm pools #2460

When these are complete OpenShell will have support (with the k8s driver) for a pool of Pods to act as sandboxes.

My question relates to how the pool behaves when separate Sandboxes are requested.

I can create a provider:

openshell provider create --name my-provider --type openai --credential OPENAI_API_KEY=<an-api-key>

I can create the inference for the provider:

openshell inference set \
    --provider my-provider \
    --model <a-model-name>

I can create a Sandbox template:

openshell sandbox template create openshell-warm-pool-example ...

I can finally create a sandbox:

openshell sandbox create --template openshell-warm-pool-example -- \
    curl https://inference.local/v1/chat/completions \
    --json '{"messages":[{"role":"user","content":"hello"}]}'

IIUC that ties all sandboxes from the pool to a single provider and inference configuration.

I have a need to use transient sandboxes with different provider and inference configurations.

Cold-provisioning would introduce unacceptable latency and I hoped the warm_pool would give me access to a pool of pre-provisioned hollow/empty sandboxes that I could configure with specific provider and inference configurations when instantiating a sandbox from the pool.

I just want to understand how the pool is to be used.

[manstis]

@mrunalp I see the issue was reported by yourself; IDK if you know the answers?

I cannot ping @grs the PR author here and don't want to venture outside the prescribed communication channels.