Spot capacity and direct SSH
View MarkdownChoose Instances → Launch instance → Capacity → Spot (interruptible) for interruptible compute. Existing managed offerings retain their terminal, VS Code and Jupyter access; dedicated VM and GPU-container quotes show Direct SSH, with the environment shown before launch. Quotes do not guarantee launch capacity. Direct-SSH instances are organization-wide and appear in each project’s Instances list.
For a temporary disk, Location → Lowest eligible price is the console default. Nodus checks matching locations at launch; the summary shows the lowest current eligible quote. Select a specific location to keep placement there. Your GPU count, hourly ceiling and transfer consent still apply. Retained Workspaces keep their separate placement settings; selecting another region does not copy existing files.
The direct-SSH API gives your scripts a Nodus SSH command for the dedicated node. Nodes can be preempted at any
time. Your own scripts remain responsible for saving progress and restarting work during the pilot.
The API exposes execution_environment (vm or container). A GPU container does not provide host administration or nested VMs. Retained storage is available only when supports_workspace_storage is true; always check that field before planning checkpoint retention.
The controlled staging GPU-container pilot allocates a fixed 512 GB temporary disk per node. It does not support retained Workspace storage, and files on that disk do not survive instance release. Save environments, caches and checkpoints externally when they must survive replacement. This staging pilot is not general production availability.
Sign in and save an SSH key
Section titled “Sign in and save an SSH key”pip install --upgrade nodus-computenodus loginnodus create sshkey laptop --from-file ~/.ssh/id_ed25519.pubYou can also save public keys in the console’s Account → SSH keys. Keep the private key on your machine. Spot uses the same organization, API keys and billing account as your other Nodus work.
A quoted price is not a reservation. A location can be unavailable when no admitted price ceiling fits your bid, even if its displayed market price is lower. The request’s maximum falls back to your account default; without either, a finite platform limit still applies. If capacity checks fail, launch is refused safely.
A GPU route that fails to become ready is temporarily excluded for 15 minutes, including when startup reaches its 15-minute deadline. Its price remains visible with unavailable status. Other eligible routes can be selected within your maximum and storage constraints; no on-demand fallback is used. Releasing a node yourself or reaching your price limit does not trigger this cooldown. Failed startup incurs no compute charge; retained storage is billed separately.
Only routes enabled for public launch can receive new requests. A listed price alone does not mean a GPU and location is available to launch. Existing instances keep their access, billing and release behavior when new launches for their route are disabled.
Check prices and launch
Section titled “Check prices and launch”nodus spot pricesnodus spot up h200x8 --max-price YOUR_NODE_PRICE_CAP --waitFor training, create a retained Workspace below and pass --workspace WORKSPACE_ID. A launch without it
has only a temporary boot disk; releasing or preempting it does not retain training files.
Prices and ceilings are USD per complete node-hour. The quote table shows available GPU configurations and named locations; a quote does not reserve capacity. The command prints an idempotency key, waits for capacity and SSH readiness, then prints an SSH command. Run that command from the machine with your private key.
Use --ssh-key PATH.pub to override saved keys, --regions eu-1 to restrict placement to Finland, and -o json
for machine-readable output. Use the stable location ID shown by nodus spot prices, or a region class such
as eu. Locations keep distinct storage: Finland (eu-1), France (eu-2), Iceland (eu-3) and United States 1
(us-1) are separate choices when quoted. Without --wait, the command returns the accepted node’s current
state immediately.
nodus spot lsnodus spot rm NODE_IDRelease ends compute billing immediately and starts deletion of the instance and its temporary disk. A preempted node is cleaned up automatically. Save needed files outside the boot disk before releasing it. A terminal node status reports the end of compute; resource cleanup can continue in the background.
Choose transfer billing explicitly (pilot)
Section titled “Choose transfer billing explicitly (pilot)”Some container quotes offer or require a separate data transfer contract. Run nodus spot prices -o json
and read transfer_billing_disclosure, transfer_markup_percent and transfer_billing_required before accepting it.
The console displays that disclosure with an unchecked consent checkbox. Changing launch choices clears consent.
For a container that becomes ready, this contract charges reported transfer costs over its entire lifetime, including startup and shutdown traffic, plus the transfer service fee frozen at launch (7.5% for this contract). Charges are provisional and may be debited or credited later, including after deletion. Never-ready startups are not charged. When a startup ends without becoming ready, its unused compute and transfer reserves return to your available balance. Resource cleanup can continue in the background; this does not authorize another launch automatically. The hourly compute ceiling does not cap transfer charges; monitored spending controls are not a guaranteed total cap.
Accept only within your approved transfer spending scope:
nodus spot up h100x1 --max-price YOUR_NODE_PRICE_CAP --transfer-billing actual_cost_plus_markup_v1 --waitPython takes transfer_billing="actual_cost_plus_markup_v1" in client.spot.request(). MCP takes the same
transfer_billing value in spot_request, separately from confirmed. An available quote does not grant consent.
Omitting the option keeps the legacy contract; a quote requiring it cannot launch without acceptance.
Waiting, capacity retries and idempotent replays preserve the exact selection. CLI resume uses the saved contract.
Node details show transfer_cost_total as the net lifetime transfer charge, separately from compute-only
cost_so_far. transfer_cost_provisional indicates that later reports can revise it. The accepted contract,
service fee and disclosure remain on the node. Usage date filters apply to compute intervals; transfer totals remain lifetime totals.
Connect through Nodus
Section titled “Connect through Nodus”nodus spot ssh NODE_ID -- nvidia-sminodus spot ssh NODE_ID --identity ~/.ssh/id_ed25519nodus spot ssh NODE_ID --configThe command carries SSH through the authenticated Nodus endpoint. Your private key stays on your computer.
Use --identity for a nondefault key that matches one installed at launch. Review the SSH host key on first
connection. --config prints an entry you can add to ~/.ssh/config for plain SSH, SCP, rsync or VS Code.
Keep your Nodus CLI session or NODUS_API_KEY available when using that entry. The key needs both
spot:read and spot:write; project-restricted keys cannot access these organization-wide nodes.
Upgrade the CLI and Python SDK before using this connection contract. The deprecated ip field is empty;
read ssh_command after the node becomes running. Scripts that connect directly to node["ip"] must use
nodus spot ssh NODE_ID instead. SSH port forwarding works through the same connection. Releasing the node
closes existing relay sessions after the next authorization check.
Use an AI agent through MCP (pilot)
Section titled “Use an AI agent through MCP (pilot)”Connect your agent using the MCP setup guide. Use an organization-wide key with
spot:read and spot:write, and save your public SSH key first. The agent can use the same hosted connection
or nodus mcp as other Nodus work.
- Call
spot_pricesand choose a GPU configuration within your approved budget. Prices are for the entire node. - Call
spot_requestwithgpu,max_price_per_node_hr, a uniqueidempotency_key, andconfirmed: trueonce the request is within your approved spending scope. Omitregionsfor the cheapest eligible location. Passworkspace_idto retain files; without it, the node has temporary storage. - Save the returned node ID. Poll
spot_getuntil it isrunningwithssh_command, then run that command on the machine holding your private SSH key. MCP returns connection instructions; it does not upload private keys. - Use
spot_usageto inspect charges andspot_releasewhen finished. Pollspot_getforcleaned_atto confirm compute cleanup. Retained Workspace storage is deleted separately and keeps billing while retained.
A no-capacity error includes a retry delay. An uncertain request must be inspected with spot_list before
requesting again; retain its original key. These tools do not run a background retry or training recovery loop.
Retain a Workspace (pilot)
Section titled “Retain a Workspace (pilot)”A Workspace in Spot mode owns regional storage mounted at /workspace on linked nodes. Creating it starts
storage allocation and billing. It does not start a machine. This differs from a managed development
Workspace, which runs a session and saves its home directory.
In the console, open Workspaces → New retained storage. Choose the project, location, capacity and access, review the storage rate and reserved credit, then create it. Capacity choices include 512 GiB, 1 TiB, 2 TiB and 4 TiB, plus Custom for any whole number from 512 through 4096 GiB. For example, 1222 GiB is valid and the estimate uses exactly 1222 GiB. One instance uses a Block slot; Shared by instances uses SharedFilesystem. Each retained slot is billed for its allocated capacity. Open the Workspace to see retained regional slots and charges, and choose Launch linked instance. Only compatible regional Workspaces appear in a direct-SSH launch. Managed home storage cannot attach to these VMs. Capacity and access mode are fixed at creation; storage does not grow automatically. Compute release keeps linked storage and its charges; deleting the Workspace requires a separate confirmation. The launch form lets you reuse a slot and choose whether to wait for home-region capacity. Choose Temporary disk for disposable files, or Keep files in a workspace to attach retained storage.
apiVersion: nodus.dev/v1kind: Workspacemetadata: name: spot-trainingspec: mode: Spot regionalStorage: provider: nodus homeRegion: eu-1 sizeGiB: 512 mode: BlockApply the manifest, read its metadata.uid with nodus get workspace spot-training -o json, then launch:
nodus spot up h200x8 --workspace WORKSPACE_ID --placement local_only --max-price YOUR_NODE_PRICE_CAP --wait --timeout 30mSave files under /workspace. Releasing the node retains those files and their storage charge. A replacement
in the same region and slot mounts the retained filesystem before becoming ready. A block slot permits one
writer; 409 volume_in_use means its previous attachment has not cleared. For independent block disks use
--workspace-slot worker-2; each slot allocates and bills its own capacity. SharedFilesystem supports
simultaneous writers to the default slot in a region when enabled for the pilot.
The default local_first placement waits up to five minutes for home-region capacity before considering
other regions. Change the bound with --local-wait 5m (zero to 15 minutes), choose local_only to keep files
local, or choose cheapest to select by price immediately. Placement always respects your compute price
ceiling. A different region receives separate empty storage; files are not copied automatically.
Python accepts the same options: client.spot.request("h200x1", workspace_id="wsp_...", workspace_slot="default", placement="local_only", local_wait_s=300, max_price=3).
Storage bills allocated GiB over retained time, separately from GPU usage, including while no node is running.
The ordinary object storage free allowance does not apply. Storage quota and funded credit must cover each allocation;
For 429 storage_quota, an Owner or Admin can raise the storage limit in Settings › Quotas up to the account’s
approved ceiling, release storage that is no longer needed, or request a higher ceiling. Five separate 512 GiB
workspaces require at least 2,560 GiB of quota. Stopping compute does not delete retained files or free their storage
allocation. Settings › Compute separately shows active Spot nodes, simultaneous starts and their editable
account limits. Nodes awaiting cleanup remain counted; raising a limit does not guarantee available capacity.
To remove the data, release linked nodes and run nodus delete workspace spot-training --confirm.
The API requires DELETE .../workspaces/spot-training?confirm=true; Python supports
client.api.delete(..., confirm=True). Deletion waits for attachments and confirmed storage removal. This is
permanent deletion of the Workspace’s regional files, including any model checkpoints saved there.
In the console, deletion requires typing the Workspace name and confirming that its files will be lost.
Retained files do not restart a training process. During this pilot your command must write valid checkpoints, load them on restart, and request replacements. Managed checkpoint recovery is a separate next stage.
Retry initial capacity acquisition from the CLI
Section titled “Retry initial capacity acquisition from the CLI”Retry is opt-in and bounded. This command permits at most three accepted startup attempts within one 30-minute deadline, including the first attempt. It uses saved SSH keys and temporary boot storage:
nodus spot up h100x8 --max-price 12 --wait --retry-startup --max-attempts 3 --timeout 30m --state ./h100-acquisition.jsonOnly a confirmed capacity_unavailable failure before SSH readiness can be retried, and only after
cleaned_at confirms compute cleanup. Each attempt has a new node ID and request key; previous nodes remain
in history with zero compute charge. Quotes, limits and funding are checked again. Supply is not guaranteed.
The command honors the location, GPU and price constraints; it never falls back to on-demand or reserved compute.
Explicit --ssh-key public keys remain unchanged. If omitted, each attempt uses the account’s currently saved keys.
Keep the state file and its .lock companion. They contain public request and node data, not API credentials,
and are private to your user. After an interruption, use the printed resume command with the same API/account:
nodus spot up h100x8 --wait --retry-startup --state ./h100-acquisition.json --resumeAfter a node is accepted, CLI waiting retries temporary connection failures and HTTP 429, 502, 503 or 504
by reading the same node. It honors Retry-After within the original timeout. These read retries do not
request another node or release the accepted node. Authentication errors and other HTTP failures stop the wait.
Resume retains the original deadline and attempt limit. Time spent with your computer asleep counts toward
that deadline. A lost POST response without a saved node ID stops
for manual inspection with nodus spot ls; the CLI cannot safely resend that request. Timeout or cancellation
does not release an accepted node. Inspect it and use nodus spot rm NODE_ID when appropriate.
With --workspace, retries stay in the first accepted attachment’s location and keep the same slot; files
are never moved to another location. Use --placement local_only to keep initial placement at the Workspace’s
home. There is no retry after a node has ever become ready, and no automatic restart of training or recovery
from running-node preemption. The console retains its single-accepted-node behavior. Python waiting uses the separate timeout-bounded behavior below.
Poll safely from Python
Section titled “Poll safely from Python”Set SPOT_WORKSPACE_ID to the retained Workspace UID, SPOT_MAX_NODE_PRICE to your approved price ceiling,
and SPOT_REQUEST_KEY to a unique identifier for one logical request. Keep the key for inspection and recovery. After an ambiguous initial POST, inspect node history before resending. This example polls every 30 seconds and releases compute after inspecting its GPUs and mount.
"""Request one node with retained storage, inspect its GPUs, then release compute."""
import osimport subprocess
import nodus
client = nodus.Client()node = client.spot.request( "h200x8", max_price=float(os.environ["SPOT_MAX_NODE_PRICE"]), workspace_id=os.environ["SPOT_WORKSPACE_ID"], placement="local_only", poll=30, timeout=1800, idempotency_key=os.environ["SPOT_REQUEST_KEY"],)try: subprocess.run( [ "nodus", "spot", "ssh", node["id"], "--", "nvidia-smi && nvidia-smi topo -m && df -h /workspace", ], check=True, )finally: client.spot.release(node["id"])client.spot.request() waits by default, with a 15-minute timeout. Set wait=False for one immediate POST,
or set timeout=1800 for a longer wait. Waiting polls the accepted node, with the same temporary read-error
retries and Retry-After handling as the CLI. A typed capacity_unavailable startup
diagnosis on a start_failed or capacity_unavailable node permits another attempt only after cleaned_at is
present. The replacement uses a fresh request key, preserves the original timeout and price ceiling, and keeps
retained storage in the first accepted location. A key you supply identifies the first attempt; save node IDs for
later inspection. Other ended nodes, including preemptions
after readiness, raise nodus.errors.SpotNodeEnded; a timeout raises nodus.errors.SpotTimeout with node and
idempotency_key for recovery. Timeout or cancellation does not release the node. Inspect or release it
before starting a different request. Async callers use await client.spot.request.aio(...).
Other methods are client.spot.prices(), list(), get(node_id), release(node_id) and
usage(from_="2026-10-24T00:00:00Z", to="2026-10-25T00:00:00Z"). Results are ordinary JSON dictionaries
and lists. Optional schemas are available in nodus.api.models.spot.
Preemption webhooks
Section titled “Preemption webhooks”In Settings → Compute → Preemption webhooks, add your public HTTPS URL and save the signing secret shown
once. The form selects spot.preempted; existing endpoints subscribed to that type, spot.* or all events
also receive it. Owners and admins can edit, disable or rotate these endpoints using the same webhook controls.
The signed notification arrives after Nodus observes preemption. Its data.object contains the node’s id,
gpu, status: "preempted" and ended_at billing cutoff. It carries no advance eviction warning and does not
restart work. Verify the signature and deduplicate webhook-id; failed deliveries
retry for up to three days, and each endpoint exposes delivery history and replay controls.
API and usage
Section titled “API and usage”Send your existing bearer token to the platform API. The Spot OpenAPI document
describes the REST schemas and the SSH WebSocket handshake. SSH returns HTTP 101 and carries binary SSH
traffic after the upgrade; use nodus spot ssh rather than a generated JSON HTTP client for this connection.
| Method | Path | Result |
|---|---|---|
| GET | /v1/spot/prices |
Current node-hour quotes |
| POST | /v1/spot/nodes |
Accepted node or a capacity/payment refusal |
| GET | /v1/spot/nodes |
Your nodes |
| GET | /v1/spot/nodes/{id} |
Status, Nodus SSH command and accrued cost |
| GET | /v1/spot/nodes/{id}/ssh |
Authenticated binary WebSocket carrying SSH (spot:write) |
| DELETE | /v1/spot/nodes/{id} |
Release and cleanup status |
| GET | /v1/spot/usage?from=...&to=... |
Per-node price intervals and charges |
Include Idempotency-Key or idempotency_key in every POST. While the key is retained, a retry returns the same node.
Preemption and failed-start cleanup can release the key, so it does not permanently prevent a replacement.
After a running node is preempted and cleaned up, use a new key for a replacement. A diagnosed pre-start capacity
failure retains its key until physical cleanup is confirmed. If a launch response is lost or unreadable, the console
locks the launch form and offers Check instances. Inspect node history before making another request;
a terminal state alone is not proof of cleanup. A POST without a key reuses an existing
provisioning node of the same GPU type in your account.
503 no_capacity means no node was created. Read its message before retrying: a quote can fit your
maximum while its available price limit does not. In that case the message gives a minimum compatible
maximum in USD per node-hour, separate from the current quote. Choose another location, wait, or raise
your maximum only within your approved budget. Raising the maximum does not guarantee capacity or
change the current rate. For a capacity retry, wait retry_after seconds and reuse the same key.
503 price_above_max requires a cheaper quote within your approved ceiling. 402 payment_required needs action in
billing, while 429 node_limit requires releasing a node or changing your account limit.
An accepted node can still fail before SSH is ready. When capacity failure is confirmed, its JSON includes
startup_failure_code: "capacity_unavailable" and a plain-language startup_failure_message; the console
and CLI show that explanation. The node remains failed with no compute charge. A retryable failure can have
end_reason: "start_failed" or "capacity_unavailable"; only the typed diagnosis plus cleaned_at permits a
replacement. The cleanup of "capacity_unavailable" also exposes retry_after. Missing diagnostic fields
mean the specific cause is unknown. By default the CLI does not replace an accepted failed node. The opt-in
initial-acquisition mode waits for confirmed cleanup before a capacity retry; an available quote does not
guarantee that the next startup will succeed.
Compute billing begins when SSH is ready. Price changes appear as separate intervals in usage, and charges appear under Spot GPU in the existing billing history. The instance detail page shows accrued cost; posted billing usage updates on the metering cycle. Provisioning failures are unbilled. A billing warning includes a release time so you can save work and restore funding before the node is released.
Organizations approved for invoice terms see their credit limit, outstanding invoices, payment links and due dates in Usage & billing. Invoice credits become available when the approved invoice is issued; paying it settles that advance without adding the credits a second time. ACH processing can take several days. Overdue invoices or returned payments pause new launches and can trigger a release warning for running nodes.