Signature
Parameters
Optional[str]
template hash (required, but Note: if you use this field, you can skip search_params, as they are automatically inferred from the template)
Optional[int]
template id (optional)
bool
Disable default search param query args
Optional[str]
launch args string for create instance ex: “—onstart onstart_wget.sh —env ‘-e ONSTART_PATH=https://s3.amazonaws.com/vast.ai/onstart_OOBA.sh’ —image atinoda/text-generation-webui:default-nightly —disk 64”
Optional[str]
deployment endpoint name (allows multiple workergroups to share same deployment endpoint)
Optional[int]
deployment endpoint id (allows multiple workergroups to share same deployment endpoint)
Optional[int]
default:"3"
number of workers to create to get an performance estimate for while initializing workergroup (default 3)
Optional[float]
estimated GPU RAM req (independent of search string)
Optional[str]
search param string for search offers ex: “gpu_ram>=23 num_gpus=2 gpu_name=RTX_4090 inet_down>200 direct_port_count>2 disk_space>=64”
Optional[float]
[NOTE: this field isn’t currently used at the workergroup level] minimum floor load in perf units/s (token/s for LLms)
Optional[float]
[NOTE: this field isn’t currently used at the workergroup level] target capacity utilization (fraction, max 1.0, default 0.9)
Optional[float]
[NOTE: this field isn’t currently used at the workergroup level]cold/stopped instance capacity target as multiple of hot capacity target (default 2.0)
Optional[int]
min number of workers to keep ‘cold’ for this workergroup
Returns
dict