parsl.providers.KubernetesProvider¶
- class parsl.providers.KubernetesProvider(image: str, namespace: str = 'default', nodes_per_block: int = 1, init_blocks: int = 4, min_blocks: int = 0, max_blocks: int = 10, max_cpu: float = 2, max_mem: str = '500Mi', init_cpu: float = 1, init_mem: str = '250Mi', parallelism: float = 1, worker_init: str = '', pod_name: Optional[str] = None, user_id: Optional[str] = None, group_id: Optional[str] = None, run_as_non_root: bool = False, secret: Optional[str] = None, persistent_volumes: List[Tuple[str, str]] = [])[source]¶
Kubernetes execution provider
- Parameters
namespace (str) – Kubernetes namespace to create deployments.
image (str) – Docker image to use in the deployment.
nodes_per_block (int) – Nodes to provision per block.
init_blocks (int) – Number of blocks to provision at the start of the run. Default is 1.
min_blocks (int) – Minimum number of blocks to maintain.
max_blocks (int) – Maximum number of blocks to maintain.
max_cpu (float) – CPU limits of the blocks (pods), in cpu units. This is the cpu “limits” option for resource specification. Check kubernetes docs for more details. Default is 2.
max_mem (str) – Memory limits of the blocks (pods), in Mi or Gi. This is the memory “limits” option for resource specification on kubernetes. Check kubernetes docs for more details. Default is 500Mi.
init_cpu (float) – CPU limits of the blocks (pods), in cpu units. This is the cpu “requests” option for resource specification. Check kubernetes docs for more details. Default is 1.
init_mem (str) – Memory limits of the blocks (pods), in Mi or Gi. This is the memory “requests” option for resource specification on kubernetes. Check kubernetes docs for more details. Default is 250Mi.
parallelism (float) – Ratio of provisioned task slots to active tasks. A parallelism value of 1 represents aggressive scaling where as many resources as possible are used; parallelism close to 0 represents the opposite situation in which as few resources as possible (i.e., min_blocks) are used.
worker_init (str) – Command to be run first for the workers, such as
python start.py
.secret (str) – The Kubernetes ImagePullSecret secret to use to pull images
pod_name (str) – The name for the pod, will be appended with a timestamp. Default is None, meaning parsl automatically names the pod.
user_id (str) – Unix user id to run the container as.
group_id (str) – Unix group id to run the container as.
run_as_non_root (bool) – Run as non-root (True) or run as root (False).
persistent_volumes (list[(str, str)]) – List of tuples describing persistent volumes to be mounted in the pod. The tuples consist of (PVC Name, Mount Directory).
- __init__(image: str, namespace: str = 'default', nodes_per_block: int = 1, init_blocks: int = 4, min_blocks: int = 0, max_blocks: int = 10, max_cpu: float = 2, max_mem: str = '500Mi', init_cpu: float = 1, init_mem: str = '250Mi', parallelism: float = 1, worker_init: str = '', pod_name: Optional[str] = None, user_id: Optional[str] = None, group_id: Optional[str] = None, run_as_non_root: bool = False, secret: Optional[str] = None, persistent_volumes: List[Tuple[str, str]] = []) None [source]¶
Methods
__init__
(image[, namespace, ...])cancel
(job_ids)Cancels the jobs specified by a list of job ids Args: job_ids : [<job_id> ...] Returns : [True/False...] : If the cancel operation fails the entire list will be False.
status
(job_ids)Get the status of a list of jobs identified by the job identifiers returned from the submit request.
submit
(cmd_string, tasks_per_node[, job_name])Submit a job :param - cmd_string: (String) - Name of the container to initiate :param - tasks_per_node: command invocations to be launched per node :type - tasks_per_node: int
Attributes
cores_per_node
Number of cores to provision per node.
Provides the label for this provider
mem_per_node
Real memory to provision per node in GB.
Returns the interval, in seconds, at which the status method should be called.
- cancel(job_ids)[source]¶
Cancels the jobs specified by a list of job ids Args: job_ids : [<job_id> …] Returns : [True/False…] : If the cancel operation fails the entire list will be False.
- status(job_ids)[source]¶
Get the status of a list of jobs identified by the job identifiers returned from the submit request. :param - job_ids: A list of job identifiers :type - job_ids: list
- Returns
A list of JobStatus objects corresponding to each job_id in the job_ids list.
- Raises
- ExecutionProviderExceptions or its subclasses –
- property status_polling_interval[source]¶
Returns the interval, in seconds, at which the status method should be called.
- Returns
the number of seconds to wait between calls to status()
- submit(cmd_string, tasks_per_node, job_name='parsl')[source]¶
Submit a job :param - cmd_string: (String) - Name of the container to initiate :param - tasks_per_node: command invocations to be launched per node :type - tasks_per_node: int
- Kwargs:
job_name (String): Name for job, must be unique
- Returns
At capacity, cannot provision more - job_id: (string) Identifier for the job
- Return type
None