parsl.providers.KubernetesProvider

class parsl.providers.KubernetesProvider(image: str, namespace: str = 'default', nodes_per_block: int = 1, init_blocks: int = 4, min_blocks: int = 0, max_blocks: int = 10, max_cpu: float = 2, max_mem: str = '500Mi', init_cpu: float = 1, init_mem: str = '250Mi', parallelism: float = 1, worker_init: str = '', pod_name: str | None = None, user_id: str | None = None, group_id: str | None = None, run_as_non_root: bool = False, secret: str | None = None, persistent_volumes: List[Tuple[str, str]] = [])[source]

Kubernetes execution provider

Parameters:

  • namespace (str) – Kubernetes namespace to create deployments.

  • image (str) – Docker image to use in the deployment.

  • nodes_per_block (int) – Nodes to provision per block.

  • init_blocks (int) – Number of blocks to provision at the start of the run. Default is 1.

  • min_blocks (int) – Minimum number of blocks to maintain.

  • max_blocks (int) – Maximum number of blocks to maintain.

  • max_cpu (float) – CPU limits of the blocks (pods), in cpu units. This is the cpu “limits” option for resource specification. Check kubernetes docs for more details. Default is 2.

  • max_mem (str) – Memory limits of the blocks (pods), in Mi or Gi. This is the memory “limits” option for resource specification on kubernetes. Check kubernetes docs for more details. Default is 500Mi.

  • init_cpu (float) – CPU limits of the blocks (pods), in cpu units. This is the cpu “requests” option for resource specification. Check kubernetes docs for more details. Default is 1.

  • init_mem (str) – Memory limits of the blocks (pods), in Mi or Gi. This is the memory “requests” option for resource specification on kubernetes. Check kubernetes docs for more details. Default is 250Mi.

  • parallelism (float) – Ratio of provisioned task slots to active tasks. A parallelism value of 1 represents aggressive scaling where as many resources as possible are used; parallelism close to 0 represents the opposite situation in which as few resources as possible (i.e., min_blocks) are used.

  • worker_init (str) – Command to be run first for the workers, such as python start.py.

  • secret (str) – The Kubernetes ImagePullSecret secret to use to pull images

  • pod_name (str) – The name for the pod, will be appended with a timestamp. Default is None, meaning parsl automatically names the pod.

  • user_id (str) – Unix user id to run the container as.

  • group_id (str) – Unix group id to run the container as.

  • run_as_non_root (bool) – Run as non-root (True) or run as root (False).

  • persistent_volumes (list[(str, str)]) – List of tuples describing persistent volumes to be mounted in the pod. The tuples consist of (PVC Name, Mount Directory).

__init__(image: str, namespace: str = 'default', nodes_per_block: int = 1, init_blocks: int = 4, min_blocks: int = 0, max_blocks: int = 10, max_cpu: float = 2, max_mem: str = '500Mi', init_cpu: float = 1, init_mem: str = '250Mi', parallelism: float = 1, worker_init: str = '', pod_name: str | None = None, user_id: str | None = None, group_id: str | None = None, run_as_non_root: bool = False, secret: str | None = None, persistent_volumes: List[Tuple[str, str]] = []) → None[source]

Methods

__init__(image[, namespace, ...])

cancel(job_ids)

Cancels the jobs specified by a list of job ids Args: job_ids : [<job_id> ...] Returns : [True/False...] : If the cancel operation fails the entire list will be False.

status(job_ids)

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

submit(cmd_string, tasks_per_node[, job_name])

Submit a job :param - cmd_string: (String) - Name of the container to initiate :param - tasks_per_node: command invocations to be launched per node :type - tasks_per_node: int

Attributes

cores_per_node

Number of cores to provision per node.

label

Provides the label for this provider

mem_per_node

Real memory to provision per node in GB.

status_polling_interval

Returns the interval, in seconds, at which the status method should be called.
cancel(job_ids)[source]

Cancels the jobs specified by a list of job ids Args: job_ids : [<job_id> …] Returns : [True/False…] : If the cancel operation fails the entire list will be False.

property label[source]

Provides the label for this provider

status(job_ids)[source]

Get the status of a list of jobs identified by the job identifiers returned from the submit request. :param - job_ids: A list of job identifiers :type - job_ids: list

Returns:

  • A list of JobStatus objects corresponding to each job_id in the job_ids list.

Raises:

- ExecutionProviderExceptions or its subclasses –

property status_polling_interval[source]

Returns the interval, in seconds, at which the status method should be called.

Returns:

the number of seconds to wait between calls to status()

submit(cmd_string, tasks_per_node, job_name='parsl')[source]

Submit a job :param - cmd_string: (String) - Name of the container to initiate :param - tasks_per_node: command invocations to be launched per node :type - tasks_per_node: int

Kwargs:

  • job_name (String): Name for job, must be unique

Returns:

At capacity, cannot provision more - job_id: (string) Identifier for the job

Return type:

  • None