parsl.providers.provider_base.ExecutionProvider

class parsl.providers.provider_base.ExecutionProvider

Execution providers are responsible for managing execution resources that have a Local Resource Manager (LRM). For instance, campus clusters and supercomputers generally have LRMs (schedulers) such as Slurm, Torque/PBS, Condor and Cobalt. Clouds, on the other hand, have API interfaces that allow much more fine-grained composition of an execution environment. An execution provider abstracts these types of resources and provides a single uniform interface to them.

The providers abstract away the interfaces provided by various systems to request, monitor, and cancel compute resources.

                      +------------------
                      |
script_string ------->|  submit
     id      <--------|---+
                      |
[ ids ]       ------->|  status
[statuses]   <--------|----+
                      |
[ ids ]       ------->|  cancel
[cancel]     <--------|----+
                      |
                      +-------------------

__init__()

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__()	Initialize self.
cancel(job_ids)	Cancels the resources identified by the job_ids provided by the user.
status(job_ids)	Get the status of a list of jobs identified by the job identifiers returned from the submit request.
submit(command, tasks_per_node[, job_name])	The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as IPP engine or even Swift-T engines).

Attributes

cores_per_node	Number of cores to provision per node.
label	Provides the label for this provider
mem_per_node	Real memory to provision per node in GB.
status_polling_interval	Returns the interval, in seconds, at which the status method should be called.

abstract cancel(job_ids: List[Any]) → List[bool]

Cancels the resources identified by the job_ids provided by the user.

Parameters

job_ids (-) – A list of job identifiers

Returns

• A list of status from cancelling the job which can be True, False

Raises

- ExecutionProviderException or its subclasses –

property cores_per_node

Number of cores to provision per node.

Providers which set this property should ask for cores_per_node cores when provisioning resources, and set the corresponding environment variable PARSL_CORES before executing submitted commands.

If this property is set, executors may use it to calculate how many tasks can run concurrently per node. This information is used by dataflow.Strategy to estimate the resources required to run all outstanding tasks.

abstract property label

Provides the label for this provider

property mem_per_node

Real memory to provision per node in GB.

Providers which set this property should ask for mem_per_node of memory when provisioning resources, and set the corresponding environment variable PARSL_MEMORY_GB before executing submitted commands.

If this property is set, executors may use it to calculate how many tasks can run concurrently per node. This information is used by dataflow.Strategy to estimate the resources required to run all outstanding tasks.

abstract status(job_ids: List[Any]) → List[parsl.providers.provider_base.JobStatus]

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

Parameters

job_ids (-) – A list of job identifiers

Returns

• A list of JobStatus objects corresponding to each job_id in the job_ids list.

Raises

- ExecutionProviderException or its subclasses –

abstract property status_polling_interval

Returns the interval, in seconds, at which the status method should be called.

Returns

the number of seconds to wait between calls to status()

abstract submit(command: str, tasks_per_node: int, job_name: str = 'parsl.auto') → Any

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as IPP engine or even Swift-T engines).

Args :

• command (str) : The bash command string to be executed

• tasks_per_node (int) : command invocations to be launched per node

KWargs:

• job_name (str) : Human friendly name to be assigned to the job request

Returns

• A job identifier, this could be an integer, string etc or None or any other object that evaluates to boolean false if submission failed but an exception isn’t thrown.

Raises

- ExecutionProviderException or its subclasses –