parsl.providers.provider_base.ExecutionProvider

class parsl.providers.provider_base.ExecutionProvider[source]

Execution providers are responsible for managing execution resources that have a Local Resource Manager (LRM). For instance, campus clusters and supercomputers generally have LRMs (schedulers) such as Slurm, Torque/PBS, Condor and Cobalt. Clouds, on the other hand, have API interfaces that allow much more fine-grained composition of an execution environment. An execution provider abstracts these types of resources and provides a single uniform interface to them.

The providers abstract away the interfaces provided by various systems to request, monitor, and cancel compute resources.

                      +------------------
                      |
script_string ------->|  submit
     id      <--------|---+
                      |
[ ids ]       ------->|  status
[statuses]   <--------|----+
                      |
[ ids ]       ------->|  cancel
[cancel]     <--------|----+
                      |
                      +-------------------
__init__()[source]

Methods

__init__()

cancel(job_ids)

Cancels the resources identified by the job_ids provided by the user.

status(job_ids)

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

submit(command, tasks_per_node[, job_name])

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as for HighThroughputExecutor or WorkQueueExecutor).

Attributes

cores_per_node

Number of cores to provision per node.

label

Provides the label for this provider

mem_per_node

Real memory to provision per node in GB.

status_polling_interval

Returns the interval, in seconds, at which the status method should be called.

abstract cancel(job_ids: List[Any]) List[bool][source]

Cancels the resources identified by the job_ids provided by the user.

Parameters

job_ids (-) – A list of job identifiers

Returns

  • A list of status from cancelling the job which can be True, False

Raises

- ExecutionProviderException or its subclasses

property cores_per_node: Optional[int][source]

Number of cores to provision per node.

Providers which set this property should ask for cores_per_node cores when provisioning resources, and set the corresponding environment variable PARSL_CORES before executing submitted commands.

If this property is set, executors may use it to calculate how many tasks can run concurrently per node.

abstract property label: str[source]

Provides the label for this provider

property mem_per_node: Optional[float][source]

Real memory to provision per node in GB.

Providers which set this property should ask for mem_per_node of memory when provisioning resources, and set the corresponding environment variable PARSL_MEMORY_GB before executing submitted commands.

If this property is set, executors may use it to calculate how many tasks can run concurrently per node.

abstract status(job_ids: List[Any]) List[parsl.providers.provider_base.JobStatus][source]

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

Parameters

job_ids (-) – A list of job identifiers

Returns

  • A list of JobStatus objects corresponding to each job_id in the job_ids list.

Raises

- ExecutionProviderException or its subclasses

abstract property status_polling_interval: int[source]

Returns the interval, in seconds, at which the status method should be called.

Returns

the number of seconds to wait between calls to status()

abstract submit(command: str, tasks_per_node: int, job_name: str = 'parsl.auto') Any[source]

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as for HighThroughputExecutor or WorkQueueExecutor).

Args :
  • command (str) : The bash command string to be executed

  • tasks_per_node (int) : command invocations to be launched per node

KWargs:
  • job_name (str) : Human friendly name to be assigned to the job request

Returns

  • A job identifier, this could be an integer, string etc or None or any other object that evaluates to boolean false if submission failed but an exception isn’t thrown.

Raises

- ExecutionProviderException or its subclasses