parsl.executors.GlobusComputeExecutor

class parsl.executors.GlobusComputeExecutor(executor: Executor, label: str = 'GlobusComputeExecutor')[source]

GlobusComputeExecutor enables remote execution on Globus Compute endpoints

GlobusComputeExecutor is a thin wrapper over globus_compute_sdk.Executor Refer to globus-compute user documentation and reference documentation for more details.

Note

As a remote execution system, Globus Compute relies on serialization to ship tasks and results between the Parsl client side and the remote Globus Compute Endpoint side. Serialization is unreliable across python versions, and wrappers used by Parsl assume identical Parsl versions across on both sides. We recommend using matching Python, Parsl and Globus Compute version on both the client side and the endpoint side for stable behavior.

__init__(executor: Executor, label: str = 'GlobusComputeExecutor')[source]
Parameters:
  • executor (globus_compute_sdk.Executor) – Pass a globus_compute_sdk Executor that will be used to execute tasks on a globus_compute endpoint. Refer to globus-compute docs

  • label – a label to name the executor

Methods

__init__(executor[, label])

param executor:

Pass a globus_compute_sdk Executor that will be used to execute

monitor_resources()

Should resource monitoring happen for tasks on running on this executor?

shutdown()

Clean-up the resources associated with the Executor.

start()

Start the Globus Compute Executor

submit(func, resource_specification, *args, ...)

Submit func to globus-compute

Attributes

hub_address

Address to the Hub for monitoring.

hub_zmq_port

Port to the Hub for monitoring.

label

radio_mode

run_dir

Path to the run directory.

run_id

UUID for the enclosing DFK.

submit_monitoring_radio

Local radio for sending monitoring messages

shutdown()[source]

Clean-up the resources associated with the Executor.

GCE.shutdown will cancel all futures that have not yet registered with Globus Compute and will not wait for the launched futures to complete. This method explicitly shutsdown the result_watcher thread to avoid it waiting for outstanding futures at thread exit.

start() None[source]

Start the Globus Compute Executor

submit(func: Callable, resource_specification: Dict[str, Any], *args: Any, **kwargs: Any) Future[source]

Submit func to globus-compute

Parameters:
  • func (Callable) – Python function to execute remotely

  • resource_specification (Dict[str, Any]) – Resource specification can be used specify MPI resources required by MPI applications on Endpoints configured to use globus compute’s MPIEngine. GCE also accepts user_endpoint_config to configure endpoints when the endpoint is a Multi-User Endpoint

  • args – Args to pass to the function

  • kwargs – kwargs to pass to the function

Return type:

Future