parsl.executors.GlobusComputeExecutor
- class parsl.executors.GlobusComputeExecutor(executor: Executor, label: str = 'GlobusComputeExecutor')[source]
GlobusComputeExecutor enables remote execution on Globus Compute endpoints
GlobusComputeExecutor is a thin wrapper over globus_compute_sdk.Executor Refer to globus-compute user documentation and reference documentation for more details.
Note
As a remote execution system, Globus Compute relies on serialization to ship tasks and results between the Parsl client side and the remote Globus Compute Endpoint side. Serialization is unreliable across python versions, and wrappers used by Parsl assume identical Parsl versions across on both sides. We recommend using matching Python, Parsl and Globus Compute version on both the client side and the endpoint side for stable behavior.
- __init__(executor: Executor, label: str = 'GlobusComputeExecutor')[source]
- Parameters:
executor (globus_compute_sdk.Executor) – Pass a globus_compute_sdk Executor that will be used to execute tasks on a globus_compute endpoint. Refer to globus-compute docs
label – a label to name the executor
Methods
__init__
(executor[, label])- param executor:
Pass a globus_compute_sdk Executor that will be used to execute
monitor_resources
()Should resource monitoring happen for tasks on running on this executor?
shutdown
()Clean-up the resources associated with the Executor.
start
()Start the Globus Compute Executor
submit
(func, resource_specification, *args, ...)Submit func to globus-compute
Attributes
hub_address
Address to the Hub for monitoring.
hub_zmq_port
Port to the Hub for monitoring.
label
radio_mode
run_dir
Path to the run directory.
run_id
UUID for the enclosing DFK.
submit_monitoring_radio
Local radio for sending monitoring messages
- shutdown()[source]
Clean-up the resources associated with the Executor.
GCE.shutdown will cancel all futures that have not yet registered with Globus Compute and will not wait for the launched futures to complete. This method explicitly shutsdown the result_watcher thread to avoid it waiting for outstanding futures at thread exit.
- submit(func: Callable, resource_specification: Dict[str, Any], *args: Any, **kwargs: Any) Future [source]
Submit func to globus-compute
- Parameters:
func (Callable) – Python function to execute remotely
resource_specification (Dict[str, Any]) – Resource specification can be used specify MPI resources required by MPI applications on Endpoints configured to use globus compute’s MPIEngine. GCE also accepts user_endpoint_config to configure endpoints when the endpoint is a Multi-User Endpoint
args – Args to pass to the function
kwargs – kwargs to pass to the function
- Return type:
Future