parsl.executors.swift_t.TurbineExecutor

class parsl.executors.swift_t.TurbineExecutor(label='turbine', storage_access=None, working_dir=None, managed=True)[source]

The Turbine executor.

Bypass the Swift/T language and run on top off the Turbine engines in an MPI environment.

Here is a diagram

             |  Data   |  Executor   |   IPC      | External Process(es)
             |  Flow   |             |            |
        Task | Kernel  |             |            |
      +----->|-------->|------------>|outgoing_q -|-> Worker_Process
      |      |         |             |            |    |         |
Parsl<---Fut-|         |             |            |  result   exception
          ^  |         |             |            |    |         |
          |  |         |   Q_mngmnt  |            |    V         V
          |  |         |    Thread<--|incoming_q<-|--- +---------+
          |  |         |      |      |            |
          |  |         |      |      |            |
          +----update_fut-----+
__init__(label='turbine', storage_access=None, working_dir=None, managed=True)[source]

Initialize the thread pool.

Trying to implement the emews model.

Methods

__init__([label, storage_access, …]) Initialize the thread pool.
scale_in(blocks) Scale in the number of active blocks by specified amount.
scale_out([blocks]) Scales out the number of active workers by 1.
set_bad_state_and_fail_all(exception) Allows external error handlers to mark this executor as irrecoverably bad and cause all tasks submitted to it now and in the future to fail.
shutdown() Shutdown method, to kill the threads and workers.
start() Start the executor.
status() Return the status of all jobs/blocks currently known to this executor.
submit(func, *args, **kwargs) Submits work to the the outgoing_q.
weakref_cb([q]) We do not use this yet.

Attributes

bad_state_is_set Returns true if this executor is in an irrecoverable error state.
executor_exception Returns an exception that indicates why this executor is in an irrecoverable state.
hub_address Address to the Hub for monitoring.
hub_port Port to the Hub for monitoring.
provider
run_dir Path to the run directory.
scaling_enabled Specify if scaling is enabled.
tasks Contains a dictionary mapping task IDs to the corresponding Future objects for all tasks that have been submitted to this executor.