parsl.executors.swift_t.TurbineExecutor

class parsl.executors.swift_t.TurbineExecutor(label='turbine', storage_access=None, working_dir=None, managed=True)[source]

The Turbine executor.

Bypass the Swift/T language and run on top off the Turbine engines in an MPI environment.

Here is a diagram

             |  Data   |  Executor   |   IPC      | External Process(es)
             |  Flow   |             |            |
        Task | Kernel  |             |            |
      +----->|-------->|------------>|outgoing_q -|-> Worker_Process
      |      |         |             |            |    |         |
Parsl<---Fut-|         |             |            |  result   exception
          ^  |         |             |            |    |         |
          |  |         |   Q_mngmnt  |            |    V         V
          |  |         |    Thread<--|incoming_q<-|--- +---------+
          |  |         |      |      |            |
          |  |         |      |      |            |
          +----update_fut-----+
__init__(label='turbine', storage_access=None, working_dir=None, managed=True)[source]

Initialize the thread pool.

Trying to implement the emews model.

Methods

__init__([label, storage_access, …]) Initialize the thread pool.
scale_in(workers) Scale in the number of active blocks by specified amount.
scale_out([workers]) Scales out the number of active workers by 1.
shutdown() Shutdown method, to kill the threads and workers.
start() Start the executor.
submit(func, *args, **kwargs) Submits work to the the outgoing_q.
weakref_cb([q]) We do not use this yet.

Attributes

scaling_enabled Specify if scaling is enabled.