parsl.providers.GridEngineProvider

class parsl.providers.GridEngineProvider(channel=LocalChannel(envs={}, script_dir=None, userhome='/home/docs/checkouts/readthedocs.org/user_builds/parsl/checkouts/latest/docs'), nodes_per_block=1, init_blocks=1, min_blocks=0, max_blocks=1, parallelism=1, walltime='00:10:00', scheduler_options='', worker_init='', launcher=SingleNodeLauncher(debug=True, fail_on_any=False), cmd_timeout: int = 60, queue=None)[source]

A provider for the Grid Engine scheduler.

Parameters
  • channel (Channel) – Channel for accessing this provider. Possible channels include LocalChannel (the default), SSHChannel, or SSHInteractiveLoginChannel.

  • nodes_per_block (int) – Nodes to provision per block.

  • min_blocks (int) – Minimum number of blocks to maintain.

  • max_blocks (int) – Maximum number of blocks to maintain.

  • parallelism (float) – Ratio of provisioned task slots to active tasks. A parallelism value of 1 represents aggressive scaling where as many resources as possible are used; parallelism close to 0 represents the opposite situation in which as few resources as possible (i.e., min_blocks) are used.

  • walltime (str) – Walltime requested per block in HH:MM:SS.

  • scheduler_options (str) – String to prepend to the #$$ blocks in the submit script to the scheduler.

  • worker_init (str) – Command to be run before starting a worker, such as ‘module load Anaconda; source activate env’.

  • launcher (Launcher) – Launcher for this provider. Possible launchers include SingleNodeLauncher (the default),

  • cmd_timeout (int) – Timeout for commands made to the scheduler in seconds

__init__(channel=LocalChannel(envs={}, script_dir=None, userhome='/home/docs/checkouts/readthedocs.org/user_builds/parsl/checkouts/latest/docs'), nodes_per_block=1, init_blocks=1, min_blocks=0, max_blocks=1, parallelism=1, walltime='00:10:00', scheduler_options='', worker_init='', launcher=SingleNodeLauncher(debug=True, fail_on_any=False), cmd_timeout: int = 60, queue=None)[source]

Methods

__init__([channel, nodes_per_block, ...])

cancel(job_ids)

Cancels the resources identified by the job_ids provided by the user.

execute_wait(cmd[, timeout])

get_configs(command, tasks_per_node)

Compose a dictionary with information for writing the submit script.

status(job_ids)

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

submit(command, tasks_per_node[, job_name])

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as IPP engine or even Swift-T engines).

Attributes

cores_per_node

Number of cores to provision per node.

label

Provides the label for this provider

mem_per_node

Real memory to provision per node in GB.

status_polling_interval

Returns the interval, in seconds, at which the status method should be called.

cancel(job_ids)[source]

Cancels the resources identified by the job_ids provided by the user.

Parameters

job_ids (-) – A list of job identifiers

Returns

  • A list of status from cancelling the job which can be True, False

Raises

- ExecutionProviderException or its subclasses

get_configs(command, tasks_per_node)[source]

Compose a dictionary with information for writing the submit script.

property status_polling_interval[source]

Returns the interval, in seconds, at which the status method should be called.

Returns

the number of seconds to wait between calls to status()

submit(command, tasks_per_node, job_name='parsl.sge')[source]

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as IPP engine or even Swift-T engines).

Args :
  • command (str) : The bash command string to be executed.

  • tasks_per_node (int) : command invocations to be launched per node

KWargs:
  • job_name (str) : Human friendly name to be assigned to the job request

Returns

  • A job identifier, this could be an integer, string etc

Raises

- ExecutionProviderException or its subclasses