parsl.providers.AdHocProvider

class parsl.providers.AdHocProvider(channels=[], worker_init='', cmd_timeout=30, parallelism=1, move_files=None)[source]

Ad-hoc execution provider

This provider is used to provision execution resources over one or more ad hoc nodes that are each accessible over a Channel (say, ssh) but otherwise lack a cluster scheduler.

Parameters

  • channels (list of Channel ojects) – Each channel represents a connection to a remote node

  • worker_init (str) – Command to be run before starting a worker, such as ‘module load Anaconda; source activate env’. Since this provider calls the same worker_init across all nodes in the ad-hoc cluster, it is recommended that a single script is made available across nodes such as ~/setup.sh that can be invoked.

  • cmd_timeout (int) – Duration for which the provider will wait for a command to be invoked on a remote system. Defaults to 30s

  • parallelism (float) – Determines the ratio of workers to tasks as managed by the strategy component

__init__(channels=[], worker_init='', cmd_timeout=30, parallelism=1, move_files=None)[source]

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([channels, worker_init, …])

Initialize self.

cancel(job_ids)

Cancel a list of jobs with job_ids

status(job_ids)

Get status of the list of jobs with job_ids

submit(command, tasks_per_node[, job_name])

Submits the command onto a channel from the list of channels

Attributes

cores_per_node

Number of cores to provision per node.

label

Provides the label for this provider

mem_per_node

Real memory to provision per node in GB.

scaling_enabled

status_polling_interval

Returns the interval, in seconds, at which the status method should be called.
cancel(job_ids)[source]

Cancel a list of jobs with job_ids

Parameters

job_ids (list of strings) – List of job id strings

Returns

list of confirmation bools

Return type

[True, False…]

property label[source]

Provides the label for this provider

property scaling_enabled[source]
status(job_ids)[source]

Get status of the list of jobs with job_ids

Parameters

job_ids (list of strings) – List of job id strings

Returns

Return type

list of JobStatus objects

property status_polling_interval[source]

Returns the interval, in seconds, at which the status method should be called.

Returns

the number of seconds to wait between calls to status()

submit(command, tasks_per_node, job_name='parsl.adhoc')[source]

Submits the command onto a channel from the list of channels

Submit returns an ID that corresponds to the task that was just submitted.

Parameters

  • command ((String)) – Commandline invocation to be made on the remote side.

  • tasks_per_node ((int)) – command invocations to be launched per node

  • job_name ((String)) – Name of the job. Default : parsl.adhoc

Returns

  • None – At capacity, cannot provision more

  • job_id ((string)) – Identifier for the job