parsl.providers.AdHocProvider¶
-
class
parsl.providers.
AdHocProvider
(channels=[], worker_init='', cmd_timeout=30, parallelism=1, move_files=None)[source]¶ Ad-hoc execution provider
This provider is used to provision execution resources over one or more ad hoc nodes that are each accessible over a Channel (say, ssh) but otherwise lack a cluster scheduler.
- Parameters
channels (list of Channel ojects) – Each channel represents a connection to a remote node
worker_init (str) – Command to be run before starting a worker, such as ‘module load Anaconda; source activate env’. Since this provider calls the same worker_init across all nodes in the ad-hoc cluster, it is recommended that a single script is made available across nodes such as ~/setup.sh that can be invoked.
cmd_timeout (int) – Duration for which the provider will wait for a command to be invoked on a remote system. Defaults to 30s
parallelism (float) – Determines the ratio of workers to tasks as managed by the strategy component
-
__init__
(channels=[], worker_init='', cmd_timeout=30, parallelism=1, move_files=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
Methods
__init__
([channels, worker_init, …])Initialize self.
cancel
(job_ids)Cancel a list of jobs with job_ids
status
(job_ids)Get status of the list of jobs with job_ids
submit
(command, tasks_per_node[, job_name])Submits the command onto a channel from the list of channels
Attributes
cores_per_node
Number of cores to provision per node.
Provides the label for this provider
mem_per_node
Real memory to provision per node in GB.
Returns the interval, in seconds, at which the status method should be called.
-
cancel
(job_ids)[source]¶ Cancel a list of jobs with job_ids
- Parameters
job_ids (list of strings) – List of job id strings
- Returns
list of confirmation bools
- Return type
[True, False…]
-
status
(job_ids)[source]¶ Get status of the list of jobs with job_ids
- Parameters
job_ids (list of strings) – List of job id strings
- Returns
- Return type
list of JobStatus objects
-
property
status_polling_interval
[source]¶ Returns the interval, in seconds, at which the status method should be called.
- Returns
the number of seconds to wait between calls to status()
-
submit
(command, tasks_per_node, job_name='parsl.adhoc')[source]¶ Submits the command onto a channel from the list of channels
Submit returns an ID that corresponds to the task that was just submitted.
- Parameters
command ((String)) – Commandline invocation to be made on the remote side.
tasks_per_node ((int)) – command invocations to be launched per node
job_name ((String)) – Name of the job. Default : parsl.adhoc
- Returns
None – At capacity, cannot provision more
job_id ((string)) – Identifier for the job