parsl.providers.CondorProvider

class parsl.providers.CondorProvider(channel: Channel = LocalChannel(envs={}, script_dir=None, userhome='/home/docs/checkouts/readthedocs.org/user_builds/parsl/checkouts/latest/docs'), nodes_per_block: int = 1, cores_per_slot: int | None = None, mem_per_slot: float | None = None, init_blocks: int = 1, min_blocks: int = 0, max_blocks: int = 1, parallelism: float = 1, environment: Dict[str, str] | None = None, project: str = '', scheduler_options: str = '', transfer_input_files: List[str] = [], walltime: str = '00:10:00', worker_init: str = '', launcher: Launcher = SingleNodeLauncher(debug=True, fail_on_any=False), requirements: str = '', cmd_timeout: int = 60, cmd_chunk_size: int = 100)[source]

HTCondor Execution Provider.

Parameters:

  • channel (Channel) – Channel for accessing this provider. Possible channels include LocalChannel (the default), SSHChannel, or SSHInteractiveLoginChannel.

  • nodes_per_block (int) – Nodes to provision per block.

  • cores_per_slot (int) – Specify the number of cores to provision per slot. If set to None, executors will assume all cores on the node are available for computation. Default is None.

  • mem_per_slot (float) – Specify the real memory to provision per slot in GB. If set to None, no explicit request to the scheduler will be made. Default is None.

  • init_blocks (int) – Number of blocks to provision at time of initialization

  • min_blocks (int) – Minimum number of blocks to maintain

  • max_blocks (int) – Maximum number of blocks to maintain.

  • parallelism (float) – Ratio of provisioned task slots to active tasks. A parallelism value of 1 represents aggressive scaling where as many resources as possible are used; parallelism close to 0 represents the opposite situation in which as few resources as possible (i.e., min_blocks) are used.

  • environment (dict of str) – A dictionary of environmant variable name and value pairs which will be set before running a task.

  • project (str) – Project which the job will be charged against

  • scheduler_options (str) – String to add specific condor attributes to the HTCondor submit script.

  • transfer_input_files (list(str)) – List of strings of paths to additional files or directories to transfer to the job

  • worker_init (str) – Command to be run before starting a worker.

  • requirements (str) – Condor requirements.

  • launcher (Launcher) – Launcher for this provider. Possible launchers include SingleNodeLauncher (the default),

  • cmd_timeout (int) – Timeout for commands made to the scheduler in seconds.

  • cmd_chunk_size (int) – Calls to the scheduler will be made for chunks of blocks with this size.

__init__(channel: Channel = LocalChannel(envs={}, script_dir=None, userhome='/home/docs/checkouts/readthedocs.org/user_builds/parsl/checkouts/latest/docs'), nodes_per_block: int = 1, cores_per_slot: int | None = None, mem_per_slot: float | None = None, init_blocks: int = 1, min_blocks: int = 0, max_blocks: int = 1, parallelism: float = 1, environment: Dict[str, str] | None = None, project: str = '', scheduler_options: str = '', transfer_input_files: List[str] = [], walltime: str = '00:10:00', worker_init: str = '', launcher: Launcher = SingleNodeLauncher(debug=True, fail_on_any=False), requirements: str = '', cmd_timeout: int = 60, cmd_chunk_size: int = 100) → None[source]

Methods

__init__([channel, nodes_per_block, ...])

cancel(job_ids)

Cancels the jobs specified by a list of job IDs.

execute_wait(cmd[, timeout])

status(job_ids)

Get the status of a list of jobs identified by their ids.

submit(command, tasks_per_node[, job_name])

Submits the command onto an Local Resource Manager job.

Attributes

cores_per_node

Number of cores to provision per node.

label

Provides the label for this provider

mem_per_node

Real memory to provision per node in GB.

status_polling_interval

Returns the interval, in seconds, at which the status method should be called.
cancel(job_ids)[source]

Cancels the jobs specified by a list of job IDs.

Parameters:

job_ids (list of str) – The job IDs to cancel.

Returns:

Each entry in the list will be True if the job is cancelled succesfully, otherwise False.

Return type:

list of bool

status(job_ids)[source]

Get the status of a list of jobs identified by their ids.

Parameters:

job_ids (list of int) – Identifiers of jobs for which the status will be returned.

Returns:

Status codes for the requested jobs.

Return type:

List of int

property status_polling_interval[source]

Returns the interval, in seconds, at which the status method should be called.

Returns:

the number of seconds to wait between calls to status()

submit(command, tasks_per_node, job_name='parsl.condor')[source]

Submits the command onto an Local Resource Manager job.

example file with the complex case of multiple submits per job:

Universe =vanilla output = out.$(Cluster).$(Process) error = err.$(Cluster).$(Process) log = log.$(Cluster) leave_in_queue = true executable = test.sh queue 5 executable = foo queue 1

$ condor_submit test.sub Submitting job(s)…… 5 job(s) submitted to cluster 118907. 1 job(s) submitted to cluster 118908.

Parameters:

  • command (str) – Command to execute

  • job_name (str) – Job name prefix.

  • tasks_per_node (int) – command invocations to be launched per node

Returns:

None if at capacity and cannot provision more; otherwise the identifier for the job.

Return type:

None or str