parsl.channels.base.Channel

class parsl.channels.base.Channel[source]

For certain resources such as campus clusters or supercomputers at research laboratories, resource requirements may require authentication. For instance some resources may allow access to their job schedulers from only their login-nodes which require you to authenticate on through SSH, GSI-SSH and sometimes even require two factor authentication. Channels are simple abstractions that enable the ExecutionProvider component to talk to the resource managers of compute facilities. The simplest Channel, LocalChannel, simply executes commands locally on a shell, while the SshChannel authenticates you to remote systems.

Channels are usually called via the execute_wait function. For channels that execute remotely, a push_file function allows you to copy over files.

                      +------------------
                      |
cmd, wtime    ------->|  execute_wait
(ec, stdout, stderr)<-|---+
                      |
src, dst_dir  ------->|  push_file
   dst_path  <--------|----+
                      |
dst_script_dir <------|  script_dir
                      |
                      +-------------------

Channels should ensure that each launched command runs in a new process group, so that providers (such as AdHocProvider and LocalProvider) which terminate long running commands using process groups can do so.

__init__()[source]

Methods

__init__()

abspath(path)

Return the absolute path.

close()

Closes the channel.

execute_wait(cmd[, walltime, envs])

Executes the cmd, with a defined walltime.

isdir(path)

Return true if the path refers to an existing directory.

makedirs(path, mode[, exist_ok])

Create a directory.

pull_file(remote_source, local_dir)

Transport file on the remote side to a local directory

push_file(source, dest_dir)

Channel will take care of moving the file from source to the destination directory

Attributes

script_dir

This is a property.

abstract abspath(path)[source]

Return the absolute path.

Parameters

path (str) – Path for which the absolute path will be returned.

abstract close()[source]

Closes the channel. Clean out any auth credentials.

Parameters

None

Returns

Bool

abstract execute_wait(cmd, walltime=None, envs={}, *args, **kwargs)[source]

Executes the cmd, with a defined walltime.

Parameters
  • cmd (-) – Command string to execute over the channel

  • walltime (-) – Timeout in seconds, optional

KWargs:
  • envs (Dict[str, str]) : Environment variables to push to the remote side

Returns

  • (exit_code, stdout, stderr) (int, optional string, optional string) If the exit code is a failure code, the stdout and stderr return values may be None.

abstract isdir(path)[source]

Return true if the path refers to an existing directory.

Parameters

path (str) – Path of directory to check.

abstract makedirs(path, mode, exist_ok=False)[source]

Create a directory.

If intermediate directories do not exist, they will be created.

Parameters
  • path (str) – Path of directory to create.

  • mode (int) – Permissions (posix-style) for the newly-created directory.

  • exist_ok (bool) – If False, raise an OSError if the target directory already exists.

abstract pull_file(remote_source, local_dir)[source]

Transport file on the remote side to a local directory

Parameters
  • remote_source (string) – remote_source

  • local_dir (string) – Local directory to copy to

Returns

destination_path (string)

abstract push_file(source, dest_dir)[source]

Channel will take care of moving the file from source to the destination directory

Parameters
  • source (string) – Full filepath of the file to be moved

  • dest_dir (string) – Absolute path of the directory to move to

Returns

destination_path (string)

abstract property script_dir[source]

This is a property. Returns the directory assigned for storing all internal scripts such as scheduler submit scripts. This is usually where error logs from the scheduler would reside on the channel destination side.

Parameters

None (-) –

Returns

  • Channel script dir