API Reference guide

Core

parsl.app.app.python_app

Decorator function for making python apps.

parsl.app.app.bash_app

Decorator function for making bash apps.

parsl.dataflow.futures.AppFuture

An AppFuture wraps a sequence of Futures which may fail and be retried.

parsl.dataflow.dflow.DataFlowKernelLoader

Manage which DataFlowKernel is active.

parsl.monitoring.MonitoringHub

Configuration

parsl.config.Config

Specification of Parsl configuration options.

parsl.set_stream_logger

Add a stream log handler.

parsl.set_file_logger

Add a stream log handler.

parsl.addresses.address_by_hostname

Returns the hostname of the local host.

parsl.addresses.address_by_interface

Returns the IP address of the given interface name, e.g.

parsl.addresses.address_by_query

Finds an address for the local host by querying ipify.

parsl.addresses.address_by_route

Finds an address for the local host by querying the local routing table for the route to Google DNS.

Channels

parsl.channels.base.Channel

For certain resources such as campus clusters or supercomputers at research laboratories, resource requirements may require authentication.

parsl.channels.LocalChannel

This is not even really a channel, since opening a local shell is not heavy and done so infrequently that they do not need a persistent channel

parsl.channels.SSHChannel

SSH persistent channel.

parsl.channels.OAuthSSHChannel

SSH persistent channel.

parsl.channels.SSHInteractiveLoginChannel

SSH persistent channel.

Data management

parsl.app.futures.DataFuture

A datafuture points at an AppFuture.

parsl.data_provider.data_manager.DataManager

The DataManager is responsible for transferring input and output data.

parsl.data_provider.data_manager.Staging

This class defines the interface for file staging providers.

parsl.data_provider.files.File

The Parsl File Class.

parsl.data_provider.ftp.FTPSeparateTaskStaging

Performs FTP staging as a separate parsl level task.

parsl.data_provider.ftp.FTPInTaskStaging

Performs FTP staging as a wrapper around the application task.

parsl.data_provider.file_noop.NoOpFileStaging

parsl.data_provider.globus.GlobusStaging

Specification for accessing data on a remote executor via Globus.

parsl.data_provider.http.HTTPSeparateTaskStaging

A staging provider that Performs HTTP and HTTPS staging as a separate parsl-level task.

parsl.data_provider.http.HTTPInTaskStaging

A staging provider that performs HTTP and HTTPS staging as in a wrapper around each task.

parsl.data_provider.rsync.RSyncStaging

This staging provider will execute rsync on worker nodes to stage in files from a remote location.

Executors

parsl.executors.base.ParslExecutor

Executors are abstractions that represent available compute resources to which you could submit arbitrary App tasks.

parsl.executors.ThreadPoolExecutor

A thread-based executor.

parsl.executors.HighThroughputExecutor

Executor designed for cluster-scale

parsl.executors.WorkQueueExecutor

Executor to use Work Queue batch system

parsl.executors.ExtremeScaleExecutor

Executor designed for leadership class supercomputer scale

parsl.executors.swift_t.TurbineExecutor

The Turbine executor.

Launchers

parsl.launchers.SimpleLauncher

Does no wrapping.

parsl.launchers.SingleNodeLauncher

Worker launcher that wraps the user’s command with the framework to launch multiple command invocations in parallel.

parsl.launchers.SrunLauncher

Worker launcher that wraps the user’s command with the SRUN launch framework to launch multiple cmd invocations in parallel on a single job allocation.

parsl.launchers.AprunLauncher

Worker launcher that wraps the user’s command with the Aprun launch framework to launch multiple cmd invocations in parallel on a single job allocation

parsl.launchers.SrunMPILauncher

Launches as many workers as MPI tasks to be executed concurrently within a block.

parsl.launchers.GnuParallelLauncher

Worker launcher that wraps the user’s command with the framework to launch multiple command invocations via GNU parallel sshlogin.

parsl.launchers.MpiExecLauncher

Worker launcher that wraps the user’s command with the framework to launch multiple command invocations via mpiexec.

parsl.launchers.JsrunLauncher

Worker launcher that wraps the user’s command with the Jsrun launch framework to launch multiple cmd invocations in parallel on a single job allocation

parsl.launchers.WrappedLauncher

Wraps the command by prepending commands before a user’s command

Providers

parsl.providers.AdHocProvider

Ad-hoc execution provider

parsl.providers.AWSProvider

A provider for using Amazon Elastic Compute Cloud (EC2) resources.

parsl.providers.CobaltProvider

Cobalt Execution Provider

parsl.providers.CondorProvider

HTCondor Execution Provider.

parsl.providers.GoogleCloudProvider

A provider for using resources from the Google Compute Engine.

parsl.providers.GridEngineProvider

A provider for the Grid Engine scheduler.

parsl.providers.LocalProvider

Local Execution Provider

parsl.providers.LSFProvider

LSF Execution Provider

parsl.providers.GridEngineProvider

A provider for the Grid Engine scheduler.

parsl.providers.SlurmProvider

Slurm Execution Provider

parsl.providers.TorqueProvider

Torque Execution Provider

parsl.providers.KubernetesProvider

Kubernetes execution provider :param namespace: Kubernetes namespace to create deployments.

parsl.providers.PBSProProvider

PBS Pro Execution Provider

Exceptions

parsl.app.errors.AppBadFormatting

An error raised during formatting of a bash function.

parsl.app.errors.AppException

An error raised during execution of an app.

parsl.app.errors.AppTimeout

An error raised during execution of an app when it exceeds its allotted walltime.

parsl.app.errors.BadStdStreamFile

Error raised due to bad filepaths specified for STDOUT/ STDERR.

parsl.app.errors.BashAppNoReturn

Bash app returned no string.

parsl.app.errors.BashExitFailure

A non-zero exit code returned from a @bash_app

parsl.app.errors.MissingOutputs

Error raised at the end of app execution due to missing output files.

parsl.app.errors.NotFutureError

A non future item was passed to a function that expected a future.

parsl.app.errors.ParslError

Base class for all exceptions.

parsl.executors.errors.ControllerError

Error raise by IPP controller.

parsl.executors.errors.ExecutorError

Base class for all exceptions.

parsl.executors.errors.ScalingFailed

Scaling failed due to error in Execution provider.

parsl.executors.errors.InsufficientMPIRanks

Error raised when attempting to launch a MPI worker pool with less than 2 ranks

parsl.executors.errors.DeserializationError

Failure at the Deserialization of results/exceptions from remote workers

parsl.executors.errors.BadMessage

Mangled/Poorly formatted/Unsupported message received

parsl.dataflow.error.DataFlowException

Base class for all exceptions.

parsl.dataflow.error.ConfigurationError

Raised when the DataFlowKernel receives an invalid configuration.

parsl.dataflow.error.DuplicateTaskError

Raised by the DataFlowKernel when it finds that a job with the same task-id has been launched before.

parsl.dataflow.error.BadCheckpoint

Error raised at the end of app execution due to missing output files.

parsl.dataflow.error.DependencyError

Error raised if an app cannot run because there was an error

parsl.launchers.error.BadLauncher

Error raised when a non callable object is provider as Launcher

parsl.providers.error.ExecutionProviderException

Base class for all exceptions Only to be invoked when only a more specific error is not available.

parsl.providers.error.OptionalModuleMissing

Error raised a required module is missing for a optional/extra provider

parsl.providers.error.ChannelRequired

Execution provider requires a channel.

parsl.providers.error.ScaleOutFailed

Generic catch.

parsl.providers.error.SchedulerMissingArgs

Error raised when the template used to compose the submit script to the local resource manager is missing required arguments

parsl.providers.error.ScriptPathError

Error raised when the template used to compose the submit script to the local resource manager is missing required arguments

parsl.channels.errors.ChannelError

Base class for all exceptions

parsl.channels.errors.BadHostKeyException

SSH channel could not be created since server’s host keys could not be verified

parsl.channels.errors.BadScriptPath

An error raised during execution of an app.

parsl.channels.errors.BadPermsScriptPath

User does not have permissions to access the script_dir on the remote site

parsl.channels.errors.FileExists

Push or pull of file over channel fails since a file of the name already exists on the destination.

parsl.channels.errors.AuthException

An error raised during execution of an app.

parsl.channels.errors.SSHException

if there was any other error connecting or establishing an SSH session

parsl.channels.errors.FileCopyException

File copy operation failed

parsl.executors.high_throughput.errors.WorkerLost

Exception raised when a worker is lost

Internal

parsl.app.app.AppBase

This is the base class that defines the two external facing functions that an App must define.

parsl.app.bash.BashApp

parsl.app.python.PythonApp

Extends AppBase to cover the Python App.

parsl.dataflow.dflow.DataFlowKernel

The DataFlowKernel adds dependency awareness to an existing executor.

parsl.dataflow.flow_control.FlowControl

Implements threshold-interval based flow control.

parsl.dataflow.memoization.Memoizer

Memoizer is responsible for ensuring that identical work is not repeated.

parsl.dataflow.strategy.Strategy

FlowControl strategy.

parsl.dataflow.flow_control.Timer

This timer is a simplified version of the FlowControl timer.