Parsl 1.1.0

Released on April 26th, 2021.

Parsl v1.1.0 includes 59 closed issues and 243 pull requests with contributions (code, tests, reviews and reports) from:

Akila Ravihansa Perera @ravihansa3000, Anna Woodard @annawoodard, @bakerjl, Ben Clifford @benclifford, Daniel S. Katz @danielskatz, Douglas Thain @dthain, @gerrick, @JG-Quarknet, Joseph Moon @jmoon1506, Kelly L. Rowland @kellyrowland, Lars Bilke @bilke, Logan Ward @WardLT, Kirill Nagaitsev @Loonride, Marcus Schwarting @meschw04, Matt Baughman @mattebaughman, Mihael Hategan @hategan, @radiantone, Rohan Kumar @rohankumar42, Sohit Miglani @sohitmiglani, Tim Shaffer @trshaffer, Tyler J. Skluzacek @tskluzac, Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi

Deprecated and Removed features

  • Python 3.5 is no longer supported.

  • Almost definitely broken Jetstream provider removed (#1821)

New Functionality

  • Allow HTEX to set CPU affinity (#1853)

  • New serialization system to replace IPP serialization (#1806)

  • Support for Python 3.9

  • @join_apps are a variation of @python_apps where an app can launch more apps and then complete only after the launched apps are also completed.

    These are described more fully in docs/userguide/joins.rst

  • Monitoring:

    hub.log is now named monitoring_router.log Remove denormalised workflow duration from monitoring db (#1774) Remove hostname from status table (#1847) Clarify distinction between tasks and tries to run tasks (#1808) Replace ‘done’ state with ‘exec_done’ and ‘memo_done’ (#1848) Use repr instead of str for monitoring fail history (#1966)

  • Monitoring visualization:

    Make task list appear under …/task/ not under …/app/ (#1762) Test that parsl-visualize does not return HTTP errors (#1700) Generate Gantt chart from status table rather than task table timestamps (#1767) Hyperlinks for app page to task pages should be on the task ID, not the app name (#1776) Use real final state to color DAG visualization (#1812)

  • Make task record garbage collection optional. (#1909)

  • Make checkpoint_files = get_all_checkpoints() by default (#1918)

Parsl 1.0.0

Released on June 11th, 2020

Parsl v1.0.0 includes 59 closed issues and 243 pull requests with contributions (code, tests, reviews and reports) from:

Akila Ravihansa Perera @ravihansa3000, Aymen Alsaadi @AymenFJA, Anna Woodard @annawoodard, Ben Clifford @benclifford, Ben Glick @benhg, Benjamin Tovar @btovar, Daniel S. Katz @danielskatz, Daniel Smith @dgasmith, Douglas Thain @dthain, Eric Jonas @ericmjonas, Geoffrey Lentner @glentner, Ian Foster @ianfoster, Kalpani Ranasinghe @kalpanibhagya, Kyle Chard @kylechard, Lindsey Gray @lgray, Logan Ward @WardLT, Lyle Hayhurst @lhayhurst, Mihael Hategan @hategan, Rajini Wijayawardana @rajiniw95, @saktar-unr, Tim Shaffer @trshaffer, Tom Glanzman @TomGlanzman, Yadu Nand Babuji @yadudoc and, Zhuozhao Li @ZhuozhaoLi

Deprecated and Removed features

  • Python3.5 is now marked for deprecation, and will not be supported after this release. Python3.6 will be the earliest Python3 version supported in the next release.

  • App decorator deprecated in 0.8 is now removed issue#1539 bash_app and python_app are the only supported App decorators in this release.

  • IPyParallelExecutor is no longer a supported executor issue#1565

New Functionality

  • WorkQueueExecutor introduced in v0.9.0 is now in beta. WorkQueueExecutor is designed as a drop-in replacement for HighThroughputExecutor. Here are some key features: * Support for packaging the python environment and shipping it to the worker side. This mechanism addresses propagating python environments in grid-like systems that lack shared-filesystems or cloud environments. * WorkQueueExecutor supports resource function tagging and resource specification * Support for resource specification kwarg issue#1675

  • Limited type-checking in Parsl internal components (as part of an ongoing effort)

  • Improvements to caching mechanism including ability to mark certain arguments to be not counted for memoization.

    • Normalize known types for memoization, and reject unknown types (#1291). This means that previous unreliable behaviour for some complex types such as dicts will become more reliable; and that other previous unreliable behaviour for other unknown complex types will now cause an error. Handling can be added for those types using parsl.memoization.id_for_memo.

    • Add ability to label some arguments in an app invocation as not memoized using the ignore_for_cache app keyword (PR 1568)

  • Special keyword args: ‘inputs’, ‘outputs’ that are used to specify files no longer support strings and now require File objects. For example, the following snippet is no longer supported in v1.0.0:

    def cat(inputs=[], outputs=[]):
         return 'cat {} > {}'.format(inputs[0], outputs[0])
    concat = cat(inputs=['hello-0.txt'],

    This is the new syntax:

    from parsl import File
    def cat(inputs=[], outputs=[]):
         return 'cat {} > {}'.format(inputs[0].filepath, outputs[0].filepath)
    concat = cat(inputs=[File('hello-0.txt')],

    Since filenames are no longer passed to apps as strings, and the string filepath is required, it can be accessed from the File object using the filepath attribute.

    from parsl import File
    def cat(inputs=[], outputs=[]):
         return 'cat {} > {}'.format(inputs[0].filepath, outputs[0].filepath)
  • New launcher: WrappedLauncher for launching tasks inside containers.

  • SSHChannel now supports a key_filename kwarg issue#1639

  • Newly added Makefile wraps several frequent developer operations such as:

    • Run the test-suite: make test

    • Install parsl: make install

    • Create a virtualenv: make virtualenv

    • Tag release and push to release channels: make deploy

  • Several updates to the HighThroughputExecutor:

    • By default, the HighThroughputExecutor will now use heuristics to detect and try all addresses when the workers connect back to the parsl master. An address can be configured manually using the HighThroughputExecutor(address=<address_string>) kwarg option.

    • Support for Mac OS. (pull#1469, pull#1738)

    • Cleaner reporting of version mismatches and automatic suppression of non-critical errors.

    • Separate worker log directories by block id issue#1508

  • Support for garbage collection to limit memory consumption in long-lived scripts.

  • All cluster providers now use max_blocks=1 by default issue#1730 to avoid over-provisioning.

  • New JobStatus class for better monitoring of Jobs submitted to batch schedulers.

Bug Fixes

  • Ignore AUTO_LOGNAME for caching issue#1642

  • Add batch jobs to PBS/torque job status table issue#1650

  • Use higher default buffer threshold for serialization issue#1654

  • Do not pass mutable default to ignore_for_cache issue#1656

  • Several improvements and fixes to Monitoring

  • Fix sites/test_ec2 failure when aws user opts specified issue#1375

  • Fix LocalProvider to kill the right processes, rather than all processes owned by user issue#1447

  • Exit htex probe loop with first working address issue#1479

  • Allow slurm partition to be optional issue#1501

  • Fix race condition with wait_for_tasks vs task completion issue#1607

  • Fix Torque job_id truncation issue#1583

  • Cleaner reporting for Serialization Errors issue#1355

  • Results from zombie managers do not crash the system, but will be ignored issue#1665

  • Guarantee monitoring will send out at least one message issue#1446

  • Fix monitoring ctrlc hang issue#1670

Parsl 0.9.0

Released on October 25th, 2019

Parsl v0.9.0 includes 199 closed issues and pull requests with contributions (code, tests, reviews and reports) from:

Andrew Litteken @AndrewLitteken, Anna Woodard @annawoodard, Ben Clifford @benclifford, Ben Glick @benhg, Daniel S. Katz @danielskatz, Daniel Smith @dgasmith, Engin Arslan @earslan58, Geoffrey Lentner @glentner, John Hover @jhover Kyle Chard @kylechard, TJ Dasso @tjdasso, Ted Summer @macintoshpie, Tom Glanzman @TomGlanzman, Levi Naden @LNaden, Logan Ward @WardLT, Matthew Welborn @mattwelborn, @MatthewBM, Raphael Fialho @rapguit, Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi

New Functionality

  • Parsl will no longer do automatic keyword substitution in @bash_app in favor of deferring to Python’s format method and newer f-strings. For example,

    # The following example worked until v0.8.0
    def cat(inputs=[], outputs=[]):
        return 'cat {inputs[0]} > {outputs[0]}' # <-- Relies on Parsl auto formatting the string
    # Following are two mechanisms that will work going forward from v0.9.0
    def cat(inputs=[], outputs=[]):
        return 'cat {} > {}'.format(inputs[0], outputs[0]) # <-- Use str.format method
    def cat(inputs=[], outputs=[]):
        return f'cat {inputs[0]} > {outputs[0]}' # <-- OR use f-strings introduced in Python3.6
  • @python_app now takes a walltime kwarg to limit the task execution time.

  • New file staging API parsl.data_provider.staging.Staging to support pluggable file staging methods. The methods implemented in 0.8.0 (HTTP(S), FTP and Globus) are still present, along with two new methods which perform HTTP(S) and FTP staging on worker nodes to support non-shared-filesystem executors such as clouds.

  • Behaviour change for storage_access parameter. In 0.8.0, this was used to specify Globus staging configuration. In 0.9.0, if this parameter is specified it must specify all desired staging providers. To keep the same staging providers as in 0.8.0, specify:

    from parsl.data_provider.data_manager import default_staging
    storage_access = default_staging + [GlobusStaging(...)]

    GlobusScheme in 0.8.0 has been renamed GlobusStaging and moved to a new module, parsl.data_provider.globus

  • WorkQueueExecutor: a new executor that integrates functionality from Work Queue is now available.

  • New provider to support for Ad-Hoc clusters parsl.providers.AdHocProvider

  • New provider added to support LSF on Summit parsl.providers.LSFProvider

  • Support for CPU and Memory resource hints to providers (github).

  • The logging_level=logging.INFO in MonitoringHub is replaced with monitoring_debug=False:

  • Managers now have a worker watchdog thread to report task failures that crash a worker.

  • Maximum idletime after which idle blocks can be relinquished can now be configured as follows:

                 max_idletime=120.0 ,  # float, unit=seconds
  • Several test-suite improvements that have dramatically reduced test duration.

  • Several improvements to the Monitoring interface.

  • Configurable port on parsl.channels.SSHChannel.

  • suppress_failure now defaults to True.

  • HighThroughputExecutor is the recommended executor, and IPyParallelExecutor is deprecated.

  • HighThroughputExecutor will expose worker information via environment variables: PARSL_WORKER_RANK and PARSL_WORKER_COUNT

Bug Fixes

  • ZMQError: Operation cannot be accomplished in current state bug issue#1146

  • Fix event loop error with monitoring enabled issue#532

  • Tasks per app graph appears as a sawtooth, not as rectangles issue#1032.

  • Globus status processing failure issue#1317.

  • Sporadic globus staging error issue#1170.

  • RepresentationMixin breaks on classes with no default parameters issue#1124.

  • File localpath staging conflict issue#1197.

  • Fix IndexError when using CondorProvider with strategy enabled issue#1298.

  • Improper dependency error handling causes hang issue#1285.

  • Memoization/checkpointing fixes for bash apps issue#1269.

  • CPU User Time plot is strangely cumulative issue#1033.

  • Issue requesting resources on non-exclusive nodes issue#1246.

  • parsl + htex + slurm hangs if slurm command times out, without making further progress issue#1241.

  • Fix strategy overallocations issue#704.

  • max_blocks not respected in SlurmProvider issue#868.

  • globus staging does not work with a non-default workdir issue#784.

  • Cumulative CPU time loses time when subprocesses end issue#1108.

  • Interchange KeyError due to too many heartbeat missed issue#1128.

Parsl 0.8.0

Released on June 13th, 2019

Parsl v0.8.0 includes 58 closed issues and pull requests with contributions (code, tests, reviews and reports)

from: Andrew Litteken @AndrewLitteken, Anna Woodard @annawoodard, Antonio Villarreal @villarrealas, Ben Clifford @benc, Daniel S. Katz @danielskatz, Eric Tatara @etatara, Juan David Garrido @garri1105, Kyle Chard @@kylechard, Lindsey Gray @lgray, Tim Armstrong @timarmstrong, Tom Glanzman @TomGlanzman, Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi

New Functionality

  • Monitoring is now integrated into parsl as default functionality.

  • parsl.AUTO_LOGNAME: Support for a special AUTO_LOGNAME option to auto generate stdout and stderr file paths.

  • File no longer behaves as a string. This means that operations in apps that treated a File as a string will break. For example the following snippet will have to be updated:

    # Old style: " ".join(inputs) is legal since inputs will behave like a list of strings
    def concat(inputs=[], outputs=[], stdout="stdout.txt", stderr='stderr.txt'):
        return "cat {0} > {1}".format(" ".join(inputs), outputs[0])
    # New style:
    def concat(inputs=[], outputs=[], stdout="stdout.txt", stderr='stderr.txt'):
        return "cat {0} > {1}".format(" ".join(list(map(str,inputs))), outputs[0])
  • Cleaner user app file log management.

  • Updated configurations using HighThroughputExecutor in the configuration section of the userguide.

  • Support for OAuth based SSH with OAuthSSHChannel.

Bug Fixes

  • Monitoring resource usage bug issue#975

  • Bash apps fail due to missing dir paths issue#1001

  • Viz server explicit binding fix issue#1023

  • Fix sqlalchemy version warning issue#997

  • All workflows are called typeguard issue#973

  • Fix ModuleNotFoundError: No module named 'monitoring' issue#971

  • Fix sqlite3 integrity error issue#920

  • HTEX interchange check python version mismatch to the micro level issue#857

  • Clarify warning message when a manager goes missing issue#698

  • Apps without a specified DFK should use the global DFK in scope at call time, not at other times. issue#697

Parsl 0.7.2

Released on Mar 14th, 2019

New Functionality

  • Monitoring: Support for reporting monitoring data to a local sqlite database is now available.

  • Parsl is switching to an opt-in model for anonymous usage tracking. Read more here: Usage statistics collection.

  • bash_app now supports specification of write modes for stdout and stderr.

  • Persistent volume support added to KubernetesProvider.

  • Scaling recommendations from study on Bluewaters is now available in the userguide.

Parsl 0.7.1

Released on Jan 18th, 2019

New Functionality

  • LowLatencyExecutor: a new executor designed to address use-cases with tight latency requirements such as model serving (Machine Learning), function serving and interactive analyses is now available.

  • New options in HighThroughputExecutor:
    • suppress_failure: Enable suppression of worker rejoin errors.

    • max_workers: Limit workers spawned by manager

  • Late binding of DFK, allows apps to pick DFK dynamically at call time. This functionality adds safety to cases where a new config is loaded and a new DFK is created.

Bug fixes

Parsl 0.7.0

Released on Dec 20st, 2018

Parsl v0.7.0 includes 110 closed issues with contributions (code, tests, reviews and reports) from: Alex Hays @ahayschi, Anna Woodard @annawoodard, Ben Clifford @benc, Connor Pigg @ConnorPigg, David Heise @daheise, Daniel S. Katz @danielskatz, Dominic Fitzgerald @djf604, Francois Lanusse @EiffL, Juan David Garrido @garri1105, Gordon Watts @gordonwatts, Justin Wozniak @jmjwozniak, Joseph Moon @jmoon1506, Kenyi Hurtado @khurtado, Kyle Chard @kylechard, Lukasz Lacinski @lukaszlacinski, Ravi Madduri @madduri, Marco Govoni @mgovoni-devel, Reid McIlroy-Young @reidmcy, Ryan Chard @ryanchard, @sdustrud, Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi

New functionality

  • HighThroughputExecutor: a new executor intended to replace the IPyParallelExecutor is now available. This new executor addresses several limitations of IPyParallelExecutor such as:

    • Scale beyond the ~300 worker limitation of IPP.

    • Multi-processing manager supports execution on all cores of a single node.

    • Improved worker side reporting of version, system and status info.

    • Supports failure detection and cleaner manager shutdown.

    Here’s a sample configuration for using this executor locally:

    from parsl.providers import LocalProvider
    from parsl.channels import LocalChannel
    from parsl.config import Config
    from parsl.executors import HighThroughputExecutor
    config = Config(

    More information on configuring is available in the Configuration section.

  • ExtremeScaleExecutor a new executor targeting supercomputer scale (>1000 nodes) workflows is now available.

    Here’s a sample configuration for using this executor locally:

    from parsl.providers import LocalProvider
    from parsl.channels import LocalChannel
    from parsl.launchers import SimpleLauncher
    from parsl.config import Config
    from parsl.executors import ExtremeScaleExecutor
    config = Config(

    More information on configuring is available in the Configuration section.

  • The libsubmit repository has been merged with Parsl to reduce overheads on maintenance with respect to documentation, testing, and release synchronization. Since the merge, the API has undergone several updates to support the growing collection of executors, and as a result Parsl 0.7.0+ will not be backwards compatible with the standalone libsubmit repos. The major components of libsubmit are now available through Parsl, and require the following changes to import lines to migrate scripts to 0.7.0:

    • from libsubmit.providers import <ProviderName> is now from parsl.providers import <ProviderName>

    • from libsubmit.channels import <ChannelName> is now from parsl.channels import <ChannelName>

    • from libsubmit.launchers import <LauncherName> is now from parsl.launchers import <LauncherName>


    This is a breaking change from Parsl v0.6.0

  • To support resource-based requests for workers and to maintain uniformity across interfaces, tasks_per_node is no longer a provider option. Instead, the notion of tasks_per_node is defined via executor specific options, for eg:

    • IPyParallelExecutor provides workers_per_node

    • HighThroughputExecutor provides cores_per_worker to allow for worker launches to be determined based on the number of cores on the compute node.

    • ExtremeScaleExecutor uses ranks_per_node to specify the ranks to launch per node.


    This is a breaking change from Parsl v0.6.0

  • Major upgrades to the monitoring infrastructure.
    • Monitoring information can now be written to a SQLite database, created on the fly by Parsl

    • Web-based monitoring to track workflow progress

  • Determining the correct IP address/interface given network firewall rules is often a nuisance. To simplify this, three new methods are now supported:

    • parsl.addresses.address_by_route

    • parsl.addresses.address_by_query

    • parsl.addresses.address_by_hostname

  • AprunLauncher now supports overrides option that allows arbitrary strings to be added to the aprun launcher call.

  • DataFlowKernel has a new method wait_for_current_tasks()

  • DataFlowKernel now uses per-task locks and an improved mechanism to handle task completions improving performance for workflows with large number of tasks.

Bug fixes (highlights)

  • Ctlr+C should cause fast DFK cleanup issue#641

  • Fix to avoid padding in wtime_to_minutes() issue#522

  • Updates to block semantics issue#557

  • Updates public_ip to address for clarity issue#557

  • Improvements to launcher docs issue#424

  • Fixes for inconsistencies between stream_logger and file_logger issue#629

  • Fixes to DFK discarding some un-executed tasks at end of workflow issue#222

  • Implement per-task locks to avoid deadlocks issue#591

  • Fixes to internal consistency errors issue#604

  • Removed unnecessary provider labels issue#440

  • Fixes to TorqueProvider to work on NSCC issue#489

  • Several fixes and updates to monitoring subsystem issue#471

  • DataManager calls wrong DFK issue#412

  • Config isn’t reloading properly in notebooks issue#549

  • Cobalt provider partition should be queue issue#353

  • bash AppFailure exceptions contain useful but un-displayed information issue#384

  • Do not CD to engine_dir issue#543

  • Parsl install fails without kubernetes config file issue#527

  • Fix import error issue#533

  • Change Local Database Strategy from Many Writers to a Single Writer issue#472

  • All run-related working files should go in the rundir unless otherwise configured issue#457

  • Fix concurrency issue with many engines accessing the same IPP config issue#469

  • Ensure we are not caching failed tasks issue#368

  • File staging of unknown schemes fails silently issue#382

  • Inform user checkpointed results are being used issue#494

  • Fix IPP + python 3.5 failure issue#490

  • File creation fails if no executor has been loaded issue#482

  • Make sure tasks in dep_fail state are retried issue#473

  • Hard requirement for CMRESHandler issue#422

  • Log error Globus events to stderr issue#436

  • Take ‘slots’ out of logging issue#411

  • Remove redundant logging issue#267

  • Zombie ipcontroller processes - Process cleanup in case of interruption issue#460

  • IPyparallel failure when submitting several apps in parallel threads issue#451

  • SlurmProvider + SingleNodeLauncher starts all engines on a single core issue#454

  • IPP engine_dir has no effect if indicated dir does not exist issue#446

  • Clarify AppBadFormatting error issue#433

  • confusing error message with simple configs issue#379

  • Error due to missing kubernetes config file issue#432

  • parsl.configs and parsl.tests.configs missing init files issue#409

  • Error when Python versions differ issue#62

  • Fixing ManagerLost error in HTEX/EXEX issue#577

  • Write all debug logs to rundir by default in HTEX/EXEX issue#574

  • Write one log per HTEX worker issue#572

  • Fixing ManagerLost error in HTEX/EXEX issue#577

Parsl 0.6.1

Released on July 23rd, 2018.

This point release contains fixes for issue#409

Parsl 0.6.0

Released July 23rd, 2018.

New functionality

  • Switch to class based configuration issue#133

    Here’s a the config for using threads for local execution

    from parsl.config import Config
    from parsl.executors.threads import ThreadPoolExecutor
    config = Config(executors=[ThreadPoolExecutor()])

    Here’s a more complex config that uses SSH to run on a Slurm based cluster

    from libsubmit.channels import SSHChannel
    from libsubmit.providers import SlurmProvider
    from parsl.config import Config
    from parsl.executors.ipp import IPyParallelExecutor
    from parsl.executors.ipp_controller import Controller
    config = Config(
  • Implicit Data Staging issue#281

    # create an remote Parsl file
    inp = File('')
    # create a local Parsl file
    out = File('file:///tmp/ARIN-STATS-FORMAT-CHANGE.txt')
    # call the convert app with the Parsl file
    f = convert(inputs=[inp], outputs=[out])
  • Support for application profiling issue#5

  • Real-time usage tracking via external systems issue#248, issue#251

  • Several fixes and upgrades to tests and testing infrastructure issue#157, issue#159, issue#128, issue#192, issue#196

  • Better state reporting in logs issue#242

  • Hide DFK issue#50

    • Instead of passing a config dictionary to the DataFlowKernel, now you can call parsl.load(Config)

    • Instead of having to specify the dfk at the time of App declaration, the DFK is a singleton loaded at call time :

      import parsl
      from parsl.tests.configs.local_ipp import config
      def double(x):
          return x * 2
      fut = double(5)
  • Support for better reporting of remote side exceptions issue#110

Bug Fixes

  • Making naming conventions consistent issue#109

  • Globus staging returns unclear error bug issue#178

  • Duplicate log-lines when using IPP issue#204

  • Usage tracking with certain missing network causes 20s startup delay. issue#220

  • task_exit checkpointing repeatedly truncates checkpoint file during run bug issue#230

  • Checkpoints will not reload from a run that was Ctrl-C’ed issue#232

  • Race condition in task checkpointing issue#234

  • Failures not to be checkpointed issue#239

  • Naming inconsitencies with maxThreads, max_threads, max_workers are now resolved issue#303

  • Fatal not a git repository alerts issue#326

  • Default kwargs in bash apps unavailable at command-line string format time issue#349

  • Fix launcher class inconsistencies issue#360

  • Several fixes to AWS provider issue#362
    • Fixes faulty status updates

    • Faulty termination of instance at cleanup, leaving zombie nodes.

Parsl 0.5.1

Released. May 15th, 2018.

New functionality

Bug Fixes

Parsl 0.5.2

Released. June 21st, 2018. This is an emergency release addressing issue#347

Bug Fixes

  • Parsl version conflict with libsubmit 0.4.1 issue#347

Parsl 0.5.0

Released. Apr 16th, 2018.

New functionality

  • Support for Globus file transfers issue#71


    This feature is available from Parsl v0.5.0 in an experimental state.

  • PathLike behavior for Files issue#174
    • Files behave like strings here :

    myfile = File("hello.txt")
    f = open(myfile, 'r')
  • Automatic checkpointing modes issue#106

    config = {
        "globals": {
            "lazyErrors": True,
            "memoize": True,
            "checkpointMode": "dfk_exit"
  • Support for containers with docker issue#45

    localDockerIPP = {
         "sites": [
             {"site": "Local_IPP",
              "auth": {"channel": None},
              "execution": {
                  "executor": "ipp",
                  "container": {
                      "type": "docker",     # <----- Specify Docker
                      "image": "app1_v0.1", # <------Specify docker image
                  "provider": "local",
                  "block": {
                      "initBlocks": 2,  # Start with 4 workers
         "globals": {"lazyErrors": True}        }


    This feature is available from Parsl v0.5.0 in an experimental state.

  • Cleaner logging issue#85
    • Logs are now written by default to runinfo/RUN_ID/parsl.log.

    • INFO log lines are more readable and compact

  • Local configs are now packaged issue#96

    from parsl.configs.local import localThreads
    from parsl.configs.local import localIPP

Bug Fixes

  • Passing Files over IPP broken issue#200

  • Fix DataFuture.__repr__ for default instantiation issue#164

  • Results added to appCache before retries exhausted issue#130

  • Missing documentation added for Multisite and Error handling issue#116

  • TypeError raised when a bad stdout/stderr path is provided. issue#104

  • Race condition in DFK issue#102

  • Cobalt provider broken on Cooley.alfc issue#101

  • No blocks provisioned if parallelism/blocks = 0 issue#97

  • Checkpoint restart assumes rundir issue#95

  • Logger continues after cleanup is called issue#93

Parsl 0.4.1

Released. Feb 23rd, 2018.

New functionality

  • GoogleCloud provider support via libsubmit

  • GridEngine provider support via libsubmit

Bug Fixes

  • Cobalt provider issues with job state issue#101

  • Parsl updates config inadvertently issue#98

  • No blocks provisioned if parallelism/blocks = 0 issue#97

  • Checkpoint restart assumes rundir bug issue#95

  • Logger continues after cleanup called enhancement issue#93

  • Error checkpointing when no cache enabled issue#92

  • Several fixes to libsubmit.

Parsl 0.4.0

Here are the major changes included in the Parsl 0.4.0 release.

New functionality

  • Elastic scaling in response to workflow pressure. issue#46 Options minBlocks, maxBlocks, and parallelism now work and controls workflow execution.

    Documented in: Elasticity

  • Multisite support, enables targetting apps within a single workflow to different sites issue#48

    @App('python', dfk, sites=['SITE1', 'SITE2'])
    def my_app(...):
  • Anonymized usage tracking added. issue#34

    Documented in: Usage statistics collection

  • AppCaching and Checkpointing issue#43

    # Set cache=True to enable appCaching
    @App('python', dfk, cache=True)
    def my_app(...):
    # To checkpoint a workflow:

    Documented in: Checkpointing, App caching

  • Parsl now creates a new directory under ./runinfo/ with an incrementing number per workflow invocation

  • Troubleshooting guide and more documentation

  • PEP8 conformance tests added to travis testing issue#72

Bug Fixes

  • Missing documentation from libsubmit was added back issue#41

  • Fixes for script_dir | scriptDir inconsistencies issue#64
    • We now use scriptDir exclusively.

  • Fix for caching not working on jupyter notebooks issue#90

  • Config defaults module failure when part of the option set is provided issue#74

  • Fixes for network errors with usage_tracking issue#70

  • PEP8 conformance of code and tests with limited exclusions issue#72

  • Doc bug in recommending max_workers instead of maxThreads issue#73

Parsl 0.3.1

This is a point release with mostly minor features and several bug fixes

  • Fixes for remote side handling

  • Support for specifying IPythonDir for IPP controllers

  • Several tests added that test provider launcher functionality from libsubmit

  • This upgrade will also push the libsubmit requirement from 0.2.4 -> 0.2.5.

Several critical fixes from libsubmit are brought in:

  • Several fixes and improvements to Condor from @annawoodard.

  • Support for Torque scheduler

  • Provider script output paths are fixed

  • Increased walltimes to deal with slow scheduler system

  • Srun launcher for slurm systems

  • SSH channels now support file_pull() method

    While files are not automatically staged, the channels provide support for bi-directional file transport.

Parsl 0.3.0

Here are the major changes that are included in the Parsl 0.3.0 release.

New functionality

  • Arguments to DFK has changed:

    # Old dfk(executor_obj)

    # New, pass a list of executors dfk(executors=[list_of_executors])

    # Alternatively, pass the config from which the DFK will #instantiate resources dfk(config=config_dict)

  • Execution providers have been restructured to a separate repo: libsubmit

  • Bash app styles have changes to return the commandline string rather than be assigned to the special keyword cmd_line. Please refer to RFC #37 for more details. This is a non-backward compatible change.

  • Output files from apps are now made available as an attribute of the AppFuture. Please refer #26 for more details. This is a non-backward compatible change

    # This is the pre 0.3.0 style
    app_fu, [file1, file2] = make_files(x, y, outputs=['f1.txt', 'f2.txt'])
    #This is the style that will be followed going forward.
    app_fu = make_files(x, y, outputs=['f1.txt', 'f2.txt'])
    [file1, file2] = app_fu.outputs
  • DFK init now supports auto-start of IPP controllers

  • Support for channels via libsubmit. Channels enable execution of commands from execution providers either locally, or remotely via ssh.

  • Bash apps now support timeouts.

  • Support for cobalt execution provider.

Bug fixes

  • Futures have inconsistent behavior in bash app fn body #35

  • Parsl dflow structure missing dependency information #30

Parsl 0.2.0

Here are the major changes that are included in the Parsl 0.2.0 release.

New functionality

  • Support for execution via IPythonParallel executor enabling distributed execution.

  • Generic executors

Parsl 0.1.0

Here are the major changes that are included in the Parsl 0.1.0 release.

New functionality

  • Support for Bash and Python apps

  • Support for chaining of apps via futures handled by the DataFlowKernel.

  • Support for execution over threads.

  • Arbitrary DAGs can be constructed and executed asynchronously.

Bug Fixes

  • Initial release, no listed bugs.

Libsubmit Changelog

As of Parsl 0.7.0 the libsubmit repository has been merged into Parsl.

Libsubmit 0.4.1

Released. June 18th, 2018. This release folds in massive contributions from @annawoodard.

New functionality

  • Several code cleanups, doc improvements, and consistent naming

  • All providers have the initialization and actual start of resources decoupled.

Libsubmit 0.4.0

Released. May 15th, 2018. This release folds in contributions from @ahayschi, @annawoodard, @yadudoc

New functionality

  • Several enhancements and fixes to the AWS cloud provider (#44, #45, #50)

  • Added support for python3.4

Bug Fixes

  • Condor jobs left in queue with X state at end of completion issue#26

  • Worker launches on Cori seem to fail from broken ENV issue#27

  • EC2 provider throwing an exception at initial run issue#46