Released. Feb 23rd, 2018.
- GoogleCloud provider support via libsubmit
- GridEngine provider support via libsubmit
- Cobalt provider issues with job state #101
- Parsl updates config inadvertently #98
- No blocks provisioned if parallelism/blocks = 0 #97
- Checkpoint restart assumes rundir bug #95
- Logger continues after cleanup called enhancement #93
- Error checkpointing when no cache enabled #92
- Several fixes to libsubmit.
Here are the major changes included in the Parsl 0.4.0 release.
Elastic scaling in response to workflow pressure. issue#46 Options minBlocks, maxBlocks, and parallelism now work and controls workflow execution.
Documented in: Elasticity
Multisite support, enables targetting apps within a single workflow to different sites issue#48
@App('python', dfk, sites=['SITE1', 'SITE2']) def my_app(...): ...
Anonymized usage tracking added. issue#34
Documented in: Usage Statistics Collection
AppCaching and Checkpointing issue#43
Parsl now creates a new directory under ./runinfo/ with an incrementing number per workflow invocation
Troubleshooting guide and more documentation
PEP8 conformance tests added to travis testing issue#72
- Missing documentation from libsubmit was added back issue#41
- Fixes for script_dir | scriptDir inconsistencies issue#64 We now use scriptDir exclusively.
- Fix for caching not working on jupyter notebooks issue#90
- Config defaults module failure when part of the option set is provided issue#74
- Fixes for network errors with usage_tracking issue#70
- PEP8 conformance of code and tests with limited exclusions issue#72
- Doc bug in recommending max_workers instead of maxThreads issue#73
This is a point release with mostly minor features and several bug fixes
- Fixes for remote side handling
- Support for specifying IPythonDir for IPP controllers
- Several tests added that test provider launcher functionality from libsubmit
- This upgrade will also push the libsubmit requirement from 0.2.4 -> 0.2.5.
Several critical fixes from libsubmit are brought in:
- Several fixes and improvements to Condor from @annawoodard.
- Support for Torque scheduler
- Provider script output paths are fixed
- Increased walltimes to deal with slow scheduler system
- Srun launcher for slurm systems
- SSH channels now support file_pull() method
- While files are not automatically staged, the channels provide support for bi-directional file transport.
Here are the major changes that are included in the Parsl 0.3.0 release.
Arguments to DFK has changed:
# Old dfk(executor_obj)
# New, pass a list of executors dfk(executors=[list_of_executors])
# Alternatively, pass the config from which the DFK will #instantiate resources dfk(config=config_dict)
Execution providers have been restructured to a separate repo: libsubmit
Bash app styles have changes to return the commandline string rather than be assigned to the special keyword cmd_line. Please refer to RFC #37 for more details. This is a non-backward compatible change.
Output files from apps are now made available as an attribute of the AppFuture. Please refer #26 for more details. This is a non-backward compatible change
# This is the pre 0.3.0 style app_fu, [file1, file2] = make_files(x, y, outputs=['f1.txt', 'f2.txt']) #This is the style that will be followed going forward. app_fu = make_files(x, y, outputs=['f1.txt', 'f2.txt']) [file1, file2] = app_fu.outputs
DFK init now supports auto-start of IPP controllers
Support for channels via libsubmit. Channels enable execution of commands from execution providers either locally, or remotely via ssh.
Bash apps now support timeouts.
Support for cobalt execution provider.
Here are the major changes that are included in the Parsl 0.2.0 release.
- Support for execution via IPythonParallel executor enabling distributed execution.
- Generic executors
Here are the major changes that are included in the Parsl 0.1.0 release.
- Support for Bash and Python apps
- Support for chaining of apps via futures handled by the DataFlowKernel.
- Support for execution over threads.
- Arbitrary DAGs can be constructed and executed asynchronously.
- Initial release, no listed bugs.