Running Workflows

Parsl workflows are a Python “main” program that defines Apps, how the Apps are invoked, and how results are passed between different Apps.

The core concept of workflows is that Parsl Apps produce Futures with all features from those in Python’s concurrent.futures module and more.

• Futures
  • AppFutures
  • DataFutures
• Example parallel patterns
  • Bag of Tasks
  • Sequential workflows
  • Parallel workflows
  • Parallel workflows with loops
  • MapReduce
  • Caching expensive initialisation between tasks
• Error handling
  • Exceptions
  • Retries
  • Lazy fail
  • Retry handlers
• Lifted operators
  • Lifted [] operator
  • Lifted . operator
• Memoization and checkpointing
  • App caching
  • Checkpointing