parsl.config.Config

class parsl.config.Config(executors: Optional[List[parsl.executors.base.ParslExecutor]] = None, app_cache: bool = True, checkpoint_files: Optional[Sequence[str]] = None, checkpoint_mode: Union[None, typing_extensions.Literal[task_exit], typing_extensions.Literal[periodic], typing_extensions.Literal[dfk_exit], typing_extensions.Literal[manual]] = None, checkpoint_period: Optional[str] = None, garbage_collect: bool = True, internal_tasks_max_threads: int = 10, retries: int = 0, retry_handler: Optional[Callable[[Exception, parsl.dataflow.taskrecord.TaskRecord], float]] = None, run_dir: str = 'runinfo', strategy: Optional[str] = 'simple', max_idletime: float = 120.0, monitoring: Optional[parsl.monitoring.monitoring.MonitoringHub] = None, usage_tracking: bool = False, initialize_logging: bool = True)[source]

Specification of Parsl configuration options.

Parameters

  • executors (list of ParslExecutor, optional) – List of ParslExecutor instances to use for executing tasks. Default is [ThreadPoolExecutor()].

  • app_cache (bool, optional) – Enable app caching. Default is True.

  • checkpoint_files (sequence of str, optional) – List of paths to checkpoint files. See parsl.utils.get_all_checkpoints() and parsl.utils.get_last_checkpoint() for helpers. Default is None.

  • checkpoint_mode (str, optional) – Checkpoint mode to use, can be 'dfk_exit', 'task_exit', 'periodic' or 'manual'. If set to None, checkpointing will be disabled. Default is None.

  • checkpoint_period (str, optional) – Time interval (in “HH:MM:SS”) at which to checkpoint completed tasks. Only has an effect if checkpoint_mode='periodic'.

  • garbage_collect (bool. optional.) – Delete task records from DFK when tasks have completed. Default: True

  • internal_tasks_max_threads (int, optional) – Maximum number of threads to allocate for submit side internal tasks such as some data transfers or @joinapps Default is 10.

  • monitoring (MonitoringHub, optional) – The config to use for database monitoring. Default is None which does not log to a database.

  • retries (int, optional) – Set the number of retries (or available retry budget when using retry_handler) in case of failure. Default is 0.

  • retry_handler (function, optional) – A user pluggable handler to decide if/how a task retry should happen. If no handler is specified, then each task failure incurs a retry cost of 1.

  • run_dir (str, optional) – Path to run directory. Default is ‘runinfo’.

  • strategy (str, optional) – Strategy to use for scaling blocks according to workflow needs. Can be ‘simple’, ‘htex_auto_scale’, ‘none’ or None. If ‘none’ or None, dynamic scaling will be disabled. Default is ‘simple’. The literal value None is deprecated.

  • max_idletime (float, optional) – The maximum idle time allowed for an executor before strategy could shut down unused blocks. Default is 120.0 seconds.

  • usage_tracking (bool, optional) – Set this field to True to opt-in to Parsl’s usage tracking system. Parsl only collects minimal, non personally-identifiable, information used for reporting to our funding agencies. Default is False.

  • initialize_logging (bool, optional) – Make DFK optionally not initialize any logging. Log messages will still be passed into the python logging system under the parsl logger name, but the logging system will not by default perform any further log system configuration. Most noticeably, it will not create a parsl.log logfile. The use case for this is when parsl is used as a library in a bigger system which wants to configure logging in a way that makes sense for that bigger system as a whole.

__init__(executors: Optional[List[parsl.executors.base.ParslExecutor]] = None, app_cache: bool = True, checkpoint_files: Optional[Sequence[str]] = None, checkpoint_mode: Union[None, typing_extensions.Literal[task_exit], typing_extensions.Literal[periodic], typing_extensions.Literal[dfk_exit], typing_extensions.Literal[manual]] = None, checkpoint_period: Optional[str] = None, garbage_collect: bool = True, internal_tasks_max_threads: int = 10, retries: int = 0, retry_handler: Optional[Callable[[Exception, parsl.dataflow.taskrecord.TaskRecord], float]] = None, run_dir: str = 'runinfo', strategy: Optional[str] = 'simple', max_idletime: float = 120.0, monitoring: Optional[parsl.monitoring.monitoring.MonitoringHub] = None, usage_tracking: bool = False, initialize_logging: bool = True) → None[source]

Methods

__init__([executors, app_cache, ...])

Attributes

executors

property executors: Sequence[parsl.executors.base.ParslExecutor][source]