parsl.config.Config

class parsl.config.Config(executors: Iterable[ParslExecutor] | None = None, app_cache: bool = True, checkpoint_files: Sequence[str] | None = None, checkpoint_mode: None | Literal['task_exit'] | Literal['periodic'] | Literal['dfk_exit'] | Literal['manual'] = None, checkpoint_period: str | None = None, dependency_resolver: DependencyResolver | None = None, exit_mode: Literal['cleanup', 'skip', 'wait'] = 'cleanup', garbage_collect: bool = True, internal_tasks_max_threads: int = 10, retries: int = 0, retry_handler: Callable[[Exception, TaskRecord], float] | None = None, run_dir: str = 'runinfo', std_autopath: Callable | None = None, strategy: str | None = 'simple', strategy_period: float | int = 5, max_idletime: float = 120.0, monitoring: MonitoringHub | None = None, usage_tracking: int = 0, project_name: str | None = None, initialize_logging: bool = True)

Specification of Parsl configuration options.

Parameters:

• executors (sequence of ParslExecutor, optional) – List (or other iterable) of ParslExecutor instances to use for executing tasks. Default is (ThreadPoolExecutor(),).

• app_cache (bool, optional) – Enable app caching. Default is True.

• checkpoint_files (sequence of str, optional) – List of paths to checkpoint files. See parsl.utils.get_all_checkpoints() and parsl.utils.get_last_checkpoint() for helpers. Default is None.

• checkpoint_mode (str, optional) – Checkpoint mode to use, can be 'dfk_exit', 'task_exit', 'periodic' or 'manual'. If set to None, checkpointing will be disabled. Default is None.

• checkpoint_period (str, optional) – Time interval (in “HH:MM:SS”) at which to checkpoint completed tasks. Only has an effect if checkpoint_mode='periodic'.

• dependency_resolver (plugin point for custom dependency resolvers. Default: only resolve Futures,) – using the SHALLOW_DEPENDENCY_RESOLVER.

• exit_mode (str, optional) –

  When Parsl is used as a context manager (using with parsl.load syntax) then this parameter controls what will happen to running tasks and exceptions at exit. The options are:

  • cleanup: cleanup the DFK on exit without waiting for any tasks

  • skip: skip all shutdown behaviour when exiting the context manager

  • wait: wait for all tasks to complete when exiting normally, but exit immediately when exiting due to an exception.

  Default is cleanup.

• garbage_collect (bool. optional.) – Delete task records from DFK when tasks have completed. Default: True

• internal_tasks_max_threads (int, optional) – Maximum number of threads to allocate for submit side internal tasks such as some data transfers or @joinapps Default is 10.

• monitoring (MonitoringHub, optional) – The config to use for database monitoring. Default is None which does not log to a database.

• retries (int, optional) – Set the number of retries (or available retry budget when using retry_handler) in case of failure. Default is 0.

• retry_handler (function, optional) – A user pluggable handler to decide if/how a task retry should happen. If no handler is specified, then each task failure incurs a retry cost of 1.

• run_dir (str, optional) – Path to run directory. Default is ‘runinfo’.

• std_autopath (function, optional) – Sets the function used to generate stdout/stderr specifications when parsl.AUTO_LOGPATH is used. If no function is specified, generates paths that look like: rundir/NNN/task_logs/X/task_{id}_{name}{label}.{out/err}

• strategy (str, optional) – Strategy to use for scaling blocks according to workflow needs. Can be ‘simple’, ‘htex_auto_scale’, ‘none’ or None. If ‘none’ or None, dynamic scaling will be disabled. Default is ‘simple’. The literal value None is deprecated.

• strategy_period (float or int, optional) – How often the scaling strategy should be executed. Default is 5 seconds.

• max_idletime (float, optional) – The maximum idle time allowed for an executor before strategy could shut down unused blocks. Default is 120.0 seconds.

• usage_tracking (int, optional) – Set this field to 1, 2, or 3 to opt-in to Parsl’s usage tracking system. The value represents the level of usage tracking detail to be collected. Setting this field to 0 will disable usage tracking. Default (this field is not set): usage tracking is not enabled. Parsl only collects minimal, non personally-identifiable, information used for reporting to our funding agencies.

• project_name (str, optional) – Option to deanonymize usage tracking data. If set, this value will be used as the project name in the usage tracking data and placed on the leaderboard.

• initialize_logging (bool, optional) – Make DFK optionally not initialize any logging. Log messages will still be passed into the python logging system under the parsl logger name, but the logging system will not by default perform any further log system configuration. Most noticeably, it will not create a parsl.log logfile. The use case for this is when parsl is used as a library in a bigger system which wants to configure logging in a way that makes sense for that bigger system as a whole.

__init__(executors: Iterable[ParslExecutor] | None = None, app_cache: bool = True, checkpoint_files: Sequence[str] | None = None, checkpoint_mode: None | Literal['task_exit'] | Literal['periodic'] | Literal['dfk_exit'] | Literal['manual'] = None, checkpoint_period: str | None = None, dependency_resolver: DependencyResolver | None = None, exit_mode: Literal['cleanup', 'skip', 'wait'] = 'cleanup', garbage_collect: bool = True, internal_tasks_max_threads: int = 10, retries: int = 0, retry_handler: Callable[[Exception, TaskRecord], float] | None = None, run_dir: str = 'runinfo', std_autopath: Callable | None = None, strategy: str | None = 'simple', strategy_period: float | int = 5, max_idletime: float = 120.0, monitoring: MonitoringHub | None = None, usage_tracking: int = 0, project_name: str | None = None, initialize_logging: bool = True) → None

Methods

__init__([executors, app_cache, ...])
get_usage_information()
validate_usage_tracking(level)

Attributes

executors

property executors: Sequence[ParslExecutor]

get_usage_information()

validate_usage_tracking(level: int) → None