parsl.executors.taskvine.TaskVineFactoryConfig

class parsl.executors.taskvine.TaskVineFactoryConfig(factory_timeout: int = 300, scratch_dir: str | None = None, min_workers: int = 1, max_workers: int = 1, workers_per_cycle: int = 1, worker_options: str | None = None, worker_executable: str = 'vine_worker', worker_timeout: int = 300, cores: int | None = None, gpus: int | None = None, memory: int | None = None, disk: int | None = None, python_env: str | None = None, batch_type: str = 'local', condor_requirements: str | None = None, batch_options: str | None = None, _project_port: int = 0, _project_address: str | None = None, _project_name: str | None = None, _project_password_file: str | None = None)[source]

Configuration of a TaskVine factory. Default is to spawn one local worker.

Parameters:
  • factory_timeout (int) – Exit if no manager seen in $factory_timeout seconds. Kill all workers on upon exit.

  • scratch_dir (Optional[str]) – Directory for factory to operate. Default is None, which TaskVine factory will use a directory residing inside Parsl logging directory.

  • min_workers (int) – Number of minimum workers connecting to manager at all times. Default is 1.

  • max_workers (int) – Number of maximum workers connecting to manager at all times. Default is 1.

  • workers_per_cycle (int) – Maximum number of new workers per 30s.

  • worker_options (Optional[str]) – Additional options to pass to workers. Run vine_worker --help for more details. Default is None.

  • worker_executable (str) – Executable to run workers. Default is ‘vine_worker’, which finds the executable on $PATH.

  • worker_timeout (int) – Number of seconds for workers to wait for manager’s contact before exiting. Default is 300.

  • cores (Optional[int]) – Number of cores a worker should have. Default is None, which allows a worker to use all available cores of the machine it lands on.

  • gpus (Optional[int]) – Number of gpus a worker should have. Default is None, which allows a worker to use all available gpus of the machine it lands on.

  • memory (Optional[int]) – Amount of memory in MBs a worker should have. Default is None, which allows a worker to use all available memory of the machine it lands on.

  • disk (Optional[int]) – Amount of disk in MBs a worker should have. Default is None, which allows a worker to use all available disk space of the machine it lands on.

  • python_env (Optional[str]) – Run workers inside this python environment tarball. Default is None.

  • batch_type (str) – Batch systems the factory should submit workers as jobs. ‘local’: spawn local workers. ‘condor’: Submit workers through HTCondor batch system. ‘sge’: Submit workers through SGE batch system. ‘slurm’: Submit workers through SLURM batch system. Default is ‘local’.

  • condor_requirements (Optional[str]) – String describing the condor requirements for slots that workers should land on. Default is None.

  • batch_options (Optional[str]) – String describing generic batch options. Varies with the type of batch system. Default is None.

__init__(factory_timeout: int = 300, scratch_dir: str | None = None, min_workers: int = 1, max_workers: int = 1, workers_per_cycle: int = 1, worker_options: str | None = None, worker_executable: str = 'vine_worker', worker_timeout: int = 300, cores: int | None = None, gpus: int | None = None, memory: int | None = None, disk: int | None = None, python_env: str | None = None, batch_type: str = 'local', condor_requirements: str | None = None, batch_options: str | None = None, _project_port: int = 0, _project_address: str | None = None, _project_name: str | None = None, _project_password_file: str | None = None) None[source]

Methods

__init__([factory_timeout, scratch_dir, ...])

Attributes

batch_options

batch_type

condor_requirements

cores

disk

factory_timeout

gpus

max_workers

memory

min_workers

python_env

scratch_dir

worker_executable

worker_options

worker_timeout

workers_per_cycle

batch_options: str | None = None[source]
batch_type: str = 'local'[source]
condor_requirements: str | None = None[source]
cores: int | None = None[source]
disk: int | None = None[source]
factory_timeout: int = 300[source]
gpus: int | None = None[source]
max_workers: int = 1[source]
memory: int | None = None[source]
min_workers: int = 1[source]
python_env: str | None = None[source]
scratch_dir: str | None = None[source]
worker_executable: str = 'vine_worker'[source]
worker_options: str | None = None[source]
worker_timeout: int = 300[source]
workers_per_cycle: int = 1[source]