parsl.providers.AWSProvider¶
-
class
parsl.providers.
AWSProvider
(image_id, key_name, init_blocks=1, min_blocks=0, max_blocks=10, nodes_per_block=1, parallelism=1, worker_init='', instance_type='t2.small', region='us-east-2', spot_max_bid=0, key_file=None, profile=None, iam_instance_profile_arn='', state_file=None, walltime='01:00:00', linger=False, launcher=SingleNodeLauncher())[source]¶ A provider for using Amazon Elastic Compute Cloud (EC2) resources.
One of 3 methods are required to authenticate: keyfile, profile or environment variables. If neither keyfile or profile are set, the following environment variables must be set:
AWS_ACCESS_KEY_ID
(the access key for your AWS account),AWS_SECRET_ACCESS_KEY
(the secret key for your AWS account), and (optionaly) theAWS_SESSION_TOKEN
(the session key for your AWS account).Parameters: - image_id (str) – Identification of the Amazon Machine Image (AMI).
- worker_init (str) – String to append to the Userdata script executed in the cloudinit phase of instance initialization.
- walltime (str) – Walltime requested per block in HH:MM:SS.
- key_file (str) – Path to json file that contains ‘AWSAccessKeyId’ and ‘AWSSecretKey’.
- nodes_per_block (int) – This is always 1 for ec2. Nodes to provision per block.
- profile (str) – Profile to be used from the standard aws config file ~/.aws/config.
- nodes_per_block – Nodes to provision per block. Default is 1.
- init_blocks (int) – Number of blocks to provision at the start of the run. Default is 1.
- min_blocks (int) – Minimum number of blocks to maintain. Default is 0.
- max_blocks (int) – Maximum number of blocks to maintain. Default is 10.
- instance_type (str) – EC2 instance type. Instance types comprise varying combinations of CPU, memory, . storage, and networking capacity For more information on possible instance types,. see here Default is ‘t2.small’.
- region (str) – Amazon Web Service (AWS) region to launch machines. Default is ‘us-east-2’.
- key_name (str) – Name of the AWS private key (.pem file) that is usually generated on the console to allow SSH access to the EC2 instances. This is mostly used for debugging.
- spot_max_bid (float) – Maximum bid price (if requesting spot market machines).
- iam_instance_profile_arn (str) – Launch instance with a specific role.
- state_file (str) – Path to the state file from a previous run to re-use.
- walltime – Walltime requested per block in HH:MM:SS. This option is not currently honored by this provider.
- launcher (Launcher) – Launcher for this provider. Possible launchers include
SingleNodeLauncher
(the default),SrunLauncher
, orAprunLauncher
- linger (Bool) – When set to True, the workers will not
halt
. The user is responsible for shutting down the node.
-
__init__
(image_id, key_name, init_blocks=1, min_blocks=0, max_blocks=10, nodes_per_block=1, parallelism=1, worker_init='', instance_type='t2.small', region='us-east-2', spot_max_bid=0, key_file=None, profile=None, iam_instance_profile_arn='', state_file=None, walltime='01:00:00', linger=False, launcher=SingleNodeLauncher())[source]¶ Initialize self. See help(type(self)) for accurate signature.
Methods
__init__
(image_id, key_name[, init_blocks, …])Initialize self. cancel
(job_ids)Cancel the jobs specified by a list of job ids. config_route_table
(vpc, internet_gateway)Configure route table for Virtual Private Cloud (VPC). create_session
()Create a session. create_vpc
()Create and configure VPC get_instance_state
([instances])Get states of all instances on EC2 which were started by this file. goodbye
()initialize_boto_client
()Initialize the boto client. read_state_file
(state_file)Read the state file, if it exists. security_group
(vpc)Create and configure a new security group. show_summary
()Print human readable summary of current AWS state to log and to console. shut_down_instance
([instances])Shut down a list of instances, if provided. spin_up_instance
(command, job_name)Start an instance in the VPC in the first available subnet. status
(job_ids)Get the status of a list of jobs identified by their ids. submit
([command, tasks_per_node, job_name])Submit the command onto a freshly instantiated AWS EC2 instance. teardown
()Teardown the EC2 infastructure. write_state_file
()Save information that must persist to a file. xstr
(s)Attributes
current_capacity
Returns the current blocksize. label
Provides the label for this provider AWSProvider.scaling_enabled