AWSProvider(image_id, init_blocks=1, min_blocks=0, max_blocks=10, nodes_per_block=1, parallelism=1, worker_init='', instance_type='t2.small', region='us-east-2', spot_max_bid=0, key_name=None, key_file=None, profile=None, iam_instance_profile_arn='', state_file=None, walltime='01:00:00', linger=False, launcher=SingleNodeLauncher())¶
A provider for using Amazon Elastic Compute Cloud (EC2) resources.
One of 3 methods are required to authenticate: keyfile, profile or environment variables. If neither keyfile or profile are set, the following environment variables must be set:
AWS_ACCESS_KEY_ID(the access key for your AWS account),
AWS_SECRET_ACCESS_KEY(the secret key for your AWS account), and (optionaly) the
AWS_SESSION_TOKEN(the session key for your AWS account).
- image_id (str) – Identification of the Amazon Machine Image (AMI).
- worker_init (str) – String to append to the Userdata script executed in the cloudinit phase of instance initialization.
- walltime (str) – Walltime requested per block in HH:MM:SS.
- key_file (str) – Path to json file that contains ‘AWSAccessKeyId’ and ‘AWSSecretKey’.
- nodes_per_block (int) – This is always 1 for ec2. Nodes to provision per block.
- profile (str) – Profile to be used from the standard aws config file ~/.aws/config.
- nodes_per_block – Nodes to provision per block. Default is 1.
- init_blocks (int) – Number of blocks to provision at the start of the run. Default is 1.
- min_blocks (int) – Minimum number of blocks to maintain. Default is 0.
- max_blocks (int) – Maximum number of blocks to maintain. Default is 10.
- instance_type (str) – EC2 instance type. Instance types comprise varying combinations of CPU, memory, . storage, and networking capacity For more information on possible instance types,. see here Default is ‘t2.small’.
- region (str) – Amazon Web Service (AWS) region to launch machines. Default is ‘us-east-2’.
- key_name (str) – Name of the AWS private key (.pem file) that is usually generated on the console to allow SSH access to the EC2 instances. This is mostly used for debugging.
- spot_max_bid (float) – Maximum bid price (if requesting spot market machines).
- iam_instance_profile_arn (str) – Launch instance with a specific role.
- state_file (str) – Path to the state file from a previous run to re-use.
- walltime – Walltime requested per block in HH:MM:SS. This option is not currently honored by this provider.
- launcher (Launcher) – Launcher for this provider. Possible launchers include
- linger (Bool) – When set to True, the workers will not
halt. The user is responsible for shutting down the node.
__init__(image_id, init_blocks=1, min_blocks=0, max_blocks=10, nodes_per_block=1, parallelism=1, worker_init='', instance_type='t2.small', region='us-east-2', spot_max_bid=0, key_name=None, key_file=None, profile=None, iam_instance_profile_arn='', state_file=None, walltime='01:00:00', linger=False, launcher=SingleNodeLauncher())¶
Initialize self. See help(type(self)) for accurate signature.
__init__(image_id[, init_blocks, …])
Cancel the jobs specified by a list of job ids.
Configure route table for Virtual Private Cloud (VPC).
Create a session.
Create and configure VPC
Get states of all instances on EC2 which were started by this file.
Initialize the boto client.
Read the state file, if it exists.
Create and configure a new security group.
Print human readable summary of current AWS state to log and to console.
Shut down a list of instances, if provided.
Start an instance in the VPC in the first available subnet.
Get the status of a list of jobs identified by their ids.
submit([command, blocksize, tasks_per_node, …])
Submit the command onto a freshly instantiated AWS EC2 instance.
Teardown the EC2 infastructure.
Save information that must persist to a file.
Returns the current blocksize.
The callers of ParslExecutors need to differentiate between Executors and Executors wrapped in a resource provider