AWS Crash Course P2

Now for something only slightly different...

So the last time I picked this up, I was manually provisioning instances, and as nice as it is that they've made the UI so user friendly, I'm more interested in creating/configuring/deploying-to instances with code, so I'm going to start looking at the documentation for the Python version of Boto3, which is an SDK for AWS.

My goal for this session is to do the same thing I did last time, but using the SDK to do all the work via a python script.  Should be fun!

Problem 1

In the quick start, and normal API docs for boto3, an access key and secret were mentioned. I was looking all over the place for the access keys I had not yet created, and there wasn't any mention of where to generate the keys other than installing the AWS CLI.  Not that I'm against that, I just want to try to do it all from python.

So I was getting this error at first:

>>> ec2 = boto3.resource('ec2')

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/__init__.py", line 92, in resource

    return _get_default_session().resource(*args, **kwargs)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/__init__.py", line 72, in _get_default_session

    setup_default_session()

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/__init__.py", line 34, in setup_default_session

    DEFAULT_SESSION = Session(**kwargs)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/session.py", line 80, in __init__

    self._setup_loader()

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/session.py", line 113, in _setup_loader

    self._loader = self._session.get_component('data_loader')

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 691, in get_component

    return self._components.get_component(name)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 885, in get_component

    self._components[name] = factory()

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 179, in <lambda>

    lambda:  create_loader(self.get_config_variable('data_path')))

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 259, in get_config_variable

    elif self._found_in_config_file(methods, var_config):

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 280, in _found_in_config_file

    return var_config[0] in self.get_scoped_config()

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 342, in get_scoped_config

    profile_map = self._build_profile_map()

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 214, in _build_profile_map

    self._profile_map = self.full_config['profiles']

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/session.py", line 380, in full_config

    cred_file)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/configloader.py", line 131, in raw_config_parse

    raise botocore.exceptions.ConfigParseError(path=path)

botocore.exceptions.ConfigParseError: Unable to parse config file:

...which really makes sense now, since my config was mangel-wurzel at this point.

So I found the Access Key widget in the AWS account info/management for my account.  Problem solved:

>>> ec2 = boto3.resource('ec2')

>>> dir(ec2)

[u'ClassicAddress', u'DhcpOptions', u'Image', u'Instance', u'InternetGateway', u'KeyPair', u'NetworkAcl', u'NetworkInterface', u'NetworkInterfaceAssociation', u'PlacementGroup', u'Route', u'RouteTable', u'RouteTableAssociation', u'SecurityGroup', u'Snapshot', u'Subnet', u'Tag', u'Volume', u'Vpc', u'VpcAddress', u'VpcPeeringConnection', '__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', u'classic_addresses', u'create_dhcp_options', u'create_instances', u'create_internet_gateway', u'create_key_pair', u'create_network_acl', u'create_network_interface', u'create_placement_group', u'create_route_table', u'create_security_group', u'create_snapshot', u'create_subnet', u'create_tags', u'create_volume', u'create_vpc', u'create_vpc_peering_connection', u'dhcp_options_sets', u'disassociate_route_table', 'get_available_subresources', u'images', u'import_key_pair', u'instances', u'internet_gateways', u'key_pairs', 'meta', u'network_acls', u'network_interfaces', u'placement_groups', u'register_image', u'route_tables',  u'security_groups', u'snapshots', u'subnets', u'volumes', u'vpc_addresses', u'vpc_peering_connections', u'vpcs']

So now lets take a look at the instances I already have:

>>> instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])

>>> instances

ec2.instancesCollection(ec2.ServiceResource(), ec2.Instance)

>>> for i in instances:

...     print(i.id, i.instance_type)

... 

('i-011b26ac17c23c5ac', 't2.micro')

This appears to be the instance I created manually.  Cool beans, how about the instance(s) status?

>>> for status in ec2.meta.client.describe_instance_status()['InstanceStatuses']:

...     print(status)

... 

{u'InstanceId': 'i-011b26ac17c23c5ac', u'InstanceState': {u'Code': 16, u'Name': 'running'}, u'AvailabilityZone': 'us-west-2b', u'SystemStatus': {u'Status': 'ok', u'Details': [{u'Status': 'passed', u'Name': 'reachability'}]}, u'InstanceStatus': {u'Status': 'ok', u'Details': [{u'Status': 'passed', u'Name': 'reachability'}]}}

Cool, now I want to create another instance using the same AMI I used to create the last instance:

>>> ec2.create_instances(ImageId='ami-00bb6f60', MinCount=1, MaxCount=1)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/resources/factory.py", line 520, in do_action

    response = action(self, *args, **kwargs)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/boto3/resources/action.py", line 83, in __call__

    response = getattr(parent.meta.client, operation_name)(**params)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/client.py", line 159, in _api_call

    return self._make_api_call(operation_name, kwargs)

  File "/Users/adam/Library/Python/2.7/lib/python/site-packages/botocore/client.py", line 494, in _make_api_call

    raise ClientError(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (InvalidParameterCombination) when calling the RunInstances operation: Non-Windows instances with a virtualization type of 'hvm' are currently not supported for this instance type.

Hrm, that isn't a pretty exception, but it does give me the info I need to move along. Using a different API which allows for PV (found in public images section of AWS management console)

>>> ec2.create_instances(ImageId='ami-01940631', MinCount=1, MaxCount=1)

[ec2.Instance(id='i-005edb19b06e5f76c')]

...and boom: I've got another instance now:

>>> for status in ec2.meta.client.describe_instance_status()['InstanceStatuses']:

...     print(status)

... 

{u'InstanceId': 'i-011b26ac17c23c5ac', u'InstanceState': {u'Code': 16, u'Name': 'running'}, u'AvailabilityZone': 'us-west-2b', u'SystemStatus': {u'Status': 'ok', u'Details': [{u'Status': 'passed', u'Name': 'reachability'}]}, u'InstanceStatus': {u'Status': 'ok', u'Details': [{u'Status': 'passed', u'Name': 'reachability'}]}}

{u'InstanceId': 'i-005edb19b06e5f76c', u'InstanceState': {u'Code': 16, u'Name': 'running'}, u'AvailabilityZone': 'us-west-2b', u'SystemStatus': {u'Status': 'initializing', u'Details': [{u'Status': 'initializing', u'Name': 'reachability'}]}, u'InstanceStatus': {u'Status': 'initializing', u'Details': [{u'Status': 'initializing', u'Name': 'reachability'}]}}

On to adjusting the states, snapshots, and instance deletion.