AWS EC2: Elastic Compute Cloud 2nd part
EC2- INSTANCE TERMINATION
When you terminate a running instance the instance states change as follow
Running- Shutting down- Terminated
By default, EBS root device volumes, (created automatically when the instance is launched) are deleted automatically when the EC2 instance is terminated.
Any additional volumes attached to the instance (those you attach to by the instance during launch or later), by default, persist after the instance is terminated.
You can modify both behaviors by modifying the “Delete on Termination” attribute of any EBS volume during instance launch or while running.
You can view EBS root volume Delete on Termination behavior from “Block device Mapping”.
EC2 TERMINATION PROTECTION
This is a feature you can enable such that an EC2 instance is protected against accidental termination through API, Console or CLI.
This can be enabled for instance store backed and EBS backed instances.
Cloud watch can only terminate EC2 instances if they do not have the termination protection enabled.
If you want to terminate an instance that has termination protection turned on, you can do so by choosing OS shutdown and configure OS shutdown as instance termination.
This can be configured during launch when the instance is running or stopped.
AWS recommends that after you launch an EC2 instance, you check its status to confirm that it moved from pending to running, and not to a terminated state.
Possible reasons that a launched instances immediately terminates are:
- The instance store backed AMI you used to launch the instance is missing a required part.
- You have reached your EBS volume limit.
- An EBS snapshot is corrupt.
To find the reason for the termination is:
- From AWS Console: Go to instances > Description tab> State Transition reason
- From CLI use the “describe-instance” command
EC2 INSTANCE METADATA
This is instance data that you can use to configure or manage the instance.
Examples: IPV4 address, IPV6 address, DNS hostnames, AMI ID, Instance ID, Instance type, Local hostname, public keys, Security Groups
Metadata can be only viewed from within the instance itself
i.e you have to logon to the instance
Metadata is not protected by encryption, anyone that has access to the instance can view this data.
To view an EC2 instance’s Metadata ( from the EC2 instance console)
To view a specific metadata parameter, an example to view local hostname
GET http:// 220.127.116.11/latest/meta- data/host-name/
EC2 INSTANCE USER DATA
Is data supplied by the user at instance launch in the form of a script to be executed during the instance boot.
User data is limited to 16 KB.
User data can only be viewed from within the instance itself.
You can change user data
- To do so, you need to stop the instance first (EBS backed)
Instance>actions> Instance setting> view/change user data
User data is not protected by encryption, do not include passwords or sensitive data in your user data scripts.
You are not charged for requests to read user data or metadata.
EC2 -VM IMPORT/EXPORT
Can be used to migrate VMware, Microsoft, XEN VM's to the cloud (Import)
Can be used to convert EC2 instances to Vmware, Microsoft, XEN VMs to use on-premise (Export)
- Windows and Linux VM’s
- Vmware ESX VMDK (Virtual Machine Disk) and OVA (Open Virtualization Appliance) images for export only.
- Citrix XEN VHD
- Microsoft Hyper V VHD
VM Import/Export is supported through API, CLI but not through AWS console
Before generating the VMDK or VHD images, make sure the VM is stopped and not in suspended or paused states.
For vmware, AWS has a VM connector which is a plugin to vmware vCenter
This allows the migration of VMs to AWS S3
Convert it to EC2 AMI
And progress can be tracked in vCenter
For an EC2 instance to have access to other AWS services you need to configure an IAM role, which will have an IAM policy attached, under the EC2 instance
- Applications on the EC2 instance will get this role permission from the EC2 instance’s metadata.
EC2 BASTION HOST FOR LINUX
For inbound, secure, connectivity to your VPC to manage and administer public and or private EC2 instances, you can use a bastion host (or a jump box/stone)
- The Bastion host is an EC2 instance, whose interface will have a security group allowing inbound SSH or RDP for windows instances.
- Bastion host can have auto-assigned public IP address or Elastic IP addresses.
- Using security groups you can further limit which IP CIDRs can access the Bastion Host.
To configure a bastion host in high availability, you can use auto-scaling group as follows:
- Create the ASG with desired capacity of 2
choose multiple availability zones
using elastic IPs on each
This is the recommended HA way
(Not an HA but saves on cost) Create an ASG with desired capacity 1, minimum 1, maximum 1, such that if the bastion instance fails, or gets terminated, the ASG will launch another one.
- Downside is, you have only one at a time, and you may have downtime until ASG launches another one, but since this is for management/ administration, downtime can be acceptable.
EC2 LAUNCH MODELS AND COST OPTIMIZATION
AWS offers a broad range of resource types and configurations to suit a plethora of use cases
- AWS services such as EC2, RDS, Redshift and Elastic search give a lot of choice of instance type
In some cases, selecting the cheapest type that suits workload’s requirements might be the best.
In other cases using larger instance type can help reduce the overall cost for better performance
Best is to use benchmarking to select the right instance size/type which depends primarily on how the workload utilizes CPU,RAM, network, storage size and I/O.
EC2 PURCHASE OPTIONS (LAUNCH MODELS)
Amazon EC2 provides the following purchasing options to enable you to optimize your costs based on your needs.
pay, by the second, for the instance that you launch
Purchase, at a significant discount, instances that are always available, for a term from one to three years.
Scheduled (Reserved) Instances
Purchase instances that are always available on the specified recurring schedule, for one year term.
Request unused EC2 instances, which can lower your Amazon EC2 costs significantly.
Pay for physical host that is fully dedicated to running your instance, and bring your existing per socket, per core, or per VM software licenses to reduce costs.
Pay, by the hour, for instances that run on single-tenant hardware.
Reserve capacity for your EC2 instances in a specific availability zone for any duration.
Purchase at a significant discount, instances that are always available, for a term from one to three years.
- EC2 Reserved Instances allow you to reserve Amazon EC2 computing capacity in exchange for a significantly discounted hourly rate.
- Depending on your Reserved Instance purchases, the discounts will be reflected in the monthly bill.
There is technically no difference between an On-Demand EC2 instance and a Reserved instance. The difference lies in the way you pay for instances that you reserve.
This is ideal for applications with predictable minimum capacity requirements.
- Reserved capacity options exist for other services as well.
You can take advantage of tools like the AWS Trusted Advisor or EC2 usage reports to identify the compute resources that you use most of the time that you should consider reserving.
RI best practice
You should not commit to Reserved Instance purchases before sufficiently benchmarking your application in production. After you have purchased reserved capacity, you can use the Reserved Instance utilization reports to ensure you are still making the most of your reserved capacity.
RI- OFFERING CLASSES
If your computer needs change, you may be able to modify or exchange your Reserved Instance, depending on the offering class. Offering class may also have additional restrictions or limitations.
Standard RI: (can be zonal or regional)
You cannot change the instance type during the term.
You can change instance size.
You can’t exchange it with another.
It can be sold in the RI market place.
Convertible RI: (can be zonal or regional)
It can be exchanged during the term for another convertible reserved instance with new attributes including instance family, instance type, platform, scope or tenancy.
It cannot be sold in the RI marketplace
SCHEDULED RESERVED INSTANCES
Purchase instances that are always available on the specified recurring schedule, for a one year term.
Use when purchasing capacity reservations that recur on a daily, weekly, monthly basis with a specified start time and duration.
You reserve the capacity in advance, so you know it is available when you need it.
You pay for the time that the instances are scheduled, even if you do not use them.
Amazon EC2 spot instances allow you to bid on spare Amazon EC2 community capacity.
- By requesting the unused EC2 instances, customers can lower your Amazon EC2 costs significantly.
Spot instances are often available at a discount compared to on-demand pricing, which can significantly reduce the cost of running applications on AWS.
- The hourly price for a Spot Instance is called a spot price.
The Spot Instance is launched when the bid price exceeds the current Spot market price
- It will continue run until you choose to terminate it, or until the spot market price exceeds your bid.
- If the Spot market price increases about your bid price, your instance will be terminated automatically, and you will not be charged for the partial hour that your instance has run.
As a result, Spot instances are great for workloads that have the tolerance to interruption.
- Use for workloads with flexible start and end times.
SPOT INSTANCE USAGE STRATEGIES
AWS charges the Spot market price for as long as the spot instance runs.
The bidding strategy could be bid much higher than that with the expectation that even if the market price occasionally spikes, customers would still be saving a lot of cost in the long term.
Mixed with On Demand:
Consider mixing Reserved, On-Demand and spot instance to combine a predictable minimum capacity with opportunistic access to additional compute resources depending on the spot market price.
This is a great way to improve throughput or application performance.
Spot Blocks for Defined Duration Workloads:
Customers can also bid for fixed duration Spot Instances
- These have different hourly pricing but allow you to specify a duration requirement.
If the bid is accepted your instance will continue to run until you choose to terminate it, or until the specified duration has ended.
- Therefore, spot instances will not be terminated due to changes in the spot price.
Spot instance can be launched independently, with Auto Scaling or EMR.
- Redshift do not use spot instances.
An Amazon EC2 Dedicated Host is a physical server with EC2 instance capacity fully dedicated to one AWS customer use.
Dedicated host allows the customer to use his existing per socket, or per VM software licenses, including
- Windows server
- Microsoft SQL server
- SUSE, Linux Enterprise Server
- Customer is billed per Host
- Doesn't support automatic instance recovery
- Supports BOYL
- It provides visibility at the sockets, physical core, and how instances are placed on the server.
Dedicated Instances are Amazon EC2 instances that run in a VPC on hardware that’s dedicated to a single customer.
Dedicated instances that belong to different AWS accounts are physically isolated at the hardware level.
Dedicated Instances that belong to AWS accounts that are linked to a single-payer account are also physically isolated at the hardware level.
- However, Dedicated instances may share hardware with other instances from the same AWS account that are not Dedicated instances.
BothDedicated Hosts and Dedicated instances can be used to launch Amazon EC2 instances onto physical servers that are dedicated to one AWS customer's use.
- There are no performance, security, or physical differences between Dedicated Instances on Dedicated Hosts.
ON DEMAND CAPACITY RESERVATIONS
It enables you to reserve capacity for your Amazon EC2 instances in a specific availability zone for any duration.
This gives you the ability to create and manage capacity reservations independently from the billing discounts offered by Reserved Instances (RI).
By creating Capacity Reservation, you ensure that you always have access to EC2 capacity when you need it, for as long as you need it.
Capacity Reservations can be created at any time, without entering into a one year or three-year term commitment, and the capacity is available immediately.
When you no longer need the reservation, cancel the capacity reservation to stop incurring charges for it.
WHEN TO USE WHAT?
Amazon EC2 On-Demand instance pricing gives you maximum flexibility with no long term commitments, use it when you want a few hours per day usage, or on/off usage but require availability when you need it.
If you require a capacity reservation, purchase:
Reserved Instance or
Capacity Reservation for a specific availability zone or
Purchased Schedules instances.
Spot instances are a cost-effective choice if you can be flexible about when your applications run and they can be interrupted.
Dedicated Hosts can help you address compliance requirements and reduce costs by using your existing server bound software licenses.
In my next blog, I will continue with more EC2 functions. So stay connected. Thankyou.