Cloudibn News

Be updated with Technology

  • BIG Data & Analytics
  • CLOUD
  • Data Center
  • IOT
  • Machine Learning & AI
  • SECURITY
  • Blockchain
  • Virtualization
You are here: Home / CLOUD / Burst credits of t2 EC2 instances need monitoring

Burst credits of t2 EC2 instances need monitoring

March 13, 2018 by cbn Leave a Comment

EC2 is one of the fundamental services on AWS. If you are not 100% Serverless, your application health depends on the health of your EC2 instances. When I do AWS architecture reviews for our clients, I check that CPU burst capacity is monitored for EC2 instances of type t2. A widespread mistake is that credits of burstable EC2 instances (t2 family) are not monitored. If your burstable EC2 instance runs out of credits, the performance drops by 70-95%.

t2 performance can drop by 95%! Are you prepared?

Why are t2 instances special?

Let’s look at the t2.large in more detail. Your baseline performance is 0.6 vCPU (70% performance drop) while you can burst up to 2 vCPUs as long as you have credits. But the baseline can be much worse. Let’s look at the t2.nano, the cheapest instance type. Your baseline is 0.05 vCPU (95% performance drop) while you can burst up to 1 vCPU. The following table shows the information for all t2 instance types.

instance type performance drop baseline vCPUs maximum vCPUs
t2.nano 95% 0.05 1
t2.micro 90% 0.10 1
t2.small 80% 0.20 1
t2.medium 80% 0.40 2
t2.large 70% 0.60 2
t2.xlarge 77.5% 0.90 4
t2.2xlarge 83.1% 1.35 8

Why does this matter?

Your t2 instance only consumes credits if your application requires more than the baseline performance. This means that you can only run out of credits if your instance is bursting. In other words, credits are only consumed if your application needs performance.

Now imagine what happens if the t2 instance is no longer able to burst and drops to the baseline from one second to the other. This is precisely what happens if you run out of burst credits. In other words, performance drops by up to 95% from one second to the other. This will have a significant impact on your application. Most likely, the application will not be responsive anymore.

How can you avoid the performance drop?

First of all, your t2 instances are earning credits while not bursting. For Example, a t2.large earns 36 credits per hour. One CPU credit is equal to one vCPU running at 100% utilization for one minute. As long as you don’t run out of credits, performance will not drop to the baseline. But how do you know? First, each t2 EC2 instance publishes the CPUCreditBalance metric to CloudWatch. The metric reports the remaining CPU credits available. Second, you can define a CloudWatch Alarm that continuously monitors the CPUCreditBalance metric. As soon as you run out of credits, an alert is triggered and you can react. E.g., increase capacity or reschedule work.

CloudWatch Alarms and marbot

We found that emails are not a good way to handle alerts. In a team, multiple people are responsible. If you send an email to a group email address:

  1. Your team has no idea if someone already started to work on solving the issue.
  2. You disturb the whole team for each alert.
  3. It’s easy to ignore an email.
  4. You have no statistics about how many alerts are generated. Too many alerts are an indication that your team is no longer able to handle them.
  5. No help to investigate the issue is available, like links to the AWS Management Console.

To solve the problem, we built marbot: a Slack bot supporting your DevOps team to detect and solve incidents on AWS.

marbot forwards alerts to Slack

marbot sends alerts to a single user from the Slack channel via a direct message. If the user doesn’t acknowledge the alert within 5 minutes, marbot will escalate to the next level. Escalations minimize distraction while keeping response time low. Try marbot for free now.

CloudFormation template

We developed a CloudFormation template to monitor an EC2 instance in any region (includes CPUCreditBalance metric monitoring). The template integrates with marbot, but you can modify it to send out emails. The template is available on GitHub for free. We also offer a version that works with a fleet of EC2 instances managed by an Auto Scaling Group.

If you have already installed marbot, you can also ask marbot to monitor your EC2 instance or read more detailed setup instructions. Otherwise: Try marbot for free now.

Summary

t2 EC2 instances are cheap, but they increase complexity. You have to monitor burst credits to ensure that you will not suffer from a 95% performance drop. We usually advise not to use the t2 family in production systems that serve user traffic. But we like t2 instances for test environments and internal applications like Jenkins. If you want to stay with the low-cost t2 family, you can enable T2 Unlimited which provides a way to continue to burst without credits but with an additional charge.

Share on FacebookShare on TwitterShare on LinkedinShare on Pinterest

Filed Under: CLOUD

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Archives

  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • May 2015

Recent Posts

  • Tigera’s Calico Cloud Now Available in AWS Marketplace
  • Key metrics to consider when assessing the performance of your VDI/ DaaS environment
  • insightsoftware Acquires Izenda, Diving Deeper into Embedded Analytics
  • Kaspersky Cited as a “Vendor to Watch” for Software-Defined Vehicles
  • The Secret IR Insider’s Diary – from Sunburst to DarkSide

Recent Comments

  • +905443535397 on Announcing Cognitive Search: Azure Search + cognitive capabilities

Categories

  • Artificial intelligence
  • BIG Data & Analytics
  • BlockChain
  • CLOUD
  • Data Center
  • IOT
  • Machine Learning
  • SECURITY
  • Storage
  • Uncategorized
  • Virtualization

Categories

  • Artificial intelligence (51)
  • BIG Data & Analytics (33)
  • BlockChain (331)
  • CLOUD (1,742)
  • Data Center (10)
  • IOT (2,091)
  • Machine Learning (149)
  • SECURITY (425)
  • Storage (25)
  • Uncategorized (63)
  • Virtualization (923)

Subscribe Our Newsletter

0% Complete

Copyright © 2021 · News Pro Theme on Genesis Framework · WordPress · Log in