In this blog post, I explain the difference between the EBS volumes GP2 and GP3. I explain why switching to GP3 promises better performance at lower costs and how straightforward such a migration can be. In addition, I will provide you with a script that makes the changeover easier and allows you to calculate your cost savings in advance.
Introduction to EBS with GP2 and GP3
What is EBS?
Elastic Block Store (EBS) is a block storage service from Amazon Web Services that provides persistent storage for EC2 instances. EBS volumes can be dynamically created, scaled and attached to EC2 instances and support features such as snapshots and encryption for data security and recovery.
So what are GP2 and GP3?
GP2 and GP3 are two types of SSD-based EBS volumes designed for general purpose and differ in performance and cost. GP2 are general purpose SSDs (Solid State Drives) that offer a balance between performance and cost. The performance of GP2 volumes scales with the size of the volume; larger volumes offer higher throughputs and more IOPS.
GP2 is well suited for a variety of general use cases, from development and test environments to databases. GP3 is the newer generation of general-purpose SSDs that offers consistent performance regardless of volume size.
GP3 volumes come with baseline performance that is independent of volume size, and users can scale IOPS and throughput independently. This results in cost savings as one does not need to add more volume size to achieve the desired performance. GP3 also offers higher maximum IOPS and throughput compared to GP2.
In summary, the new GP3 generation offers a more cost-effective storage solution that has advantages over GP2 with the same or even higher performance.
What are IOPS and Throughput?
IOPS (Input/Output Operations Per Second) measures the number of read and write operations a storage system like Amazon EBS can handle per second. It's crucial for workloads needing fast and frequent data access, like databases.
Throughput quantifies the data transfer rate to and from storage, typically in MB/s, important for applications that move large data volumes, such as data processing tasks.
Different EBS volume types balance these metrics to suit various applications—high IOPS for transactional workloads and high throughput for data-intensive tasks.
When more IOPS is needed: Consider a high-transaction database that powers an e-commerce website with thousands of transactions per minute, each involving multiple read and write operations. The performance of this website relies on how quickly each operation can be processed. For such scenarios, a higher IOPS capacity is essential to ensure fast, consistent transaction processing and a smooth user experience.
When more Throughput is needed: Imagine a video editing company that regularly works with large video files. When editors are rendering or transferring these files, the bottleneck isn't the number of operations per second, but rather how much data can be moved at a time. In this case, higher throughput is necessary to quickly read from and write to storage, enabling efficient handling of large data transfers.
How to check EBS Volumens IOPS and Throughput?
To gain a general overview of all EBS volumes, AWS Compute Optimizer
is a useful tool. It comes at no additional charge for EBS and provides a separate view for EBS volumes. AWS Compute Optimizer displays details for each EBS volume, including the current and recommended IOPS, as well as throughput. It also shows the current volume type, such as GP2 or GP3.
Should you discover potential EBS resizing findings in AWS Compute Optimizer regarding IOPS and Throughput, or if you wish to examine individual EBS volumes more closely, it's advisable to take a look at CloudWatch. There, you can view the detailed historical trends of IOPS and Throughput for each EBS volume.
Unfortunately, CloudWatch does not have a direct metric for IOPS and Throughput. Instead, it provides metrics for VolumeReadOps, VolumeWriteOps, VolumeReadBytes, and VolumeWriteBytes. To create a metric for IOPS and Throughput, one must define a custom function.
For IOPS, the function is “(VolumeReadOps + VolumeWriteOps) / PERIOD”, where PERIOD is the interval selected for both VolumeReadOps and VolumeWriteOps. In this example, it's 1 minute, so we divide by 60 seconds. The statistic chosen is “Average”. Once the function is created, the metrics VolumeReadOps and VolumeWriteOps are assigned an ID, which can then be utilized; thus, the expression in this case is “(m1+m2)/60”.
I ran a stress test using the Linux tool “fio” for this screenshot in order to validate the functionality. The storage volume here is a GP3 with 3000 IOPS, and as shown, I've fully utilized this limit.
For Throughput, the formula is ((VolumeReadBytes + VolumeWriteBytes) / 1048576) / PERIOD. The reason we divide by 1,048,576 is that VolumeReadBytes and VolumeWriteBytes are given in bytes. Since 1,024 bytes equal 1 KiB, and 1,024 KiB equals 1 MiB, multiplying these together (1024 * 1024) gives us 1,048,576. This conversion is necessary because EBS volumes report their Throughput in MiB. The statistic chosen is “Sum”.
For this screenshot, I have also conducted a load test with the Linux tool “fio” to push the limits, in this case, achieving a throughput of 125 MiB/s on the GP3 volume.
The expression used here is “((m1+m2)/1048576)/60”.
How can I migrate from GP2 to GP3?
Unfortunately, I still see AWS environments using GP2, mostly for historical reasons or due to a lack of current information. Although GP3 only offers advantages over GP2 and the migration from GP2 to GP3 can be done without downtime, many are sticking with the older technology. For environments that were built with Infrastructure as Code, the conversion should be carried out in the code, in the corresponding templates from CloudFormation, CDK, Terraform, etc. For statically structured and manually created environments, it is possible to convert the EBS volumes of the EC2 instances automatically and with minimal effort using a script. I show how this works in this GitHub repository. I also explain how a cost comparison can be created before the migration from GP2 to GP3, so that it becomes clear how high the cost savings are.
The script also allows me to decide whether I want to use the same IOPS and throughputs with GP3 as with GP2. I would then have better performance for EBS volumes under 1000 GB and the same performance for volumes over 1000 GB, with lower costs in each case with GP3.
Before proceeding with the migration of EBS volumes, it is essential to verify whether their utilization exceeds the baseline performance levels of GP3, which are 3000 IOPS and 125 MiB/s throughput. To facilitate this assessment, you can utilize tools such as the AWS Compute Optimizer and the CloudWatch metrics previously mentioned.
If there are no activities over the GP3 Baseline, the export of the script can be adapted so that the baseline values can be used for the migration from GP2 to GP3. This can achieve a saving of 20%, see middle column. The script uses the “matching configuration” by default, see right column:
Once the migration from GP2 to GP3 has been started with the script, the conversion of a volume with a size of 30 GB takes around 5–10 minutes and for a volume of 300 GB around 30 minutes. As already mentioned, there is no downtime or impact on performance.
Conclusion
I hope I have been able to show you the differences between GP2 and GP3 EBS volumes and the simplicity of migrating from GP2 to GP3. I would appreciate it if you could use my script for the migration, and I hope to convince you to adopt the newer GP3 EBS volumes. For example, I was able to migrate around 500 GP2 EBS volumes to GP3 for one customer using the script, saving almost $4000 a month in AWS costs.