Week 2, 2022 - EC2 Tags in Metadata; Cloudflare R2; AWS pricing cut

By (7 minutes read)

Over the coming weeks I’m likely to discuss a mix of old and new announcements, mostly so I get a chance to discuss things that happened when I wasn’t writing. This week we start with the great news around EC2 tags and then dive back to an announcement from Cloudflare and how that impacted AWS pricing.

EC2 Tags in Metadata

The first week of January usually isn’t the time where I feel that a release might be one of my favourites of the year. But when AWS announced that you can now get an instance’s tags through it’s metadata endpoint I became very happy. This solves an issue I’ve run into a number of times.

If you haven’t run into this before, the issue with EC2 tags previously was that you can’t see what tags are attached to an instance without using the AWS CLI (or SDK etc). And for that you needed an IAM permission. Unfortunately, there is no way to say in IAM “only allow this action on the instance it’s run from” so unless you create a separate IAM role for every instance you’d end up with a role that lets you read the tags of other instances as well

1. Which obviously isn’t ideal, but until last week was the only way we could do this.

Of course, the question would then be, why do you need access to the tags in the first place? That can be for a variety of reasons really, but usually it’s because you want to run some software on the instance that needs this information. The exact use case will differ, but an example where I’ve used it is with configuration scripts and security software that reported back to a centralised system and therefore needed the tags to more easily identify where these instances were running.

So, it’s great to see that this is now a possibility. However, it isn’t perfect as it comes with a little caveat. You need to enable it first. If you don’t enable it, when you call the metadata endpoint you just get the same list as always:

$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
reservation-id
security-groups
services/

So, you need to enable it. Which you need to do by changing the metadata options. If you use infrastructure as code

2, this needs to be done through a launch template as that’s not something you can do using the standard EC2 resource
3 but as of this writing neither CloudFormation nor Terraform have been updated to support this yet. Which is kind of a bummer to be honest and prevents most uses until it’s been updated. Until then you can test it however by running the below for an existing instance (or see the documentation for doing it for a new instance)

$ aws ec2 modify-instance-metadata-options \
    --instance-id $YOURINSTANCEID \
    --instance-metadata-tags enabled

And then when you call the metadata endpoint you see tags at the end of the list:

$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
reservation-id
security-groups
services/
tags/

And calling the tags/ endpoint then gives you an instance endpoint, containing a list of tags which you can go through as always when using the metadata endpoint.

$ curl http://169.254.169.254/latest/meta-data/tags/instance/Name
arjen-demo-Instance

I’m hoping that the fact that it includes instance/ means we can expect more resources in there later on, such as volumes or security groups, but the instance tags are already the most important information so I’m very happy to see this.

Cloudflare R2

Back in September, Cloudflare pre-announced their R2 object storage service. R2 is clearly aimed as an S3 competitor. It’s not the first of its kind, and likely won’t be the last. So, why is this one worth calling out? Because of the pricing model.

To be clear, R2 is not yet available and so we don’t really know if anything about it will change. The main difference however is the price as Cloudflare’s claim is that R2 will be quite a bit cheaper for storage than the standard tier of S3

4. Most importantly though, egress traffic to the Internet is free which compares quite well to the $ 0.09 per GB you pay for a bucket in a US region, or $ 0.114 per GB for a bucket in the Sydney region
5. Obviously, this doesn’t mean a lot with small amounts of data but as Corey Quinn pointed out, with large amounts of traffic the difference is quite substantial.

As a non-commercial example, let’s say you self-host your weekly podcast. If you store a 50MB episode file on S3 and get about 10.000 listeners

6 that would be about 500GB of traffic which is $45 dollars per week when served from the US. Not much for a company, but for something you do as a hobby it would sure be nice if you can spend that kind of money elsewhere.

AWS' free tier expansion

Without mentioning Cloudflare’s blogpost and therefore completely unrelated

7, AWS announced an expansion of their free tier just before re:Invent. Where before the free tier for data transfer was temporary, it is now perpetual per account and has increased numbers. Traffic to the Internet from your region is now free for the first 100GB (across all your regions) where before it was 1GB per region. This includes S3 traffic, but also other regional services like your EC2 instances and load balancers. And in addition, the CloudFront free tier went up to 1TB per month as well as the number of HTTP(S) requests
8.

The first reaction for this is obviously “Yay! More free stuff”! We can all see our bills go down! Except, even with my simple example above regarding the podcast hosting it doesn’t actually make that much of a difference. Assuming no other months exist than February, there are 4 releases of the podcast per month which gives a total of 2TB of data traffic. So we now pay 5% less if we serve directly from S3 but a respectable 50% if we instead served it from CloudFront. And obviously if you do anything truly at scale you’ll barely notice the difference.

So, it’s definitely a nice change, but I admit that if I had a podcast with 10.000 listeners I’d switch hosting its files to R2 the moment it becomes available

9. Let’s be clear though, there are many features and integrations that S3 has that I don’t see coming to R2 anytime soon so it’s not usually as dry cut as this example for some simple file hosting. And obviously my wallet is definitely grateful for Cloudflare announcing R2 and AWS making a completely unrelated change to their free tier for data transfer.

  1. You can add some limitations, but in the end it’s still wider than the current instance. ↩︎

  2. As you obviously should. ↩︎

  3. Since introducing launch templates, AWS has had the tendency to only add new features for EC2 instances there. So it’s highly recommended to switch to using them, especially for AutoScaling groups but also for standalone instances. ↩︎

  4. S3 has a lot of different access tiers, especially if you include Glacier. Always make sure to review your data retrieval requirements to see if you can save money there. ↩︎

  5. Again, S3 pricing is complex and there are a bunch of things you do get for free such as egress traffic to an AWS service in the same region. ↩︎

  6. Let’s be clear, this is not a number I get on any of mine. ↩︎

  7. As it says in the article “as part of our long tradition of AWS price reductions”. The timing is obviously coincidence. ↩︎

  8. Yes, the pricing is complicated, which is why I don’t often write about it. I’m also pretty sure that, while you could make the argument this is because it means you only pay for what you need, this complexity isn’t exactly customer friendly. ↩︎

  9. Well, honestly I’d just use a service that handles it all for me as that’s easier. ↩︎

Read more like this:

Or always get the latest by subscribing through RSS, Twitter, or email!