Over the coming weeks I’m likely to discuss a mix of old and new announcements, mostly so I get a chance to discuss things that happened when I wasn’t writing. This week we start with the great news around EC2 tags and then dive back to an announcement from Cloudflare and how that impacted AWS pricing.

EC2 Tags in Metadata

The first week of January usually isn’t the time where I feel that a release might be one of my favourites of the year. But when AWS announced that you can now get an instance’s tags through it’s metadata endpoint I became very happy. This solves an issue I’ve run into a number of times.

If you haven’t run into this before, the issue with EC2 tags previously was that you can’t see what tags are attached to an instance without using the AWS CLI (or SDK etc). And for that you needed an IAM permission. Unfortunately, there is no way to say in IAM “only allow this action on the instance it’s run from” so unless you create a separate IAM role for every instance you’d end up with a role that lets you read the tags of other instances as well¹. Which obviously isn’t ideal, but until last week was the only way we could do this.

Of course, the question would then be, why do you need access to the tags in the first place? That can be for a variety of reasons really, but usually it’s because you want to run some software on the instance that needs this information. The exact use case will differ, but an example where I’ve used it is with configuration scripts and security software that reported back to a centralised system and therefore needed the tags to more easily identify where these instances were running.

So, it’s great to see that this is now a possibility. However, it isn’t perfect as it comes with a little caveat. You need to enable it first. If you don’t enable it, when you call the metadata endpoint you just get the same list as always:

$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
reservation-id
security-groups
services/ 

So, you need to enable it. Which you need to do by changing the metadata options. If you use infrastructure as code², this needs to be done through a launch template as that’s not something you can do using the standard EC2 resource³ but as of this writing neither CloudFormation nor Terraform have been updated to support this yet. Which is kind of a bummer to be honest and prevents most uses until it’s been updated. Until then you can test it however by running the below for an existing instance (or see the documentation for doing it for a new instance)

$ aws ec2 modify-instance-metadata-options \
    --instance-id $YOURINSTANCEID \
    --instance-metadata-tags enabled

And then when you call the metadata endpoint you see tags at the end of the list:

$ curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id
instance-life-cycle
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
reservation-id
security-groups
services/
tags/

And calling the tags/ endpoint then gives you an instance endpoint, containing a list of tags which you can go through as always when using the metadata endpoint.

$ curl http://169.254.169.254/latest/meta-data/tags/instance/Name
arjen-demo-Instance

I’m hoping that the fact that it includes instance/ means we can expect more resources in there later on, such as volumes or security groups, but the instance tags are already the most important information so I’m very happy to see this.

Cloudflare R2

Back in September, Cloudflare pre-announced their R2 object storage service. R2 is clearly aimed as an S3 competitor. It’s not the first of its kind, and likely won’t be the last. So, why is this one worth calling out? Because of the pricing model.

To be clear, R2 is not yet available and so we don’t really know if anything about it will change. The main difference however is the price as Cloudflare’s claim is that R2 will be quite a bit cheaper for storage than the standard tier of S3⁴. Most importantly though, egress traffic to the Internet is free which compares quite well to the $ 0.09 per GB you pay for a bucket in a US region, or $ 0.114 per GB for a bucket in the Sydney region⁵. Obviously, this doesn’t mean a lot with small amounts of data but as Corey Quinn pointed out, with large amounts of traffic the difference is quite substantial.

As a non-commercial example, let’s say you self-host your weekly podcast. If you store a 50MB episode file on S3 and get about 10.000 listeners⁶ that would be about 500GB of traffic which is $45 dollars per week when served from the US. Not much for a company, but for something you do as a hobby it would sure be nice if you can spend that kind of money elsewhere.

AWS' free tier expansion

Without mentioning Cloudflare’s blogpost and therefore completely unrelated⁷, AWS announced an expansion of their free tier just before re:Invent. Where before the free tier for data transfer was temporary, it is now perpetual per account and has increased numbers. Traffic to the Internet from your region is now free for the first 100GB (across all your regions) where before it was 1GB per region. This includes S3 traffic, but also other regional services like your EC2 instances and load balancers. And in addition, the CloudFront free tier went up to 1TB per month as well as the number of HTTP(S) requests⁸.

The first reaction for this is obviously “Yay! More free stuff”! We can all see our bills go down! Except, even with my simple example above regarding the podcast hosting it doesn’t actually make that much of a difference. Assuming no other months exist than February, there are 4 releases of the podcast per month which gives a total of 2TB of data traffic. So we now pay 5% less if we serve directly from S3 but a respectable 50% if we instead served it from CloudFront. And obviously if you do anything truly at scale you’ll barely notice the difference.

So, it’s definitely a nice change, but I admit that if I had a podcast with 10.000 listeners I’d switch hosting its files to R2 the moment it becomes available⁹. Let’s be clear though, there are many features and integrations that S3 has that I don’t see coming to R2 anytime soon so it’s not usually as dry cut as this example for some simple file hosting. And obviously my wallet is definitely grateful for Cloudflare announcing R2 and AWS making a completely unrelated change to their free tier for data transfer.