This week AWS released several useful features for existing services. Step Functions now support Dynamic Parallelism, and you can now replicate your S3 buckets within the same region.

Step Functions Dynamic Parallelism

The Step Functions team has been on a roll lately, last month they added nested step functions, and now they’ve added Dynamic Parallelism as well. Now, dynamic parallelism may not be the clearest explanation¹ of what this new feature does, so I’ll try to break it down.

What it lets you do is configure parts of your state machines to potentially run in parallel. It was already possible to run various parts of your state machine in parallel, for example sending a notification to Slack and an email at the same time. Those were hardcoded parallelisations though, and generally different functionalities. What dynamic parallelism gives you, however, is the ability to run the same functions in parallel.

The example given in the announcement is an online store, where for each item in an order the Step Function can check the stock and prepare it for shipping simultaneously. While a reasonably simplified example, it does illustrate how you can speed up some processes.

Another effect of it is that it can potentially clean up your configuration as well since you don’t need to build your own loop to iterate over a set of items. The dynamic parallelism, which happily uses the Iterator keyword takes care of that. And when you then combine that with nested step functions, you can end up with a pretty clean and concise state machine².

S3 Same-Region Replication

Four-and-a-half years ago³, AWS released cross-region replication for S3. And now it’s possible to replicate S3 within a region as well. As a side-effect, they now refer to both functionalities together as simply S3 Replication.

There are a couple of clear use cases for using same-region replication instead of cross-region replication. Possibly the most important one for anyone living in a country without multiple regions is data sovereignty. Sometimes you may store data that for legal reasons isn’t allowed to cross borders. This is a quite common restriction here in Australia, for example, and not having to build my own solution for replication is definitely a good thing.

And no, replicating data to a different account isn’t quite as disaster-proof⁴ as replicating it to a different region. It will help protect against accidental (or wilful) deletion, however, which is far more likely to happen. Some other use cases are collating logs from different accounts, and replicating production assets to your development environments.

Ambassador Corner

• Cloud in your Datacenter - AWS Outposts
• Choosing the size and plugging the hole of your leaky bucket: Picking the right (burstable) instance
• Walkthrough - ECS Local: Bringing ECS to your local environment 
• Pulling Puppet’s Strings with AWS OpsWorks CM
• Relationalize Unstructured Data In AWS Athena with GrokSerDe | The Data Guy