My 3 favourite re:invent serverless announcements
09 Dec 2021Once a year AWS holds their re:invent conference where they announce new features and services. In this article I will go over my favourite serverless announcements.
AWS Lambda Event Filters for DynamoDB Streams
Gone are the days when we have 10 Lambda functions attached to a stream, with each one having filtering logic in the beginning.
Anyone played with or put into prod the new filtered #DynamoDB stream in Lambda Functions? If you have not created one, it’s really nice! #serverless #NoSQL
— Kirk Kirkconnell (@NoSQLKnowHow) December 1, 2021
Navigate to your Lambda Function -> Configuration -> Triggers and add new. Then go to additional settings.
We may now specify filters that Lambda applies BEFORE calling our function.
Confirmed with @ajaynairthinks that the Lambda Service eats the polling cost, so you only get charged for what makes it into your function.
— Jeremy Daly (@jeremy_daly) December 1, 2021
In applications where there are many more Lambda functions attached to the same stream, this can be a major cost savings opportunity. Imagine having 100 functions, and only one actually does something for each change. The cheapest code is the one that doesn’t run.
If you’re using the Serverless Framework, there’s already support for this filtering.
New Storage Tiers for S3 and DynamoDB
I’ve written about archiving data to save on storage cost before: Archive your AWS data to reduce storage cost
That article now needs an update, as there are more options, and more automations. Below you can find three major changes for long-lived data.
S3 - Glacier
There’s the new S3 Glacier Instant Retrieval with “the same throughput and milliseconds access as the S3 Standard”. If you stick with the previous S3 Glacier tier, you pay 10% less for storage. The previous tier is now called S3 Glacier Flexible Retrieval and offers “retrieval in minutes or free bulk retrievals in 5-12 hours”.
This is great if you have data like medical images or tax receipts. You rarely need to access them, but if so, you need them immediately. This is now possible with Glacier which has a far lower cost than the standard S3 tiers.
S3 Glacier Deep Archive did not change with this announcement and continues to offer “data retrieval from 12-48 hours” at the lowest storage prices of S3.
S3 - Intelligent Tiering
If you don’t know the access patterns for your S3 data, then the Intelligent Tiering comes in handy. Intelligent Tiering picks the best storage tier for your objects, and now also includes the archived tier.
This means that you can tell S3 to monitor access patterns. If it finds a better storage tier for an object, S3 will move the object. This can lead to cost reduction for you, but keep in mind that you get charged for monitoring access patterns.
DynamoDB
I heard you like not pay less. DynamoDB joins the party with the Infrequent Access table class. This is available for On-Demand capacity only.
Well, looky there. AWS just cut costs by 60% for infrequently accessed data on @dynamodb.
— Danielle Royston (@TelcoDR) December 1, 2021
That's great news for @totogi who's built their charging system on @dynamodb! WOO HOO!#singletabledesign #reinventhttps://t.co/zb7EAcIKrJ
By switching from the default class to the infrequent access class, you get the following price changes:
- Write Capacity Units: +25% ($1.25 per million to 1.56$ per million)
- Read Capacity Units: +25% ($0.25 per million to $0.31 per million)
- GB per month: -60% ($0.25 per GB-month to $0.1 per GB-month)
When changing table storage class, you are limited to two changes per 30-day period:
Note that no more than two table class updates on your table are allowed in a 30-day trailing period.
Kinesis On-Demand
“Announcing Amazon Kinesis Data Streams On-Demand”
Kinesis has a new On-Demand mode where you’re charged by the hour and for throughput. Gone are the days of having to manage shards.
In on-demand mode, pricing is based on the volume of data ingested and retrieved along with a per-hour charge for each data stream in your account […]
But is it really serverless if you’re charged by the hour, not only for usage?
Does it scale to zero?
— Jeremy Daly (@jeremy_daly) November 30, 2021
This announcement might not follow everyone’s definition of serverless. However, I’m glad that I don’t have to configure and manage stream shards.
You do not need to specify […] throughput […]. Kinesis Data Streams instantly accommodates your workloads as they ramp up or down.
While an idle cost of $30 per month is not great for projects that run at nearly 0$ per month, I still welcome this new option. So far the operational effort to manage throughput has been a major factor in deciding against Kinesis and for SQS. Now that blocker is gone.
Conclusion
Not many new features this year, but continuous improvement to existing services. This gives me the feeling that engineers can make great improvements to existing services (that also help with their promotions), without having to launch a new feature.
To lower spending and better services!
Enjoyed this article? I publish a new article every month. Connect with me on Twitter and sign up for new articles to your inbox!