Networking is still hard in 2026
Hey ๐๐ฝ
Important off topic things first: Sandro got married! ๐
We were in Munich for it and it was a fantastic day! โ๏ธ
Highly recommend a wedding over sprint planning or fighting with AWS shenanigans!
Now to something less fun: A story I keep hearing, and it always starts the same way.
A cluster gets noticeably more expensive over two days.
1. Nothing was deployed and traffic is flat.
2. All dashboards stay green: pods Running, nodes Ready, no alerts and no errors.
3. But the bill still goes up... significantly ๐
The only thing that changed is where the nodes are running. That's enough.
In this issue: how a Spot shortage quietly rearranges a cluster across three availability zones, and why that makes internal traffic a costly liability!
|
|
Search without the cluster
Typesense โ the open source alternative to Algolia
|
Typo-tolerant search from a single binary. No shard count to pick, no JVM heap to tune, no cluster to keep alive โ and natural language search is built in.
Smaller learning curve than Elasticsearch ย โขย Django & Laravel Scout integrations
Check it out on GitHub โSponsored by Typesense, a tool we run ourselves.
๐ This Week's Deep Dive
Three things are true about a lot of EKS clusters using Karpenter (great choice!) as their scheduler: Then Spot capacity for that family runs out in the region. Karpenter does exactly what it was told: it falls back to On-Demand, and it places new nodes wherever it can still get instances. Within a day, a fleet that had effectively lived in one availability zone is spread across three. Nothing about that is wrong. Spreading across zones is what you are supposed to want. Except the services still talk to each other constantly, and now roughly two thirds of every internal call crosses a zone boundary at $0.01 per GB in each direction. The traffic pattern never changed. Only the placement did. And there is no Kubernetes signal for placement: no restarts, no failing probes, zero errors. In Cost Explorer it hides inside EC2-Other. The full story is on the blog: how a NodePool quietly became a placement policy, why one instance family is the thing that set it up, and the four changes that stop it from happening again. |
10 Most Common AWS Savings Opportunities
Not a sponsor and not paid, we just know the team and rate the work. No affiliate link either. They went through more than 100 AWS accounts and wrote up what they actually found, with how often each thing shows up and the median yearly waste attached to it.
Oversized EC2 instances in 46% of accounts. Redundant NAT gateways in another 46%. Unused EKS clusters at a median $876 a year. Every item comes with the CLI commands to check your own account, which is what makes it worth the 18 minutes.
Read the breakdown โ
๐ฐ This Week in AWS๐ฉบEC2 application status checks
๐The AWS sign-in page is changing
๐ฐManaged dashboards in Billing and Cost Management
|
Go read your own NodePool with this in mind!
If it names exactly one instance family, you already know what to do.
And if you have had a cost surprise that no dashboard warned you about, hit reply and tell me.
I collect these!
See you next week! ๐
Tobi & Sandro