Comment by topher200
4 days ago
I have a preempt-able workload for which I could use Spot instances or Savings Plans.
Does anyone have experience running Spot in 2025? If you were to start over, would you keep using Spot?
- I observe with pricing that Spot is cheaper
- I am running on three different architectures, which should limit Spot unavailability
- I've been running about 50 Spot EC2 instances for a month without issue. I'm debating turning it on for many more instances
In terms of cost, from cheapest to most expensive:
1. Spot with autoscaling to adjust to demand and a savings plan that covers the ~75th percentile scale
2. On-demand with RIs (RIs will definitely die some day)
3. On-demand with savings-plans (More flexible but more expensive than RIs)
3. Spot
4. On-demand
I definitely recommend spot instances. If you're greenfielding a new service and you're not tied to AWS, some other providers have hilariously cheap spot markets - see http://spot.rackspace.com/. If you're using AWS, definitely auto-scaling spot with savings plans are the way to go. If you're using Kubernetes, the AWS Karpenter project (https://karpenter.sh/) has mechanisms for determining the cheapest spot price among a set of requirements.
Overall tho, in my experience, ec2 is always pretty far down the list of AWS costs. S3, RDS, Redshift, etc wind up being a bigger bill in almost all past-early-stage startups.
To "me, too" this, it's not like that AWS spot instance just go "poof," they do actually warn you (my recollection is 60s in advance of the TerminateInstance call), and so a resiliency plane on top of the workloads (such as the cited Kubernetes) can make that a decided "non-event". Shout out to the reverse uptime crew, a subset of Chaos Engineering
The spot termination warnings arrive 120 seconds before termination https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-ins...