Comment by 0xbadcafebee
3 years ago
Which is great until there's a security vuln in an end-of-life piece of core software (the distro, the kernel, lxc, etc) and you need to upgrade the whole thing, and then it's a 4+ week slog of building a new server, testing the new software, fixing bugs, moving the apps, finding out you missed some stuff and moving that stuff, shutting down the old one. Better to occasionally upgrade/reinstall the whole thing with a script and get used to not making one-off changes on servers.
If I were to buy one big server, it would be as a hypervisor. Run Xen or something and that way I can spin up and down VMs as I choose, LVM+XFS for snapshots, logical disk management, RAID, etc. But at that point you're just becoming a personal cloud provider; might as well buy smaller VMs from the cloud with a savings plan, never have to deal with hardware, make complex changes with a single API call. Resizing an instance is one (maybe two?) API call. Or snapshot, create new instance, delete old instance: 3 API calls. Frickin' magic.
"the EC2 Instance Savings Plans offer up to 72% savings compared to On-Demand pricing on your Amazon EC2 Instances" - https://aws.amazon.com/savingsplans/
Huh? Using lxd would be identical to what you suggest (VMs on Xen) from a security upgrade and management perspective. Architecturally and operationally they're basically the equivalent, except that VMs need memory slicing up but lxd containers don't. There are security isolation differences but you're not talking about that here?
I would want the memory slicing + isolation, plus a hypervisor like Xen doesn't need an entire host OS so there's less complexity, vulns, overhead, etc, and I'm not aware if LXD does the kind of isolation that ex. allows for IKE IPSec tunnels? Non-hypervisors don't allow for it iirc. Would rather use Docker for containers because the whole container ecosystem is built around it.
> I would want the memory slicing + isolation...
Fine, but then that's your reason. "until there's a security vuln in an end-of-life piece of core software...and then it's a 4+ week slog of building a new server" isn't a difference in the context of comparing Xen VMs and lxd containers. As an aside, lxd does support cgroup memory slicing. It has the advantage that it's not mandatory like it is in VMs, but you can do it if you want it.
> Would rather use Docker for containers because the whole container ecosystem is built around it.
This makes no sense. You're hearing the word "container" and inferring an equivalence that does not exist. The "whole container ecosystem" is something that exists for Docker-style containers, and is entirely irrelevant for lxd containers.
lxd containers are equivalent to full systems, and exist in the "Use one big server" ecosystem. If you're familiar with running a full system into a VM, then you're familiar with the inside of a lxd container. They're the same. In userspace, there's no significant difference.