I have a server with a bunch of services just as Docker containers. I see that Proxmox is popular among the self-hosting community. I was wondering why?
I understand that running things in a VM provides better security than running them in a container. But is the difference so important given the relatively low risk that an exploit happens inside a container that leads to doing damage to the host machine?
There’s also obviously the additional overhead of using Proxmox. It wouldn’t be an issue for me as I should have enough resources to, say replace all my Docker containers with VMs. I’m more wondering if the security difference is so massive, or if there is another reason I’m missing why people use Proxmox.
Or am I misunderstanding how people use Proxmox? I was assuming people would use it like how you use Docker, i.e. different services get their own VM/container. If you have a different kind of setup I’d be interested in hearing it.
Edit: I would appreciate if people stop being pedantic and actually read the post. Obviously I am aware that you can run containers in VMs, or containers on bare metal alongside VMs. That’s not what the question is and you know it.
I found proxmox and docker to be fairly incompatible, and went through many iterations of different things to make it work well. Docker in VMs, Docker in LXC, Docker on the host (which felt redundant as hell). Proxmox is an amazing hypervisor, but then I realized I didn’t really need a hypervisor since I was mostly running containers.
My recommendations:
No need for VMs Just run debian and run containers on it
Some VMs, Mostly containers, 1 host Run proxmox, and create a VM in proxmox for your contianer workloads
Some VMs, Mostly containers, >1 host, easy mode Same as above, but make one host debian and the other one proxmox
Some VMs, Mostly Containers, >1 host, hard mode but worth it after 2 years Use kubernetes, I use k3s. Some nodes are just debian with k3s on them, others are running in VMs on proxmox using the extra compute available. This has a massive learning curve though, it took me well into a year to finally having it at a state I like it - but I’ll never go back.
Same here. I used proxmox for 8 years and have recently dumped it in favour of a couple of incus machines running OCI and LXC containers.
Much lighter, much faster, and to be honest, more straightforward when it comes to storage abstraction, which I think proxmox does in a very… convoluted way.
What did you find to be incompatible between proxmox and docker? I get that it’s essentially an extra layer of complexity if all you’re doing is running docker containers, but I don’t see how that makes them incompatible.
Docker in LXC can be a pain, especially when using backups as the Overlay2 filesystems don’t really jive with the way Proxmox does backups. And forget about running Docker in an unprivileged LXC.
Running in a VM is perfectly fine though; not sure what issues anyone has there. I ran on big beefy servers with 24 cores and tons of RAM though.
It was nice to be able to move my services between machines using a live migration while doing updates though; but again you have to be set up for that. My entire network was managed with twin OPNSense routers as VMs in Proxmox; they handled their own failover and so I could just shut down one at a time to run updates, even to Proxmox itself, and when it came back up then I could work on the other one. But, I wanted to learn all that and have zero downtime so the wife wouldn’t get mad every time I botched something (which, especially in the beginning, was often)
If you don’t have the money or time and just have one server box with a normal amount of RAM and disk; Proxmox is probably overkill unless you want to experiment with VMs or Linux containers. It’s an awesome product and I will sing its praises all day, but if you just want some docker containers you can make a far simpler setup; although I will say that the “overhead” is way less than you might think. It’s just more complicated (not hard, there’s just more going on than vanilla Debian or something)
That thing about docker being so badly behaved in unprivileged containers seems to be a proxmox problem, not an LXC problem, as I’ve discovered running LXC in a non-proxmox environment.
I run a majority of my docker containers within an unprivileged LXC, even gpu pass through and it works great.
On what platform?
That’s unfortunate. I know they do change some things for both security hardening as well as for convinience of the platform, it’s a double-edged sword apparently.