Seconding this. Especially if you’re still learning and making mistakes, it’s so nice to just be able to destroy a VM/CT and start over, rather then potentially breaking other things or the OS itself.

Ah, that’s good to know! I’ll give those other options a shot. Thank you so much for taking the time to help me with that! I’m very new to the whole LLM things, and sorta figuring it out as I go

It has a Intel Xeon E3-1225 V2, 20gb of ram, and a Strix GTX 970 with 4gb of VRAM. I’ve actually tried Mistral 7b and Decapoda Llama 7b, running them in Python with Huggingface’s Transformers library (from local models)

Hm… Alright, I’ll have to take another look at it. I kinda gave up, figuring my old server just didn’t have the specs for it

Show as in I waited a few minutes and finally killed it when it didn’t seem like it was going anywhere. And this was with the 7b model…

I tried doing that on my home server, but running it on the CPU is super slow, and the model won’t fit on the GPU. Not sure what I’m doing wrong

They’re being useless, but what I do is use Proxmox and just install my stuff each in their own LXC

Nice! Glad to hear that works. I’ll have to give it another go. I had spent the whole day trying to get Mullvad (without WireGuard) working, but it kept failing to create the tun device, so by the time I got it working with WireGuard I didn’t really feel like trying to figure out the VLAN thing too lol.

Ugh, I wish I could be more help on that, but I couldn’t get Mullvad to work that way either. I think what needs to be done is to use pfsense or something to create a virtual LAN, set the container running Mullvad to be the gateway on that network, then give each container a virtual network bridge connected to that virtual network. What I ended up doing was just installing Mullvad (through WireGuard) on the same container as qBitTorrent and telling qBitTorrent to use the virtual network device that Mullvad creates.
Fortunately, that’s the only thing that really needs to run through it for me (I think your Real Debrid will need to as well). AFAIK, the *arr stuff doesn’t need to be hidden.
As to getting things to talk to each other in containers, where were you having trouble? You should just be able to give all the *arr stuff the addresses where you reach the other ones. That may just be their IP address, or I run PiHole so I can have a local DNS and give them all their own hostnames.

Edit: I’m doing all this in Debian LXCs

I’ve been working on the same thing over the past month, with some minor differences. I skipped portainer and am just running LXCs on Proxmox, and built it from the beginning as a *arr/Plex box, so it has 4x4TB internal drives in ZFS RAID6, with the OS on an SSD. I still need to try out the TrueNAS thing, but I’m running a Minecraft server on it, and I just spent the better part of a day figuring out how to run Mullvad on it and force all my torrent traffic to use it.

I use Proxmox, so I just use the PVE web interface

Ah, yeah, I’m running Proxmox. And I’m definitely seeing I have a lot of research to do before I open up to the internet!

I’d love to hear more about your Authelia setup. I’m using Authentik, but planning to do the same thing. I haven’t opened my server up to the Internet yet (just built it on Friday), but what I’d like to do is have a webserver that supplies a login page, and you can’t access anything else until you’ve logged in

Well now I’m concerned. I have plans to do something similar, but I’m only spending a few hundred on my setup. By the time it’s done it’ll be:

• Supermicro X9SAE Mobo
• Xeon E3-1225 v3
• 32gb ram
• 2x 240gb ssds in RAID1 for the OS
• 4x 4TB Ironwolf HDDs for media storage

Is that not going to be enough for pretty much the same use case?