So I’ve been self-hosting for a few years now…got a decent setup…
But I only really got into docker-compose in the last few months… I mean, I’ve used it through the “Apps” in TureNAS, but never directly…
I ran a mastodon host on a VM (giant pain in the ass) and a few other things that weren’t available as apps, but never mainstream…
Once I got into it, I managed to get everything moved over to truenas as docker containers…
Last week it saved my ass…the AC in the house failed…and I was able to shut down everything save the Truenas (with a fan pointed at the front of the server) and keep everything up and running throughout…
So it’s been helpful to know, and an awesome learning experience. (I’m happiest when I’m learning new things)
My question is this…
How do you back them up? I mean, I have snapshot backups of my docker-volume ZFS dataset, and replicate that to a remote host…but I’m not even sure I’d know how to recover it if I had to…is it just a file copy and restart?
You must log in or # to comment.
I use a volume bind mount for the container’s config directory, then back that up. And the docker-compose file.
Yeah, I bind mount the config inside a config directory just below the compose file, and the state (databases, files, etc) to a seperate storage location that makes sense. The compose and config gets added to git and pushed to a forge, while the storage directories are backed up like normal (in my case, zfs snapshots pushed to another computer and then restic backs up those cold files to b2. The volumes are ephemeral.
I have another container running the proxmox backup client that does a filesystem dump daily… Not bad, but I think postgres may not recover from that…
Right now 2 weeks of daily replicated snapshots are probably as good as I’m going to get…
Works fine… Most of the time. As long as the file gets saved at an instant in time (e.g. zfs snapshot), postgres sees it like a power failure and restores. You want to have several recent backups if you do this because it will occasionally fail.
Saw an article that says proxmox backup client can be set up to be ZFS aware…will have to do more reading… (for backups)
Replication is always a point in time, and I keep two weeks worth of dailies.
Test it, find out.
Well that worked… Amaze! I copied the lemmy directory to my user directory, made the necessary changes in docker-compose, and boom, it came up. If I were to script the changes using sed, I could even have a full DR failover.
Not a bad idea…I’ll have to make some network changes in the docker compose and build a new macvlan for it…but it should be easily addressable by IP address. I did to a test startup of my mastodon DB and that did seem to work…
And I should probably block outbound traffic just to be sure it’s not trying to update the fediverse…
Cool, now I know what I’m playing with tonight. :)
I have a single dataset running off of an ssd pool that stores config and application data. Large media gets stored on a separate pool on spinning disks. The app dataset gets snapshotted and backed up, but I am yet to create a separate offsite backup.
I am also in the process of switching away from TrueNas “Apps” to actual Docker containers running from compose files in Dockge. These compose files all get version controlled in Git as well. Apps have been really difficult to configure initially as many of the fields have odd requirements you can’t see until it errors on startup. I have found it easier to follow the documentation directly from the application I am installing so compose files have been much more straightforward.
I’m just running docker-compose on the truenas itself… bonus points is docker ps shows me the truenas apps as well as the ones I’ve created… Downside is, no GUI indication of the docker-compose stuff.
I have a similar setup. 2x 250G hardware mirrored for boot, 4x 2TB SSD’s ZFS mirrored for apps, 12x 22TB Raid-Z2 for mostly static data (plex, etc) All in a single Dell PowerEdge R730 with 256G of ram and an RTX3050 in it, so I have plenty of room for apps and such.
By today’s standards it’s an expensive setup, but I only paid $189/each for the 22T drives back when I built it, so it wasn’t so bad then.
Damn that is a nice setup lol. I only built mine a couple years ago and only managed to get 5x 16TB drives at $169 each. I have 4 in the system and one cold spare.
I have tended to avoid cli on TrueNas mostly out of fear that I’ll mess something up and I’ll be too lazy to fix it. That was my main motivation behind switching to Dockge for containers, it even shows apps running through other container management systems, although you can’t manage them through there.
Truenas is hardened enough that it’s pretty hard to screw it up… you can’t install packages or anything like that… My goal was to have one server that’s “Production” so that I can nuke the lab from time to time as needed. (I have a proxmox server, a VMWare server, and one that bounces back and forth between nutanix and whatever else I want to play with at the time…
Also have another R730XD that I use for proxmox backup server, 6x 12TB drives (that were my nas drives before I got the 22TB upgrades) PBS is an amazing application, even when I’m not using the proxmox VM’s, the filesystem backups work like a champ, and it replicates backups to an NFS datastore in my other house (running on Truenas on a T630 under my desk)
Yeah I am pretty comfortable with CLI on linux, but I just never bothered with TrueNas. Also I am lazy and once something works I just leave it there even when I could improve it lol.
I have been meaning to venture into Proxmox as well but I don’t have the time or spare parts to get a new machine running atm. Maybe someday I’ll get around to it though as it seems like a nice playground without breaking things.
It’s just Debian, no surprises.
