I am fairly new in the selfhosting business. I host a few services like Jellyfin or AdGuard, but most of my data just sits on plain SMB shares on my NAS.
I am interested in setting up things like Paperless-ngx, Nextcloud maybe Immich as well. One thing I am struggling with is how to organize all this so I don’t have the data scattered around multiple places or in the worst-case even duplicated. I like the simplicity of network shares and the fact that I am not relying on a third-party application to keep being maintained. I am currently not sure I am willing to give that up. However, I am also intrigued by the features these services provide. I know Nextcloud has the option to mount external storage, but I don’t know which downsides come with that. It gets more complicated with Paperless. As far as I know you typically have a consume directory where you throw in your data and Paperless stores it using its own system in the media directory. This means I either throw in everything or I suddenly have two locations where my documents are stored. If I would mount that media directory in things like Nextcloud I probably wouldn’t be able to find anything because of the different structure and naming scheme.
The idea that I had in mind is that I have a single directory structure of my data that can be used on its own and all these tools are just different frontends and provide different views and information of the same data. Maybe this approach is just something from the past and I should move on.
I should add that I am not planning to expose any of these services to the public. All of this is only accessed from inside my house or using a Wireguard VPN.
- How do you guys handle all this?
- How do you avoid data duplication?
- How do you avoid multiple potential file locations? Is a document in my Nextcloud, in my Paperless or just on the network share?
- Do you prepare in some ways in case an application stops being maintained?
- How has this reliance on multiple services impacted other things e.g. your backups?
I run everything in docker. My docker volumes are all subdirectories under /media, which is the root of my NAS. If an app only needs something specific, docker only mounts, eg /media/movies. If the app needs many subdirectories (like backups), it gets the full /media mount.
All the docker data/configs are under /docker/volumes (eg /docker/volumes/jellyfin), and every container is configured with compose in /docker/docker-compose.yml.
The biggest hurdle for me was creating a sane docker-compose file that defines where everything lives. Once that is standardized, adding and maintaining your apps becomes easy.
If you’re very new to this, even a dumb LLM like ChatGPT can be very helpful to explain what’s inconsistent, redundant, or incorrect with your config. If you are lost I would strongly suggest this as a way to get oriented. But DO NOT copy and paste anything into the LLM containing passwords, tokens, keys, etc. if you accident do, change those keys before finishing up your project.
With docker there’s no duplication- you have one copy of everything you need and you just point each container at the same data. This makes it trivial to keep file consistent across your apps and doesn’t waste space.
I already use Docker and docker-compose for all the services I host. They are also all configured to use bind mounts instead of volume mounts, so that I am in control where everything is located. I mainly use the official documentation although I sometimes like to check https://github.com/Haxxnet/Compose-Examples/tree/main. I didn’t came across any major problems with that workflow so far, but that is also besides the point of my original question.
Jellyfin is a good example of the workflow I had in mind. I have a media share on my NAS and just point Jellyfin to that share in the docker-compose. But that is not how things like Paperless work, at least not from my (still limited) understanding of it. I think there are other workflows, but the main one of Paperless is a consume directory and a data/media directory. Everything in the consume directory gets ingested by Paperless and is moved with (I think) a different hierarchical structure and filename to the media directory. So if I point the consume directory to my current documents share on the NAS everything is moved away. I am sure there are ways to specify how Paperless organizes the media directory, but it most certainly will not be exactly the same as before. So making the data/media directory function as my new NAS “share” will also result in a change. In addition to that, my documents share on the NAS also contains data that I probably don’t need to track with Paperless e.g. some spreadsheets. I am not sure how Paperless handles documents in the media directory that were not originally processed using the consume folder or if that is even a good idea in the first place.