Link + path = Linkopath
I ran into the same problem and ended up switching to an S3 with Vultr. It’s been a while since I did it but here are the links that I used to figure it out. I’m deployed using Lemmy-Easy-Deploy.
I used a combination of:
https://lemmy.world/post/538280
https://github.com/ubergeek77/Lemmy-Easy-Deploy/blob/main/ADVANCED_CONFIGURATION.md
https://git.asonix.dog/asonix/pict-rs/#user-content-filesystem-to-object-storage-migration
Good luck!
Maybe look into self hosted llm. I’ve used two recently to help analyze a large volume of books, by ingesting them into the data set, then chat with the bot for specifics. It worked pretty well but there are some limitations, such as token length and general hallucinations. But they both use citations of the data they used, so it helps to check their work.
PrivateGPT - https://github.com/imartinez/privateGPT
Llamaindex - https://github.com/run-llama/llama_index
Both have simple selfhosted webui or actual applications. So, in theory you should be able to ingest data and then see if it then matches any submissions you submit later. But I have not really tried it for this though, so it might not work.
It’s been a while since I set mine up but I know that most public indexes using a proxy will get blocked by cloudflare at one time or another. Away around that is to use flaresolverr. Not totally sure if that is your problem but it might be a good place to look.
www.gifable.club
Then click on get started and register an account to see what it looks like.
This might help, haven’t tried it but their proof of concept website looks promising.
If you are wanting to self-host outside of your home-lab and use a VPS, it is pretty simple. Ubergeek77 has compiled a docker image to easily install it all in like 5 steps. Take a look, https://github.com/ubergeek77/Lemmy-Easy-Deploy
#Lemmy-Easy-Deploy @ubergeek77@lemmy.ubergeek77.chat
This is the one I use to host on a vps. No clue on it’s deployability on a pi.
https://github.com/ubergeek77/Lemmy-Easy-Deploy
It’s auto installs and updates. Just need to forward the DNS for your instance to whatever domain name you like. It’s pretty straight forward from the documentation.