oh freaking awesome, this looks amazing! Thank you so much for this!

For most of you suggesting hosting a repository - yes but,

Host forgejo. Just host the git mirror. It comes with a package repo out of the box. Then you have the source code and the docker images

Generally there are not LLMs that do this, but you start building up a workflow. You speak, one service reads in the audio and translates it to text. Then you feed that into an LLM, it responds in text, and you have another service translate that into audio.

Home Assistant is the easiest way to get them all put together.

https://www.home-assistant.io/integrations/assist_pipeline

Edit agree with others below. Use the apps that are made for it.

• Whisper for STT
• Any hosted LLM can work, text-generation-webui or tabbyapi
• I use xttsv2 for TTS

This post is written as if there’s only one “community”. Why does there need to be a primary? I’m here and I’m happy. If I have questions I search online or ask here, same as any other community

Seconded with Matrix. All I’m wanting for it is for someone to make a Discord/Revolt UI frontend for Matrix 2.0 and it’ll be a drop in replacement

Oh yeah, critical component. And vram, in fact I would only consider LLMs on a 3000+ card right now, they require quite a bit of vram

NVidia is great in a server, drivers are a pain but do-able. I have a 3000 series that I use regularly and pass into my kubernetes cluster. NVidia on a gaming rig linux is fine, but there is more overhead with the drivers.

AMD is great in gaming servers, but doesn’t have CUDA, so it’s not as useful in a server environment in my experience - if you’re thinking of doing CUDA workloads like hosting LLMs.

1060 will be a noticeable step in Jellyfin

DNS is of course the preferred approach

I wouldn’t worry about mounting your nfs shares directly to those host unless you need to. Compose has an operator similar to k8s that lets docker itself manage the shares, which is insanely useful if you lose your host. Then you don’t have to have piles of scripts to mount them.

https://stackoverflow.com/questions/45282608/how-to-directly-mount-nfs-share-volume-in-container-using-docker-compose-v3

version: "3.2"

services:
  rsyslog:
    image: jumanjiman/rsyslog
    ports:
      - "514:514"
      - "514:514/udp"
    volumes:
      - type: volume
        source: example
        target: /nfs
        volume:
          nocopy: true
volumes:
  example:
    driver_opts:
      type: "nfs"
      o: "addr=10.40.0.199,nolock,soft,rw"
      device: ":/docker/example"

LLMs use a ton of VRAM, the more VRAM you have the better.

If you just need an API, then TabbyAPI is pretty great.

If you need a full UI, then Oogabooga’s TextGenration WebUI is a good place to start

Personally, if you’re considering it already, kubernetes might be something to look into. It’s a lot. Like a lot to learn. But I can honestly say I could do it for a job now with how much I’ve learned. Then it’s less about how to set up machines and more about just reapplying your infrastructure.

Is this really feature flags as a service? FFAAS?

Tried it and it works pretty well! I’ll have to keep playing with it, but so far so good!

How well do you find it works? I’m not afraid of the fee, but I don’t want to spend time setting it up and paying the fee to only find out that it won’t do most things

The number one thing that most of these don’t do well for me is the connection with banks. You mentioned that there is bank syncing, how well does that work? Can I say, just click my bank and do an oauth connection, and it will store it? I really loved Mint, and essentially want it to be done the same way

I like the look of exui, but is there a way to run it without torch or needing a GPU? I have tabby running on a separate computer, like SillyTavern I just want to connect to the API, not host it locally

Another great point, if I lose my Linux isos, sucks but I’ll redownload. If I lose my family videos, sucks but I’ll log into my backups and resync. If I lose my credentials I’m fucked. Plain fucked. I can’t decrypt my backups, can’t log into services, it’s done.

I don’t, specifically because I don’t trust myself to host that. I know what people will say here, but I trust 1pass way more than I could do it myself.

1pass uses your password plus a secret key to generate your full “password”, meaning you need both to access your vault. The password you memorize, the key you keep safe somewhere (inside the vault is even good, since you probably have it open on another device should you need it). They publish their docs, and show how they encrypt your vaults. To them, your vaults are truly just random bytes they store in blob storage. They don’t store your key, they don’t store your password, they will not help you out if you lock yourself out. That’s the level of security I want for a password vault. If they ever get breached, which hey, it can happen, the most someone will get is a random blob of data, which then I’d go and probably generate a new password and reencrypt everything again anyway.

Vs me hosting myself, I’m sure the code is good - but I don’t trust myself to host that data. There’s too many points of failure. I could set up encryption wrong, I could expose a bad port, if someone gained access to my network I don’t trust that they wouldn’t find some way to access my vaults. It’s just too likely I have a bad config somewhere that would open everything up. Plus then it’s on me to upgrade immediately if there’s a zero day, something I’m more likely to miss.

I know, on the selfhosted community this is heresy, but this is the one thing I don’t self host, I leave it to true security researchers.

Hell I don’t even buy “New” on Amazon anymore, it’s all way too shady. If I want a 3d printed novelty I’ll go there, but something I need to know won’t catch fire? No way.

Neat! I’ll check it out!