• 0 Posts
  • 53 Comments
Joined 3 years ago
cake
Cake day: July 29th, 2023

help-circle

  • About time this hit enterprise.

    Been saying the downfall of OpenAI and Anthropic will be local models that can accomplish the same thing without needing a dstacenter worth of GPUs to run.

    Plus you have no total token limit on self hosted models. You can let a slop agent run for weeks without ballooning costs or getting rate limited.

    And no model censoring, so you can go build that nuclear bomb you always wanted lol.

    Both AI giants are already losing a ton of money because their pricing isn’t viable for the amount of training they’re doing trying to be on top. If they readjust, they’re gonna suffocate against much cheaper solutions.


  • vLLM to host and openwebui for a frontend that I really only use to demo models to friends.

    harness

    Opencode running with full access inside a rootless podman container with only the directories I want mounted (usually a single project, sometimes read only).

    So far though, I’ve only used it to document and report on project repos because Qwen is not that good at actually coding (although it can give pretty decent conf file examples, but not always optimal).

    Also useful for parsing pita data dump type files that I’m too lazy to construct multiline awk commands for.



  • Uh no, most if not all of the paid indexers are not worth it compared to just waiting for an open invite from a quality one.

    Not even the gold tier quality ones allow paid entry AFAIK. They only allow people who are long time seeders and uploaders with reputation.

    The one thing you might have to pay for is a VPN in case you want to torrent on public indexers, which is often worth it because not everything will end up on private indexers, and you usually have to wait a couple of months before you can join a solid private indexer.


  • I can trip the measly 15 amp breaker by drawing 1.5kW on two servers + turning on an amplifier that is somehow on the same circuit despite being in a different room quite far from where my servers are lol.

    On average though I only draw like 350W. The huge spike only happens if I run a miner/llm on both of my servers or the during the startup spike.

    The 1kW chunk comes from some RTX Quadro 6000s which each eat 250W each when maxxed out.

    EDIT: Want to mention that most machines report their power draw (from mono) without IPMI, so you can use your favorite stats collector use it to form a rough estimate of kWh total in case you don’t have a proper meter that records usage.





  • I don’t want to run a separate database or anything complex, so things like Immich or Photoprism are not an option.

    Why not just use podman and let it handle itself? Unless you’re planning on hosting like maybe several million photos, I doubt you would ever need to manually set up a DB.

    Just download the compose file, add your photo directory paths to it, and let it start and never think about it again until you feel like updating for some new feature months later.


  • If its internal network traffic only, you can just use stalwart and add the DNS records to your network so that stuff can use SMTP.

    Otherwise, (also with stalwart or your preferred software), you will want to use an SMTP relay so that outgoing messages will be delivered without the hassle of getting your domain reputational enough for sent messages not getting junked by the big email providers.

    I’m extra lazy and haven’t set this up, so I just have a public mail server that only receives messages because my outgoing SMTP port is blocked by my ISP.


  • Saw the title and I already knew IPv6 was still gonna be “not enough adoption” lol.

    The overly aggressive lockdown from NAT is not entirely misplaced though. Windows is already notorious for default hosting an atrocious level of protocols that are anywhere between “I can get your NetNTLMv2 hash for free” to “gg ez RCE as SYSTEM user”.

    Linux and Unix were also not magically protected either, like every FTP server implementation has had several RCE vulns and it was so bad that Linus himself pointed out how easy it was to abuse buffer overflow vulns even after the addition of NX mem protection with return to libc, several years before ASLR finally landed in Linux and Windows.

    Seriously, the early 2000s was the wild west of vulns because everything was still written in C++ so you could just waltz into any binary, find some stack/heap overflow, and you’re on your way to an RCE.

    NAT (and firewall) most definitely did prevent millions of users from getting rekt by a metric ton of malware that operated on the ability to connect to those services.






  • mlg@lemmy.worldtoSelfhosted@lemmy.world•Recommended mini pc for a homelab?
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    5 months ago

    If you want something that fits the SFF cube shape so you can throw it next to a TV or desk for video output, you can probably go for the SYS-A22GA-NBRT.

    Easy to upgrade CPU & RAM later down the line if you start doing more stuff with it, plus space for dedicated GPU if you ever want to do heavy media server stuff.

    Would avoid pi due to the underperformance for the price. Plus best bang for buck usable storage will always be HDDs. SD cards are nice but you have to disable journaling to keep the writes low as to not wear down usable blocks.