Nothing wrong with used servers, that’s the only thing I’ve ever run. Ebay has provided a ton of equipment to me.

I’ve never used TrueNAS, but my experience with ZFS is that it could care less what order the drives are detected by the operating system. You could always shut down the machine, swap two drives around, boot back up, and see if the pool comes back online. If it fails, shut it back down and put the drives in their original locations.

If you are moving your data to new (larger) drives, before anything else you should take the opportunity to play with the new drives and find the ZFS settings that work well. I think recordsize is autodetected these days, but maybe for your use things like dedup, atime, and relatime can be turned off, and do you need xattr? If you’re using 4096 block sizes did you partition the drives starting at sector 2048? Did you turn off compression if you don’t need it? Also consider your hardware, like if you have multiple connection ports, can you get a speed increase by spreading out the drives so you don’t saturate any particular channel?

Newer hardware by itself can make a huge difference too. My last upgrade took me from PCIe x4 to x16 slots, allowing me to upgrade to SAS3 cards, and overall went from around 70MB/s to 460MB/s transfer speeds with enough hardware to manage up to 40 drives. Turns out the new configuration also uses much less power, so a big win all around.

Good point, POE might be the only choice then. Although these cameras do provide x264 and x265 compression so that would save quite a lot.

You got me thinking, so I did a search and ran across this page: https://www.hanssonit.se/nextcloud-vm/ I’m not sure how old these releases are, but at the very least it might provide some hints for building your own? I’m going to keep looking to see if I can find an image built on Debian, but at least now I know some options are out there.

[Edit] I also ran across across this page which builds a VM for you using an Ubuntu machine, so I’m guessing I could probably adjust it to a Debian setup fairly easily. https://github.com/nextcloud/vm

I was considering POE as an option, and this camera does have an ethernet port (although I can’t tell yet if that’s only for configuration or if the video will also stream over it directly). I don’t really need a constant stream and this camera also provides motion options so maybe it would only send video as needed (although during a heavy storm all of the cameras would probably fire at once).

I played with Zoneminder years ago but would like to get something set up for home security. I have a full internal network plus servers and about 60TB of free storage space so there’s really no limitations to what I could set up. Ideally I’d like to just hit a local IP from a cell phone to check the cameras (and remote access isn’t really needed), so that’s where I was trying to go with my previous questions.

The software side seems easy enough, but finding compatible IP cameras has been stumping me. I see the Reolink 4K TrackMix wifi cameras on Amazon for $130, and other than a few hiccups it looks likely that this piece of hardware would work, unless anyone knows of any “gotchas” that I’ve missed? Otherwise I’ll do a bit more research and then order one of the cameras to see how far I can get with it.

Ah gotcha, so you’re not trying to to live-view the feed on them. Thanks for the camera suggestion anyway, the brand still seems to be decent enough even if they don’t support Linux.

I looked up this camera and the reviews for the outdoor model with night vision seem pretty promising. Can I ask how you are using your camera(s)? Like do you have a local server to record the data? Can you access them with a cell phone without using any special apps?

A lot of it will depend on what age of hardware you are looking for, especially the price. Last year I upgraded all my machines to Poweredge R620 servers. These are old enough that you can find a lot of options for CPUs and memory dirt cheap, and you can find them with either 2.5" or 3.5" internal hot-swap bays. The 6xx series is 1U and the 7xx series is 2U chassis. If power is a concern, I run some VM servers with around 10 VMs in use, 64GB of memory, and a pair of 12-core Xeon E5-2630L v2 processors (2.4GHz, low power) at around 84W, but there’s plenty of options to customize to your needs unless you need current-generation horsepower. The PERC controller in them can be flashed to IT mode for full control, and I run ZFS through it for some of my machines. I built these machines for around $150-200 each and picked them up from ebay (there’s a US seller I can recommend if you’re interested in going that route).

Keep in mind the R* series are rack servers, but Dell also has tower versions of the same machines available – I think those are labeled as M620?

I’m curious what you’ve heard, this is the first recommendation I’ve seen against filezilla? You certainly have to get the server side set up right for it to work, and plain FTP is a security risk, but otherwise am I missing something?

If you can work from the command line (and assuming you have a linux server) then SSH is simple – really all it does is give you a secure connection to the command line. You should get familiar with it because if something goes wrong with your server that may be the only way you can connect to it.

Next you need tools to transfer files to the server. While wget is useful for grabbing stuff from other web servers, while something like scp can get you to any host that also accepts ssh. I use this all the time to transfer files between home and work. Or you might set up an sFTP service to accept a GUI connection from a client like FileZilla.

As for what you can put on your web server… Well if you install php then you can run any php code. If you write javascript code then the web browser interprets that, so nothing to add to your server, but NodeJS code would require some installation. You also want to take some time to learn about security practices. For example if you have pages that use a database, an attacker can write a URL to gain access to your server if the code simply accepts any random input. There’s not really any limit to what can be run, but some things (like the php example) require you to install more components on your server.

Alternatively, there are also functional services you can run that have nothing to do with web pages. For example, a caldav service would allow you to host your own calendar that can be shared between multiple people or locations. Or maybe you want to start up a chat server like IRC or Matrix? Maybe you want to start a Mosquitto server for your personal IoT content? Think of it this way – literally anything and everything that makes the internet run is something you can host yourself.

There was no such thing as a default firewall, but even now when I set up a new Debian machine there are no firewall rules, just the base iptables installed so you CAN add rules. Back then we also had insecure things like telnet installed by default and exposed to the world, so there’s really no telling exactly how they managed to get into my machine. It’s still good to learn about network security up front rather than relying on any default settings if someone is planning on self-hosting.

This was back in '99 and I didn’t know much about linux (or servers) at the time, so I’m not exactly sure what they did… but one morning I woke up and noticed my web service wasn’t working. I had an active login on the terminal but was just getting garbage from it, and I couldn’t log in remotely at all. My guess was that someone hacked in, but hacked the system so badly that they basically trashed it. I was able to recover a little data straight from the drive but I didn’t know anything about analyzing the damage to figure out what happened. so I finally ended up wiping the drive and starting over.

At that point I did a sped-run of learning how to set up a firewall, and noticed right away all kinds of attempts to hit my IP. It took time to learn more about IDS and trying not to be too wreckless in setting up my web pages, but apparently it was enough to thwart however that first attacker got in. Eventually I moved to a dedicated firewall in front of multiple servers.

Since then I’ve had a couple instances where someone cracked a user password and started sending spam through, but fail2ban stopped that. And boy are there a LOT of attempts at trying to get into the servers. I should probably bump up fail2ban to block IPs faster and over a longer period when they use invalid user names since attacks these days happen from such a wider range of IPs.

I see a number of comments to use a virtual server host, but I have not seen any mention of the main reason WHY this is advisable… If you want to host something from your home, people need a way to reach you. There are two options for this – use a DDNS service (generally frowned upon for permanent installations), or get a static IP address from your provider.

DDNS means you have to monitor whenever your local IP address changes, send out updated records, and wait for those changes to propagate across the internet. This generally will mean several minutes or more of down time where nobody can reach your server, and can happen at completely random times.

A static IP is reliable, but they cost money, and some providers won’t even give you the option unless you get a business-class connection, which costs even more money. However this cost is usually already rolled into the price of a virtual machine.

Keep in mind also that when hosting at home, simply using a laptop to stay online 24/7 is not enough, you also need a battery backup for your network equipment. You will want to learn about setting up a firewall and some kind of IDS to protect the front end of your services, but for starting out you can host this on the same machine as your other services. And if you really want to be safe, set up a second internal machine that you can perform regular backups to, so when your machine gets hacked you have a way to restore the information.

My first server was online for two whole weeks before someone blew it up. Learn security first, everything after that will be easy.

I dunno, like I said zfs is pretty damn good at recovery. If the drives simply drop out but there’s no hardware fault you should be able to clear the errors and bring the pool back up again. And the chances of two drives failing at the same time are pretty low. One of these days I do need to buy a spare to have on hand though. Maybe I’ll even swap out one drive just to see how long it takes to rebuild.

My current setup is eight 18TB Exos drives, all purchased from Amazon’s refurb shop, and running in a RAIDz2. I’m pulling about 450MB/s through various tests on a system that is in use. I’ve been running this about a year now and smartd hasn’t detected any issues. I have almost never run new drives for my storage and the only time I’ve ever lost data was back when I was running mdadm and a power glitch broke the sync on multiple drives so the array couldn’t be recovered. With zfs I have even run a RAID0 with five drives which saw multiple power incidents (before I got a redundant power supply) and I never once lost anything because of zfs’ awesome error detection.

So yes, used drives can be just fine as long as you do your research on the drive models, have a very solid power supply, and are configured for hot-swapping so you can replace a drive when they fail. Of course that’s solid advice even for brand new drives, but my last set of used drives (also from ebay) lasted about a decade before it was time to upgrade. Sure, individual drives took a dump over that time, this was another set of eight and I replaced three of them, but the data was always safe.

What do you consider a fair amount? My current server has 64GB of ram but arc_summary says ZFS is only using 6.35GB on a system with three ZFS pools totaling over 105TB of storage under pretty much constant usage.

No matter how you go about it, getting these drives set up to be reliable isn’t going to be cheap. If you want to run without an enclosure, at the very least (and assuming you are running Linux) you are going to want something like LSI SAS cards with external ports, preferably a 4-port card (around $50-$100, each port will run four drives) that you can flash into IT mode. You will need matching splitter cables (3x $25 each). And most importantly you need a VERY solid power supply, preferably something with redundancy (probably $100 or more). These prices are based on used hardware from ebay, except for the cables, and you’ll have to do some considerable research to learn how to flash the SAS cards, and which ones can be flashed.

Of course this is very bare-bones, you won’t have a case to mount the drives in, and splitter cables from the power supply can be finicky, but with time and experience it can be made to work very well. My current NAS is capable of handling up to 32 external and 8 internal drives and I’m using 3D-printed drive cages with some cheap SATA2 backplanes to finally get a rock-solid setup. It takes a lot of work and experience to do things cheaply.

This right here. As a member of the OpenNIC project, I used to run an open resolver and this required a lot of hands-on maintenance. Basically what happens is someone sends a very small packet requesting the lookup of something which returns a huge amount of data (like DNSSEC records). They can make thousands of these requests in a short period, attempting to flood out the target domain’s DNS servers and effectively take them offline, by using your open server as the attacker.

At the very least, you need to have strict rate-limiting controls on DNS lookups. And since the requests come in through UDP, they can spoof their IP address so you can’t simply block an attacker. When I ran into this issue, I wrote up scripts to monitor for a lot of requests to the same domain name and outright block those until the attack stopped. It wasn’t a great solution, but it did at least make sure my system wasn’t contributing to an attack.

Your best bet is to only respond to DNS requests for your own domain(s). If you really want an open resolver, think about limiting it by creating some sort of sign-up method (for instance, ddns servers use a specific URL to register the changing IP of known users), but still keep the rate-limiting in place.

You might want to use a code block instead of bullet points for your table, the way you presented it is unreadable but I found the info on your blog page.

One of my criteria for video formats is the portability. Like sometimes I might watch something through a web browser which natively supports x264. Yeah x265 provides better compression, and AV1 certainly looks interesting, but they both require the addition of codecs on most of my viewing devices and in some cases that’s not possible.

For most cases I’ve found that CRF25 with x264 works reasonably well. I tend to download 720p videos to watch on our 1080p TV and don’t notice the difference except in very minor situations like rapid motion on a solid-color background (usually only seen on movie studio logo screens). Any sort of animated shows can go even lower without noticeable degradation.