@AverageGoob The watchdog saves me from rebooting the host manually, but at the risk of data loss (though not more than a locked up SD card). I configured a custom script that writes to a file, when the card has problems, the watchdog kicks in. To keep the script from stressing the card even more, the script only writes to the file every few minutes.
As you said it’s only a workaround. I’ll move the stuff on the problematic host to a VM with SSD shortly.
@a_fancy_kiwi I agree, same here. This is the last pi that’s running off an SD card with services that do “significant” disk I/O. I have a few zeros that only really write to the card for OS updates. Their job is to collect data and send it via the network. I haven’t had issues with that kind of workload using micro SD cards.
Edit: For Pis with write workloads I’m using basic USB3 SSDs. Didn’t have good results with USB sticks though.