I’ve got a lot of services, some in docker, some in LXC or a VM in proxmox. Currently I’ve got no monitoring service. Recently a service went down and I didn’t notice for quite a while so now I’ve got a bunch of missing data. What monitoring tools do you all use? Looking for something that works with docker and plain Linux CTs/VMs and can notify me if a website is down, docker container crashed, VM is offline, etc.

and as a bonus feature something that I can run on two machines so if an entire machine dies, the other will notice and I’ll still receive a notification.

notification can be anything, email, sms, push, etc.

  • nitrolife@rekabu.ruEnglish
    8·
    11 months ago

    I use:

    1. Monitoring server - prometheus
    2. Alert manager for prometheus - alertmanager. You can write any triggers here.
    3. Web UI for prometheus - Grafana
    4. Exporters for prometheus - node-exporter, blackbox-exporter, mysql-exporter, psql-exporter etc. You can find exporter for everything you need.
    5. Some services native support pormetheus. Docker for example: https://docs.docker.com/config/daemon/prometheus/

    If you whant cluster you can install thanos on prometheus.

    • peregus@lemmy.worldEnglish
      2·
      11 months ago

      I’d like to explore Prometheus (I’ve never used it). Right now I use InfluxDB to store some data (ping times, temperature, servers load, etc.) can Prometheus read those values and react if something is off or should I store everything twice?

      • nitrolife@rekabu.ruEnglish
        3·
        11 months ago

        prometheus use own time series database. you can connect influxdb to grafana and send alarms from grafana, but alertmanager better i think. node-explorer can collect all this data (sensors, VM/PC load etc.)

        • peregus@lemmy.worldEnglish
          2·
          11 months ago

          I’ve never used alerting in Grafana, how do they work? Is it possible to get alert if a ping is higher than xx for a period of time? What are alertmanager and node-explorer? Plugins or standalone tools? Sorry for all the questions! 😁 And thanks for the info!

          • nitrolife@rekabu.ruEnglish
            1·
            11 months ago

            Grafana sends an email screenshot of the graph when an event is triggered on the graph. You can see alerts part on any graph for understand.