What to watch on a home server
A Mac that runs services and long jobs at home raises three questions. Is it up? Is it working? Or is it stuck? In this course I answer them with metrics on a dashboard and an alert on your phone when the answer is no. Before you start you need Docker with Compose version 2 from Colima, Docker Desktop or Orbstack and two tools from Homebrew, NodeExporter, with Brew install NodeExporter, and UV, the Python package manager. Three commands check them. There are three kinds of signals. Logs tell you what happened line by line. Traces follow one request through several services. Metrics are numbers measured again and again. How many jobs wait? How much disk is free? For the three questions, metrics are enough and they are cheap to keep for weeks. I collect them with OpenTelemetry, an open standard for telemetry, and its protocol OTLP, the OpenTelemetry protocol. Your code sends metrics in one format and the backends can change later. The starter repository is called Homelab monitoring it has one commit, a compose file for the monitoring stack, the pro, me, theos and grafana settings, and four scripts, two of them standing for the real work of a homelab. The worker script takes jobs from a queue folder one at a time. While it works it holds a log folder then it moves the job to a done folder. One job in five fails and is written to a log. Replace it in your head with your renders, backups or builds. Grafana needs an admin password. It goes in an env file that git ignores, readable only by me. Now I start the worker in the background and queue six jobs. The The status script answers the three questions once. In the terminal are docker, node exporter and the work app. How many jobs wait, how many are done or failed, what the worker does right now and how much disk is free. That is the baseline. It works but only while you look at it, it keeps no history and it tells nobody when the queue stops moving. So the plan is the same answers as metrics every 30 seconds kept for 15 days shown on one dashboard with three alerts, a service down, job stack and the disk filling up.