Uptime Kuma push monitoring (VM stats)

Use this runbook to monitor VM CPU, memory, and disk usage via Uptime Kuma Push monitors. The VM runs a small script on a systemd timer and pushes status + a numeric ping value for graphing.

Entry points

• Uptime Kuma (hosted on np-tools): http://tools.core.lef:3001/
• Push API base: http://tools.core.lef:3001/api/push/<monitor-id>

Prerequisites

• You can access Uptime Kuma (VPN/LAN) and create monitors.
• curl is installed on the VM.
• Root access to create files under /opt and /etc/systemd/system.

Monitor layout (recommended)

• One Group monitor per VM (notifications disabled)
• Three Push monitors under the group:
  • <vm-name> CPU (<95%)
  • <vm-name> MEM (<95%)
  • <vm-name> DISK (<95%)

Suggested settings for Push monitors:

• Heartbeat interval: 305 seconds
• Retries: 1
• Retry interval: 305 seconds
• Resend notification: 4 (every ~20 minutes)

Treat Push monitor IDs as secrets (anyone with the ID can spoof status). Store them in Vault and copy them onto the VM via a root-only file.

Install the pusher

1. Create the directory:

sudo mkdir -p /opt/kuma-pusher

2. Create the script at /opt/kuma-pusher/kuma-pusher.sh:

sudo nano /opt/kuma-pusher/kuma-pusher.sh

#!/usr/bin/env bash
set -euo pipefail

# ---- config ----
cpu_max=95
disk_max=95

# memory: adjust as needed
mem_avail_crit_mb=512        # down if < 512 MiB available
mem_avail_crit_pct=1         # or < 1% available

kuma_base="${KUMA_BASE:-http://tools.core.lef:3001/api/push}"
mon_cpu="${KUMA_PUSH_ID_CPU:?set KUMA_PUSH_ID_CPU}"
mon_mem="${KUMA_PUSH_ID_MEM:?set KUMA_PUSH_ID_MEM}"
mon_disk="${KUMA_PUSH_ID_DISK:?set KUMA_PUSH_ID_DISK}"

# ---- helpers ----
pct() { awk -v n="${1:-0}" -v d="${2:-0}" 'BEGIN{ if(d>0){printf "%.0f",(n/d)*100}else{print 0} }'; }

get_cpu_usage() {
  # calculate CPU% over ~1s
  read -r _ u1 n1 s1 i1 w1 ir1 si1 st1 _ < <(grep '^cpu ' /proc/stat)
  idle1=$((i1 + w1)); non1=$((u1 + n1 + s1 + ir1 + si1 + st1)); tot1=$((idle1 + non1))
  sleep 1
  read -r _ u2 n2 s2 i2 w2 ir2 si2 st2 _ < <(grep '^cpu ' /proc/stat)
  idle2=$((i2 + w2)); non2=$((u2 + n2 + s2 + ir2 + si2 + st2)); tot2=$((idle2 + non2))
  d_idle=$((idle2 - idle1)); d_tot=$((tot2 - tot1))
  if [ "$d_tot" -gt 0 ]; then
    echo $(( 100 * (d_tot - d_idle) / d_tot ))
  else
    echo 0
  fi
}

get_mem_total_kb()      { awk '/MemTotal:/      {print $2}' /proc/meminfo; }
get_mem_available_kb()  { awk '/MemAvailable:/   {print $2}' /proc/meminfo; }

push_status() {
  # $1=resource $2=status(up|down) $3=message $4=monitor_id $5=ping
  local r="$1" s="$2" m="$3" id="$4" p="${5:-1}"
  curl -fsS -m 10 --retry 1 -o /dev/null --get \
    --data-urlencode "status=${s}" \
    --data-urlencode "msg=${m}" \
    --data-urlencode "ping=${p}" \
    "${kuma_base}/${id}"
}

# ---- collect ----
cpu=$(get_cpu_usage)

mt_kb=$(get_mem_total_kb)
ma_kb=$(get_mem_available_kb)
ma_mb=$(( ma_kb / 1024 ))
ma_pct=$(pct "$ma_kb" "$mt_kb")

disk_pct=$(df / | awk 'END{gsub("%",""); print $5}')

# ---- decide cpu ----
cpu_status="up"; cpu_msg="cpu ${cpu}%"; cpu_ping="$cpu"
[ "$cpu" -ge "$cpu_max" ] && cpu_status="down" && cpu_msg="high ${cpu}%"

# ---- decide mem ----
mem_status="up"
mem_msg="avail ${ma_mb}MiB (${ma_pct}%)"
mem_ping=$((100 - ma_pct)); [ "$mem_ping" -lt 1 ] && mem_ping=1

if [ "$ma_mb" -lt "$mem_avail_crit_mb" ] || [ "$ma_pct" -lt "$mem_avail_crit_pct" ]; then
  mem_status="down"
  mem_msg="low avail ${ma_mb}MiB (${ma_pct}%)"
  mem_ping=1500
fi

# ---- decide disk ----
disk_status="up"; disk_msg="root ${disk_pct}% used"; disk_ping="$disk_pct"
if [ "$disk_pct" -ge "$disk_max" ]; then
  disk_status="down"; disk_msg="root high ${disk_pct}%"
elif [ "$disk_pct" -ge 80 ]; then
  disk_msg="root elevated ${disk_pct}%"
fi

# ---- push ----
push_status "cpu"  "$cpu_status" "$cpu_msg" "$mon_cpu"  "$cpu_ping"
push_status "mem"  "$mem_status" "$mem_msg" "$mon_mem"  "$mem_ping"
push_status "disk" "$disk_status" "$disk_msg" "$mon_disk" "$disk_ping"

3. Create the environment file at /etc/kuma-pusher.env (root-only):

sudo nano /etc/kuma-pusher.env
sudo chmod 600 /etc/kuma-pusher.env

KUMA_BASE="http://tools.core.lef:3001/api/push"
KUMA_PUSH_ID_CPU="<push-id>"
KUMA_PUSH_ID_MEM="<push-id>"
KUMA_PUSH_ID_DISK="<push-id>"

4. Make the script executable:

sudo chmod +x /opt/kuma-pusher/kuma-pusher.sh

Systemd units

Create the service:

sudo nano /etc/systemd/system/kuma-pusher.service

[Unit]
Description=Push system health to Uptime Kuma
After=network-online.target local-fs.target

[Service]
Type=oneshot
EnvironmentFile=/etc/kuma-pusher.env
ExecStart=/opt/kuma-pusher/kuma-pusher.sh

Create the timer:

sudo nano /etc/systemd/system/kuma-pusher.timer

[Unit]
Description=Run kuma-pusher health monitor every 5 minutes

[Timer]
OnBootSec=30s
OnUnitActiveSec=300s
AccuracySec=1s
Persistent=true
Unit=kuma-pusher.service

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now kuma-pusher.timer

Validate

sudo /opt/kuma-pusher/kuma-pusher.sh
systemctl list-timers | grep kuma
journalctl -u kuma-pusher.service

Known risks / failure modes

• Wrong KUMA_BASE (DNS/firewall) or wrong Push IDs → pushes fail.
• Push IDs leak → someone can spoof an up status; keep /etc/kuma-pusher.env root-only and store IDs in Vault.
• Non-standard root filesystem (not /) → adjust the df line in the script.

Related

Operations Monitoring Servers & logins Servers