Introduction

When dealing with power outages in home lab environments, proper shutdown procedures are crucial to avoid data corruption and hardware damage. However, not all UPS (Uninterruptible Power Supply) systems support Network UPS Tools (NUT), making it difficult to determine when a server should be shut down.

For example, I have a power station with UPS mode for camping. When I’m not camping, I use it as a UPS for my non-tier-1 devices/servers in my homelab.

To solve this problem, we can leverage IoT devices that lose power when the main supply goes down. By periodically pinging these devices, we can detect when a power outage has occurred and initiate a safe shutdown process after a specified threshold.

This blog post walks through a Bash script designed for this purpose, explaining how it works and how you can implement it.

How the Script Works

This Bash script continuously monitors IoT devices (or any networked devices) by pinging them. If all the defined devices fail to respond for a set duration (e.g., 20 minutes), it safely shuts down the server.

Key Features:

✔️ Works without NUT – Ideal for UPS systems that don’t support monitoring.
✔️ Uses standard ping checks – No additional dependencies beyond basic network tools.
✔️ Customizable threshold – Configure how long the server should wait before shutting down.
✔️ Runs as a cron job – Automatically executes at regular intervals.

Breaking Down the Script

Here’s the complete script:

#!/bin/bash

# Define the target host. If ANY host is up, the server will NOT be shut down.
TARGET_HOSTS=("example.com" "device1.iot.example.com")
THRESHOLD=20  # 20 * 1 min = 20 minutes (1 min is the crontab interval)
# different distro may have different shutdown command. 
# e.g. Proxmox needs absolute path '/usr/sbin/shutdown -h now'; Synology is fine with 'shutdown -h now'
SHUTDOWN_CMD="shutdown -h now" 

# Define a status file to track the downtime duration
STATUS_FILE="/tmp/host_down_count"

# Ensure the script is run as root
if [ "$(id -u)" -ne 0 ]; then
    echo "This script must be run as root!"
    exit 1
fi

# Ensure ping is installed
if ! command -v ping &> /dev/null; then
    echo "Ping not found. Installing..."
    apt update && apt install -y iputils-ping
fi

# Function to check if any host is up
is_any_host_up() {
    local hosts=("$@")
    for host in "${hosts[@]}"; do
        if ping -c 2 -W 2 "$host" > /dev/null 2>&1; then
            echo "$(date): Host $host is up."
            return 0  # At least one host is up
        else
            echo "$(date): Host $host is down."
        fi
    done
    echo "$(date): No host is up."
    return 1  # No host is up
}

# Check if any of the hosts is up
if is_any_host_up "${TARGET_HOSTS[@]}"; then
    echo "$(date): At least one host is up."
    rm -f "$STATUS_FILE"  # Reset status file if any host is up
    exit 0
else
    echo "$(date): All hosts are down."
    
    # Increase downtime count
    if [ -f "$STATUS_FILE" ]; then
        COUNT=$(cat "$STATUS_FILE")
        COUNT=$((COUNT + 1))
    else
        COUNT=1
    fi
    echo "$COUNT" > "$STATUS_FILE"
    echo "$(date): Count: $COUNT"

    # If the server has been down for $THRESHOLD * 1 minutes (1-minute intervals → $THRESHOLD checks)
    if [ "$COUNT" -ge "$THRESHOLD" ]; then
        echo "$(date): All target hosts have been down for $THRESHOLD minutes. Shutting down server..."
        rm -f "$STATUS_FILE"  # Reset status file before shutdown
        $SHUTDOWN_CMD
    fi
    exit 1
fi