A server can be "up" and still be quietly failing you. Disk filling up over weeks, memory creeping toward a swap death spiral, a backup job that's been silently erroring for days — none of that shows up in a basic uptime check. The only way to catch it is to actually watch the numbers over time, not just whether the server responds to a ping.
This guide sets up a self-hosted observability stack — Prometheus for metrics collection and storage, Node Exporter for exposing system-level statistics, and Grafana for visualizing all of it on dashboards — on a bare-metal Ubuntu 24.04 server. By the end, you'll have live graphs of CPU load, memory pressure, disk usage, and network throughput, with no dependency on a third-party SaaS monitoring platform.
How the Pieces Fit Together
Before installing anything, it helps to know what each component actually does and where data flows between them:
[Node Exporter :9100] ──scraped by──► [Prometheus :9090] ──queried by──► [Grafana :3000]
Node Exporter is a lightweight agent that reads hardware and OS-level statistics straight from the kernel (via
/procand/sys) and exposes them on a plain HTTP endpoint.Prometheus is the collection and storage engine. Rather than waiting for data to be pushed to it, it pulls (scrapes) metrics from each exporter on a schedule and stores them as time-series data.
Grafana is the visualization layer. It queries Prometheus using PromQL and renders the results as dashboards, graphs, and alerts.
This pull-based model is one reason the stack scales well on bare metal — adding a new server to watch just means pointing Prometheus at one more exporter endpoint, with no agent needing to know where to send anything.
Prerequisites
A bare-metal or dedicated server running Ubuntu 24.04 LTS.
A non-root user with
sudoprivileges.Open inbound access (locally or via firewall rule) on ports 9090 (Prometheus), 3000 (Grafana), and 9100 (Node Exporter) — or, better, restrict these to your management network and reach Grafana through a reverse proxy, covered at the end.
Step 1: Create Dedicated System Users
Running monitoring services under their own unprivileged accounts limits what damage a compromised process could do — a sound default for anything reading system-level data with elevated visibility.
sudo useradd --no-create-home --shell /bin/false prometheus
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus /var/lib/prometheus
Step 2: Install Prometheus
Prometheus ships as a single static binary, so installation is just downloading, extracting, and placing it.
cd /tmp
curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest \
| grep browser_download_url | grep linux-amd64 | cut -d '"' -f 4 | wget -qi -
tar xvf prometheus-*.linux-amd64.tar.gz
cd prometheus-*.linux-amd64
sudo mv prometheus promtool /usr/local/bin/
sudo mv consoles console_libraries /etc/prometheus/
sudo mv prometheus.yml /etc/prometheus/prometheus.yml
sudo chown -R prometheus:prometheus /etc/prometheus /usr/local/bin/prometheus /usr/local/bin/promtool
Create a systemd unit so Prometheus runs as a managed background service and restarts automatically if it crashes:
sudo tee /etc/systemd/system/prometheus.service > /dev/null <A status showing Active: active (running) means it started cleanly. Prometheus's own web interface is now reachable at http://your-server-ip:9090 — it scrapes its own internal metrics by default, so you should already see one target listed under Status → Targets.
Step 3: Install Node Exporter
Node Exporter follows the same binary-and-systemd pattern, exposing the actual hardware and OS metrics — CPU time, memory pressure, disk I/O, filesystem usage, network counters — that Prometheus will go on to collect.
cd /tmp
curl -s https://api.github.com/repos/prometheus/node_exporter/releases/latest \
| grep browser_download_url | grep linux-amd64 | cut -d '"' -f 4 | wget -qi -
tar xvf node_exporter-*.linux-amd64.tar.gz
sudo mv node_exporter-*.linux-amd64/node_exporter /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
sudo tee /etc/systemd/system/node_exporter.service > /dev/null <Confirm it's exposing data by visiting http://your-server-ip:9100/metrics — you should see a long plaintext list of metric names and values. This page is Node Exporter's entire interface; there's no dashboard here, just raw numbers waiting to be scraped.
Step 4: Point Prometheus at Node Exporter
Right now Prometheus only knows about itself. Edit /etc/prometheus/prometheus.yml and add Node Exporter as a scrape target under scrape_configs:
scrape_configs:
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
- job_name: "node"
static_configs:
- targets: ["localhost:9100"]
If you're monitoring multiple bare-metal servers from one central Prometheus instance, add each one's IP under the same node job:
- job_name: "node"
static_configs:
- targets: ["10.0.0.11:9100", "10.0.0.12:9100", "10.0.0.13:9100"]
Restart Prometheus to pick up the change:
sudo systemctl restart prometheus
Back in the Prometheus web UI under Status → Targets, you should now see two entries — prometheus and node — both reporting state UP. If node shows DOWN, double-check the target's firewall allows inbound traffic on port 9100 from the Prometheus host specifically.
Step 5: Install Grafana
Grafana is the layer that turns raw time-series numbers into something a human actually wants to look at.
sudo apt-get install -y apt-transport-https software-properties-common wget gnupg2
wget -q -O - https://packages.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/grafana.gpg > /dev/null
echo "deb [signed-by=/etc/apt/trusted.gpg.d/grafana.gpg] https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install -y grafana
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server
Visit http://your-server-ip:3000. The default login is username admin, password admin — you'll be prompted to set a new password immediately, and you should, since this dashboard will be looking at fairly sensitive operational data about your infrastructure.
Step 6: Connect Grafana to Prometheus
Inside Grafana:
- Go to Connections → Data Sources → Add data source
- Choose Prometheus
- Set the URL to
http://localhost:9090(same server) orhttp://<prometheus-ip>:9090if Grafana is running elsewhere - Click Save & Test — a confirmation that the API query succeeded means the connection is live
Step 7: Import a Pre-Built Dashboard
Building dashboard panels from scratch works, but for general system monitoring, the community-maintained Node Exporter Full dashboard covers CPU, memory, disk, network, and load averages out of the box.
- Go to Dashboards → New → Import
- Enter dashboard ID 1860
- Select your Prometheus data source when prompted
- Click Import
You should immediately see live graphs populate — CPU usage by core, memory breakdown (used, cached, available), disk space per mount point, network throughput per interface. This single dashboard is usually enough to catch the slow-burn problems a simple uptime check would miss entirely.
Step 8: Set Up Basic Alerting
A dashboard nobody's watching at 3 AM doesn't help when disk usage crosses a dangerous threshold overnight. Grafana's built-in alerting can notify you instead of requiring someone to be looking.
In Grafana, go to Alerting → Alert rules → New alert rule. A practical starting alert — disk usage climbing past a safe margin — uses a query against the node_filesystem_avail_bytes and node_filesystem_size_bytes metrics that Node Exporter already exposes. Configure a contact point (email, Slack webhook, or similar) under Alerting → Contact points so the rule actually reaches you when it fires.
Securing the Stack for Production
The setup above is functional but wide open — Prometheus and Grafana both default to no authentication on Prometheus's own UI and a default admin password on Grafana's. Before exposing this beyond your own management network:
Restrict ports 9090 and 9100 to internal/management traffic only via your firewall (
ufworiptables) — these don't need to be reachable from the public internet at all; only Grafana needs a path to Prometheus, and that can happen entirely over a private network.Put Grafana behind a reverse proxy with TLS (Nginx or Caddy) rather than exposing port 3000 directly, so login traffic and dashboard data aren't sent in plaintext.
Change Grafana's default admin credentials immediately — this was already prompted in Step 5, but it's worth confirming explicitly.
Frequently Asked Questions
What's the difference between Prometheus and Grafana — do I need both?
Prometheus collects and stores the metrics; Grafana visualizes them. Prometheus does have a basic built-in graphing interface, usable in a pinch, but it's not designed for building the kind of multi-panel, shareable dashboards that Grafana specializes in. Running Prometheus alone gets you data; pairing it with Grafana is what makes that data actually legible at a glance.
Why does Prometheus pull metrics instead of having servers push them?
The pull model means Prometheus itself controls the scrape schedule and immediately knows if a target stops responding — a target going silent is itself a signal worth alerting on. It also keeps configuration centralized: adding a new server to monitor means updating one config file on the Prometheus server, not deploying new push configuration to every monitored machine.
How much disk space does Prometheus's time-series data actually use?
It depends on how many metrics you're scraping and at what interval, but a single node's system-level metrics (CPU, memory, disk, network) at a default 15-second scrape interval typically uses a modest amount of disk per day. Usage scales with the number of exporters and targets being scraped. Prometheus's default retention period and `--storage.tsdb.retention.time` flag control how long old data is kept before being purged.
Can this monitor more than one server from a single Prometheus instance?
Yes — that's the standard pattern. Install Node Exporter on every server you want visibility into, then add each one's IP and port 9100 as a target under the same `job_name: "node"` block in `prometheus.yml`. One central Prometheus and Grafana pair can comfortably watch a small fleet of bare-metal servers this way.
Is self-hosted monitoring actually worth it instead of a SaaS dashboard?
It depends on what you're optimizing for. Self-hosting keeps your operational metrics entirely under your own control, with no recurring per-host or per-metric billing from a third party. The trade-off is that you're now responsible for keeping the monitoring stack itself running and scaled.
