# Observability

Metrics and logs both surface in Grafana, keeping dashboards and log queries in one place.

---

## Metrics

kube-prometheus-stack runs in the `prometheus` namespace (ArgoCD-managed). Prometheus scrapes all nodes, pods, and control plane components. Grafana has dashboards for cluster overview, node resources, Longhorn, ArgoCD, and Traefik.

Node Exporter is deployed via Ansible on every VM including `docker-host11` and the edge VPS, so coverage isn't limited to what's inside Kubernetes.

Goldilocks and VPA run alongside and analyze actual resource usage to suggest better request/limit values.

Alertmanager routes alerts to Ntfy via a custom webhook bridge.

---

## Logs

Loki runs in the `loki` namespace (single-node, filesystem storage). Promtail runs as a DaemonSet and ships pod logs from every cluster node to Loki. Logs are queryable in Grafana alongside metrics.

---

## Alerting flow

```
Prometheus Alertmanager ──► Ntfy (push notification)
```