Fact-check docs against live cluster and repo history
Corrected stale claims found by cross-referencing git history and live kubectl/SSH state: - Drop Terraform provisioning claim (VMs are created by the ansible-homelab proxmox role via community.proxmox.proxmox_kvm; terraform/ is empty) - Bump k3s badge to the version actually running (v1.36) - Drop Elastic Stack as a live logging path - it was disabled in favor of agentmemory and has no namespace or ArgoCD Application anymore - Drop zeroclaw from the app list - manifests exist but it was never wired into ArgoCD's app registry, so it has never actually run Added detail confirmed against live state: Longhorn's Hetzner backup target, node_cleanup containerd-pruning timer, MikroTik static DHCP leases, Prometheus cardinality/false-positive-alert tuning, actual CNPG cluster inventory, corrected MetalLB pool size, aya01's real NFS capacity, k3s-server11's post-rebuild spec drift, and the naruto01 etcd quorum concentration.
This commit is contained in:
Tuan-Dat Tran
36
README.md
36
README.md
@@ -1,8 +1,8 @@
|
||||
# Homelab
|
||||
|
||||
17-node Kubernetes cluster on four bare-metal Proxmox hosts, provisioned with Terraform and Ansible, managed through ArgoCD GitOps. Runs my home automation, media stack, photo backup, documents, and a few side projects.
|
||||
17-node Kubernetes cluster on four bare-metal Proxmox hosts, provisioned and configured with Ansible, managed through ArgoCD GitOps. Runs my home automation, media stack, photo backup, documents, and a few side projects.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -77,11 +77,13 @@ Secrets are committed to Git too, encrypted via Sealed Secrets. Only the in-clus
|
||||
|
||||
No ports are open on the home router. Internal load balancing goes through MetalLB + Traefik. External access uses Cloudflare tunnels or a WireGuard VPN routed through the edge VPS.
|
||||
|
||||
Longhorn handles block storage by replicating volumes across all 14 agent nodes. The media library lives on a dedicated NFS host instead — latency matters when Jellyfin is reading large video files, and NFS is simpler for that.
|
||||
Longhorn handles block storage by replicating volumes across all 14 agent nodes, with backup snapshots shipped off-site to a Hetzner Storage Box over CIFS. The media library lives on a dedicated NFS host instead — latency matters when Jellyfin is reading large video files, and NFS is simpler for that.
|
||||
|
||||
Metrics go to Prometheus + Grafana. Logs go to Loki, also surfaced in Grafana — Promtail runs as a DaemonSet and ships pod logs from every node.
|
||||
|
||||
All VMs are provisioned with Terraform and configured by Ansible. Rebuilding from scratch doesn't require remembering anything.
|
||||
Not every HA guarantee holds up under scrutiny — two of the three etcd control-plane VMs currently run on the same physical Proxmox host (`naruto01`), a single point of failure for cluster quorum that surfaced during an incident and is still on the list to rebalance.
|
||||
|
||||
All VMs are provisioned and configured by Ansible — the `proxmox` role calls the Proxmox API directly (`community.proxmox.proxmox_kvm`) to clone, resize, and boot VMs from cloud-init templates, then the rest of the playbooks configure them. Rebuilding from scratch doesn't require remembering anything.
|
||||
|
||||
---
|
||||
|
||||
@@ -89,18 +91,18 @@ All VMs are provisioned with Terraform and configured by Ansible. Rebuilding fro
|
||||
|
||||
| Layer | Host | Role | Resources |
|
||||
|-------|------|------|-----------|
|
||||
| Physical | `aya01` | Proxmox node + NFS server | Dedicated storage — no VMs |
|
||||
| Physical | `aya01` | Proxmox node + NFS server (49 TB share, ~60% used) | Dedicated storage — no VMs |
|
||||
| Physical | `lulu` | Proxmox node | k3s agents |
|
||||
| Physical | `naruto01` | Proxmox node | k3s server + agents |
|
||||
| Physical | `mii01` | Proxmox node | k3s server + agents + docker host |
|
||||
| VM | `k3s-server-{10,11,12}` | K3s control plane (HA etcd + kube-vip VIP) | 2 vCPU · 4 GB RAM · 64 GB |
|
||||
| VM | `k3s-server-{10,11,12}` | K3s control plane (HA etcd + kube-vip VIP) | 2 vCPU · 4 GB RAM · 64 GB (`k3s-server11` runs 4 vCPU · 6 GB after a rebuild — not yet reconciled with the other two) |
|
||||
| VM | `k3s-agent-{10…23}` | K3s worker nodes ×14 | 2 vCPU · 4 GB RAM · 128 GB |
|
||||
| VM | `docker-host11` | Docker host | 2 vCPU · 4 GB RAM · 192 GB |
|
||||
| VM | `docker-host11` | Docker host (AMD Ryzen 7 5700U — CPU-only, no iGPU passthrough for Jellyfin) | 2 vCPU · 4 GB RAM · 192 GB |
|
||||
| VM | `docker-lb` | Caddy reverse proxy (LAN only) | 1 vCPU · 2 GB RAM |
|
||||
| VPS | `mii` | Edge node (Netcup) | WireGuard · Traefik · Pangolin |
|
||||
| SBC | `naruto` | Raspberry Pi | Zigbee2MQTT · Mosquitto MQTT broker |
|
||||
|
||||
All VMs run Debian 12 on `virtio` network bridges, provisioned from cloud-init templates via Terraform + Ansible. The Raspberry Pi runs Docker Compose, managed by the same Ansible setup.
|
||||
All VMs run Debian 12 on `virtio` network bridges, provisioned from cloud-init templates via Ansible. The Raspberry Pi runs Docker Compose, managed by the same Ansible setup. A MikroTik router is also under Ansible's control — it backs up its RouterOS config to Git on every run and gets static DHCP leases pinned for the k3s nodes, so a lease renewal can't silently move a node's IP.
|
||||
|
||||
---
|
||||
|
||||
@@ -116,9 +118,9 @@ All VMs run Debian 12 on `virtio` network bridges, provisioned from cloud-init t
|
||||
| **Cert-Manager** | Helm (ArgoCD) | Automated TLS via Let's Encrypt DNS-01 (Cloudflare API) |
|
||||
| **Sealed Secrets** | Helm (ArgoCD) | Encrypts secrets for safe storage in Git |
|
||||
| **Longhorn** | Helm (ArgoCD) | Distributed block storage (RWO + RWX) across all 14 agents |
|
||||
| **CloudNativePG** | Operator (ArgoCD) | HA PostgreSQL — used by Immich and LiteLLM |
|
||||
| **CloudNativePG** | Operator (ArgoCD) | HA PostgreSQL — one cluster per app (Immich, Vaultwarden, N8n, Paperless, LiteLLM); Vaultwarden's is 3-instance HA, the rest single-instance |
|
||||
| **Kube-Prometheus-Stack** | Helm (ArgoCD) | Prometheus + Grafana monitoring |
|
||||
| **Loki + Promtail** | Helm (ArgoCD) | Log aggregation — Promtail DaemonSet ships pod logs to Loki |
|
||||
| **Loki + Promtail** | Helm (ArgoCD) | Log aggregation — Promtail DaemonSet ships pod logs to Loki (14-day retention) |
|
||||
| **Goldilocks + VPA** | Helm (ArgoCD) | Resource usage analysis and request/limit rightsizing |
|
||||
| **Istio (Ambient)** | Helm (ArgoCD) | Service mesh — ztunnel DaemonSet on all nodes (L4); no Waypoint proxies yet |
|
||||
| **K3s Upgrade Controller** | Operator (ArgoCD) | Automated rolling K3s version upgrades |
|
||||
@@ -141,12 +143,12 @@ All VMs run Debian 12 on `virtio` network bridges, provisioned from cloud-init t
|
||||
| **Stirling PDF** | PDF tools | – |
|
||||
| **Karakeep** | Bookmark manager | – |
|
||||
| **Gitea** | Self-hosted Git (source of truth for ArgoCD) | Kubernetes · Longhorn PVC · SSH via MetalLB · Pangolin tunnel (git.tudattr.dev) |
|
||||
| **Gitea Runner** | CI/CD runner | – |
|
||||
| **Arr Stack** | Media automation suite | Prowlarr · Sonarr · Radarr · Unpackarr |
|
||||
| **Download clients** | VPN-isolated download clients (×2) | Gluetun sidecar |
|
||||
| **Gitea Runner** | CI/CD runner, ×2 (one for the in-cluster Gitea, one for the external instance) | Docker-in-Docker, rootless, shared RWX Longhorn volume |
|
||||
| **Arr Stack** | Media automation suite | Prowlarr · Sonarr · Radarr · Recyclarr · Unpackarr |
|
||||
| **Download clients** | VPN-isolated + clearnet qBittorrent pair, run side by side | Gluetun sidecar (ProtonVPN) on the VPN-isolated one |
|
||||
| **Jellyfin** | Media server, CPU-only transcoding | Docker |
|
||||
| **LiteLLM** | OpenAI-compatible LLM proxy — routes to Gemini, DeepSeek, and other providers | CloudNativePG |
|
||||
| **agentmemory** | Persistent memory server for AI agents | Longhorn PVC · private Gitea registry |
|
||||
| **LiteLLM** | OpenAI-compatible LLM proxy — routes to Gemini 2.5 Flash and DeepSeek | CloudNativePG |
|
||||
| **agentmemory** | Persistent memory server for AI agents — consolidation, graph extraction, reflection | Longhorn PVC · private Gitea registry · local embeddings |
|
||||
| **athome** | Personal homepage | – |
|
||||
|
||||
---
|
||||
@@ -163,7 +165,9 @@ ansible-homelab/
|
||||
│ ├── docker_host/ # Docker + Compose services
|
||||
│ ├── proxmox/ # Proxmox node setup
|
||||
│ ├── edge_vps/ # VPS: WireGuard, Traefik, Pangolin
|
||||
│ └── raspberry_pi/ # Raspberry Pi: Zigbee2MQTT + Mosquitto via Docker Compose
|
||||
│ ├── raspberry_pi/ # Raspberry Pi: Zigbee2MQTT + Mosquitto via Docker Compose
|
||||
│ ├── mikrotik/ # router config backup + static DHCP leases
|
||||
│ └── node_cleanup/ # systemd timer: prunes containerd images, vacuums journal logs
|
||||
└── playbooks/
|
||||
|
||||
argocd-homelab/
|
||||
|
||||
Reference in New Issue
Block a user