homelab-docs/docs/storage.md

Storage

Overview

Three storage tiers serve different workloads:

Tier	System	Access	Used by
Distributed block	Longhorn	RWO + RWX	All stateful K8s workloads
Relational	CloudNativePG	In-cluster Postgres	Immich
Network file	NFS (bare-metal)	NFS mount	Jellyfin media library

---

Longhorn

Longhorn provides distributed block storage across all 14 agent nodes. Each volume is replicated (default: 3 replicas) across different nodes.

• RWO (ReadWriteOnce) — used for most services (Vaultwarden, Paperless, etc.)
• RWX (ReadWriteMany) — used where multiple pods need shared access
• Volumes are backed by the local disk on each agent node (128 GB each)
• Longhorn manager runs as a DaemonSet; the CSI plugin integrates with the K8s storage layer
• Snapshots and backups are supported via the Longhorn UI

Control plane nodes (k3s-server-*) are tainted NoSchedule — Longhorn manager tolerates this taint and runs everywhere, but user workloads are pushed to agent nodes only.

---

CloudNativePG

The CNPG operator manages HA PostgreSQL clusters as first-class Kubernetes resources. Currently used by:

• Immich — primary database (photos, albums, users, ML embeddings)

CNPG handles streaming replication, failover, and scheduled backups. Data is stored on Longhorn PVCs.

---

NFS

A dedicated physical node (aya01) runs a bare-metal NFS server. This serves the media library to Jellyfin.

• Movies, TV shows, and music live on aya01
• docker-host11 (where Jellyfin runs) mounts the NFS share
• Separating media storage from the compute host means the Jellyfin VM can be rebuilt without touching the library
• NFS is not used for K8s workloads — Longhorn handles all PVC-backed storage

---

Secret Storage

Kubernetes secrets are managed with Sealed Secrets (Bitnami). The workflow:

1. Create a regular K8s Secret
2. Encrypt it with kubeseal using the cluster's public key → produces a SealedSecret
3. Commit the SealedSecret to Git — it is safe to store publicly
4. The in-cluster Sealed Secrets controller decrypts it into a regular Secret at apply time

Ansible secrets (VM credentials, API tokens) are encrypted with Ansible Vault and stored in vars/group_vars/*/secrets_*.yaml.