44 lines
1.9 KiB
Markdown
44 lines
1.9 KiB
Markdown
# Storage
|
|
|
|
## Overview
|
|
|
|
Three storage tiers, each doing a different job:
|
|
|
|
| Tier | System | Access | Used by |
|
|
|------|--------|--------|---------|
|
|
| Distributed block | Longhorn | RWO + RWX | All stateful K8s workloads |
|
|
| Relational | CloudNativePG | In-cluster Postgres | Immich |
|
|
| Network file | NFS (bare-metal) | NFS mount | Jellyfin media library |
|
|
|
|
---
|
|
|
|
## Longhorn
|
|
|
|
Longhorn gives distributed block storage across all 14 agent nodes. Each volume is replicated (default: 3 replicas) across different nodes, using the local disk on each agent (128 GB each).
|
|
|
|
RWO (ReadWriteOnce) covers most services. RWX (ReadWriteMany) is used where multiple pods need access to the same volume. Snapshots and backups are available through the Longhorn UI.
|
|
|
|
Control plane nodes are tainted `NoSchedule` — Longhorn manager tolerates this and runs everywhere, but user workloads stay on agent nodes.
|
|
|
|
---
|
|
|
|
## CloudNativePG
|
|
|
|
CloudNativePG manages HA PostgreSQL clusters as Kubernetes resources. Immich uses it for its primary database (photos, albums, users, ML embeddings). CNPG handles streaming replication, failover, and scheduled backups, with data stored on Longhorn PVCs.
|
|
|
|
---
|
|
|
|
## NFS
|
|
|
|
`aya01` is a dedicated bare-metal NFS server. Jellyfin mounts the share from `docker-host11` to access movies, TV shows, and music. Keeping the media library on a separate host means the Jellyfin VM can be rebuilt without touching the data.
|
|
|
|
NFS is not used for K8s workloads — Longhorn handles all PVC-backed storage.
|
|
|
|
---
|
|
|
|
## Secrets
|
|
|
|
Kubernetes secrets go through Sealed Secrets (Bitnami). The workflow: create a regular `Secret`, encrypt it with `kubeseal` using the cluster's public key into a `SealedSecret`, then commit that to Git. Only the in-cluster controller can decrypt it.
|
|
|
|
Ansible secrets (VM credentials, API tokens) are encrypted with Ansible Vault and live in `vars/group_vars/*/secrets_*.yaml`.
|