# Storage

## Overview

Three storage tiers serve different workloads:

| Tier | System | Access | Used by |
|------|--------|--------|---------|
| Distributed block | Longhorn | RWO + RWX | All stateful K8s workloads |
| Relational | CloudNativePG | In-cluster Postgres | Immich |
| Network file | NFS (bare-metal) | NFS mount | Jellyfin media library |

---

## Longhorn

Longhorn provides distributed block storage across all 14 agent nodes. Each volume is replicated (default: 3 replicas) across different nodes.

- **RWO** (ReadWriteOnce) — used for most services (Vaultwarden, Paperless, etc.)
- **RWX** (ReadWriteMany) — used where multiple pods need shared access
- Volumes are backed by the local disk on each agent node (128 GB each)
- Longhorn manager runs as a DaemonSet; the CSI plugin integrates with the K8s storage layer
- Snapshots and backups are supported via the Longhorn UI

Control plane nodes (`k3s-server-*`) are tainted `NoSchedule` — Longhorn manager tolerates this taint and runs everywhere, but user workloads are pushed to agent nodes only.

---

## CloudNativePG

The CNPG operator manages HA PostgreSQL clusters as first-class Kubernetes resources. Currently used by:

- **Immich** — primary database (photos, albums, users, ML embeddings)

CNPG handles streaming replication, failover, and scheduled backups. Data is stored on Longhorn PVCs.

---

## NFS

A dedicated physical node (`aya01`) runs a bare-metal NFS server. This serves the media library to Jellyfin.

- Movies, TV shows, and music live on `aya01`
- `docker-host11` (where Jellyfin runs) mounts the NFS share
- Separating media storage from the compute host means the Jellyfin VM can be rebuilt without touching the library
- NFS is not used for K8s workloads — Longhorn handles all PVC-backed storage

---

## Secret Storage

Kubernetes secrets are managed with **Sealed Secrets** (Bitnami). The workflow:

1. Create a regular K8s `Secret`
2. Encrypt it with `kubeseal` using the cluster's public key → produces a `SealedSecret`
3. Commit the `SealedSecret` to Git — it is safe to store publicly
4. The in-cluster Sealed Secrets controller decrypts it into a regular `Secret` at apply time

Ansible secrets (VM credentials, API tokens) are encrypted with **Ansible Vault** and stored in `vars/group_vars/*/secrets_*.yaml`.