ansible/docs/runbooks/arr-cleanup/findings.md

arr-stack Downloads Cleanup — Investigation Findings

Storage Layout (aya01)

Device	FS	Size	Used	Mount
/dev/sdc3	btrfs	1.9T	177G (10%)	/ (system)
/dev/sda1	btrfs proxmox	2.8T	1.3T (48%)	/opt
/dev/sdd1	ext4	17T	15T (92%)	/mnt/hdd0
/dev/sde1	ext4	17T	15T (92%)	/mnt/hdd2
/dev/sdf1	ext4	17T	15T (92%)	/mnt/hdd1
mergerfs	fuse	49T	43T (92%)	/media

/media is a mergerfs union of hdd0 + hdd1 + hdd2. All three HDDs were at ~92% capacity before cleanup.

After cleanup (2026-04-23):

Device	Used	Avail	Use%
/dev/sdd1 (hdd0)	9.4T	6.2T	61%
/dev/sdf1 (hdd1)	9.3T	6.3T	60%
/dev/sde1 (hdd2)	7.8T	7.8T	51%
mergerfs /media	27T	21T	57%

~16T freed total (92% → 57% on the mergerfs pool).

/media Breakdown (before cleanup)

Directory	Size
downloads	22T
series	16T
movies	5T

Root Cause: No Hardlinks → All Imports Are Copies

Zero hardlinked files exist anywhere across all three HDDs. Confirmed by inspecting the Kubernetes manifests in argocd-homelab/services/arr-stack/ and by inode comparison of 1365 download/media file pairs (0 shared inodes found).

All three services mount the mergerfs /media/ path via NFS:

sonarr:      NFS 192.168.20.12:/media/downloads  → /downloads
             NFS 192.168.20.12:/media/series     → /tv
radarr:      NFS 192.168.20.12:/media/downloads  → /downloads
             NFS 192.168.20.12:/media/movies     → /movies
qbit:        NFS 192.168.20.12:/media/downloads  → /downloads

mergerfs does not support hardlinks across underlying filesystems. When qBit downloads to /media/downloads/sonarr/ (lands on e.g. hdd1) and Sonarr imports to /media/series/ (lands on e.g. hdd0), the hardlink attempt crosses a physical disk boundary → falls back to copy. Every import doubles the data.

Cleanup Performed (2026-04-23)

Three passes using the scripts in this directory:

Pass 1 — Orphans (not in Sonarr at all)

Script: cleanup-orphans.py

Deleted 49 entries totalling 461.6G — downloads with no matching Sonarr series and no series directory on disk. Includes Game of Thrones (all 8 seasons), Sex Education (all 4 seasons), Love Death & Robots (multiple duplicate copies), and various anime episode files.

111 entries were SKIPPED (series dir found on disk, needs manual review) — includes Bleach, House, Lucifer, You, Detective Conan episodes, What If, etc. See cleanup.log for full list.

Pass 2 — Confirmed-imported Sonarr downloads

Script: cleanup.py --arr sonarr

Deleted 1106 entries, 0 failed. These were downloads where Sonarr confirmed episodeFileCount > 0 AND the series directory was verified to exist on disk at the time of verify.py run.

Pass 3 — Confirmed-imported Radarr downloads

Script: cleanup.py --arr radarr

Deleted 259 entries, 0 failed. These were downloads where Radarr confirmed hasFile=True AND the file/directory path was verified to exist on disk.

Totals

Pass	Entries	Space
Orphans (cleanup-orphans.py)	49	~461G
Sonarr imports (cleanup.py)	1106	~12T (estimated)
Radarr imports (cleanup.py)	259	~4T (estimated)
Total	1414	~16T freed

All deletions logged to cleanup.log with UTC timestamp, size, title, path, outcome.

Verification Results (via API + disk path check)

API keys stored in ../sonarr.api.env and ../radarr.api.env. Access via kubectl -n arr-stack port-forward svc/sonarr 8989:8989 and svc/radarr 7878:7878.

Container path mappings:

• Sonarr /tv/ → /media/series/
• Radarr /movies/ → /media/movies/

	Safe to delete	Orphans (not in arr)	Keep
Radarr (289 items, ~5.2T)	265	25	0
Sonarr (1439 items, ~17T)	1106	333	0

"Safe to delete" = API confirms hasFile=True (Radarr) or episodeFileCount > 0 (Sonarr), AND the reported file/directory path was verified to exist on disk via SSH.

Radarr Orphans (25) — not matched in Radarr, not deleted

• Constantine (2005)
• Cowboy Bebop: Knockin' on Heaven's Door (2001)
• Les Misérables (2012)
• Pokémon Detective Pikachu (2019)
• Code Geass: Fukkatsu no Lelouch (2019)
• Eiga Go-Toubun no Hanayome (2022)
• Gisaengchung / Parasite (Korean title — matching failure)
• Dune: Part One (2021) — matching failure, is in Radarr
• Harry Potter (older/duplicate copies — matching failure)
• Porco Rosso / Kurenai no buta — matching failure
• Castle in the Sky / Laputa — matching failure
• Steins;Gate: The Movie — matching failure
• Project Silence / Talchul — matching failure
• Digimon: Frontier & Savers films
• One Piece films (several)
• Paripi Koumei movie
• Fantastic Four (2025) extra copies (3)
• JJK DCP trailer file

6 Radarr "path mismatch" entries (all confirmed safe, deleted)

Flagged due to path comparison artifacts, manually verified on disk:

• Star Wars Episode IV/V/VI/IX — each is a separate Radarr entry; all directories exist
• WALL·E — · middle-dot character caused comparison failure; file exists

Pending Decisions

Bleach USBD Remux TL (1.8T)

/media/downloads/sonarr/Bleach USBD Remux TL — full lossless Bluray remux S00–S16 (-ZR- group). Currently in SKIPPED (series dir /media/series/Bleach (2004) {imdb-tt0434665}/ exists, 310G imported). Most seasons were imported from x265 Bluray packs (-iVy group) rather than from this remux. S11 has no imported content at all. S13, S14 partially imported. Decision: keep (for quality imports once disk freed) or delete (free 1.8T, accept x265 quality). See memory file for full per-season breakdown.

SKIPPED downloads (111 Sonarr entries)

Downloads where the series directory exists on disk but the series is not currently in Sonarr. Likely removed series (House, Lucifer, You, Black Clover, etc.) or ongoing shows with stale episodes. These need manual review — series may have been intentionally removed from Sonarr.

Fix (not applied — future reference)

Mount per-HDD NFS paths instead of the mergerfs path, so downloads and media share the same physical filesystem and hardlinks work:

# sonarr/radarr/qtun deployments — change NFS path from:
path: /media/downloads   →   path: /mnt/hdd0/downloads
path: /media/series      →   path: /mnt/hdd0/series
path: /media/movies      →   path: /mnt/hdd0/movies

Jellyfin/Plex continue reading from /media/ (mergerfs). New imports hardlink within hdd0.