Compare commits
2 Commits
6eef96b302
...
7c33148688
| Author | SHA1 | Date | |
|---|---|---|---|
|
7c33148688 | ||
|
|
fe3f1749c5 |
129
README.md
129
README.md
@@ -3,90 +3,69 @@
|
||||
**I do not recommend this project being used for ones own infrastructure, as
|
||||
this project is heavily attuned to my specific host/network setup**
|
||||
|
||||
The Ansible Project to provision fresh Debian VMs for my Proxmox instances.
|
||||
This Ansible project automates the setup of a K3s Kubernetes cluster on Proxmox VE. It also includes playbooks for configuring Docker hosts, load balancers, and other services.
|
||||
|
||||
## Configuration
|
||||
## Repository Structure
|
||||
|
||||
The configuration of this project is done via files in the `./vars` directory.
|
||||
The inventory is composed of `.ini` files in the `./vars` directory. Each `.ini` file represents an inventory and can be used with the `-i` flag when running playbooks.
|
||||
The repository is organized into the following main directories:
|
||||
|
||||
The variables for the hosts and groups are defined in the `./vars/group_vars` directory. The structure of this directory is as follows:
|
||||
- `playbooks/`: Contains the main Ansible playbooks for different setup scenarios.
|
||||
- `roles/`: Contains the Ansible roles that are used by the playbooks.
|
||||
- `vars/`: Contains variable files, including group-specific variables.
|
||||
|
||||
```
|
||||
vars/
|
||||
├── group_vars/
|
||||
│ ├── all/
|
||||
│ │ ├── secrets.yml
|
||||
│ │ └── vars.yml
|
||||
│ ├── <group_name>/
|
||||
│ │ ├── *.yml
|
||||
├── docker.ini
|
||||
├── k3s.ini
|
||||
├── kubernetes.ini
|
||||
├── proxmox.ini
|
||||
└── vps.ini
|
||||
## Playbooks
|
||||
|
||||
The following playbooks are available:
|
||||
|
||||
- `proxmox.yml`: Provisions VMs and containers on Proxmox VE.
|
||||
- `k3s-servers.yml`: Sets up the K3s master nodes.
|
||||
- `k3s-agents.yml`: Sets up the K3s agent nodes.
|
||||
- `k3s-loadbalancer.yml`: Configures a load balancer for the K3s cluster.
|
||||
- `k3s-storage.yml`: Configures storage for the K3s cluster.
|
||||
- `docker.yml`: Sets up Docker hosts and their load balancer.
|
||||
- `docker-host.yml`: Configures the docker hosts.
|
||||
- `docker-lb.yml`: Configures a load balancer for Docker services.
|
||||
- `kubernetes_setup.yml`: A meta-playbook for setting up the entire Kubernetes cluster.
|
||||
|
||||
## Roles
|
||||
|
||||
The following roles are defined:
|
||||
|
||||
- `common`: Common configuration tasks for all nodes.
|
||||
- `proxmox`: Manages Proxmox VE, including VM and container creation.
|
||||
- `k3s_server`: Installs and configures K3s master nodes.
|
||||
- `k3s_agent`: Installs and configures K3s agent nodes.
|
||||
- `k3s_loadbalancer`: Configures an Nginx-based load balancer for the K3s cluster.
|
||||
- `k3s_storage`: Configures storage solutions for Kubernetes.
|
||||
- `docker_host`: Installs and configures Docker.
|
||||
- `kubernetes_argocd`: Deploys Argo CD to the Kubernetes cluster.
|
||||
- `node_exporter`: Installs the Prometheus Node Exporter for monitoring.
|
||||
- `reverse_proxy`: Configures a Caddy-based reverse proxy.
|
||||
|
||||
## Usage
|
||||
|
||||
1. **Install dependencies:**
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
ansible-galaxy install -r requirements.yml
|
||||
```
|
||||
|
||||
The `all` group contains variables that are common to all hosts. Each other directory in `group_vars` corresponds to a group defined in the inventory files and contains variables specific to that group.
|
||||
2. **Configure variables:**
|
||||
|
||||
## Run Playbook
|
||||
- Create an inventory file (e.g., `vars/k3s.ini`).
|
||||
- Adjust variables in `vars/group_vars/` to match your environment.
|
||||
|
||||
To run a playbook, you need to specify the inventory file and the playbook file. For example, to run the `k3s-servers.yml` playbook with the `k3s.ini` inventory, you can use the following command:
|
||||
3. **Run playbooks:**
|
||||
|
||||
```sh
|
||||
ansible-playbook -i vars/k3s.ini playbooks/k3s-servers.yml
|
||||
```bash
|
||||
# To provision VMs on Proxmox
|
||||
ansible-playbook -i vars/proxmox.ini playbooks/proxmox.yml
|
||||
|
||||
# To set up the K3s cluster
|
||||
ansible-playbook -i vars/k3s.ini playbooks/kubernetes_setup.yml
|
||||
```
|
||||
|
||||
## After successful k3s installation
|
||||
## Disclaimer
|
||||
|
||||
To access our Kubernetes cluster from our host machine to work on it via
|
||||
flux and such we need to manually copy a k3s config from one of our server nodes to our host machine.
|
||||
Then we need to install `kubectl` on our host machine and optionally `kubectx` if we're already
|
||||
managing other Kubernetes instances.
|
||||
Then we replace the localhost address inside of the config with the IP of our load balancer.
|
||||
Finally we'll need to set the KUBECONFIG variable.
|
||||
|
||||
```sh
|
||||
mkdir ~/.kube/
|
||||
scp k3s-server00:/etc/rancher/k3s/k3s.yaml ~/.kube/config
|
||||
chown $USER ~/.kube/config
|
||||
sed -i "s/127.0.0.1/192.168.20.22/" ~/.kube/config
|
||||
export KUBECONFIG=~/.kube/config
|
||||
```
|
||||
|
||||
Install flux and continue in the flux repository.
|
||||
|
||||
## Longhorn Nodes
|
||||
|
||||
To create longhorn nodes from existing kubernetes nodes we want to increase
|
||||
their storage capacity. Since we're using VMs for our k3s nodes we can
|
||||
resize the root-disk of the VMs in the proxmox GUI.
|
||||
|
||||
Then we have to resize the partitions inside of the VM so the root partition
|
||||
uses the newly available space.
|
||||
When we have LVM-based root partition we can do the following:
|
||||
|
||||
```sh
|
||||
# Create a new partition from the free space.
|
||||
sudo fdisk /dev/sda
|
||||
# echo "n\n\n\n\n\nw\n"
|
||||
# n > 5x\n > w > \n
|
||||
# Create a LVM volume on the new partition
|
||||
sudo pvcreate /dev/sda3
|
||||
sudo vgextend k3s-vg /dev/sda3
|
||||
# Use the newly available storage in the root volume
|
||||
sudo lvresize -l +100%FREE -r /dev/k3s-vg/root
|
||||
```
|
||||
|
||||
## Cloud Init VMs
|
||||
|
||||
```sh
|
||||
# On Hypervisor Host
|
||||
qm resize <vmid> scsi0 +32G
|
||||
# On VM
|
||||
sudo fdisk -l /dev/sda # To check
|
||||
echo 1 | sudo tee /sys/class/block/sda/device/rescan
|
||||
sudo fdisk -l /dev/sda # To check
|
||||
# sudo apt-get install cloud-guest-utils
|
||||
sudo growpart /dev/sda 1
|
||||
```
|
||||
This project is highly customized for the author's specific environment. Using it without modification is not recommended.
|
||||
|
||||
69
blog.md
Normal file
69
blog.md
Normal file
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: "Automating My Homelab: From Bare Metal to Kubernetes with Ansible"
|
||||
date: 2025-07-27
|
||||
author: "TuDatTr"
|
||||
tags: ["Ansible", "Proxmox", "Kubernetes", "K3s", "IaC", "Homelab"]
|
||||
---
|
||||
|
||||
## The Homelab Dream: Repeatable, Automated, and Powerful
|
||||
|
||||
For many tech enthusiasts, a homelab is a playground for learning, experimenting, and self-hosting services. But as the complexity grows, so does the management overhead. Manually setting up virtual machines, configuring networks, and deploying applications becomes a tedious and error-prone process. My journey into building the ultimate homelab led me to a powerful solution: Infrastructure as Code (IaC) with Ansible.
|
||||
|
||||
This blog post walks you through my Ansible project, which automates the entire lifecycle of my homelab—from provisioning VMs on Proxmox to deploying a production-ready K3s Kubernetes cluster.
|
||||
|
||||
## Why Ansible?
|
||||
|
||||
When I decided to automate my infrastructure, I considered several tools. I chose Ansible for its simplicity, agentless architecture, and gentle learning curve. Writing playbooks in YAML felt declarative and intuitive, and the vast collection of community-supported modules meant I wouldn't have to reinvent the wheel.
|
||||
|
||||
## The Architecture: A Multi-Layered Approach
|
||||
|
||||
My Ansible project is designed to be modular and scalable, with a clear separation of concerns. It's built around a collection of roles, each responsible for a specific component of the infrastructure.
|
||||
|
||||
### Layer 1: Proxmox Provisioning
|
||||
|
||||
The foundation of my homelab is Proxmox VE. The `proxmox` role is the first step in the automation pipeline. It handles:
|
||||
|
||||
- **VM and Container Creation:** Using a simple YAML definition in my `vars` files, I can specify the number of VMs and containers to create, their resources (CPU, memory, disk), and their base operating system images.
|
||||
- **Cloud-Init Integration:** For VMs, I leverage Cloud-Init to perform initial setup, such as setting the hostname, creating users, and injecting SSH keys for Ansible to connect to.
|
||||
- **Hardware Passthrough:** The role also configures hardware passthrough for devices like Intel Quick Sync for video transcoding in my media server.
|
||||
|
||||
### Layer 2: The K3s Kubernetes Cluster
|
||||
|
||||
With the base VMs ready, the next step is to build the Kubernetes cluster. I chose K3s for its lightweight footprint and ease of installation. The setup is divided into several roles:
|
||||
|
||||
- `k3s_server`: This role bootstraps the first master node and then adds additional master nodes to create a highly available control plane.
|
||||
- `k3s_agent`: This role joins the worker nodes to the cluster.
|
||||
- `k3s_loadbalancer`: A dedicated VM running Nginx is set up to act as a load balancer for the K3s API server, ensuring a stable endpoint for `kubectl` and other clients.
|
||||
|
||||
### Layer 3: Applications and Services
|
||||
|
||||
Once the Kubernetes cluster is up and running, it's time to deploy applications. My project includes roles for:
|
||||
|
||||
- `docker_host`: For services that are better suited to run in a traditional Docker environment, this role sets up and configures Docker hosts.
|
||||
- `kubernetes_argocd`: I use Argo CD for GitOps-based continuous delivery. This role deploys Argo CD to the cluster and configures it to sync with my application repositories.
|
||||
- `reverse_proxy`: Caddy is my reverse proxy of choice, and this role automates its installation and configuration, including obtaining SSL certificates from Let's Encrypt.
|
||||
|
||||
## Putting It All Together: The Power of Playbooks
|
||||
|
||||
The playbooks in the `playbooks/` directory tie everything together. For example, the `kubernetes_setup.yml` playbook runs all the necessary roles in the correct order to bring up the entire Kubernetes cluster from scratch.
|
||||
|
||||
```yaml
|
||||
# playbooks/kubernetes_setup.yml
|
||||
---
|
||||
- name: Set up Kubernetes Cluster
|
||||
hosts: all
|
||||
gather_facts: true
|
||||
roles:
|
||||
- role: k3s_server
|
||||
- role: k3s_agent
|
||||
- role: k3s_loadbalancer
|
||||
- role: kubernetes_argocd
|
||||
```
|
||||
|
||||
## Final Thoughts and Future Plans
|
||||
|
||||
This Ansible project has transformed my homelab from a collection of manually configured machines into a fully automated and reproducible environment. I can now tear down and rebuild my entire infrastructure with a single command, which gives me the confidence to experiment without fear of breaking things.
|
||||
|
||||
While the project is highly tailored to my specific needs, I hope this overview provides some inspiration for your own automation journey. The principles of IaC and the power of tools like Ansible can be applied to any environment, big or small.
|
||||
|
||||
What's next? I plan to explore more advanced Kubernetes concepts, such as Cilium for networking and policy, and integrate more of my self-hosted services into the GitOps workflow with Argo CD. The homelab is never truly "finished," and that's what makes it so much fun.
|
||||
Reference in New Issue
Block a user