92 lines
3.1 KiB
Markdown
92 lines
3.1 KiB
Markdown
# TuDatTr IaC
|
|
|
|
**I do not recommend this project being used for ones own infrastructure, as
|
|
this project is heavily attuned to my specific host/network setup**
|
|
|
|
The Ansible Project to provision fresh Debian VMs for my Proxmox instances.
|
|
|
|
## Configuration
|
|
|
|
The configuration of this project is done via files in the `./vars` directory.
|
|
The inventory is composed of `.ini` files in the `./vars` directory. Each `.ini` file represents an inventory and can be used with the `-i` flag when running playbooks.
|
|
|
|
The variables for the hosts and groups are defined in the `./vars/group_vars` directory. The structure of this directory is as follows:
|
|
|
|
```
|
|
vars/
|
|
├── group_vars/
|
|
│ ├── all/
|
|
│ │ ├── secrets.yml
|
|
│ │ └── vars.yml
|
|
│ ├── <group_name>/
|
|
│ │ ├── *.yml
|
|
├── docker.ini
|
|
├── k3s.ini
|
|
├── kubernetes.ini
|
|
├── proxmox.ini
|
|
└── vps.ini
|
|
```
|
|
|
|
The `all` group contains variables that are common to all hosts. Each other directory in `group_vars` corresponds to a group defined in the inventory files and contains variables specific to that group.
|
|
|
|
## Run Playbook
|
|
|
|
To run a playbook, you need to specify the inventory file and the playbook file. For example, to run the `k3s-servers.yml` playbook with the `k3s.ini` inventory, you can use the following command:
|
|
|
|
```sh
|
|
ansible-playbook -i vars/k3s.ini playbooks/k3s-servers.yml
|
|
```
|
|
|
|
## After successful k3s installation
|
|
|
|
To access our Kubernetes cluster from our host machine to work on it via
|
|
flux and such we need to manually copy a k3s config from one of our server nodes to our host machine.
|
|
Then we need to install `kubectl` on our host machine and optionally `kubectx` if we're already
|
|
managing other Kubernetes instances.
|
|
Then we replace the localhost address inside of the config with the IP of our load balancer.
|
|
Finally we'll need to set the KUBECONFIG variable.
|
|
|
|
```sh
|
|
mkdir ~/.kube/
|
|
scp k3s-server00:/etc/rancher/k3s/k3s.yaml ~/.kube/config
|
|
chown $USER ~/.kube/config
|
|
sed -i "s/127.0.0.1/192.168.20.22/" ~/.kube/config
|
|
export KUBECONFIG=~/.kube/config
|
|
```
|
|
|
|
Install flux and continue in the flux repository.
|
|
|
|
## Longhorn Nodes
|
|
|
|
To create longhorn nodes from existing kubernetes nodes we want to increase
|
|
their storage capacity. Since we're using VMs for our k3s nodes we can
|
|
resize the root-disk of the VMs in the proxmox GUI.
|
|
|
|
Then we have to resize the partitions inside of the VM so the root partition
|
|
uses the newly available space.
|
|
When we have LVM-based root partition we can do the following:
|
|
|
|
```sh
|
|
# Create a new partition from the free space.
|
|
sudo fdisk /dev/sda
|
|
# echo "n\n\n\n\n\nw\n"
|
|
# n > 5x\n > w > \n
|
|
# Create a LVM volume on the new partition
|
|
sudo pvcreate /dev/sda3
|
|
sudo vgextend k3s-vg /dev/sda3
|
|
# Use the newly available storage in the root volume
|
|
sudo lvresize -l +100%FREE -r /dev/k3s-vg/root
|
|
```
|
|
|
|
## Cloud Init VMs
|
|
|
|
```sh
|
|
# On Hypervisor Host
|
|
qm resize <vmid> scsi0 +32G
|
|
# On VM
|
|
sudo fdisk -l /dev/sda # To check
|
|
echo 1 | sudo tee /sys/class/block/sda/device/rescan
|
|
sudo fdisk -l /dev/sda # To check
|
|
# sudo apt-get install cloud-guest-utils
|
|
sudo growpart /dev/sda 1
|
|
``` |