[Kubernetes] Install Kubernetes using Kubespray(v2.22.2) on VirtualBox

Kubespray? #

Kubespray는 Ansible 플레이북, 인벤토리, 프로비저닝 도구와 일반적인 운영체제, 쿠버네티스 클러스터의 설정 관리 작업에 대한 도메인 지식의 결합으로 만들어졌다. Kubespray는 아래와 같은 기능을 제공한다.
Kubespray 지원 사항
- 고가용성을 지닌 클러스터
- 구성 가능 (인스턴스를 위한 네트워크 플러그인 선택)
- 대부분의 인기있는 리눅스 배포판들에 대한 지원
  - Flatcar Container Linux by Kinvolk
  - Debian Bullseye, Buster, Jessie, Stretch
  - Ubuntu 16.04, 18.04, 20.04, 22.04
  - CentOS/RHEL 7, 8, 9
  - Fedora 35, 36
  - Fedora CoreOS
  - openSUSE Leap 15.x/Tumbleweed
  - Oracle Linux 7, 8, 9
  - Alma Linux 8, 9
  - Rocky Linux 8, 9
  - Kylin Linux Advanced Server V10
  - Amazon Linux 2
- 지속적인 통합 (CI) 테스트

Vagrant 설정 및 실행 #

Vagrant 설정 #

vagrant init

Vagrantfile 작성 #

require "yaml"  

CONFIG = YAML.load_file(File.join(File.dirname(__FILE__), "config.yaml"))

Vagrant.configure("2") do |config|
  # Use the same SSH key for all machines
  config.ssh.insert_key = false

  # masters
  CONFIG["masters"].each do |master|
    config.vm.define master["name"] do |cfg|
      cfg.vm.box = master["box"]
      cfg.vm.network "private_network", ip: master["ip"], virtualbox_intnet: true
      cfg.vm.hostname = master["hostname"]

      cfg.vm.provider "virtualbox" do |v|
        v.memory = master["memory"]
        v.cpus = master["cpu"]
        v.name = master["name"]
        v.customize ['modifyvm', :id, '--graphicscontroller', 'vmsvga']
        v.customize ['modifyvm', :id, '--hwvirtex', 'on']
      end
      cfg.vm.provision "shell", inline: <<-SCRIPT
        sed -i -e "s/PasswordAuthentication no/PasswordAuthentication yes/g" /etc/ssh/sshd _config
        systemctl restart sshd
      SCRIPT

      # set timezone & disable swap memory, ufw & enable ip forwarding
      cfg.vm.provision "shell", inline: <<-SCRIPT
        sudo apt-get update
        sudo timedatectl set-timezone "Asia/Seoul"
        sudo swapoff -a
        sudo sed -i "/swap/d" /etc/fstab
        sudo systemctl stop ufw
        sudo systemctl disable ufw
        sudo sed -i "s/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/" /etc/sysctl.conf
        sudo sysctl -p
      SCRIPT

      # install python
      cfg.vm.provision "shell", inline: <<-SCRIPT
        sudo apt install python3-pip python3-setuptools virtualenv -y
      SCRIPT
    end
  end
  
  # worker nodes
  CONFIG["workers"].each do |worker|
    config.vm.define worker["name"] do |cfg|
      cfg.vm.box = worker["box"]
      cfg.vm.network "private_network", ip: worker["ip"], virtualbox_intnet: true
      cfg.vm.hostname = worker["hostname"]
      
      cfg.vm.provider "virtualbox" do |v|
        v.memory = worker["memory"]
        v.cpus = worker["cpu"]
        v.name = worker["name"]
        v.customize ['modifyvm', :id, '--graphicscontroller', 'vmsvga']
        v.customize ['modifyvm', :id, '--hwvirtex', 'on']
      end
      cfg.vm.provision "shell", inline: <<-SCRIPT
        sed -i -e "s/PasswordAuthentication no/PasswordAuthentication yes/g" /etc/ssh/sshd_config
        systemctl restart sshd
      SCRIPT

      # set timezone & disable swap memory & ufw & enable ip forwarding
      cfg.vm.provision "shell", inline: <<-SCRIPT
        sudo apt-get update
        sudo timedatectl set-timezone "Asia/Seoul"
        sudo swapoff -a
        sudo sed -i "/swap/d" /etc/fstab
        sudo systemctl stop ufw
        sudo systemctl disable ufw
        sudo sed -i "s/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/" /etc/sysctl.conf
        sudo sysctl -p
      SCRIPT
    end
  end
end

config.yaml 작성 #

 1masters:
 2  - name: ks-master
 3    box: generic/ubuntu2004
 4    hostname: ks-master
 5    ip: 192.168.10.100
 6    memory: 4096
 7    cpu: 4
 8
 9workers:
10  - name: ks-worker-1
11    box: generic/ubuntu2004
12    hostname: ks-worker-1
13    ip: 192.168.10.210
14    memory: 4096
15    cpu: 4
16
17  - name: ks-worker-2
18    box: generic/ubuntu2004
19    hostname: ks-worker-2
20    ip: 192.168.10.220
21    memory: 4096
22    cpu: 4

Vagrant 실행 #

vagrant up

SSH 생성 및 설정 #

vagrant ssh ks-master

ssh-keygen -t rsa

ls -al .ssh/
cat .ssh/id_rsa.pub

ssh-copy-id vagrant@192.168.10.100
ssh-copy-id vagrant@192.168.10.210
ssh-copy-id vagrant@192.168.10.220

virtualenv --python=python3 venv

. venv/bin/activate

git clone https://github.com/kubernetes-sigs/kubespray
cd kubespray
git checkout v2.22.2

pip install -r requirements.txt

ansible --version

cp -rfp inventory/sample inventory/mycluster

declare -a IPS=(192.168.10.100 192.168.10.210 192.168.10.220)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

ansible all -m ping -i inventory/mycluster/hosts.yaml

vi inventory/mycluster/group_vars/k8s_cluster/addons.yml

ansible-playbook -i inventory/mycluster/hosts.yaml  --become --become-user=root cluster.yml

deactivate

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubectl get nodes
kubectl get componentstatus
kubectl get --raw='/readyz?verbose'

vi inventory/mycluster/hosts.yaml #

 1all:
 2  hosts:
 3    ks-master:
 4      ansible_host: 192.168.10.100
 5      ip: 192.168.10.100
 6      access_ip: 192.168.10.100
 7    ks-worker-1:
 8      ansible_host: 192.168.10.210
 9      ip: 192.168.10.210
10      access_ip: 192.168.10.210
11    ks-worker-2:
12      ansible_host: 192.168.10.220
13      ip: 192.168.10.220
14      access_ip: 192.168.10.220
15  children:
16    kube_control_plane:
17      hosts:
18        ks-master:
19    kube_node:
20      hosts:
21        ks-worker-1:
22        ks-worker-2:
23    etcd:
24      hosts:
25        ks-master:
26    k8s_cluster:
27      children:
28        kube_control_plane:
29        kube_node:
30    calico_rr:
31      hosts: {}