KyungRyeol Yoon KyungRyeol Yoon
Kubernetes · Cluster ·

[Kubernetes] Install Kubernetes using Kubekey(v3.0.7) Artifact on VirtualBox

· 10분 읽기

offline 설치 위한 artifact 참고

script 다운로드 #

1curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.7 sh -

artifact-3.0.7.yaml 작성 #

  1apiVersion: kubekey.kubesphere.io/v1alpha2
  2kind: Manifest
  3metadata:
  4  name: artifact-v3.0.7
  5spec:
  6  arches:
  7  - amd64
  8  operatingSystems:
  9  - arch: amd64
 10    type: linux
 11    id: ubuntu
 12    version: "20.04"
 13    osImage: Ubuntu 20.04.4 LTS
 14    repository:
 15      iso:
 16        localPath: ""
 17        url: "https://github.com/kubesphere/kubekey/releases/download/v3.0.7/ubuntu-20.04-debs-amd64.iso"
 18  kubernetesDistributions:
 19  - type: kubernetes
 20    version: v1.24.9
 21  components:
 22    helm:
 23      version: v3.9.0
 24    cni:
 25      version: v0.9.1
 26    etcd:
 27      version: v3.4.13
 28    calicoctl:
 29      version: v3.23.2
 30    ## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list.
 31    ## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained.
 32    containerRuntimes:
 33    - type: docker
 34      version: 20.10.8
 35    - type: containerd
 36      version: 1.6.4
 37    crictl:
 38      version: v1.24.0
 39    docker-registry:
 40      version: "2"
 41    harbor:
 42      version: v2.5.3
 43    docker-compose:
 44      version: v2.2.2
 45  images:
 46  - docker.io/kubesphere/kube-apiserver:v1.27.2
 47  - docker.io/kubesphere/kube-apiserver:v1.26.5
 48  - docker.io/kubesphere/kube-apiserver:v1.25.10
 49  - docker.io/kubesphere/kube-apiserver:v1.24.9
 50  - docker.io/kubesphere/kube-controller-manager:v1.27.2
 51  - docker.io/kubesphere/kube-controller-manager:v1.26.5
 52  - docker.io/kubesphere/kube-controller-manager:v1.25.10
 53  - docker.io/kubesphere/kube-controller-manager:v1.24.9
 54  - docker.io/kubesphere/kube-scheduler:v1.27.2
 55  - docker.io/kubesphere/kube-scheduler:v1.26.5
 56  - docker.io/kubesphere/kube-scheduler:v1.25.10
 57  - docker.io/kubesphere/kube-scheduler:v1.24.9
 58  - docker.io/kubesphere/kube-proxy:v1.27.2
 59  - docker.io/kubesphere/kube-proxy:v1.26.5
 60  - docker.io/kubesphere/kube-proxy:v1.25.10
 61  - docker.io/kubesphere/kube-proxy:v1.24.9
 62  - docker.io/kubesphere/pause:3.8
 63  - docker.io/kubesphere/pause:3.7
 64  - docker.io/kubesphere/pause:3.6
 65  - docker.io/kubesphere/pause:3.5
 66  - docker.io/kubesphere/pause:3.4.1
 67  - docker.io/coredns/coredns:1.8.6
 68  - docker.io/coredns/coredns:1.8.0
 69  - docker.io/calico/cni:v3.23.2
 70  - docker.io/calico/kube-controllers:v3.23.2
 71  - docker.io/calico/node:v3.23.2
 72  - docker.io/calico/pod2daemon-flexvol:v3.23.2
 73  - docker.io/calico/typha:v3.23.2
 74  - docker.io/kubesphere/flannel:v0.12.0
 75  - docker.io/openebs/provisioner-localpv:3.3.0
 76  - docker.io/openebs/linux-utils:3.3.0
 77  - docker.io/library/haproxy:2.3
 78  - docker.io/kubesphere/nfs-subdir-external-provisioner:v4.0.2
 79  - docker.io/kubesphere/k8s-dns-node-cache:1.15.12
 80  # https://github.com/kubesphere/ks-installer/releases/download/v3.3.2/images-list.txt
 81  ##kubesphere-images
 82  - docker.io/kubesphere/ks-installer:v3.4.1
 83  - docker.io/kubesphere/ks-installer:v3.3.2
 84  - docker.io/kubesphere/ks-apiserver:v3.4.1
 85  - docker.io/kubesphere/ks-apiserver:v3.3.2
 86  - docker.io/kubesphere/ks-console:v3.4.1
 87  - docker.io/kubesphere/ks-console:v3.3.2
 88  - docker.io/kubesphere/ks-controller-manager:v3.4.1
 89  - docker.io/kubesphere/ks-controller-manager:v3.3.2
 90  - docker.io/kubesphere/kubectl:v1.22.0
 91  - docker.io/kubesphere/kubectl:v1.20.0
 92  - docker.io/kubesphere/kubefed:v0.8.1
 93  - docker.io/kubesphere/tower:v0.2.1
 94  - docker.io/kubesphere/tower:v0.2.0
 95  - docker.io/minio/minio:RELEASE.2019-08-07T01-59-21Z
 96  - docker.io/minio/mc:RELEASE.2019-08-07T23-14-43Z
 97  - docker.io/csiplugin/snapshot-controller:v4.0.0
 98  - docker.io/kubesphere/nginx-ingress-controller:v1.3.1
 99  - docker.io/kubesphere/nginx-ingress-controller:v1.1.0
100  - docker.io/mirrorgooglecontainers/defaultbackend-amd64:1.4
101  - docker.io/kubesphere/metrics-server:v0.4.2
102  - docker.io/library/redis:5.0.14-alpine
103  - docker.io/library/haproxy:2.0.25-alpine
104  - docker.io/library/alpine:3.14
105  - docker.io/osixia/openldap:1.3.0
106  - docker.io/kubesphere/netshoot:v1.0
107  ##kubeedge-images
108  - docker.io/kubeedge/cloudcore:v1.13.0
109  - docker.io/kubeedge/cloudcore:v1.9.2
110  - docker.io/kubesphere/iptables-manager:v1.13.0
111  - docker.io/kubeedge/iptables-manager:v1.9.2
112  - docker.io/kubesphere/edgeservice:v0.3.0
113  - docker.io/kubesphere/edgeservice:v0.2.0
114  ##gatekeeper-images
115  - docker.io/openpolicyagent/gatekeeper:v3.5.2
116  ##openpitrix-images
117  - docker.io/kubesphere/openpitrix-jobs:v3.3.2
118  ##kubesphere-devops-images
119  - docker.io/kubesphere/devops-apiserver:ks-v3.4.1
120  - docker.io/kubesphere/devops-apiserver:ks-v3.3.2
121  - docker.io/kubesphere/devops-controller:ks-v3.4.1
122  - docker.io/kubesphere/devops-controller:ks-v3.3.2
123  - docker.io/kubesphere/devops-tools:ks-v3.4.1
124  - docker.io/kubesphere/devops-tools:ks-v3.3.2
125  - docker.io/kubesphere/ks-jenkins:v3.4.0-2.319.3-1
126  - docker.io/kubesphere/ks-jenkins:v3.3.0-2.319.1
127  - docker.io/jenkins/inbound-agent:4.10-2
128  - docker.io/kubesphere/builder-base:v3.2.2
129  - docker.io/kubesphere/builder-nodejs:v3.2.0
130  - docker.io/kubesphere/builder-maven:v3.2.1-jdk11
131  - docker.io/kubesphere/builder-maven:v3.2.0
132  - docker.io/kubesphere/builder-python:v3.2.0
133  - docker.io/kubesphere/builder-go:v3.2.2-1.18
134  - docker.io/kubesphere/builder-go:v3.2.2-1.17
135  - docker.io/kubesphere/builder-go:v3.2.2-1.16
136  - docker.io/kubesphere/builder-go:v3.2.0
137  - docker.io/kubesphere/builder-base:v3.2.2-podman
138  - docker.io/kubesphere/builder-nodejs:v3.2.0-podman
139  - docker.io/kubesphere/builder-maven:v3.2.1-jdk11-podman
140  - docker.io/kubesphere/builder-maven:v3.2.0-podman
141  - docker.io/kubesphere/builder-python:v3.2.0-podman
142  - docker.io/kubesphere/builder-go:v3.2.0-podman
143  - docker.io/kubesphere/builder-go:v3.2.2-1.18-podman
144  - docker.io/kubesphere/builder-go:v3.2.2-1.17-podman
145  - docker.io/kubesphere/builder-go:v3.2.2-1.16-podman
146  - docker.io/kubesphere/s2ioperator:v3.2.1
147  - docker.io/kubesphere/s2irun:v3.2.0
148  - docker.io/kubesphere/s2i-binary:v3.2.0
149  - docker.io/kubesphere/tomcat85-java11-centos7:v3.2.0
150  - docker.io/kubesphere/tomcat85-java11-runtime:v3.2.0
151  - docker.io/kubesphere/tomcat85-java8-centos7:v3.2.0
152  - docker.io/kubesphere/tomcat85-java8-runtime:v3.2.0
153  - docker.io/kubesphere/java-11-centos7:v3.2.0
154  - docker.io/kubesphere/java-11-runtime:v3.2.0
155  - docker.io/kubesphere/java-8-centos7:v3.2.0
156  - docker.io/kubesphere/java-8-runtime:v3.2.0
157  - docker.io/kubesphere/nodejs-8-centos7:v3.2.0
158  - docker.io/kubesphere/nodejs-6-centos7:v3.2.0
159  - docker.io/kubesphere/nodejs-4-centos7:v3.2.0
160  - docker.io/kubesphere/python-36-centos7:v3.2.0
161  - docker.io/kubesphere/python-35-centos7:v3.2.0
162  - docker.io/kubesphere/python-34-centos7:v3.2.0
163  - docker.io/kubesphere/python-27-centos7:v3.2.0
164  - quay.io/argoproj/argocd:v2.3.3
165  - quay.io/argoproj/argocd-applicationset:v0.4.1
166  - ghcr.io/dexidp/dex:v2.30.2
167  - docker.io/library/redis:6.2.6-alpine
168  ##kubesphere-monitoring-images
169  - docker.io/jimmidyson/configmap-reload:v0.7.1
170  - docker.io/jimmidyson/configmap-reload:v0.5.0
171  - docker.io/prom/prometheus:v2.39.1
172  - docker.io/prom/prometheus:v2.34.0
173  - docker.io/kubesphere/prometheus-config-reloader:v0.55.1
174  - docker.io/kubesphere/prometheus-operator:v0.55.1
175  - docker.io/kubesphere/kube-rbac-proxy:v0.11.0
176  - docker.io/kubesphere/kube-state-metrics:v2.6.0
177  - docker.io/kubesphere/kube-state-metrics:v2.5.0
178  - docker.io/prom/node-exporter:v1.3.1
179  - docker.io/prom/alertmanager:v0.23.0
180  - docker.io/thanosio/thanos:v0.31.0
181  - docker.io/thanosio/thanos:v0.25.2
182  - docker.io/grafana/grafana:8.3.3
183  - docker.io/kubesphere/kube-rbac-proxy:v0.11.0
184  - docker.io/kubesphere/kube-rbac-proxy:v0.8.0
185  - docker.io/kubesphere/notification-manager-operator:v2.3.0
186  - docker.io/kubesphere/notification-manager-operator:v1.4.0
187  - docker.io/kubesphere/notification-manager:v2.3.0
188  - docker.io/kubesphere/notification-manager:v1.4.0
189  - docker.io/kubesphere/notification-tenant-sidecar:v3.2.0
190  ##kubesphere-logging-images
191  - docker.io/kubesphere/elasticsearch-curator:v5.7.6
192  - docker.io/kubesphere/opensearch-curator:v0.0.5
193  - docker.io/kubesphere/elasticsearch-oss:6.8.22
194  - docker.io/opensearchproject/opensearch:2.6.0
195  - docker.io/opensearchproject/opensearch-dashboards:2.6.0
196  - docker.io/kubesphere/fluentbit-operator:v0.14.0
197  - docker.io/kubesphere/fluentbit-operator:v0.13.0
198  - docker.io/library/docker:19.03
199  - docker.io/kubesphere/fluent-bit:v1.9.4
200  - docker.io/kubesphere/fluent-bit:v1.8.11
201  - docker.io/kubesphere/log-sidecar-injector:v1.2.0
202  - docker.io/elastic/filebeat:6.7.0
203  - docker.io/kubesphere/kube-events-operator:v0.6.0
204  - docker.io/kubesphere/kube-events-operator:v0.4.0
205  - docker.io/kubesphere/kube-events-exporter:v0.6.0
206  - docker.io/kubesphere/kube-events-exporter:v0.4.0
207  - docker.io/kubesphere/kube-events-ruler:v0.6.0
208  - docker.io/kubesphere/kube-events-ruler:v0.4.0
209  - docker.io/kubesphere/kube-auditing-operator:v0.2.0
210  - docker.io/kubesphere/kube-auditing-webhook:v0.2.0
211  ##istio-images
212  - docker.io/istio/pilot:1.14.6
213  - docker.io/istio/pilot:1.11.1
214  - docker.io/istio/proxyv2:1.14.6
215  - docker.io/istio/proxyv2:1.11.1
216  - docker.io/jaegertracing/jaeger-operator:1.29
217  - docker.io/jaegertracing/jaeger-operator:1.27
218  - docker.io/jaegertracing/jaeger-agent:1.29
219  - docker.io/jaegertracing/jaeger-agent:1.27
220  - docker.io/jaegertracing/jaeger-collector:1.29
221  - docker.io/jaegertracing/jaeger-collector:1.27
222  - docker.io/jaegertracing/jaeger-query:1.29
223  - docker.io/jaegertracing/jaeger-query:1.27
224  - docker.io/jaegertracing/jaeger-es-index-cleaner:1.29
225  - docker.io/jaegertracing/jaeger-es-index-cleaner:1.27
226  - docker.io/kubesphere/kiali-operator:v1.50.1
227  - docker.io/kubesphere/kiali-operator:v1.38.1
228  - docker.io/kubesphere/kiali:v1.50
229  - docker.io/kubesphere/kiali:v1.38
230  # ##example-images
231  # - docker.io/library/busybox:1.31.1
232  # - docker.io/library/nginx:1.14-alpine
233  # - docker.io/joosthofman/wget:1.0
234  # - docker.io/nginxdemos/hello:plain-text
235  # - docker.io/library/wordpress:4.8-apache
236  # - docker.io/mirrorgooglecontainers/hpa-example:latest
237  # - docker.io/fluent/fluentd:v1.4.2-2.0
238  # - docker.io/library/perl:latest
239  # - docker.io/kubesphere/examples-bookinfo-productpage-v1:1.16.2
240  # - docker.io/kubesphere/examples-bookinfo-reviews-v1:1.16.2
241  # - docker.io/kubesphere/examples-bookinfo-reviews-v2:1.16.2
242  # - docker.io/kubesphere/examples-bookinfo-details-v1:1.16.2
243  # - docker.io/kubesphere/examples-bookinfo-ratings-v1:1.16.3
244  # ##weave-scope-images
245  # - docker.io/weaveworks/scope:1.13.0
246  registry:
247    auths:
248      "docker.io":
249        username: "username"
250        password: "password"

Export Artifact #

1sudo ./kk artifact export -m artifact-3.0.7.yaml -o artifact-3.0.7.tar.gz

Cluster 설치를 위한 config 파일 생성 및 작성 #

  • config 파일 생성

    1sudo ./kk create config --with-kubesphere v3.3.2 --with-kubernetes v1.24.9 -f config-sample.yaml

  • config 파일 작성

      1apiVersion: kubekey.kubesphere.io/v1alpha2
  2kind: Cluster
  3metadata:
  4  name: sample
  5spec:
  6  hosts:
  7  - {name: manage-master, address: 192.168.10.100, internalAddress: 192.168.10.100, user: root, password: vagrant}
  8  - {name: manage-worker-1, address: 192.168.10.110, internalAddress: 192.168.10.110, user: root, password: vagrant}
  9  - {name: manage-worker-2, address: 192.168.10.120, internalAddress: 192.168.10.120, user: root, password: vagrant}
 10  roleGroups:
 11    etcd:
 12    - manage-master
 13    control-plane:
 14    - manage-master
 15    worker:
 16    - manage-worker-1
 17    - manage-worker-2
 18    registry:
 19    - manage-worker-1
 20  controlPlaneEndpoint:
 21    ## Internal loadbalancer for apiservers
 22    # internalLoadbalancer: haproxy
 23
 24    #domain: lb.kubesphere.local
 25    domain: 192.168.10.100
 26    address: ""
 27    port: 6443
 28  kubernetes:
 29    version: v1.24.9
 30    clusterName: cluster.local
 31    autoRenewCerts: true
 32    containerManager: containerd
 33  etcd:
 34    type: kubekey
 35  network:
 36    plugin: calico
 37    kubePodsCIDR: 10.233.64.0/18
 38    kubeServiceCIDR: 10.233.0.0/18
 39    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
 40    multusCNI:
 41      enabled: false
 42  registry:
 43    type: harbor
 44    auths:
 45      "dockerhub.kubekey.local":
 46        username: admin
 47        password: Harbor12345
 48    privateRegistry: "dockerhub.kubekey.local"
 49    namespaceOverride: "kubesphereio"
 50    registryMirrors: []
 51    insecureRegistries: []
 52  addons: []
 53---
 54apiVersion: installer.kubesphere.io/v1alpha1
 55kind: ClusterConfiguration
 56metadata:
 57  name: ks-installer
 58  namespace: kubesphere-system
 59  labels:
 60    version: v3.3.2
 61spec:
 62  persistence:
 63    storageClass: ""
 64  authentication:
 65    jwtSecret: ""
 66  zone: ""
 67  local_registry: ""
 68  namespace_override: ""
 69  # dev_tag: ""
 70  etcd:
 71    monitoring: false
 72    endpointIps: localhost
 73    port: 2379
 74    tlsEnable: true
 75  common:
 76    core:
 77      console:
 78        enableMultiLogin: true
 79        port: 30880
 80        type: NodePort
 81    # apiserver:
 82    #  resources: {}
 83    # controllerManager:
 84    #  resources: {}
 85    redis:
 86      enabled: false
 87      volumeSize: 2Gi
 88    openldap:
 89      enabled: false
 90      volumeSize: 2Gi
 91    minio:
 92      volumeSize: 20Gi
 93    monitoring:
 94      # type: external
 95      endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
 96      GPUMonitoring:
 97        enabled: false
 98    gpu:
 99      kinds:
100      - resourceName: "nvidia.com/gpu"
101        resourceType: "GPU"
102        default: true
103    es:
104      # master:
105      #   volumeSize: 4Gi
106      #   replicas: 1
107      #   resources: {}
108      # data:
109      #   volumeSize: 20Gi
110      #   replicas: 1
111      #   resources: {}
112      logMaxAge: 7
113      elkPrefix: logstash
114      basicAuth:
115        enabled: false
116        username: ""
117        password: ""
118      externalElasticsearchHost: ""
119      externalElasticsearchPort: ""
120  alerting:
121    enabled: false
122    # thanosruler:
123    #   replicas: 1
124    #   resources: {}
125  auditing:
126    enabled: false
127    # operator:
128    #   resources: {}
129    # webhook:
130    #   resources: {}
131  devops:
132    enabled: false
133    # resources: {}
134    jenkinsMemoryLim: 8Gi
135    jenkinsMemoryReq: 4Gi
136    jenkinsVolumeSize: 8Gi
137  events:
138    enabled: false
139    # operator:
140    #   resources: {}
141    # exporter:
142    #   resources: {}
143    # ruler:
144    #   enabled: true
145    #   replicas: 2
146    #   resources: {}
147  logging:
148    enabled: false
149    logsidecar:
150      enabled: true
151      replicas: 2
152      # resources: {}
153  metrics_server:
154    enabled: false
155  monitoring:
156    storageClass: ""
157    node_exporter:
158      port: 9100
159      # resources: {}
160    # kube_rbac_proxy:
161    #   resources: {}
162    # kube_state_metrics:
163    #   resources: {}
164    # prometheus:
165    #   replicas: 1
166    #   volumeSize: 20Gi
167    #   resources: {}
168    #   operator:
169    #     resources: {}
170    # alertmanager:
171    #   replicas: 1
172    #   resources: {}
173    # notification_manager:
174    #   resources: {}
175    #   operator:
176    #     resources: {}
177    #   proxy:
178    #     resources: {}
179    gpu:
180      nvidia_dcgm_exporter:
181        enabled: false
182        # resources: {}
183  multicluster:
184    clusterRole: none
185  network:
186    networkpolicy:
187      enabled: false
188    ippool:
189      type: none
190    topology:
191      type: none
192  openpitrix:
193    store:
194      enabled: false
195  servicemesh:
196    enabled: false
197    istio:
198      components:
199        ingressGateways:
200        - name: istio-ingressgateway
201          enabled: false
202        cni:
203          enabled: false
204  edgeruntime:
205    enabled: false
206    kubeedge:
207      enabled: false
208      cloudCore:
209        cloudHub:
210          advertiseAddress:
211            - ""
212        service:
213          cloudhubNodePort: "30000"
214          cloudhubQuicNodePort: "30001"
215          cloudhubHttpsNodePort: "30002"
216          cloudstreamNodePort: "30003"
217          tunnelNodePort: "30004"
218        # resources: {}
219        # hostNetWork: false
220      iptables-manager:
221        enabled: true
222        mode: "external"
223        # resources: {}
224      # edgeService:
225      #   resources: {}
226  terminal:
227    timeout: 600

registry 설치 #

1sudo ./kk init registry -f config-sample.yaml -a artifact-3.0.7.tar.gz

[ERROR] ssh error

  • 각 node 별로 ssh가 안될시 root passwd가 맞지 않아 발생함.
  • vagrant에서 vm이 생성되면 root 비번을 설정해줘야 하는 듯
    1sudo passwd root

{: .prompt-info }

Harbor 인증서 복사 및 업데이트 (harbor curl: (60) SSL certificate problem: unable to get local issuer certificate) #

 1sudo cp /etc/docker/certs.d/dockerhub.kubekey.local/ca.crt /usr/local/share/ca-certificates/harbor-ca.crt
 2scp -i /home/vagrant/.ssh/id_rsa /usr/local/share/ca-certificates/harbor-ca.crt root@192.168.10.110:/usr/local/share/ca-certificates/harbor-ca.crt
 3scp -i /home/vagrant/.ssh/id_rsa /usr/local/share/ca-certificates/harbor-ca.crt root@192.168.10.120:/usr/local/share/ca-certificates/harbor-ca.crt
 4
 5# 각 node 별로 아래 작업
 6sudo update-ca-certificates
 7
 8# 인증서 적용 확인
 9ls -lrt /etc/ssl/certs
10- harbor-ca.pem -> /usr/local/share/ca-certificates/harbor-ca.crt
11- ca-certificates.crt
12
13systemctl restart containerd

harbor 주소 : [harbor 설치한 ip]:80 {: .prompt-info }

Harbor 프로젝트 생성 및 수정 #

1curl -O https://raw.githubusercontent.com/kubesphere/ks-installer/master/scripts/create_project_harbor.sh

Harbor 프로젝트 수정 및 url 수정(https://dockerhub.kubekey.local) #

  • 파일 편집

    1vi create_project_harbor.sh

  • url 수정(https://dockerhub.kubekey.local)

     1#!/usr/bin/env bash
 2
 3# Copyright 2018 The KubeSphere Authors.
 4#
 5# Licensed under the Apache License, Version 2.0 (the "License");
 6# you may not use this file except in compliance with the License.
 7# You may obtain a copy of the License at
 8#
 9#     http://www.apache.org/licenses/LICENSE-2.0
10#
11# Unless required by applicable law or agreed to in writing, software
12# distributed under the License is distributed on an "AS IS" BASIS,
13# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14# See the License for the specific language governing permissions and
15# limitations under the License.
16
17url="https://dockerhub.kubekey.local"  #Change the value of url to https://dockerhub.kubekey.local.
18user="admin"
19passwd="Harbor12345"
20
21harbor_projects=(library
22    kubesphereio
23    kubesphere
24    argoproj
25    calico
26    coredns
27    openebs
28    csiplugin
29    minio
30    mirrorgooglecontainers
31    osixia
32    prom
33    thanosio
34    jimmidyson
35    grafana
36    elastic
37    istio
38    jaegertracing
39    jenkins
40    weaveworks
41    openpitrix
42    joosthofman
43    nginxdemos
44    fluent
45    kubeedge
46    openpolicyagent
47)
48
49for project in "${harbor_projects[@]}"; do
50    echo "creating $project"
51    curl -u "${user}:${passwd}" -X POST -H "Content-Type: application/json" "${url}/api/v2.0/projects" -d "{ \"project_name\": \"${project}\", \"public\": true}" -k #Add -k at the end of the curl command.
52done

  • 파일 권한 변경

    1chmod +x create_project_harbor.sh

  • 실행

    1./create_project_harbor.sh

image 별도로 push 방법 #

1sudo ./kk artifact image push -f config-sample.yaml -a artifact-3.0.7.tar.gz

[ERROR] Harbor에 image push 할 때 Unauthorized 에러 발생 때

  • 다시 로그인
1docker login [your.host.com]:port -u username -p password

{: .prompt-info }

Cluster 설치 #

1sudo ./kk create cluster -f config-sample.yaml -a artifact-3.0.7.tar.gz

Install operating system packages

1sudo ./kk create cluster -f config-sample.yaml -a artifact-3.0.7.tar.gz --with-packages

{: .prompt-tip }

--skip-push-images를 추가하면 harbor에 image를 push하는 과정으로 생략할 수 있다.

1sudo ./kk create cluster -f config-sample.yaml -a artifact-3.0.7.tar.gz --skip-push-images

{: .prompt-tip }

kubekey command 참고

Cluster 설치하면서 log 확인 #

1kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

Kubernetes 일반 유저 일 때 #

1mkdir -p $HOME/.kube
2sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
3sudo chown $(id -u):$(id -g) $HOME/.kube/config

만약 일반 계정에서 아래와 sudo 명령어 없이 kubectl 명령어 사용시 아래와 같은 오류가 발생하면

  • [ERROR] error loading config file /etc/kubernetes/admin.conf: open /etc/kubernetes/admin.conf: permission denied
    • 아래 명령어를 입력하면 sudo 없이 사용 가능하다.
      1export KUBECONFIG=$HOME/.kube/config

{: .prompt-danger }

[ERROR] error making pod data directories: mkdir /var/lib/kubelet/pods/86cfe394-ba32-4a9f-ad65-1fb21f98a4ba: read-only file system

1chown -R kubelet:kubelet /var/lib/kubelet/pods
2chmod 750 /var/lib/kubelet/pods
3systemctl restart kubelet

{: .prompt-danger }

Advertisement
관련 글