Saturday, 20 April 2019

Install Kubernetes Cluster with Docker CE on CentOS 7

Install Kubernetes Cluster with Docker CE on CentOS 7

Kubernetes or k8s is an open-source container orchestration system for automated application deployment, management and scaling across clusters of hosts. Kubernetes is initially developed by Google, but now maintained by Cloud Native Computing Foundation. Kubernetes requires a container runtime interface (CRI) for orchestration. Kubernetes supports different CRIs including Docker, containerd and cri-o.

In our previous article, we have configured a Docker Swarm Cluster on CentOS 7 for container orchestration. Now, in this article, we are installing a two node Kubernetes / K8s cluster with Docker CE on CentOS 7.

This article is about the installation and configuration of Kubernetes on CentOS 7 and it doesn’t addresses the technical details about Kubernetes architecture and components. Therefore, if you are interested to read more about Kubernetes you should read Kubernetes in Action by Manning Publications.

 

This Article Provides:

     

    System Specification:

    We have two CentOS 7 virtual machines with following specifications.

    Hostname: kubemaster-01 kubenode-01
    IP Address: 192.168.116.160/24 192.168.116.161/24
    Cluster Role: K8s master K8s node
    CPU: 3.4 Ghz (2 cores) * 3.4 Ghz (2 cores) *
    Memory: 2 GB 2 GB
    Storage: 40 GB 40 GB
    Operating System: CentOS 7.6 CentOS 7.6
    Docker version: 18.09.5 18.09.5
    Kubernetes version: 1.14.1 1.14.1

    * We must have at least 2 cores on each node to install Kubernetes.

    Make sure the hostnames are resolvable on all nodes. You can either use DNS Server or Local DNS Resolver for this purpose.

     

    Installing Docker CE on CentOS 7:

    We are configuring Docker CE as Kubernetes CRI (Container Runtime Interface). Other choices for Kubernetes CRI are containerd, cri-o and frakti.

    Connect with Kubernetes master kubemaster-01.example.com using ssh as root user.

    Install Docker CE prerequisite packages using yum command.

    [root@kubemaster-01 ~]# yum install -y device-mapper-persistent-data lvm2 yum-utils Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.xeonbd.com * extras: mirror.xeonbd.com * updates: mirror.xeonbd.com Package device-mapper-persistent-data-0.7.3-3.el7.x86_64 already installed and latest version Package yum-utils-1.1.31-50.el7.noarch already installed and latest version Resolving Dependencies --> Running transaction check ---> Package lvm2.x86_64 7:2.02.180-8.el7 will be updated ---> Package lvm2.x86_64 7:2.02.180-10.el7_6.3 will be an update --> Processing Dependency: lvm2-libs = 7:2.02.180-10.el7_6.3 for package: 7:lvm2-2.02.180-10.el7_6.3.x86_64 --> Running transaction check ---> Package lvm2-libs.x86_64 7:2.02.180-8.el7 will be updated ---> Package lvm2-libs.x86_64 7:2.02.180-10.el7_6.3 will be an update --> Processing Dependency: device-mapper-event = 7:1.02.149-10.el7_6.3 for package: 7:lvm2-libs-2.02.180-10.el7_6.3.x86_64 --> Running transaction check ---> Package device-mapper-event.x86_64 7:1.02.149-8.el7 will be updated ---> Package device-mapper-event.x86_64 7:1.02.149-10.el7_6.3 will be an update --> Processing Dependency: device-mapper-event-libs = 7:1.02.149-10.el7_6.3 for package: 7:device-mapper-event-1.02.149-10.el7_6.3.x86_64 --> Processing Dependency: device-mapper = 7:1.02.149-10.el7_6.3 for package: 7:device-mapper-event-1.02.149-10.el7_6.3.x86_64 --> Running transaction check ---> Package device-mapper.x86_64 7:1.02.149-8.el7 will be updated --> Processing Dependency: device-mapper = 7:1.02.149-8.el7 for package: 7:device-mapper-libs-1.02.149-8.el7.x86_64 ---> Package device-mapper.x86_64 7:1.02.149-10.el7_6.3 will be an update ---> Package device-mapper-event-libs.x86_64 7:1.02.149-8.el7 will be updated ---> Package device-mapper-event-libs.x86_64 7:1.02.149-10.el7_6.3 will be an update --> Running transaction check ---> Package device-mapper-libs.x86_64 7:1.02.149-8.el7 will be updated ---> Package device-mapper-libs.x86_64 7:1.02.149-10.el7_6.3 will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Updating: lvm2 x86_64 7:2.02.180-10.el7_6.3 updates 1.3 M Updating for dependencies: device-mapper x86_64 7:1.02.149-10.el7_6.3 updates 292 k device-mapper-event x86_64 7:1.02.149-10.el7_6.3 updates 188 k device-mapper-event-libs x86_64 7:1.02.149-10.el7_6.3 updates 188 k device-mapper-libs x86_64 7:1.02.149-10.el7_6.3 updates 320 k lvm2-libs x86_64 7:2.02.180-10.el7_6.3 updates 1.1 M Transaction Summary ================================================================================ Upgrade 1 Package (+5 Dependent packages) Total download size: 3.3 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. (1/6): device-mapper-event-libs-1.02.149-10.el7_6.3.x86_64 | 188 kB 00:03 (2/6): lvm2-2.02.180-10.el7_6.3.x86_64.rpm | 1.3 MB 00:04 (3/6): device-mapper-event-1.02.149-10.el7_6.3.x86_64.rpm | 188 kB 00:04 (4/6): device-mapper-libs-1.02.149-10.el7_6.3.x86_64.rpm | 320 kB 00:05 (5/6): lvm2-libs-2.02.180-10.el7_6.3.x86_64.rpm | 1.1 MB 00:04 (6/6): device-mapper-1.02.149-10.el7_6.3.x86_64.rpm | 292 kB 00:11 -------------------------------------------------------------------------------- Total 297 kB/s | 3.3 MB 00:11 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : 7:device-mapper-1.02.149-10.el7_6.3.x86_64 1/12 Updating : 7:device-mapper-libs-1.02.149-10.el7_6.3.x86_64 2/12 Updating : 7:device-mapper-event-libs-1.02.149-10.el7_6.3.x86_64 3/12 Updating : 7:device-mapper-event-1.02.149-10.el7_6.3.x86_64 4/12 Updating : 7:lvm2-libs-2.02.180-10.el7_6.3.x86_64 5/12 Updating : 7:lvm2-2.02.180-10.el7_6.3.x86_64 6/12 Cleanup : 7:lvm2-2.02.180-8.el7.x86_64 7/12 Cleanup : 7:lvm2-libs-2.02.180-8.el7.x86_64 8/12 Cleanup : 7:device-mapper-event-1.02.149-8.el7.x86_64 9/12 Cleanup : 7:device-mapper-event-libs-1.02.149-8.el7.x86_64 10/12 Cleanup : 7:device-mapper-1.02.149-8.el7.x86_64 11/12 Cleanup : 7:device-mapper-libs-1.02.149-8.el7.x86_64 12/12 Verifying : 7:device-mapper-libs-1.02.149-10.el7_6.3.x86_64 1/12 Verifying : 7:device-mapper-event-libs-1.02.149-10.el7_6.3.x86_64 2/12 Verifying : 7:device-mapper-1.02.149-10.el7_6.3.x86_64 3/12 Verifying : 7:device-mapper-event-1.02.149-10.el7_6.3.x86_64 4/12 Verifying : 7:lvm2-libs-2.02.180-10.el7_6.3.x86_64 5/12 Verifying : 7:lvm2-2.02.180-10.el7_6.3.x86_64 6/12 Verifying : 7:lvm2-2.02.180-8.el7.x86_64 7/12 Verifying : 7:device-mapper-event-1.02.149-8.el7.x86_64 8/12 Verifying : 7:lvm2-libs-2.02.180-8.el7.x86_64 9/12 Verifying : 7:device-mapper-libs-1.02.149-8.el7.x86_64 10/12 Verifying : 7:device-mapper-1.02.149-8.el7.x86_64 11/12 Verifying : 7:device-mapper-event-libs-1.02.149-8.el7.x86_64 12/12 Updated: lvm2.x86_64 7:2.02.180-10.el7_6.3 Dependency Updated: device-mapper.x86_64 7:1.02.149-10.el7_6.3 device-mapper-event.x86_64 7:1.02.149-10.el7_6.3 device-mapper-event-libs.x86_64 7:1.02.149-10.el7_6.3 device-mapper-libs.x86_64 7:1.02.149-10.el7_6.3 lvm2-libs.x86_64 7:2.02.180-10.el7_6.3 Complete!

    Add Docker yum repository as follows:

    [root@kubemaster-01 ~]# yum-config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo Loaded plugins: fastestmirror adding repo from: https://download.docker.com/linux/centos/docker-ce.repo grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo repo saved to /etc/yum.repos.d/docker-ce.repo

    Build yum cache for Docker repository.

    [root@kubemaster-01 ~]# yum makecache fast Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.xeonbd.com * extras: mirror.xeonbd.com * updates: mirror.xeonbd.com base | 3.6 kB 00:00 docker-ce-stable | 3.5 kB 00:00 extras | 3.4 kB 00:00 updates | 3.4 kB 00:00 (1/2): docker-ce-stable/x86_64/primary_db | 27 kB 00:00 (2/2): docker-ce-stable/x86_64/updateinfo | 55 B 00:01 Metadata Cache Created

    Install Docker CE using yum command.

    [root@kubemaster-01 ~]# yum install -y docker-ce Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.xeonbd.com * extras: mirror.xeonbd.com * updates: mirror.xeonbd.com Resolving Dependencies --> Running transaction check ---> Package docker-ce.x86_64 3:18.09.5-3.el7 will be installed --> Processing Dependency: container-selinux >= 2.9 for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Processing Dependency: libseccomp >= 2.3 for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Processing Dependency: docker-ce-cli for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Processing Dependency: libcgroup for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Processing Dependency: libseccomp.so.2()(64bit) for package: 3:docker-ce-18.09.5-3.el7.x86_64 --> Running transaction check ---> Package container-selinux.noarch 2:2.74-1.el7 will be installed --> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.74-1.el7.noarch ---> Package containerd.io.x86_64 0:1.2.5-3.1.el7 will be installed ---> Package docker-ce-cli.x86_64 1:18.09.5-3.el7 will be installed ---> Package libcgroup.x86_64 0:0.41-20.el7 will be installed ---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed --> Running transaction check ---> Package policycoreutils-python.x86_64 0:2.5-29.el7_6.1 will be installed --> Processing Dependency: policycoreutils = 2.5-29.el7_6.1 for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-29.el7_6.1.x86_64 --> Running transaction check ---> Package audit-libs-python.x86_64 0:2.8.4-4.el7 will be installed ---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed ---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed ---> Package policycoreutils.x86_64 0:2.5-29.el7 will be updated ---> Package policycoreutils.x86_64 0:2.5-29.el7_6.1 will be an update ---> Package python-IPy.noarch 0:0.75-6.el7 will be installed ---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: docker-ce x86_64 3:18.09.5-3.el7 docker-ce-stable 19 M Installing for dependencies: audit-libs-python x86_64 2.8.4-4.el7 base 76 k checkpolicy x86_64 2.5-8.el7 base 295 k container-selinux noarch 2:2.74-1.el7 extras 38 k containerd.io x86_64 1.2.5-3.1.el7 docker-ce-stable 22 M docker-ce-cli x86_64 1:18.09.5-3.el7 docker-ce-stable 14 M libcgroup x86_64 0.41-20.el7 base 66 k libseccomp x86_64 2.3.1-3.el7 base 56 k libsemanage-python x86_64 2.5-14.el7 base 113 k policycoreutils-python x86_64 2.5-29.el7_6.1 updates 456 k python-IPy noarch 0.75-6.el7 base 32 k setools-libs x86_64 3.3.8-4.el7 base 620 k Updating for dependencies: policycoreutils x86_64 2.5-29.el7_6.1 updates 916 k Transaction Summary ================================================================================ Install 1 Package (+11 Dependent packages) Upgrade ( 1 Dependent package) Total download size: 57 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. (1/13): container-selinux-2.74-1.el7.noarch.rpm | 38 kB 00:01 (2/13): audit-libs-python-2.8.4-4.el7.x86_64.rpm | 76 kB 00:01 (3/13): checkpolicy-2.5-8.el7.x86_64.rpm | 295 kB 00:06 warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/docker-ce-18.09.5-3.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY Public key for docker-ce-18.09.5-3.el7.x86_64.rpm is not installed (4/13): docker-ce-18.09.5-3.el7.x86_64.rpm | 19 MB 01:07 (5/13): libsemanage-python-2.5-14.el7.x86_64.rpm | 113 kB 00:02 (6/13): libseccomp-2.3.1-3.el7.x86_64.rpm | 56 kB 00:03 (7/13): libcgroup-0.41-20.el7.x86_64.rpm | 66 kB 00:04 (8/13): python-IPy-0.75-6.el7.noarch.rpm | 32 kB 00:01 (9/13): policycoreutils-2.5-29.el7_6.1.x86_64.rpm | 916 kB 00:07 (10/13): containerd.io-1.2.5-3.1.el7.x86_64.rpm | 22 MB 01:20 (11/13): setools-libs-3.3.8-4.el7.x86_64.rpm | 620 kB 00:07 (12/13): policycoreutils-python-2.5-29.el7_6.1.x86_64.rpm | 456 kB 00:10 (13/13): docker-ce-cli-18.09.5-3.el7.x86_64.rpm | 14 MB 00:33 -------------------------------------------------------------------------------- Total 579 kB/s | 57 MB 01:41 Retrieving key from https://download.docker.com/linux/centos/gpg Importing GPG key 0x621E9F35: Userid : "Docker Release (CE rpm) <docker@docker.com>" Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35 From : https://download.docker.com/linux/centos/gpg Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : policycoreutils-2.5-29.el7_6.1.x86_64 1/14 Installing : libcgroup-0.41-20.el7.x86_64 2/14 Installing : setools-libs-3.3.8-4.el7.x86_64 3/14 Installing : checkpolicy-2.5-8.el7.x86_64 4/14 Installing : libseccomp-2.3.1-3.el7.x86_64 5/14 Installing : libsemanage-python-2.5-14.el7.x86_64 6/14 Installing : audit-libs-python-2.8.4-4.el7.x86_64 7/14 Installing : python-IPy-0.75-6.el7.noarch 8/14 Installing : policycoreutils-python-2.5-29.el7_6.1.x86_64 9/14 Installing : 2:container-selinux-2.74-1.el7.noarch 10/14 Installing : containerd.io-1.2.5-3.1.el7.x86_64 11/14 Installing : 1:docker-ce-cli-18.09.5-3.el7.x86_64 12/14 Installing : 3:docker-ce-18.09.5-3.el7.x86_64 13/14 Cleanup : policycoreutils-2.5-29.el7.x86_64 14/14 Verifying : 1:docker-ce-cli-18.09.5-3.el7.x86_64 1/14 Verifying : libcgroup-0.41-20.el7.x86_64 2/14 Verifying : containerd.io-1.2.5-3.1.el7.x86_64 3/14 Verifying : policycoreutils-python-2.5-29.el7_6.1.x86_64 4/14 Verifying : python-IPy-0.75-6.el7.noarch 5/14 Verifying : audit-libs-python-2.8.4-4.el7.x86_64 6/14 Verifying : libsemanage-python-2.5-14.el7.x86_64 7/14 Verifying : libseccomp-2.3.1-3.el7.x86_64 8/14 Verifying : 3:docker-ce-18.09.5-3.el7.x86_64 9/14 Verifying : checkpolicy-2.5-8.el7.x86_64 10/14 Verifying : 2:container-selinux-2.74-1.el7.noarch 11/14 Verifying : policycoreutils-2.5-29.el7_6.1.x86_64 12/14 Verifying : setools-libs-3.3.8-4.el7.x86_64 13/14 Verifying : policycoreutils-2.5-29.el7.x86_64 14/14 Installed: docker-ce.x86_64 3:18.09.5-3.el7 Dependency Installed: audit-libs-python.x86_64 0:2.8.4-4.el7 checkpolicy.x86_64 0:2.5-8.el7 container-selinux.noarch 2:2.74-1.el7 containerd.io.x86_64 0:1.2.5-3.1.el7 docker-ce-cli.x86_64 1:18.09.5-3.el7 libcgroup.x86_64 0:0.41-20.el7 libseccomp.x86_64 0:2.3.1-3.el7 libsemanage-python.x86_64 0:2.5-14.el7 policycoreutils-python.x86_64 0:2.5-29.el7_6.1 python-IPy.noarch 0:0.75-6.el7 setools-libs.x86_64 0:3.3.8-4.el7 Dependency Updated: policycoreutils.x86_64 0:2.5-29.el7_6.1 Complete!

    Configure Docker service for use by Kubernetes.

    [root@kubemaster-01 ~]# mkdir /etc/docker [root@kubemaster-01 ~]# cat > /etc/docker/daemon.json << EOF > { > "exec-opts": ["native.cgroupdriver=systemd"], > "log-driver": "json-file", > "log-opts": { > "max-size": "100m" > }, > "storage-driver": "overlay2", > "storage-opts": [ > "overlay2.override_kernel_check=true" > ] > } > EOF

    Start and enable Docker service.

    [root@kubemaster-01 ~]# systemctl enable docker.service Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service. [root@kubemaster-01 ~]# systemctl start docker.service

    Docker CE has been installed. Repeat the above steps to install Docker CE on kubenode-01.example.com.

     

    Installing Kubernetes on CentOS 7:

    Set following Kernel parameter as required by Kubernetes.

    [root@kubemaster-01 ~]# cat > /etc/sysctl.d/kubernetes.conf << EOF > net.ipv4.ip_forward = 1 > net.bridge.bridge-nf-call-ip6tables = 1 > net.bridge.bridge-nf-call-iptables = 1 > EOF

    Reload Kernel parameter configuration files.

    [root@kubemaster-01 ~]# modprobe br_netfilter [root@kubemaster-01 ~]# sysctl --system * Applying /usr/lib/sysctl.d/00-system.conf ... net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 * Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ... kernel.yama.ptrace_scope = 0 * Applying /usr/lib/sysctl.d/50-default.conf ... kernel.sysrq = 16 kernel.core_uses_pid = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.default.promote_secondaries = 1 net.ipv4.conf.all.promote_secondaries = 1 fs.protected_hardlinks = 1 fs.protected_symlinks = 1 * Applying /etc/sysctl.d/99-sysctl.conf ... * Applying /etc/sysctl.d/kubernetes.conf ... net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 * Applying /etc/sysctl.conf ...

    Turn off Swap for Kubernetes installation.

    [root@kubemaster-01 ~]# swapoff -a [root@kubemaster-01 ~]# sed -e '/swap/s/^/#/g' -i /etc/fstab

    Kubernetes uses following services ports on Master node.

    Port Protocol Purpose
    6443 TCP Kubernetes API server
    2379-2380 TCP etcd server client API
    10250 TCP Kubelet API
    10251 TCP kube-scheduler
    10252 TCP kube-controller-manager

    Allow Kubernetes service ports on kubemaster-01.example.com in Linux firewall.

    [root@kubemaster-01 ~]# firewall-cmd --permanent --add-port={6443,2379,2380,10250,10251,10252}/tcp success [root@kubemaster-01 ~]# firewall-cmd --reload success

    Kubernetes uses following service ports on Worker node.

    Port Protocol Purpose
    10250 TCP Kubelet API
    30000-32767 TCP NodePort Services

    Allow Kubernetes service ports on kubenode-01.example.com in Linux firewall.

    [root@kubenode-01 ~]# firewall-cmd --permanent --add-port={10250,30000-32767}/tcp success [root@kubenode-01 ~]# firewall-cmd --reload success

    Switch SELinux to Permissive mode using following commands.

    [root@kubemaster-01 ~]# setenforce 0 [root@kubemaster-01 ~]# sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

    Add Kubernetes yum repository as follows.

    [root@kubemaster-01 ~]# cat > /etc/yum.repos.d/kubernetes.repo << EOF > [kubernetes] > name=Kubernetes > baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 > enabled=1 > gpgcheck=1 > repo_gpgcheck=1 > gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg > EOF

    Build yum cache for kubernetes repository.

    [root@kubemaster-01 ~]# yum makecache fast Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.xeonbd.com * extras: mirror.xeonbd.com * updates: mirror.xeonbd.com base | 3.6 kB 00:00 docker-ce-stable | 3.5 kB 00:00 extras | 3.4 kB 00:00 kubernetes/signature | 454 B 00:00 Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg Importing GPG key 0xA7317B0F: Userid : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f From : https://packages.cloud.google.com/yum/doc/yum-key.gpg Is this ok [y/N]: y Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg kubernetes/signature | 1.4 kB 00:07 !!! updates | 3.4 kB 00:00 kubernetes/primary | 47 kB 00:00 kubernetes 339/339 Metadata Cache Created

    Install Kubernetes packages using yum command.

    [root@kubemaster-01 ~]# yum install -y kubelet kubeadm kubectl Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.xeonbd.com * extras: mirror.xeonbd.com * updates: mirror.xeonbd.com Resolving Dependencies --> Running transaction check ---> Package kubeadm.x86_64 0:1.14.1-0 will be installed --> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.14.1-0.x86_64 --> Processing Dependency: cri-tools >= 1.11.0 for package: kubeadm-1.14.1-0.x86_64 ---> Package kubectl.x86_64 0:1.14.1-0 will be installed ---> Package kubelet.x86_64 0:1.14.1-0 will be installed --> Processing Dependency: socat for package: kubelet-1.14.1-0.x86_64 --> Processing Dependency: conntrack for package: kubelet-1.14.1-0.x86_64 --> Running transaction check ---> Package conntrack-tools.x86_64 0:1.4.4-4.el7 will be installed --> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 ---> Package cri-tools.x86_64 0:1.12.0-0 will be installed ---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed ---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed --> Running transaction check ---> Package libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 will be installed ---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 will be installed ---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: kubeadm x86_64 1.14.1-0 kubernetes 8.7 M kubectl x86_64 1.14.1-0 kubernetes 9.5 M kubelet x86_64 1.14.1-0 kubernetes 23 M Installing for dependencies: conntrack-tools x86_64 1.4.4-4.el7 base 186 k cri-tools x86_64 1.12.0-0 kubernetes 4.2 M kubernetes-cni x86_64 0.7.5-0 kubernetes 10 M libnetfilter_cthelper x86_64 1.0.0-9.el7 base 18 k libnetfilter_cttimeout x86_64 1.0.0-6.el7 base 18 k libnetfilter_queue x86_64 1.0.2-2.el7_2 base 23 k socat x86_64 1.7.3.2-2.el7 base 290 k Transaction Summary ================================================================================ Install 3 Packages (+7 Dependent packages) Total download size: 56 M Installed size: 256 M Downloading packages: (1/10): conntrack-tools-1.4.4-4.el7.x86_64.rpm | 186 kB 00:02 warning: /var/cache/yum/x86_64/7/kubernetes/packages/53edc739a0e51a4c17794de26b13ee5df939bd3161b37f503fe2af8980b41a89-cri-tools-1.12.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY Public key for 53edc739a0e51a4c17794de26b13ee5df939bd3161b37f503fe2af8980b41a89-cri-tools-1.12.0-0.x86_64.rpm is not installed (2/10): 53edc739a0e51a4c17794de26b13ee5df939bd3161b37f503f | 4.2 MB 00:16 (3/10): 9e1af74c18311f2f6f8460dbd1aa3e02911105bfd455416381 | 8.7 MB 00:32 (4/10): 5c6cb3beb5142fa21020e2116824ba77a2d1389a3321601ea5 | 9.5 MB 00:31 (5/10): libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm | 18 kB 00:01 (6/10): libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm | 18 kB 00:01 (7/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm | 23 kB 00:02 (8/10): socat-1.7.3.2-2.el7.x86_64.rpm | 290 kB 00:08 (9/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc | 10 MB 00:35 (10/10): e1e8f430609698d7ec87642179ab57605925cb9aa48d406da | 23 MB 01:13 -------------------------------------------------------------------------------- Total 544 kB/s | 56 MB 01:45 Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg Importing GPG key 0xA7317B0F: Userid : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f From : https://packages.cloud.google.com/yum/doc/yum-key.gpg Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg Importing GPG key 0x3E1BA8D5: Userid : "Google Cloud Packages RPM Signing Key <gc-team@google.com>" Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5 From : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : socat-1.7.3.2-2.el7.x86_64 1/10 Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64 2/10 Installing : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 3/10 Installing : kubectl-1.14.1-0.x86_64 4/10 Installing : libnetfilter_cthelper-1.0.0-9.el7.x86_64 5/10 Installing : conntrack-tools-1.4.4-4.el7.x86_64 6/10 Installing : kubernetes-cni-0.7.5-0.x86_64 7/10 Installing : kubelet-1.14.1-0.x86_64 8/10 Installing : cri-tools-1.12.0-0.x86_64 9/10 Installing : kubeadm-1.14.1-0.x86_64 10/10 Verifying : cri-tools-1.12.0-0.x86_64 1/10 Verifying : libnetfilter_cthelper-1.0.0-9.el7.x86_64 2/10 Verifying : kubectl-1.14.1-0.x86_64 3/10 Verifying : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 4/10 Verifying : libnetfilter_queue-1.0.2-2.el7_2.x86_64 5/10 Verifying : kubeadm-1.14.1-0.x86_64 6/10 Verifying : kubelet-1.14.1-0.x86_64 7/10 Verifying : kubernetes-cni-0.7.5-0.x86_64 8/10 Verifying : socat-1.7.3.2-2.el7.x86_64 9/10 Verifying : conntrack-tools-1.4.4-4.el7.x86_64 10/10 Installed: kubeadm.x86_64 0:1.14.1-0 kubectl.x86_64 0:1.14.1-0 kubelet.x86_64 0:1.14.1-0 Dependency Installed: conntrack-tools.x86_64 0:1.4.4-4.el7 cri-tools.x86_64 0:1.12.0-0 kubernetes-cni.x86_64 0:0.7.5-0 libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 socat.x86_64 0:1.7.3.2-2.el7 Complete!

    To enable automatic completion of kubectl commands, we have to execute the script provided by kubectl command itself. You must ensure that bash-completion package is installed.

    [root@kubemaster-01 ~]# source <(kubectl completion bash)

    For making it persistent, we have to add the script in Bash Completion directory.

    [root@kubemaster-01 ~]# kubectl completion bash > /etc/bash_completion.d/kubectl

    Kubernetes has been installed. Repeat above steps to install Kubernetes on kubenode-01.example.com.

     

    Configuring Kubelet Service on Master Node:

    Use kubeadm command to pull images that are required to configure kubelet service.

    [root@kubemaster-01 ~]# kubeadm config images pull [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.14.1 [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.14.1 [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.14.1 [config/images] Pulled k8s.gcr.io/kube-proxy:v1.14.1 [config/images] Pulled k8s.gcr.io/pause:3.1 [config/images] Pulled k8s.gcr.io/etcd:3.3.10 [config/images] Pulled k8s.gcr.io/coredns:1.3.1

    Initialize and configure the kubelet service as follows:

    [root@kubemaster-01 ~]# kubeadm init [init] Using Kubernetes version: v1.14.1 [preflight] Running pre-flight checks [WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service' [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Activating the kubelet service [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubemaster-01.example.com kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.116.160] [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [kubemaster-01.example.com localhost] and IPs [192.168.116.160 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [kubemaster-01.example.com localhost] and IPs [192.168.116.160 127.0.0.1 ::1] [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. [apiclient] All control plane components are healthy after 42.152638 seconds [upload-config] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.14" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --experimental-upload-certs [mark-control-plane] Marking the node kubemaster-01.example.com as control-plane by adding the label "node-role.kubernetes.io/master=''" [mark-control-plane] Marking the node kubemaster-01.example.com as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: mm20xq.goxx7plwzrx75tv3 [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.116.160:6443 --token mm20xq.goxx7plwzrx75tv3 \ --discovery-token-ca-cert-hash sha256:00065886b183ea9cc2e9fbb68ff2a82b52574c2ab5ad8868c4fd6c2feb006d6f

    Execute following commands as suggested by above command.

    [root@kubemaster-01 ~]# mkdir -p $HOME/.kube [root@kubemaster-01 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config [root@kubemaster-01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

    Start and enable Kubelet Service.

    [root@kubemaster-01 ~]# systemctl enable kubelet.service Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service. [root@kubemaster-01 ~]# systemctl start kubelet.service

     

    Adding a node to Kubernetes Cluster on CentOS 7:

    Execute status of nodes in the Kubernetes cluster.

    [root@kubemaster-01 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION kubemaster-01.example.com NotReady master 50m v1.14.1

    Add another node to Kubernetes cluster by executing the command provided by kubeadm init command.

    [root@kubenode-01 ~]# kubeadm join 192.168.116.160:6443 --token mm20xq.goxx7plwzrx75tv3 \ > --discovery-token-ca-cert-hash sha256:00065886b183ea9cc2e9fbb68ff2a82b52574c2ab5ad8868c4fd6c2feb006d6f [preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.14" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Activating the kubelet service [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

    If you experience network errors, then you have to install a non-default network like Flannel on all nodes.

    [root@kubemaster-01 ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml podsecuritypolicy.extensions/psp.flannel.unprivileged configured clusterrole.rbac.authorization.k8s.io/flannel unchanged clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged serviceaccount/flannel unchanged configmap/kube-flannel-cfg unchanged daemonset.extensions/kube-flannel-ds-amd64 unchanged daemonset.extensions/kube-flannel-ds-arm64 unchanged daemonset.extensions/kube-flannel-ds-arm unchanged daemonset.extensions/kube-flannel-ds-ppc64le unchanged daemonset.extensions/kube-flannel-ds-s390x unchanged

    Check status of nodes in Kubernetes cluster again.

    [root@kubemaster-01 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION kubemaster-01.example.com Ready master 45m v1.14.1 kubenode-01.example.com Ready <none> 43m v1.14.1

    We have successfully installed a two node Kubernetes cluster with Docker CE on CentOS 7.

    Install Kubernetes Cluster with Docker CE on CentOS 7


    YOU MIGHT ALSO LIKE:

    11 comments:

    1. Hello!

      The kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml not working. I got this:
      unable to recognize "https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml": Get http://localhost:8080/api?timeout=32s: dial tcp [::1]:8080: connect: connection refused
      Please help!

      ReplyDelete
      Replies
      1. Execute following commands and try again.

        # mkdir -p $HOME/.kube
        # sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/conf
        # sudo chown $(id -u):$(id -g) $HOME/.kube/config

        Delete
    2. this step doesn't work - Start and enable Docker service-
      CentOS Linux release 7.6.1810 (Core)
      if I remove /etc/docker directory the service start well

      ReplyDelete
      Replies
      1. Please look for a possible typo error in /etc/docker/daemon.json.

        Delete
    3. sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/conf
      or
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      ?

      ReplyDelete
      Replies
      1. Thank you for the correction. The article has been updated accordingly.

        Delete
    4. Good day! The article is excellent. Please advise how to add a second master.

      ReplyDelete
      Replies
      1. Thanks. Please follow the instructions from the link.

        Delete
    5. how can i run ``kubeadm join`` if I installed only Docker on kubenode?

      ReplyDelete
      Replies
      1. Yes, you can.
        Because we are installing kubernetes on kubenode-01.example.com as well.

        Delete