又快，又好的利用Word制作试卷选择题的正确方法
977
2022-05-29
从零搭建Linux+Docker+Ansible+kubernetes 学习环境(1*Master+3*Node)
一直想学K8s，但是没有环境，本身K8s就有些重。上学之前租了一个阿里云的ESC，单核2G的，单机版K8s的勉强可以装上去，多节点没法搞，书里的Demo也没法学。需要多个节点，涉及到多机器操作，所以顺便温习一下ansible 。
这是一个在Win10上从零搭建学习环境的教程，包含：
通过Vmware Workstation安装四个linux系统虚拟机，一个Master管理节点，三个Node计算节点。
通过桥接模式，可以访问外网，并且可以通过win10物理机ssh远程访问。
可以通过Master节点机器ssh免密登录任意Node节点机。
配置Ansible，Master节点做controller节点，使用角色配置时间同步，使用playbook 安装配置docker K8S等。
Docker,K8s集群相关包安装，网络配置等
关于Vmware Workstation 和 Linux ios包，默认小伙伴已经拥有。Vmware Workstation 默认小伙伴已经安装好，没有的可以网上下载一下。
我所渴求的，無非是將心中脫穎語出的本性付諸生活，為何竟如此艱難呢 ------《彷徨少年時》
这里默认小伙伴已经安装了Vmware Workstation（VMware-workstation-full-15.5.6-16341506.exe）,已经准备了linux系统 安装光盘（CentOS-7-x86_64-DVD-1810.iso）。括号内是我用的版本，我们的方式：
先安装一个Node节点机器，然后通过克隆的方式得到剩余的两个Node机器和一个Master机器
1. 系统安装
2. 配置网络
配置网卡为DHCP模式(自动分配IP地址)
┌──[root@localhost.localdomain]-[~] └─$ nmcli connection modify 'ens33' ipv4.method auto connection.autoconnect yes ┌──[root@localhost.localdomain]-[~] └─$ nmcli connection up 'ens33' 连接已成功激活（D-Bus 活动路径：/org/freedesktop/NetworkManager/ActiveConnection/4） ┌──[root@localhost.localdomain]-[~] └─$ ifconfig | head -2 ens33: flags=4163
┌──[root@192.168.1.7]-[~] └─$ ifconfig ens33: flags=4163
网络配置这里，如果觉得不是特别方面，可以使用NAT模式，即通过vm1或者vmm8 做虚拟交换机来使用，这样就不用考虑ip问题了。
3. 机器克隆
我们以相同的方式，克隆剩余的一个node节点机器，和一个Master节点机。这里不做展示
4.管理控制节点到计算节点DNS配置
┌──[root@192.168.1.10]-[~] └─$ vim /etc/hosts ┌──[root@192.168.1.10]-[~] └─$ cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.7 node0 192.168.1.9 node1 192.168.1.11 node2 192.168.1.10 master ┌──[root@192.168.1.10]-[~] └─$
5.管理控制节点到计算节点SSH免密配置
ssh-keygen生成密匙，全部回车
┌──[root@192.168.1.10]-[~] └─$ ssh usage: ssh [-1246AaCfGgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec] [-D [bind_address:]port] [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11] [-i identity_file] [-J [user@]host[:port]] [-L address] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-Q query_option] [-R address] [-S ctl_path] [-W host:port] [-w local_tun[:remote_tun]] [user@]hostname [command] ┌──[root@192.168.1.10]-[~] └─$ ls -ls ~/.ssh/ ls: 无法访问/root/.ssh/: 没有那个文件或目录 ┌──[root@192.168.1.10]-[~] └─$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:qHboVj/WfMTYCDFDZ5ISf3wEcmfsz0EXJH19U6SnxbY root@node0 The key's randomart image is: +---[RSA 2048]----+ | .o+.=o+.o+*| | ..=B +. o==| | ..+o.....O| | ... .. .=.| | . S. = o.E | | o. o + o | | +... o . | | o.. + o . | | .. . . . | +----[SHA256]-----+
SSH 免密配置，使用ssh-copy-id传递密匙
ssh-copy-id root@node0 ssh-copy-id root@node1 ssh-copy-id root@node2
免密测试
ssh root@node0 ssh root@node1 ssh root@node2
到这一步，我们已经做好了linux环境的搭建，想学linux的小伙伴就可以从这里开始学习啦。这是我linux学习一路整理的笔记，有些实战，感兴趣小伙伴可以看看
这里为了方便，我们直接在物理机操作，而且我们已经配置了ssh，因为我本机的内存不够，所以我只能启三台机器了。
## 1. SSH到控制节点即192.168.1.10，配置yum源,安装ansible
┌──(liruilong㉿Liruilong)-[/mnt/e/docker] └─$ ssh root@192.168.1.10 Last login: Sat Sep 11 00:23:10 2021 ┌──[root@master]-[~] └─$ ls anaconda-ks.cfg initial-setup-ks.cfg 下载 公共 图片 文档 桌面 模板 视频 音乐 ┌──[root@master]-[~] └─$ cd /etc/yum.repos.d/ ┌──[root@master]-[/etc/yum.repos.d] └─$ ls CentOS-Base.repo CentOS-CR.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo CentOS-Media.repo CentOS-Sources.repo CentOS-Vault.repo CentOS-x86_64-kernel.repo ┌──[root@master]-[/etc/yum.repos.d] └─$ mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup ┌──[root@master]-[/etc/yum.repos.d] └─$ wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
查找 ansible安装包
┌──[root@master]-[/etc/yum.repos.d] └─$ yum list | grep ansible ansible-collection-microsoft-sql.noarch 1.1.0-1.el7_9 extras centos-release-ansible-27.noarch 1-1.el7 extras centos-release-ansible-28.noarch 1-1.el7 extras centos-release-ansible-29.noarch 1-1.el7 extras centos-release-ansible26.noarch 1-3.el7.centos extras ┌──[root@master]-[/etc/yum.repos.d]
阿里云的yum镜像没有ansible包，所以我们需要使用epel安装
┌──[root@master]-[/etc/yum.repos.d] └─$ wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo --2021-09-11 00:40:11-- http://mirrors.aliyun.com/repo/epel-7.repo Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 1.180.13.237, 1.180.13.236, 1.180.13.240, ... Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|1.180.13.237|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 664 [application/octet-stream] Saving to: ‘/etc/yum.repos.d/epel.repo’ 100%[=======================================================================================================================================================================>] 664 --.-K/s in 0s 2021-09-11 00:40:12 (91.9 MB/s) - ‘/etc/yum.repos.d/epel.repo’ saved [664/664] ┌──[root@master]-[/etc/yum.repos.d] └─$ yum install -y epel-release
查找ansible安装包，并安装
┌──[root@master]-[/etc/yum.repos.d] └─$ yum list|grep ansible Existing lock /var/run/yum.pid: another copy is running as pid 12522. Another app is currently holding the yum lock; waiting for it to exit... The other application is: PackageKit Memory : 28 M RSS (373 MB VSZ) Started: Sat Sep 11 00:40:41 2021 - 00:06 ago State : Sleeping, pid: 12522 ansible.noarch 2.9.25-1.el7 epel ansible-collection-microsoft-sql.noarch 1.1.0-1.el7_9 extras ansible-doc.noarch 2.9.25-1.el7 epel ansible-inventory-grapher.noarch 2.4.4-1.el7 epel ansible-lint.noarch 3.5.1-1.el7 epel ansible-openstack-modules.noarch 0-20140902git79d751a.el7 epel ansible-python3.noarch 2.9.25-1.el7 epel ansible-review.noarch 0.13.4-1.el7 epel ansible-test.noarch 2.9.25-1.el7 epel centos-release-ansible-27.noarch 1-1.el7 extras centos-release-ansible-28.noarch 1-1.el7 extras centos-release-ansible-29.noarch 1-1.el7 extras centos-release-ansible26.noarch 1-3.el7.centos extras Kubernetes-ansible.noarch 0.6.0-0.1.gitd65ebd5.el7 epel python2-ansible-runner.noarch 1.0.1-1.el7 epel python2-ansible-tower-cli.noarch 3.3.9-1.el7 epel vim-ansible.noarch 3.2-1.el7 epel ┌──[root@master]-[/etc/yum.repos.d] └─$ yum install -y ansible
┌──[root@master]-[/etc/yum.repos.d] └─$ ansible --version ansible 2.9.25 config file = /etc/ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] ┌──[root@master]-[/etc/yum.repos.d] └─$
查看主机清单
┌──[root@master]-[/etc/yum.repos.d] └─$ ansible 127.0.0.1 --list-hosts hosts (1): 127.0.0.1 ┌──[root@master]-[/etc/yum.repos.d]
2. ansible环境配置
我们这里使用liruilong这个普通账号，一开始装机配置的那个用户，生产中会配置特定的用户，不使用root用户;
1. 主配置文件 ansible.cfg 编写
┌──[root@master]-[/home/liruilong] └─$ su liruilong [liruilong@master ~]$ pwd /home/liruilong [liruilong@master ~]$ mkdir ansible;cd ansible;vim ansible.cfg [liruilong@master ansible]$ cat ansible.cfg [defaults] # 主机清单文件，就是要控制的主机列表 inventory=inventory # 连接受管机器的远程的用户名 remote_user=liruilong # 角色目录 roles_path=roles # 设置用户的su 提权 [privilege_escalation] become=True become_method=sudo become_user=root become_ask_pass=False [liruilong@master ansible]$
2. 主机清单：
被控机列表，可以是 域名，IP，分组([组名])，聚合（[组名:children]），也可以主动的设置用户名密码
[liruilong@master ansible]$ vim inventory [liruilong@master ansible]$ cat inventory [nodes] node1 node2 [liruilong@master ansible]$ ansible all --list-hosts hosts (2): node1 node2 [liruilong@master ansible]$ ansible nodes --list-hosts hosts (2): node1 node2 [liruilong@master ansible]$ ls ansible.cfg inventory [liruilong@master ansible]$
3. 配置liruilong用户的ssh免密
master节点上以liruilong用户对三个节点分布配置
[liruilong@master ansible]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/liruilong/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/liruilong/.ssh/id_rsa. Your public key has been saved in /home/liruilong/.ssh/id_rsa.pub. The key fingerprint is: SHA256:cJ+SHgfMk00X99oCwEVPi1Rjoep7Agfz8DTjvtQv0T0 liruilong@master The key's randomart image is: +---[RSA 2048]----+ | .oo*oB. | | o +.+ B + | | . B . + o .| | o+=+o . o | | SO=o .o..| | ..==.. .E.| | .+o .. .| | .o.o. | | o+ .. | +----[SHA256]-----+
[liruilong@master ansible]$ ssh-copy-id node1 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/liruilong/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys liruilong@node1's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'node1'" and check to make sure that only the key(s) you wanted were added.
嗯 ，node2和mater也需要配置
[liruilong@master ansible]$ ssh-copy-id node2 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/liruilong/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys liruilong@node2's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'node2'" and check to make sure that only the key(s) you wanted were added. [liruilong@master ansible]$
4. 配置liruilong普通用户提权
这里有个问题，我的机器上配置了sudo免密，但是第一次没有生效，需要输入密码，之后就不需要了，使用ansible还是不行。后来发现，在/etc/sudoers.d 下新建一个以普通用户命名的文件的授权就可以了，不知道啥原因了。
node1
┌──[root@node1]-[~] └─$ visudo ┌──[root@node1]-[~] └─$ cat /etc/sudoers | grep liruilong liruilong ALL=(ALL) NOPASSWD:ALL ┌──[root@node1]-[/etc/sudoers.d] └─$ cd /etc/sudoers.d/ ┌──[root@node1]-[/etc/sudoers.d] └─$ vim liruilong ┌──[root@node1]-[/etc/sudoers.d] └─$ cat liruilong liruilong ALL=(ALL) NOPASSWD:ALL ┌──[root@node1]-[/etc/sudoers.d] └─$
┌──[root@node2]-[~] └─$ vim /etc/sudoers.d/liruilong
node2 和 master 按照相同的方式设置
5. 测试临时命令
ansible 清单主机地址列表 -m 模块名 [-a '任务参数']
[liruilong@master ansible]$ ansible all -m ping node2 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" } node1 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" } [liruilong@master ansible]$ ansible nodes -m command -a 'ip a list ens33' node2 | CHANGED | rc=0 >> 2: ens33:
嗯，到这一步，ansible我们就配置完成了，可以在当前环境学习ansible。这是我ansible学习整理的笔记，主要是CHRE考试的笔记，有些实战，感兴趣小伙伴可以看看
关于docker以及k8s的安装，我们可以通过rhel-system-roles基于角色进行安装，也可以自定义角色进行安装，也可以直接写剧本进行安装，这里我们使用直接部署ansible剧本的方式，一步一步构建。
docker的话，感兴趣的小伙伴可以看看我的笔记。容器化技术学习笔记 我们主要看看K8S，
1. 使用ansible部署Docker
这里部署的话，一种是直接刷大佬写好的脚本，一种是自己一步一步来，这里我们使用第二种方式。
我们现在有的机器
1. 配置节点机yum源
这里因为我们要用节点机装包，所以需要配置yum源，ansible配置的方式有很多，可以通过yum_repository配置，我们这里为了方便，直接使用执行shell的方式。
[liruilong@master ansible]$ ansible nodes -m shell -a 'mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup;wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo' node2 | CHANGED | rc=0 >> --2021-09-11 11:40:20-- http://mirrors.aliyun.com/repo/Centos-7.repo Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 1.180.13.241, 1.180.13.238, 1.180.13.237, ... Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|1.180.13.241|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2523 (2.5K) [application/octet-stream] Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’ 0K .. 100% 3.99M=0.001s 2021-09-11 11:40:20 (3.99 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2523/2523] node1 | CHANGED | rc=0 >> --2021-09-11 11:40:20-- http://mirrors.aliyun.com/repo/Centos-7.repo Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 1.180.13.241, 1.180.13.238, 1.180.13.237, ... Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|1.180.13.241|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2523 (2.5K) [application/octet-stream] Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’ 0K .. 100% 346M=0s 2021-09-11 11:40:20 (346 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2523/2523] [liruilong@master ansible]$
配置好了yum源，我们需要确认一下
[liruilong@master ansible]$ ansible all -m shell -a 'yum repolist | grep aliyun' [liruilong@master ansible]$
2. 配置时间同步
这里为了方便。我们直接使用 ansible角色 安装RHEL角色软件包，拷贝角色目录到角色目录下，并创建剧本 timesync.yml
┌──[root@master]-[/home/liruilong/ansible] └─$ yum -y install rhel-system-roles 已加载插件：fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com base | 3.6 kB 00:00:00 epel | 4.7 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/2): epel/x86_64/updateinfo | 1.0 MB 00:00:00 (2/2): epel/x86_64/primary_db | 7.0 MB 00:00:01 正在解决依赖关系 There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help). --> 正在检查事务 ---> 软件包 rhel-system-roles.noarch.0.1.0.1-4.el7_9 将被 安装 --> 解决依赖关系完成 依赖关系解决 ======================================================================================================================== Package 架构 版本 源 大小 ======================================================================================================================== 正在安装: rhel-system-roles noarch 1.0.1-4.el7_9 extras 988 k 事务概要 ======================================================================================================================== 安装 1 软件包 总下载量：988 k 安装大小：4.8 M Downloading packages: rhel-system-roles-1.0.1-4.el7_9.noarch.rpm | 988 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction 正在安装 : rhel-system-roles-1.0.1-4.el7_9.noarch 1/1 验证中 : rhel-system-roles-1.0.1-4.el7_9.noarch 1/1 已安装: rhel-system-roles.noarch 0:1.0.1-4.el7_9 完毕！ ┌──[root@master]-[/home/liruilong/ansible] └─$ su - liruilong 上一次登录：六 9月 11 13:16:23 CST 2021pts/2 上 [liruilong@master ~]$ cd /home/liruilong/ansible/ [liruilong@master ansible]$ ls ansible.cfg inventory [liruilong@master ansible]$ cp -r /usr/share/ansible/roles/rhel-system-roles.timesync roles/
[liruilong@master ansible]$ ls ansible.cfg inventory roles timesync.yml [liruilong@master ansible]$ cat timesync.yml - name: timesync hosts: all vars: - timesync_ntp_servers: - hostname: 192.168.1.10 iburst: yes roles: - rhel-system-roles.timesync [liruilong@master ansible]$
3. docker环境初始化
编写 docker环境初始化的剧本 install_docker_playbook.yml
- name: install docker on node1,node2 hosts: node1,node2 tasks: - yum: name=docker state=absent - yum: name=docker state=present - yum: name=firewalld state=absent - shell: echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf - shell: sysctl -p - shell: sed -i '18 i ExecStartPort=/sbin/iptables -P FORWARD ACCEPT' /lib/systemd/system/docker.service - shell: cat /lib/systemd/system/docker.service - shell: systemctl daemon-reload - service: name=docker state=restarted enabled=yes
执行剧本
[liruilong@master ansible]$ cat install_docker_playbook.yml - name: install docker on node1,node2 hosts: node1,node2 tasks: - yum: name=docker state=absent - yum: name=docker state=present - yum: name=firewalld state=absent - shell: echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf - shell: sysctl -p - shell: sed -i '18 i ExecStartPort=/sbin/iptables -P FORWARD ACCEPT' /lib/systemd/system/docker.service - shell: cat /lib/systemd/system/docker.service - shell: systemctl daemon-reload - service: name=docker state=restarted enabled=yes [liruilong@master ansible]$ ls ansible.cfg install_docker_check.yml install_docker_playbook.yml inventory roles timesync.yml [liruilong@master ansible]$ ansible-playbook install_docker_playbook.yml
然后，我们编写一个检查的剧本install_docker_check.yml ，用来检查docker的安装情况
- name: install_docker-check hosts: node1,node2 ignore_errors: yes tasks: - shell: docker info register: out - debug: msg="{{out}}" - shell: systemctl -all | grep firewalld register: out1 - debug: msg="{{out1}}" - shell: cat /etc/sysctl.conf register: out2 - debug: msg="{{out2}}" - shell: cat /lib/systemd/system/docker.service register: out3 - debug: msg="{{out3}}"
[liruilong@master ansible]$ ls ansible.cfg install_docker_check.yml install_docker_playbook.yml inventory roles timesync.yml [liruilong@master ansible]$ cat install_docker_check.yml - name: install_docker-check hosts: node1,node2 ignore_errors: yes tasks: - shell: docker info register: out - debug: msg="{{out}}" - shell: systemctl -all | grep firewalld register: out1 - debug: msg="{{out1}}" - shell: cat /etc/sysctl.conf register: out2 - debug: msg="{{out2}}" - shell: cat /lib/systemd/system/docker.service register: out3 - debug: msg="{{out3}}" [liruilong@master ansible]$ ansible-playbook install_docker_check.yml
2. etcd 安装
安装etcd(键值型数据库)，在Kube-master上操作，创建配置网络
编写ansible剧本 install_etcd_playbook.yml
- name: install etcd or master hosts: 127.0.0.1 tasks: - yum: name=etcd state=present - lineinfile: path=/etc/etcd/etcd.conf regexp=ETCD_LISTEN_CLIENT_URLS="http://localhost:2379" line=ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" - shell: cat /etc/etcd/etcd.conf register: out - debug: msg="{{out}}" - service: name=etcd state=restarted enabled=yes
[liruilong@master ansible]$ ls ansible.cfg install_docker_playbook.yml inventory timesync.yml install_docker_check.yml install_etcd_playbook.yml roles [liruilong@master ansible]$ cat install_etcd_playbook.yml - name: install etcd or master hosts: 127.0.0.1 tasks: - yum: name=etcd state=present - lineinfile: path=/etc/etcd/etcd.conf regexp=ETCD_LISTEN_CLIENT_URLS="http://localhost:2379" line=ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" - shell: cat /etc/etcd/etcd.conf register: out - debug: msg="{{out}}" - service: name=etcd state=restarted enabled=yes [liruilong@master ansible]$ ansible-playbook install_etcd_playbook.yml
1. 创建配置网络：10.254.0.0/16
[liruilong@master ansible]$ etcdctl ls / [liruilong@master ansible]$ etcdctl mk /atomic.io/network/config '{"Network": "10.254.0.0/16", "Backend": {"Type":"vxlan"}} ' {"Network": "10.254.0.0/16", "Backend": {"Type":"vxlan"}} [liruilong@master ansible]$ etcdctl ls / /atomic.io [liruilong@master ansible]$ etcdctl ls /atomic.io /atomic.io/network [liruilong@master ansible]$ etcdctl ls /atomic.io/network /atomic.io/network/config [liruilong@master ansible]$ etcdctl get /atomic.io/network/config {"Network": "10.254.0.0/16", "Backend": {"Type":"vxlan"}} [liruilong@master ansible]$
3. flannel 安装配置（k8s所有机器上操作）
flannel是一个网络规划服务，它的功能是让k8s集群中，不同节点主机创建的docker容器，都具有在集群中唯一的虚拟IP地址。flannel 还可以在这些虚拟机IP地址之间建立一个覆盖网络，通过这个覆盖网络，实现不同主机内的容器互联互通；嗯，类似一个vlan的作用。
kube-master 管理主机上没有docker，只需要安装flannel，修改配置，启动并设置开机自启动即可。
1. ansible 主机清单添加 master节点
嗯，这里因为master节点机需要装包配置，所以我们在主机清单里加了master节点
[liruilong@master ansible]$ sudo cat /etc/hosts 192.168.1.11 node2 192.168.1.9 node1 192.168.1.10 master [liruilong@master ansible]$ ls ansible.cfg install_docker_playbook.yml inventory timesync.yml install_docker_check.yml install_etcd_playbook.yml roles [liruilong@master ansible]$ cat inventory master [nodes] node1 node2 [liruilong@master ansible]$ ansible master -m ping master | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" }
2. flannel 安装配置
编写剧本 install_flannel_playbook.yml：
- name: install flannel or all hosts: all vars: group_node: nodes tasks: - yum: name: flannel state: present - lineinfile: path: /etc/sysconfig/flanneld regexp: FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379" line: FLANNEL_ETCD_ENDPOINTS="http://192.168.1.10:2379" - service: name: docker state: stopped when: group_node in group_names - service: name: flanneld state: restarted enabled: yes - service: name: docker state: restarted when: group_node in group_names
执行剧本之前要把master的firewalld 关掉。也可以把2379端口放开
[liruilong@master ansible]$ su root 密码： ┌──[root@master]-[/home/liruilong/ansible] └─$ systemctl disable flanneld.service --now Removed symlink /etc/systemd/system/multi-user.target.wants/flanneld.service. Removed symlink /etc/systemd/system/docker.service.wants/flanneld.service. ┌──[root@master]-[/home/liruilong/ansible] └─$ systemctl status flanneld.service ● flanneld.service - Flanneld overlay address etcd agent Loaded: loaded (/usr/lib/systemd/system/flanneld.service; disabled; vendor preset: disabled) Active: inactive (dead) 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.046900 50344 manager.go:149] Using interface with name ens33 and address 192.168.1.10 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.046958 50344 manager.go:166] Defaulting external address to interface address (192.168.1.10) 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.056681 50344 local_manager.go:134] Found lease (10.254.68.0/24) for current IP (192..., reusing 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.060343 50344 manager.go:250] Lease acquired: 10.254.68.0/24 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.062427 50344 network.go:58] Watching for L3 misses 9月 12 18:34:24 master flanneld-start[50344]: I0912 18:34:24.062462 50344 network.go:66] Watching for new subnet leases 9月 12 18:34:24 master systemd[1]: Started Flanneld overlay address etcd agent. 9月 12 18:40:42 master systemd[1]: Stopping Flanneld overlay address etcd agent... 9月 12 18:40:42 master flanneld-start[50344]: I0912 18:40:42.194559 50344 main.go:172] Exiting... 9月 12 18:40:42 master systemd[1]: Stopped Flanneld overlay address etcd agent. Hint: Some lines were ellipsized, use -l to show in full. ┌──[root@master]-[/home/liruilong/ansible] └─$
┌──[root@master]-[/home/liruilong/ansible] └─$ cat install_flannel_playbook.yml - name: install flannel or all hosts: all vars: group_node: nodes tasks: - yum: name: flannel state: present - lineinfile: path: /etc/sysconfig/flanneld regexp: FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379" line: FLANNEL_ETCD_ENDPOINTS="http://192.168.1.10:2379" - service: name: docker state: stopped when: group_node in group_names - service: name: flanneld state: restarted enabled: yes - service: name: docker state: restarted when: group_node in group_names ┌──[root@master]-[/home/liruilong/ansible] └─$ ansible-playbook install_flannel_playbook.yml
3. 测试 flannel
这里也可以使用ansible 的docker相关模块处理，我们这里为了方便直接用shell模块
编写 install_flannel_check.yml
- name: flannel config check hosts: all vars: nodes: nodes tasks: - block: - shell: ifconfig docker0 | head -2 register: out - debug: msg="{{out}}" - shell: docker rm -f {{inventory_hostname}} - shell: docker run -itd --name {{inventory_hostname}} centos register: out1 - debug: msg="{{out1}}" when: nodes in group_names - shell: ifconfig flannel.1 | head -2 register: out - debug: msg="{{out}}"
执行剧本
[liruilong@master ansible]$ cat install_flannel_check.yml - name: flannel config check hosts: all vars: nodes: nodes tasks: - block: - shell: ifconfig docker0 | head -2 register: out - debug: msg="{{out}}" - shell: docker rm -f {{inventory_hostname}} - shell: docker run -itd --name {{inventory_hostname}} centos register: out1 - debug: msg="{{out1}}" when: nodes in group_names - shell: ifconfig flannel.1 | head -2 register: out - debug: msg="{{out}}" [liruilong@master ansible]$ [liruilong@master ansible]$ ansible-playbook install_flannel_check.yml PLAY [flannel config check] ************************************************************************************************************************************************************************************* TASK [Gathering Facts] ****************************************************************************************************************************************************************************************** ok: [master] ok: [node2] ok: [node1] TASK [shell] **************************************************************************************************************************************************************************************************** skipping: [master] changed: [node2] changed: [node1] TASK [debug] **************************************************************************************************************************************************************************************************** skipping: [master] ok: [node1] => { "msg": { "changed": true, "cmd": "ifconfig docker0 | head -2", "delta": "0:00:00.021769", "end": "2021-09-12 21:51:44.826682", "failed": false, "rc": 0, "start": "2021-09-12 21:51:44.804913", "stderr": "", "stderr_lines": [], "stdout": "docker0: flags=4163
验证node1上的centos容器能否ping通 node2上的centos容器
[liruilong@master ansible]$ ssh node1 Last login: Sun Sep 12 21:58:49 2021 from 192.168.1.10 [liruilong@node1 ~]$ sudo docker exec -it node1 /bin/bash [root@1c0628dcb7e7 /]# ip a 1: lo:
测试可以ping通，到这一步，我们配置了 flannel 网络，实现不同机器间容器互联互通
4. 安装部署 kube-master
嗯，网络配置好之后，我们要在master管理节点安装配置相应的kube-master。先看下有没有包
[liruilong@master ansible]$ yum list kubernetes-* 已加载插件：fastestmirror, langpacks Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com 可安装的软件包 kubernetes.x86_64 1.5.2-0.7.git269f928.el7 extras kubernetes-ansible.noarch 0.6.0-0.1.gitd65ebd5.el7 epel kubernetes-client.x86_64 1.5.2-0.7.git269f928.el7 extras kubernetes-master.x86_64 1.5.2-0.7.git269f928.el7 extras kubernetes-node.x86_64 1.5.2-0.7.git269f928.el7 extras [liruilong@master ansible]$ ls /etc/yum.repos.d/
嗯，如果有1.10的包，最好用 1.10 的，这里我们只有1.5 的就先用1.5 的试试，1.10 的yum源没找到
编写 install_kube-master_playbook.yml 剧本
- name: install kube-master or master hosts: master tasks: - shell: swapoff -a - replace: path: /etc/fstab regexp: "/dev/mapper/centos-swap" replace: "#/dev/mapper/centos-swap" - shell: cat /etc/fstab register: out - debug: msg="{{out}}" - shell: getenforce register: out - debug: msg="{{out}}" - shell: setenforce 0 when: out.stdout != "Disabled" - replace: path: /etc/selinux/config regexp: "SELINUX=enforcing" replace: "SELINUX=disabled" - shell: cat /etc/selinux/config register: out - debug: msg="{{out}}" - yum_repository: name: Kubernetes description: K8s aliyun yum baseurl: https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ gpgcheck: yes gpgkey: https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg repo_gpgcheck: yes enabled: yes - yum: name: kubernetes-master,kubernetes-client state: absent - yum: name: kubernetes-master state: present - yum: name: kubernetes-client state: present - lineinfile: path: /etc/kubernetes/config regexp: KUBE_MASTER="--master=http://127.0.0.1:8080" line: KUBE_MASTER="--master=http://192.168.1.10:8080" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1" line: KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379" line: KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.1.10:2379" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" line: KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" line: KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota" - service: name: kube-apiserver state: restarted enabled: yes - service: name: kube-controller-manager state: restarted enabled: yes - service: name: kube-scheduler state: restarted enabled: yes - shell: kubectl get cs register: out - debug: msg="{{out}}"
执行剧本
[liruilong@master ansible]$ ansible-playbook install_kube-master_playbook.yml ............ TASK [debug] ************************************************************************************************************************************************** ok: [master] => { "msg": { "changed": true, "cmd": "kubectl get cs", "delta": "0:00:05.653524", "end": "2021-09-12 23:44:58.030756", "failed": false, "rc": 0, "start": "2021-09-12 23:44:52.377232", "stderr": "", "stderr_lines": [], "stdout": "NAME STATUS MESSAGE ERROR\nscheduler Healthy ok \ncontroller-manager Healthy ok \netcd-0 Healthy {\"health\":\"true\"} ", "stdout_lines": [ "NAME STATUS MESSAGE ERROR", "scheduler Healthy ok ", "controller-manager Healthy ok ", "etcd-0 Healthy {\"health\":\"true\"} " ] } } PLAY RECAP **************************************************************************************************************************************************** master : ok=13 changed=4 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 [liruilong@master ansible]$ cat install_kube-master_playbook.yml - name: install kube-master or master hosts: master tasks: - shell: swapoff -a - replace: path: /etc/fstab regexp: "/dev/mapper/centos-swap" replace: "#/dev/mapper/centos-swap" - shell: cat /etc/fstab register: out - debug: msg="{{out}}" - shell: getenforce register: out - debug: msg="{{out}}" - shell: setenforce 0 when: out.stdout != "Disabled" - replace: path: /etc/selinux/config regexp: "SELINUX=enforcing" replace: "SELINUX=disabled" - shell: cat /etc/selinux/config register: out - debug: msg="{{out}}" - yum_repository: name: Kubernetes description: K8s aliyun yum baseurl: https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ gpgcheck: yes gpgkey: https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg repo_gpgcheck: yes enabled: yes - yum: name: kubernetes-master,kubernetes-client state: absent - yum: name: kubernetes-master state: present - yum: name: kubernetes-client state: present - lineinfile: path: /etc/kubernetes/config regexp: KUBE_MASTER="--master=http://127.0.0.1:8080" line: KUBE_MASTER="--master=http://192.168.1.10:8080" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1" line: KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379" line: KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.1.10:2379" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" line: KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" - lineinfile: path: /etc/kubernetes/apiserver regexp: KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota" line: KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota" - service: name: kube-apiserver state: restarted enabled: yes - service: name: kube-controller-manager state: restarted enabled: yes - service: name: kube-scheduler state: restarted enabled: yes - shell: kubectl get cs register: out - debug: msg="{{out}}" [liruilong@master ansible]$
5. 安装部署 kube-node
管理节点安装成功之后我们要部署相应的计算节点，kube-node 的安装 ( 在所有node服务器上部署 )
剧本编写： install_kube-node_playbook.yml
[liruilong@master ansible]$ cat - name: install kube-node or nodes hosts: nodes tasks: - shell: swapoff -a - replace: path: /etc/fstab regexp: "/dev/mapper/centos-swap" replace: "#/dev/mapper/centos-swap" - shell: cat /etc/fstab register: out - debug: msg="{{out}}" - shell: getenforce register: out - debug: msg="{{out}}" - shell: setenforce 0 when: out.stdout != "Disabled" - replace: path: /etc/selinux/config regexp: "SELINUX=enforcing" replace: "SELINUX=disabled" - shell: cat /etc/selinux/config register: out - debug: msg="{{out}}" - yum_repository: name: Kubernetes description: K8s aliyun yum baseurl: https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ gpgcheck: yes gpgkey: https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg repo_gpgcheck: yes enabled: yes - yum: name: kubernetes-node state: absent - yum: name: kubernetes-node state: present - lineinfile: path: /etc/kubernetes/config regexp: KUBE_MASTER="--master=http://127.0.0.1:8080" line: KUBE_MASTER="--master=http://192.168.1.10:8080" - lineinfile: path: /etc/kubernetes/kubelet regexp: KUBELET_ADDRESS="--address=127.0.0.1" line: KUBELET_ADDRESS="--address=0.0.0.0" - lineinfile: path: /etc/kubernetes/kubelet regexp: KUBELET_HOSTNAME="--hostname-override=127.0.0.1" line: KUBELET_HOSTNAME="--hostname-override={{inventory_hostname}}" - lineinfile: path: /etc/kubernetes/kubelet regexp: KUBELET_API_SERVER="--api-servers=http://127.0.0.1:8080" line: KUBELET_API_SERVER="--api-servers=http://192.168.1.10:8080" - lineinfile: path: /etc/kubernetes/kubelet regexp: KUBELET_ARGS="" line: KUBELET_ARGS="--cgroup-driver=systemd --kubeconfig=/etc/kubernetes/kubelet.kubeconfig" - shell: kubectl config set-cluster local --server="http://192.168.1.10:8080" - shell: kubectl config set-context --cluster="local" local - shell: kubectl config set current-context local - shell: kubectl config view register: out - debug: msg="{{out}}" - copy: dest: /etc/kubernetes/kubelet.kubeconfig content: "{{out.stdout}}" force: yes - shell: docker pull tianyebj/pod-infrastructure:latest - service: name: kubelet state: restarted enabled: yes - service: name: kube-proxy state: restarted enabled: yes - name: service check hosts: master tasks: - shell: sleep 10 async: 11 - shell: kubectl get node register: out - debug: msg="{{out}}"
执行剧本 install_kube-node_playbook.yml
[liruilong@master ansible]$ ansible-playbook install_kube-node_playbook.yml ........ ... TASK [debug] ************************************************************************************************************************************************************************************************** ok: [master] => { "msg": { "changed": true, "cmd": "kubectl get node", "delta": "0:00:00.579772", "end": "2021-09-15 02:00:34.829752", "failed": false, "rc": 0, "start": "2021-09-15 02:00:34.249980", "stderr": "", "stderr_lines": [], "stdout": "NAME STATUS AGE\nnode1 Ready 1d\nnode2 Ready 1d", "stdout_lines": [ "NAME STATUS AGE", "node1 Ready 1d", "node2 Ready 1d" ] } } PLAY RECAP **************************************************************************************************************************************************************************************************** master : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 node1 : ok=27 changed=19 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 node2 : ok=27 changed=19 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 [liruilong@master ansible]$
6. 安装部署 kube-dashboard
dashboard 镜像安装:kubernetes-dashboard 是 kubernetes 的web管理面板.这里的话一定要和K8s的版本对应，包括配置文件
[liruilong@master ansible]$ ansible node1 -m shell -a 'docker search kubernetes-dashboard' [liruilong@master ansible]$ ansible node1 -m shell -a 'docker pull docker.io/rainf/kubernetes-dashboard-amd64'
kube-dashboard.yaml 文件，修改dashboard的yaml文件，在kube-master上操作
kind: Deployment apiVersion: extensions/v1beta1 metadata: labels: app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: replicas: 1 selector: matchLabels: app: kubernetes-dashboard template: metadata: labels: app: kubernetes-dashboard # Comment the following annotation if Dashboard must not be deployed on master annotations: scheduler.alpha.kubernetes.io/tolerations: | [ { "key": "dedicated", "operator": "Equal", "value": "master", "effect": "NoSchedule" } ] spec: containers: - name: kubernetes-dashboard image: docker.io/rainf/kubernetes-dashboard-amd64 #默认的镜像是使用google的，这里改成docker仓库的 imagePullPolicy: Always ports: - containerPort: 9090 protocol: TCP args: # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. - --apiserver-host=http://192.168.1.10:8080 #注意这里是api的地址 livenessProbe: httpGet: path: / port: 9090 initialDelaySeconds: 30 timeoutSeconds: 30 --- kind: Service apiVersion: v1 metadata: labels: app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: type: NodePort ports: - port: 80 targetPort: 9090 nodePort: 30090 selector: app: kubernetes-dashboard
根据yaml文件，创建dashboard容器，在kube-master上操作
[liruilong@master ansible]$ vim kube-dashboard.yaml [liruilong@master ansible]$ kubectl create -f kubernetes-dashboard.yaml deployment "kubernetes-dashboard" created service "kubernetes-dashboard" created [liruilong@master ansible]$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system kubernetes-dashboard-1953799730-jjdfj 1/1 Running 0 6s [liruilong@master ansible]$
看一下在那个节点上，然后访问试试
[liruilong@master ansible]$ ansible nodes -a "docker ps" node2 | CHANGED | rc=0 >> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 14433d421746 docker.io/rainf/kubernetes-dashboard-amd64 "/dashboard --port..." 10 minutes ago Up 10 minutes k8s_kubernetes-dashboard.c82dac6b_kubernetes-dashboard-1953799730-jjdfj_kube-system_ea2ec370-1594-11ec-bbb1-000c294efe34_9c65bb2a afc4d4a56eab registry.access.redhat.com/rhel7/pod-infrastructure:latest "/usr/bin/pod" 10 minutes ago Up 10 minutes k8s_POD.28c50bab_kubernetes-dashboard-1953799730-jjdfj_kube-system_ea2ec370-1594-11ec-bbb1-000c294efe34_6851b7ee node1 | CHANGED | rc=0 >> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [liruilong@master ansible]$
在 node2上，即可以通过 http://192.168.1.11:30090/ 访问，我们测试一下
嗯，到这里，就完成了全部的Linux+Docker+Ansible+K8S学习环境搭建。k8s的搭建方式有些落后，但是刚开始学习，慢慢来，接下来就进行愉快的 K8S学习吧。
Docker Kubernetes Linux
版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。
相关文章
推荐文章
最近发表
