linux （centos 8.1）生产环境基于9台物理机 安装 opentstack ussuri集群以及集成ceph (已转gitee)
生活随笔
收集整理的這篇文章主要介紹了
linux （centos 8.1）生产环境基于9台物理机 安装 opentstack ussuri集群以及集成ceph (已转gitee)
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
零 修訂記錄
| 1 | 新增 | 20210429 |
| 2 | 修訂 | 20210504 |
一 摘要
本文介紹基于9臺物理機部署openstack ussuri版本 以及與ceph nautils集成。
二環境信息
（一）服務器信息
| procontroller01.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| procontroller02.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| procontroller03.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| procompute01.pro.chouniu.fun | 浪潮 SA5212M5 | 52182/1024G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| procompute02.pro.chouniu.fun | 浪潮 SA5212M5 | 52182/1024G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| procompute03.pro.chouniu.fun | 浪潮 SA5212M5 | 52182/1024G/SSD:240G2/SAS:2T 7.2K 2 /10G X7102/1G PHY卡*1/RAID卡 SAS3108 2GB | 1 |
| proceph01.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2 960G2 /SAS:8T 7.2K 6 /10G X7102/1G PHY卡1/RAID卡 SAS3108 2GB | 1 |
| proceph02.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2 960G2 /SAS:8T 7.2K 6 /10G X7102/1G PHY卡1/RAID卡 SAS3108 2GB | 1 |
| proceph03.pro.chouniu.fun | 浪潮 SA5212M5 | 42102/128G/SSD:240G2 960G2 /SAS:8T 7.2K 6 /10G X7102/1G PHY卡1/RAID卡 SAS3108 2GB | 1 |
（二）交換機信息
兩臺相同配置的交換機配置堆疊。
| A3_1F_DC_openstack_test_jieru_train-irf_b02&b03 | H3C LS-6860-54HF | 10G 光口48，40g 光口6 | 2 |
(三)軟件信息
2.3.1 操作系統
[root@localhost ~]# cat /etc/centos-release
CentOS Linux release 8.1.1911 (Core)
[root@localhost ~]# uname -a
Linux localhost.localdomain 4.18.0-147.el8.x86_64 #1 SMP Wed Dec 4 21:51:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost ~]#
2.3.2 docker
2.3.3 kolla
2.3.4 kolla-ansible
三實施
（一）部署規劃
3.1.1 部署網絡規劃
兩臺交互及做堆疊，服務器上有兩張萬兆網卡，每張網卡有兩個port，交叉制作bond0,bond1;
10.3.140.0/24 bond0 用于物理機地址及api管理
10.3.141.0/24 專用于存儲集群交換
10.3.142.0/24-10.3.149.0/24 用于虛擬機；
| procontroller01 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.11 | B02.40U | 1 | BAGG1111/LACP | access | 140 | API管理 |
| procontroller01 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 1 | BAGG1111/LACP | access | 140 | API管理 | |
| procontroller01 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 25 | BAGG25/LACP | trunck | permit 140-149 vlan | overlay |
| procontroller01 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 25 | BAGG25/LACP | trunck | permit 140-149 vlan | overlay | |
| procontroller02 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.12 | B02.40U | 2 | BAGG2/LACP | access | 140 | API管理 |
| procontroller02 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 2 | BAGG2/LACP | access | 140 | API管理 | |
| procontroller02 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 26 | BAGG26/LACP | trunck | permit 140-149 vlan | overlay |
| procontroller02 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 26 | BAGG26/LACP | trunck | permit 140-149 vlan | overlay | |
| procontroller03 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.13 | B02.40U | 3 | BAGG3/LACP | access | 140 | API管理 |
| procontroller03 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 3 | BAGG3/LACP | access | 140 | API管理 | |
| procontroller03 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 27 | BAGG27/LACP | trunck | permit 140-149 vlan | overlay |
| procontroller03 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 27 | BAGG27/LACP | trunck | permit 140-149 vlan | overlay | |
| procompute01 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.21 | B02.40U | 4 | BAGG4/LACP | access | 140 | API管理 |
| procompute01 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 4 | BAGG4/LACP | access | 140 | API管理 | |
| procompute01 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 28 | BAGG28/LACP | trunck | permit 140-149 vlan | overlay |
| procompute01 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 28 | BAGG28/LACP | trunck | permit 140-149 vlan | overlay | |
| procompute02 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.22 | B02.40U | 5 | BAGG5/LACP | access | 140 | API管理 |
| procompute02 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 5 | BAGG5/LACP | access | 140 | API管理 | |
| procompute02 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 29 | BAGG29/LACP | trunck | permit 140-149 vlan | overlay |
| procompute02 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 29 | BAGG29/LACP | trunck | permit 140-149 vlan | overlay | |
| procompute03 | 萬兆光口1 | ens35f1 | mode4 | bond0:10.3.140.23 | B02.40U | 6 | BAGG6/LACP | access | 140 | API管理 |
| procompute03 | 萬兆光口3 | ens47f1 | mode4 | B03.40U | 6 | BAGG6/LACP | access | 140 | API管理 | |
| procompute03 | 萬兆光口2 | ens35f0 | mode4 | bond1:不配置地址 | B02.40U | 30 | BAGG30/LACP | trunck | permit 140-149 vlan | overlay |
| procompute03 | 萬兆光口4 | ens47f0 | mode4 | B03.40U | 30 | BAGG30/LACP | trunck | permit 140-149 vlan | overlay | |
| proceph01 | 萬兆光口1 | enp59s0f1 | mode4 | bond0:10.3.140.31 | B02.40U | 7 | BAGG7/LACP | access | 140 | API管理 |
| proceph01 | 萬兆光口3 | enp175s0f1 | mode4 | B03.40U | 7 | BAGG7/LACP | access | 140 | API管理 | |
| proceph01 | 萬兆光口2 | enp59s0f0 | mode4 | bond1:10.3.141.31 | B02.40U | 31 | BAGG31/LACP | access | 141 | 存儲專用網絡 |
| proceph01 | 萬兆光口4 | enp175s0f0 | mode4 | B03.40U | 31 | BAGG31/LACP | access | 141 | 存儲專用網絡 | |
| proceph02 | 萬兆光口1 | enp59s0f1 | mode4 | bond0:10.3.140.32 | B02.40U | 8 | BAGG8/LACP | access | 140 | API管理 |
| proceph02 | 萬兆光口3 | enp175s0f1 | mode4 | B03.40U | 8 | BAGG8/LACP | access | 140 | API管理 | |
| proceph02 | 萬兆光口2 | enp59s0f0 | mode4 | bond1:10.3.141.32 | B02.40U | 32 | BAGG32/LACP | access | 141 | 存儲專用網絡 |
| proceph02 | 萬兆光口4 | enp175s0f0 | mode4 | B03.40U | 32 | BAGG32/LACP | access | 141 | 存儲專用網絡 | |
| proceph03 | 萬兆光口1 | enp59s0f1 | mode4 | bond0:10.3.140.33 | B02.40U | 9 | BAGG9/LACP | access | 140 | API管理 |
| proceph03 | 萬兆光口3 | enp175s0f1 | mode4 | B03.40U | 9 | BAGG9/LACP | access | 140 | API管理 | |
| proceph03 | 萬兆光口2 | enp59s0f0 | mode4 | bond1:10.3.141.33 | B02.40U | 33 | BAGG33/LACP | access | 141 | 存儲專用網絡 |
| proceph03 | 萬兆光口4 | enp175s0f0 | mode4 | B03.40U | 33 | BAGG33/LACP | access | 141 | 存儲專用網絡 |
3.1.2 部署節點功能規劃
| procontroller01.pro.chouniu.fun | bond0 | 10.3.140.11 | 10.3.140.10 | kolla-ansible kolla ansible 部署服務器 控制節點 網絡節點 存儲服務 監控節點 |
| procontroller02.pro.chouniu.fun | bond0 | 10.3.140.12 | 10.3.140.10 | 控制節點 網絡節點 存儲服務 |
| procontroller03.pro.chouniu.fun | bond0 | 10.3.140.13 | 10.3.140.10 | 控制節點 存儲服務 網絡節點 |
| procompute01.pro.chouniu.fun | bond0 | 10.3.140.21 | 本機不需要使用 | 計算節點 |
| procompute02.pro.chouniu.fun | bond0 | 10.3.140.22 | 本機不需要使用 | 計算節點 |
| procompute03.pro.chouniu.fun | bond0 | 10.3.140.23 | 本機不需要使用 | 計算節點 |
| proceph01.pro.chouniu.fun | bond0 | 10.3.140.31 | 本機不需要使用 | 存儲節點 |
| proceph02.pro.chouniu.fun | bond0 | 10.3.140.32 | 本機不需要使用 | 存儲節點 |
| proceph03.pro.chouniu.fun | bond0 | 10.3.140.33 | 本機不需要使用 | 存儲節點 |
（二）部署ceph 集群
ceph 集群部署請參照linux （centos 7.6）生產環境基于三臺物理機 安裝 ceph 集群以及集成openstack
(三) 部署openstack ussuri集群
我這里是離線部署，關于如何下載openstack ussuri包，及相關依賴
請參考openstack ussuri版本基于內網三臺物理機集群kolla-ansible部署及與ceph 集群 集成
這里主要下載兩類
1.是鏡像
2.是部署節點安裝時需要的各種依賴
3.3.1 部署前準備（控制節點和計算節點都要）
3.3.1.1 關閉防火墻以及selinux
[dev@10-3-170-32 base]$ ansible-playbook closefirewalldandselinux.yml需要重啟機器
3.3.1.2 配置dns以及修改機器名
dns 服務器上配置參考下表
| procontroller01.pro.chouniu.fun | 10.3.140.11 | 無 |
| procontroller02.pro.chouniu.fun | 10.3.140.12 | 無 |
| procontroller03.pro.chouniu.fun | 10.3.140.13 | 無 |
| procompute01.pro.chouniu.fun | 10.3.140.21 | 無 |
| procompute02.pro.chouniu.fun | 10.3.140.22 | 無 |
| procompute03.pro.chouniu.fun | 10.3.140.23 | 無 |
| proceph01.pro.chouniu.fun | 10.3.140.31 | 存儲集群部署時已執行 |
| proceph02.pro.chouniu.fun | 10.3.140.32 | 存儲集群部署時已執行 |
| proceph03.pro.chouniu.fun | 10.3.140.33 | 存儲集群部署時已執行 |
| procloud.chouniu.fun | 10.3.140.10 | 虛IP 對應域名 |
修改機器名：
[root@localhost ~]# hostnamectl set-hostname procontroller01.pro.chouniu.fun [root@localhost ~]#以上以一臺機器為例，9臺機器均需設置，其中存儲節點在ceph 安裝時已設置，故此處只需設置6臺即可。
修改服務器dns 解析地址
[dev@10-3-170-32 base]$ ansible-playbook modifydns.yml修改ssh 配置文件 /etc/ssh/sshd_config，取消登錄時調用dns,重啟ssh
#UseDNS yes UseDNS no systemctl restart sshd3.3.1.3 更新yum 源
更新操作系統yum,更新docker 源
[dev@10-3-170-32 base]$ ansible-playbook updateyum.yml[dev@10-3-170-32 base]$ ansible-playbook updateansible.yml [root@procontroller01 ~]# ll /etc/yum.repos.d total 12 -rw-r--r-- 1 root root 501 Apr 29 14:55 CentOS-81.repo -rw-r--r-- 1 root root 91 Apr 29 15:26 ansible2910forcentos8.repo -rw-r--r-- 1 root root 104 Apr 29 14:55 docker1803forcentos8.repo [root@procontroller01 ~]#3.3.1.4 同步時間服務器
[dev@10-3-170-32 base]$ ansible-playbook modifychronyclient.yml3.3.1.5 安裝docker
控制節點 計算節點 逐臺安裝,以下一控制節點一為例只需
安裝、修改配置文件、設置開機啟動
我這里目前安裝的是docker 18.03,三臺節點都需要安裝
我這里已經配置了docker yum 源，直接yum install docker-ce 即可安裝，安裝時可能會報錯，主要是依賴包兼容問題。
安裝docker 異常處理參考openstack ussuri 版本 all-in-one 離線部署
若有該包兼容性錯誤，可以刪除
yum remove podmandocker 配置文件修改
新增 /etc/systemd/system/docker.service.d/kolla.conf
[root@procontroller01 ~]# cd /etc/systemd/system/[root@procontroller01 system]# mkdir docker.service.d [root@procontroller01 system]# cd docker.service.d/ [root@procontroller01 docker.service.d]# vi kolla.conf [root@procontroller01 docker.service.d]#文件內容如下
[Service] MountFlags=shared設置開機啟動
[root@procontroller01 docker.service.d]# systemctl daemon-reload [root@procontroller01 docker.service.d]# systemctl restart docker && systemctl enable docker && systemctl status docker Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service. ● docker.service - Docker Application Container EngineLoaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)Drop-In: /etc/systemd/system/docker.service.d└─kolla.confActive: active (running) since Thu 2021-04-29 15:01:14 CST; 126ms agoDocs: https://docs.docker.comMain PID: 36902 (dockerd)Tasks: 39 (limit: 32767)Memory: 55.6MCGroup: /system.slice/docker.service├─36902 /usr/bin/dockerd└─36914 docker-containerd --config /var/run/docker/containerd/containerd.tomlApr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.185105356+08:00" level=info msg="Graph> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.185371260+08:00" level=warning msg="Yo> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.185384710+08:00" level=warning msg="Yo> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.185806534+08:00" level=info msg="Loadi> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.272931021+08:00" level=info msg="Defau> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.331678817+08:00" level=info msg="Loadi> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.363650311+08:00" level=info msg="Docke> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.363785574+08:00" level=info msg="Daemo> Apr 29 15:01:14 procontroller01.pro.chouniu.fun dockerd[36902]: time="2021-04-29T15:01:14.369294430+08:00" level=info msg="API l> Apr 29 15:01:14 procontroller01.pro.chouniu.fun systemd[1]: Started Docker Application Container Engine.3.3.1.6 安裝pip
centos8.1 自帶python3，默然安裝了pip3.6 ，我們只需要做下鏈接即可。
3.3.2.2.1 制作pip鏈接
[root@procontroller01 ~]# whereis pip3.6 pip3: /usr/bin/pip3 /usr/bin/pip3.6 /usr/share/man/man1/pip3.1.gz [root@procontroller01 ~]# ln -s /usr/bin/pip3.6 /usr/bin/pip [root@procontroller01 ~]# pip -V pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6) [root@procontroller01 ~]#3.3.1.7 禁用libvirt(僅計算節點執行)
[root@procompute03 ~]# systemctl stop libvirtd.service && systemctl disable libvirtd.service && systemctl status libvirtd.service Removed /etc/systemd/system/multi-user.target.wants/libvirtd.service. Removed /etc/systemd/system/sockets.target.wants/virtlogd.socket. Removed /etc/systemd/system/sockets.target.wants/virtlockd.socket. ● libvirtd.service - Virtualization daemonLoaded: loaded (/usr/lib/systemd/system/libvirtd.service; disabled; vendor preset: enabled)Active: inactive (dead) since Thu 2021-04-29 16:40:50 CST; 108ms agoDocs: man:libvirtd(8)https://libvirt.orgMain PID: 5342 (code=exited, status=0/SUCCESS)CGroup: /system.slice/libvirtd.service├─5814 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/>└─5815 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/>Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq-dhcp[5814]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq-dhcp[5814]: DHCP, sockets bound exclusively to interface virbr0 Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq[5814]: no servers found in /etc/resolv.conf, will retry Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq[5814]: read /etc/hosts - 2 addresses Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq[5814]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses Apr 29 14:45:01 procompute03.pro.chouniu.fun dnsmasq-dhcp[5814]: read /var/lib/libvirt/dnsmasq/default.hostsfile Apr 29 14:49:16 procompute03.pro.chouniu.fun dnsmasq[5814]: reading /etc/resolv.conf Apr 29 14:49:16 procompute03.pro.chouniu.fun dnsmasq[5814]: using nameserver 10.3.157.201#53 Apr 29 16:40:50 procompute03.pro.chouniu.fun systemd[1]: Stopping Virtualization daemon.3.3.1.8 安裝docker python sdk (所有節點都需要安裝)
[root@procompute03 ~]# tar -zxvf /tmp/dockerpython.tgz dockerpython/ dockerpython/docker-4.2.1-py2.py3-none-any.whl dockerpython/requests-2.25.1-py2.py3-none-any.whl dockerpython/websocket_client-0.58.0-py2.py3-none-any.whl dockerpython/six-1.15.0-py2.py3-none-any.whl dockerpython/idna-2.10-py2.py3-none-any.whl dockerpython/chardet-4.0.0-py2.py3-none-any.whl dockerpython/certifi-2020.12.5-py2.py3-none-any.whl dockerpython/urllib3-1.26.4-py2.py3-none-any.whl [root@procompute03 ~]# pip install --no-index --find-links=/root/dockerpython docker==4.2.1 WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Collecting docker==4.2.1 Collecting websocket-client>=0.32.0 (from docker==4.2.1) Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python3.6/site-packages (from docker==4.2.1) Requirement already satisfied: six>=1.4.0 in /usr/lib/python3.6/site-packages (from docker==4.2.1) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker==4.2.1) Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker==4.2.1) Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests!=2.18.0,>=2.14.2->docker==4.2.1) Installing collected packages: websocket-client, docker Successfully installed docker-4.2.1 websocket-client-0.58.0 [root@procompute03 ~]#3.3.2 安裝部署服務(部署服務器)
3.3.2.1 安裝ansible
3.3.2.1.1 安裝ansible
[root@procontroller01 ~]# yum install ansible Failed to set locale, defaulting to C.UTF-8 ansible [root@procontroller01 ~]# ansible --version ansible 2.9.10config file = /etc/ansible/ansible.cfgconfigured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']ansible python module location = /usr/lib/python3.6/site-packages/ansibleexecutable location = /usr/bin/ansiblepython version = 3.6.8 (default, Nov 21 2019, 19:31:34) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] [root@procontroller01 ~]#3.3.2.1.2 修改ansible 配置文件
對ansible /etc/ansible/ansible.cfg 配置文件做了優化
先備份原文件：cp /etc/ansible/ansible.cfg /etc/ansible/ansible.cfg.bak.orig
新增如下內容：
[defaults] inventory = $HOME/ansible/hosts host_key_checking=False pipelining=True forks=103.3.2.2 安裝基礎依賴
centos8 使用dnf 安裝rpm 包，安裝python3-devel libffi-devel gcc openssl-devel python3-libselinux git vim bash-completion net-tools 基礎依賴包
3.3.2.2.1 安裝
[root@procontroller01 ussuri]# cd dependencies/ [root@procontroller01 dependencies]# ll total 31224 -rw-r--r-- 1 root root 280084 Jun 17 2020 bash-completion-2.7-5.el8.noarch.rpm -rw-r--r-- 1 root root 24564532 Jun 17 2020 gcc-8.3.1-4.5.el8.x86_64.rpm -rw-r--r-- 1 root root 190956 Jun 17 2020 git-2.18.2-2.el8_1.x86_64.rpm -rw-r--r-- 1 root root 29396 Jun 17 2020 libffi-devel-3.1-21.el8.i686.rpm -rw-r--r-- 1 root root 29376 Jun 17 2020 libffi-devel-3.1-21.el8.x86_64.rpm -rw-r--r-- 1 root root 330916 Jun 17 2020 net-tools-2.0-0.51.20160912git.el8.x86_64.rpm -rw-r--r-- 1 root root 2395376 Jun 17 2020 openssl-devel-1.1.1c-2.el8_1.1.i686.rpm -rw-r--r-- 1 root root 2395344 Jun 17 2020 openssl-devel-1.1.1c-2.el8_1.1.x86_64.rpm -rw-r--r-- 1 root root 290084 Jun 17 2020 python3-libselinux-2.9-2.1.el8.x86_64.rpm -rw-r--r-- 1 root root 16570 Jun 17 2020 python36-devel-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64.rpm -rw-r--r-- 1 root root 1427224 Jun 17 2020 vim-enhanced-8.0.1763-13.el8.x86_64.rpm [root@procontroller01 dependencies]# dnf install python3-devel libffi-devel gcc openssl-devel python3-libselinux git vim bash-completion net-tools Failed to set locale, defaulting to C.UTF-83.3.2.3 安裝kolla （部署節點）
[root@procontroller01 ussuri]# unzip kolla-10.1.0.zip Archive: kolla-10.1.0.zip fbea3bf26d93e6fd784dbd9967659f549df3ec2d3.3.2.3.1離線安裝
[root@procontroller01 ussuri]# cd kolla-10.1.0/ [root@procontroller01 kolla-10.1.0]# git init Initialized empty Git repository in /root/ussuri/kolla-10.1.0/.git/ [root@procontroller01 kolla-10.1.0]#安裝依賴包
[root@procontroller01 ~]# pip install --no-index --find-links=/root/software/ussuri/kollapip -r /root/software/ussuri/kolla-10.1.0/requirements.txt WARNING: Running pip安裝kolla
[root@procontroller01 ~]# pip install /root/software/ussuri/kolla-10.1.0/ WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Processing ./software/ussuri/kolla-10.1.0驗證
[root@procontroller01 ~]# kolla-build --version 0.0.0 [root@procontroller01 ~]#3.3.2.4 安裝kolla-ansible (部署節點)
本文安裝的是kolla-ansible 10.1.0 版本
3.3.2.4.1離線安裝
解壓
[root@procontroller01 ussuri]# unzip kolla-ansible-10.1.0.zip Archive: kolla-ansible-10.1.0.zip 6bba8cc52af3a26678da48129856f80c21eb8e38git 初始化
[root@procontroller01 ussuri]# cd kolla-ansible-10.1.0/ [root@procontroller01 kolla-ansible-10.1.0]# git init Initialized empty Git repository in /root/software/ussuri/kolla-ansible-10.1.0/.git/ [root@procontroller01 kolla-ansible-10.1.0]#安裝kolla-ansible 依賴包
[root@procontroller01 kolla-ansible-10.1.0]# pip install --no-index --find-links=/root/software/ussuri//kollaansiblepip -r /root/software/ussuri/kolla-ansible-10.1.0/requirements.txt WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python3.6/site-packages (from -r /root/software/ussuri/kolla-ansible-10.1.0/requirements.txt (line 1))安裝kolla-ansible
[root@procontroller01 kolla-ansible-10.1.0]# pip install /root/software/ussuri/kolla-ansible-10.1.0 WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead. Processing /root/software/ussuri/kolla-ansible-10.1.0驗證
[root@procontroller01 kolla-ansible-10.1.0]# kolla-ansible -h Usage: /usr/local/bin/kolla-ansible COMMAND [options]Options:--inventory, -i <inventory_path> Specify path to ansible inventory file--playbook, -p <playbook_path> Specify path to ansible playbook file--configdir <config_path> Specify path to directory with globals.yml--key -k <key_path> Specify path to ansible vault keyfile--help, -h Show this usage information3.3.2.4.2 新增kolla-ansilbe 配置文件
[root@procontroller01 kolla-ansible-10.1.0]# mkdir -p /etc/kolla [root@procontroller01 kolla-ansible-10.1.0]# cp -r /root/software/ussuri/kolla-ansible-10.1.0/etc/kolla/* /etc/kolla [root@procontroller01 kolla-ansible-10.1.0]# ll /etc/kolla/ total 36 -rw-r--r-- 1 root root 25509 Apr 29 15:55 globals.yml -rw-r--r-- 1 root root 5037 Apr 29 15:55 passwords.yml [root@procontroller01 kolla-ansible-10.1.0]#3.3.2.4.2 新增ansilbe 配置文件
[root@procontroller01 kolla-ansible-10.1.0]# mkdir /root/ansible [root@procontroller01 kolla-ansible-10.1.0]# cp /root/software/ussuri/kolla-ansible-10.1.0/ansible/inventory/* /root/ansible/ [root@procontroller01 kolla-ansible-10.1.0]# ll /root/ansible/ total 24 -rw-r--r-- 1 root root 9584 Apr 29 15:56 all-in-one -rw-r--r-- 1 root root 10058 Apr 29 15:56 multinode [root@procontroller01 kolla-ansible-10.1.0]#3.3.2.5 部署節點免密登錄到所有節點
部署節點能免密登錄到所有控制節點和計算節點，
下面僅示例登錄到控制節點1，剩下六臺 參考方法
3.3.3 kolla ansible 部署openstack準備工作
下面上傳的是train 版本鏡像，ussuri 版本上傳方法類似，只是上傳的內容以及鏡像tag 不同。
這里主要是配置文件的初步修改，鏡像上傳到內網鏡像中心,
我這里內網有個docker 倉庫，而且是https方式
https://registry.chouniu.fun
非https 方式，docker 需要配置。
3.3.3.1 鏡像上傳到內網鏡像中心
3.3.3.1.1 上傳到內網
部署節點新增目錄
[root@procontroller01 software]# mkdir -p ./openstack/train/images/kolla [root@procontroller01 software]# cd openstack/train/images/kolla/ [root@procontroller01 kolla]# pwd /root/software/openstack/train/images/kolla [root@procontroller01 kolla]#openstak 鏡像導入內網
此步驟略
我這邊鏡像到上傳到部署節點,總共134個鏡像
3.3.3.1.2 解壓鏡像壓縮包
解壓腳本
#! /bin/shfor i in ` ls -al /root/software/openstack/train/images/kolla/ | grep gz | awk '{print $9}' `;do gzip -d /root/software/openstack/train/images/kolla/$i; done3.3.3.1.3 批量加載鏡像
腳本
[root@procontroller01 scripts]# cat dockerload.sh #!/bin/bashpath=$1cd $pathfor filename in `ls`doecho $filenamedocker load < $filenamedone[root@procontroller01 scripts]#執行命令：
[root@procontroller01 scripts]# ./dockerload.sh /root/software/openstack/train/images/kolla/ centos-source-aodh-api.tar 174f56854903: Loading layer [==================================================>] 211.7MB/211.7MB 1f66661b2a8e: Loading layer [==================================================>] 84.99kB/84.99kB檢查，已全部上傳到部署節點
[root@procontroller01 kolla]# docker images | grep train | wc -l 134 [root@procontroller01 kolla]# [root@procontroller01 kolla]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE kolla/centos-source-nova-compute train bb089a9b5fdf 4 weeks ago 2.08GB kolla/centos-source-horizon train a4b1597d0c85 4 weeks ago 1.19GB3.3.3.1.4 上傳到內網docker 倉庫
[root@procontroller01 scripts]# for i in `docker images|grep -v registry | grep train|awk '{print $1}'`;do docker image tag $i:train registry.chouniu.fun/$i:train;done [root@procontroller01 scripts]#經比對 上面命令少執行了一條
[root@procontroller01 kolla]# docker images | grep centos-source-glance-registry kolla/centos-source-glance-registry train f59b22154362 4 weeks ago 1.02GB [root@procontroller01 kolla]# docker images|grep centos-source-glance-registry | grep train|awk '{print $1}' kolla/centos-source-glance-registry [root@procontroller01 kolla]# for i in `docker images|grep centos-source-glance-registry | grep train|awk '{print $1}'`;do docker image tag $i:train registry.chouniu.fun/$i:train;done [root@procontroller01 kolla]#先登錄私有倉庫
[root@procontroller01 kolla]# docker login -u admin -p passwordxxx registry.chouniu.fun WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store批量推送到私有倉庫
for i in ` docker images | grep kxdigit |awk '{print $1}'`;do docker push $i;done3.3.3.2 生成密碼文件并修改登錄密碼（部署節點）
先使用kolla-genpwd 生成全套密碼，然后為了前臺便于登錄，修改keystone_admin_password 密碼 為Admin_PASS2021
[root@procontroller01 scripts]# cd /etc/kolla/ [root@procontroller01 kolla]# ll 總用量 36 -rw-r--r--. 1 root root 24901 4月 27 11:07 globals.yml -rw-r--r--. 1 root root 5217 4月 27 11:07 passwords.yml [root@procontroller01 kolla]# kolla-genpwd [root@procontroller01 kolla]# ll 總用量 56 -rw-r--r--. 1 root root 24901 4月 27 11:07 globals.yml -rw-r--r--. 1 root root 25916 4月 27 17:42 passwords.yml [root@procontroller01 kolla]# vim passwords.yml [root@procontroller01 kolla]# keepalived_password: 02u493LqhiQ8RXp0Zux4bEeZs0sr0AbxfFjb6CvZ keystone_admin_password: Admin_PASS2021 keystone_database_password: DjD2mvrS0sgii9Wriz8emSBAekdBNXjDhjwwaa983.3.3.3 修改multinode（部署節點）
主要修改點有：
[control]
[network]
[compute]
[monitoring]
[storage]
這條我這里暫時沒改，先試試能不能用
[nova-compute-ironic:children] 配置改為compute
僅放了修改的地方
[root@procontroller01 ansible]# cat multinode [control] # These hostname must be resolvable from your deployment host #control01 #control02 #control03 procontroller01.pro.chouniu.fun procontroller02.pro.chouniu.fun procontroller03.pro.chouniu.fun# The above can also be specified as follows: #control[01:03] ansible_user=kolla# The network nodes are where your l3-agent and loadbalancers will run # This can be the same as a host in the control group [network] #network01 #network02 procontroller01.pro.chouniu.fun procontroller02.pro.chouniu.fun procontroller03.pro.chouniu.fun[compute] #compute01 procompute01.pro.chouniu.fun procompute02.pro.chouniu.fun procompute03.pro.chouniu.fun[monitoring] #monitoring01 procontroller01.pro.chouniu.fun# When compute nodes and control nodes use different interfaces, # you need to comment out "api_interface" and other interfaces from the globals.yml # and specify like below: #compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1[storage] #storage01 procontroller01.pro.chouniu.fun procontroller02.pro.chouniu.fun procontroller03.pro.chouniu.fun[deployment] localhost ansible_connection=local[nova-compute-ironic:children] #nova compute3.3.3.4 修改global.yml（部署節點）
此步暫略，后面與ceph 集成時也要修改該文件，整合到一起。
3.3.4 openstack與 ceph 集成
3.3.4.1 修改global.yml
[root@procontroller01 ansible]# cat /etc/kolla/globals.yml |grep -v "^#" | grep -v ^$ --- kolla_base_distro: "centos" kolla_install_type: "source" openstack_release: "ussuri" kolla_internal_vip_address: "10.3.140.10" docker_registry: registry.chouniu.fun network_interface: "bond0" tunnel_interface: "{{ network_interface }}" neutron_external_interface: "bond1" neutron_plugin_agent: "openvswitch" openstack_logging_debug: "True" enable_openstack_core: "yes" enable_haproxy: "yes" enable_cinder: "yes" enable_manila_backend_cephfs_native: "yes" ceph_glance_keyring: "ceph.client.glance.keyring" ceph_glance_user: "glance" ceph_glance_pool_name: "images" ceph_cinder_keyring: "ceph.client.cinder.keyring" ceph_cinder_user: "cinder" ceph_cinder_pool_name: "volumes" ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring" ceph_cinder_backup_user: "cinder-backup" ceph_cinder_backup_pool_name: "backups" ceph_nova_keyring: "{{ ceph_cinder_keyring }}" ceph_nova_user: "cinder" ceph_nova_pool_name: "vms" glance_backend_ceph: "yes" cinder_backend_ceph: "yes" nova_backend_ceph: "yes" nova_compute_virt_type: "qemu" [root@procontroller01 ansible]#3.3.4.2 ceph 狀態檢查
[cephadmin@proceph01 ~]$ ceph -scluster:id: ad0bf159-1b6f-472b-94de-83f713c339a3health: HEALTH_OKservices:mon: 3 daemons, quorum proceph01,proceph02,proceph03 (age 2d)mgr: proceph01(active, since 2d), standbys: proceph02, proceph03osd: 18 osds: 18 up (since 2d), 18 in (since 2d)data:pools: 4 pools, 544 pgsobjects: 4 objects, 76 Busage: 18 GiB used, 131 TiB / 131 TiB availpgs: 544 active+clean[cephadmin@proceph01 ~]$ [cephadmin@proceph01 ~]$ ceph health detail HEALTH_OK [cephadmin@proceph01 ~]$ [cephadmin@proceph01 ~]$ ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 130.98395 root default -3 43.66132 host proceph010 hdd 7.27689 osd.0 up 1.00000 1.000001 hdd 7.27689 osd.1 up 1.00000 1.000002 hdd 7.27689 osd.2 up 1.00000 1.000003 hdd 7.27689 osd.3 up 1.00000 1.000004 hdd 7.27689 osd.4 up 1.00000 1.000005 hdd 7.27689 osd.5 up 1.00000 1.00000 -5 43.66132 host proceph026 hdd 7.27689 osd.6 up 1.00000 1.000007 hdd 7.27689 osd.7 up 1.00000 1.000008 hdd 7.27689 osd.8 up 1.00000 1.000009 hdd 7.27689 osd.9 up 1.00000 1.00000 10 hdd 7.27689 osd.10 up 1.00000 1.00000 11 hdd 7.27689 osd.11 up 1.00000 1.00000 -7 43.66132 host proceph03 12 hdd 7.27689 osd.12 up 1.00000 1.00000 13 hdd 7.27689 osd.13 up 1.00000 1.00000 14 hdd 7.27689 osd.14 up 1.00000 1.00000 15 hdd 7.27689 osd.15 up 1.00000 1.00000 16 hdd 7.27689 osd.16 up 1.00000 1.00000 17 hdd 7.27689 osd.17 up 1.00000 1.00000 [cephadmin@proceph01 ~]$3.3.4.3 配置glance 使用ceph
3.3.4.3.1 生成 ceph.client.glance.keyring
在ceph 部署節點 生成 ceph.client.glance.keyring 文件
[cephadmin@proceph01 ~]$ ceph auth get-or-create client.glance | tee /etc/ceph/ceph.client.glance.keyring [client.glance]key = AQCYS4VgKKk7MRAAQZkwE3ISG1J+jsN1AcpqRg== [cephadmin@proceph01 ~]$ ll /etc/ceph/ total 16 -rw-------. 1 cephadmin cephadmin 151 Apr 23 18:01 ceph.client.admin.keyring -rw-rw-r-- 1 cephadmin cephadmin 64 Apr 27 19:03 ceph.client.glance.keyring -rw-r--r-- 1 root root 308 Apr 25 16:57 ceph.conf -rw-r--r--. 1 cephadmin cephadmin 92 Nov 24 03:33 rbdmap -rw-------. 1 cephadmin cephadmin 0 Apr 23 17:58 tmpd8sfbW [cephadmin@proceph01 ~]$3.3.4.3.2 koll-ansible 部署節點 新建/etc/kolla/config/glance
[root@procontroller01 ~]# mkdir -p /etc/kolla/config/glance [root@procontroller01 ~]# cd /etc/kolla/config/glance/ [root@procontroller01 glance]# ll 總用量 0 [root@procontroller01 glance]# pwd /etc/kolla/config/glance [root@procontroller01 glance]#3.3.4.3.3 ceph.client.glance.keyring 文件從ceph部署節點拷貝到openstack 部署節點該目錄下/etc/kolla/config/glance
[cephadmin@proceph01 ceph]$ scp /etc/ceph/ceph.conf root@10.3.140.11:/etc/kolla/config/glance/ root@10.3.140.11's password: ceph.conf 100% 308 588.9KB/s 00:00 [cephadmin@proceph01 ceph]$只保留如下內容
[global] fsid = ad0bf159-1b6f-472b-94de-83f713c339a3 mon_host = 10.3.140.31,10.3.140.32,10.3.140.333.3.4.4 配置cinder 使用ceph
3.3.4.4.1 ceph 部署節點生成 ceph.client.cinder.keyring 和 ceph.client.cinder-backup.keyring 密鑰文件
[cephadmin@proceph01 ceph]$ ceph auth get-or-create client.cinder | tee /etc/ceph/ceph.client.cinder.keyring [client.cinder]key = AQCrS4VgBJhtDhAA5X+b5N7GJMOK/4p12IwRBg== [cephadmin@proceph01 ceph]$ ceph auth get-or-create client.cinder-backup | tee /etc/ceph/ceph.client.cinder-backup.keyring [client.cinder-backup]key = AQC5S4VgmMBDIRAA6xnBwr04Ik0lmn0WIoD20Q== [cephadmin@proceph01 ceph]$ ll /etc/ceph/ total 24 -rw-------. 1 cephadmin cephadmin 151 Apr 23 18:01 ceph.client.admin.keyring -rw-rw-r-- 1 cephadmin cephadmin 71 Apr 27 19:13 ceph.client.cinder-backup.keyring -rw-rw-r-- 1 cephadmin cephadmin 64 Apr 27 19:13 ceph.client.cinder.keyring -rw-rw-r-- 1 cephadmin cephadmin 64 Apr 27 19:03 ceph.client.glance.keyring -rw-r--r-- 1 root root 308 Apr 25 16:57 ceph.conf -rw-r--r--. 1 cephadmin cephadmin 92 Nov 24 03:33 rbdmap -rw-------. 1 cephadmin cephadmin 0 Apr 23 17:58 tmpd8sfbW [cephadmin@proceph01 ceph]$3.3.4.4.2 kolla-ansible 部署節點新增cinder 配置文件目錄
[root@procontroller01 glance]# mkdir -p /etc/kolla/config/cinder/cinder-volume/ [root@procontroller01 glance]# mkdir -p /etc/kolla/config/cinder/cinder-backup [root@procontroller01 glance]# ll /etc/kolla/config/ 總用量 0 drwxr-xr-x. 4 root root 48 4月 27 19:14 cinder drwxr-xr-x. 2 root root 57 4月 27 19:14 glance [root@procontroller01 glance]# ll /etc/kolla/config/cinder/ 總用量 0 drwxr-xr-x. 2 root root 6 4月 27 19:14 cinder-backup drwxr-xr-x. 2 root root 6 4月 27 19:14 cinder-volume [root@procontroller01 glance]#3.3.4.4.3 cinder 相關密鑰文件拷貝到 kolla-ansbile 部署節點
[cephadmin@proceph01 ceph]$ scp /etc/ceph/ceph.client.cinder.keyring root@10.3.140.11:/etc/kolla/config/cinder/cinder-backup/ root@10.3.140.11's password: ceph.client.cinder.keyring 100% 64 121.4KB/s 00:00 [cephadmin@proceph01 ceph]$ scp /etc/ceph/ceph.client.cinder.keyring root@10.3.140.11:/etc/kolla/config/cinder/cinder-volume/ root@10.3.140.11's password: ceph.client.cinder.keyring 100% 64 124.8KB/s 00:00 [cephadmin@proceph01 ceph]$ scp /etc/ceph/ceph.client.cinder-backup.keyring root@10.3.140.11:/etc/kolla/config/cinder/cinder-backup/ root@10.3.140.11's password: ceph.client.cinder-backup.keyring 100% 71 141.6KB/s 00:00 [cephadmin@proceph01 ceph]$3.3.4.4.4 cinder 配置目錄下添加ceph
因為ceph.conf 配置文件內容 同上 3.3.4.3.4 ,故直接復制過來即可
[root@procontroller01 glance]# cp /etc/kolla/config/glance/ceph.conf /etc/kolla/config/cinder/ [root@procontroller01 glance]# ll /etc/kolla/config/cinder/ 總用量 4 -rw-r--r--. 1 root root 100 4月 27 19:19 ceph.conf drwxr-xr-x. 2 root root 81 4月 27 19:17 cinder-backup drwxr-xr-x. 2 root root 40 4月 27 19:17 cinder-volume [root@procontroller01 glance]#3.3.4.5 配置nova 使用ceph
3.3.4.5.1 kolla ansible 部署節點新建/etc/kolla/config/nova目錄
[root@procontroller01 glance]# mkdir -p /etc/kolla/config/nova [root@procontroller01 glance]# ll /etc/kolla/config/ 總用量 0 drwxr-xr-x. 4 root root 65 4月 27 19:19 cinder drwxr-xr-x. 2 root root 57 4月 27 19:14 glance drwxr-xr-x. 2 root root 6 4月 27 19:20 nova [root@procontroller01 glance]#3.3.4.5.2 拷貝ceph.client.cinder.keyring 到 /etc/kolla/config/nova
[cephadmin@proceph01 ceph]$ scp /etc/ceph/ceph.client.cinder.keyring root@10.3.140.11:/etc/kolla/config/nova/ root@10.3.140.11's password: ceph.client.cinder.keyring 100% 64 126.0KB/s 00:00 [cephadmin@proceph01 ceph]$3.3.4.5.3 配置ceph.conf
[root@procontroller01 glance]# cp /etc/kolla/config/glance/ceph.conf /etc/kolla/config/nova/ [root@procontroller01 glance]# ll /etc/kolla/config/nova/ 總用量 8 -rw-r--r--. 1 root root 64 4月 27 19:21 ceph.client.cinder.keyring -rw-r--r--. 1 root root 100 4月 27 19:23 ceph.conf [root@procontroller01 glance]#3.3.4.5.4 配置nova.conf
umap可以啟用trim
writeback設置虛擬機的disk cache
3.3.5 kolla-ansible 安裝openstack ussuri
3.3.5.1 預檢查
[root@procontroller01 kolla]# kolla-ansible -v -i /root/ansible/multinode prechecks3.3.5.2 拉去依賴包
[root@procontroller01 kolla]# kolla-ansible -v -i /root/ansible/multinode pull Pulling Docker images : ansible-playbook -i /root/ansible/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=pull /usr/local/share/kolla-ansible/ansible/site.yml --verbose Using /etc/ansible/ansible.cfg as config file3.3.5.3 部署
[root@procontroller01 kolla]# kolla-ansible -v -i /root/ansible/multinode deploy3.3.5.4 安裝openstackclient(控制計算節點)
[root@procontroller01 openstackclient]# pip install --no-index --find-links=/root/software/ussuri/openstackclient python_openstackclient==5.2.03.3.5.5 生成本地環境變量
[root@procontroller01 openstackclient]# kolla-ansible -v -i /root/ansible/multinode post-deploy Post-Deploying Playbooks : ansible-playbook -i /root/ansible/multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla /usr/local/share/kolla-ansible/ansible/post-deploy.yml --verbose[root@procontroller01 openstackclient]# source /etc/kolla/admin-openrc.sh [root@procontroller01 openstackclient]# openstack server list[root@procontroller01 openstackclient]# openstack nova list openstack: 'nova list' is not an openstack command. See 'openstack --help'. Did you mean one of these?quota listquota setquota show [root@procontroller01 openstackclient]#3.3.6 實施vlan 網絡
3.3.6.1 部署前
[root@procontroller01 templates]# pwd /usr/local/share/kolla-ansible/ansible/roles/neutron/templates [root@procontroller01 templates]# [root@procontroller01 templates]# cp ml2_conf.ini.j2 ml2_conf.ini.j2.bak.orig [root@procontroller01 templates]# vim ml2_conf.ini.j2修改內容 [ml2_type_vlan] {% if enable_ironic | bool %} network_vlan_ranges = physnet1 {% else %} network_vlan_ranges = physnet1:140:149 {% endif %}若在上一步驟已經部署過，且系統里沒有數據，可以destroy ，重新部署，重新推送配置文件
這一步謹慎執行，因為也可以直接修改配置文件
3.3.6.2 部署后修改文件
3.3.6.2.1 控制節點（網絡節點）配置文件
其實第一步也可以放在這里手動修改
/etc/kolla/neutron-server/ml2_conf.ini
主要修改了此句：tenant_network_types = vxlan,vlan,flat
重啟控制節點服務
docker restart neutron_server neutron_openvswitch_agent [root@procontroller03 ~]# docker exec -it -u root neutron_openvswitch_agent /bin/bash (neutron-openvswitch-agent)[root@procontroller03 /]# ovs-vsctl show 081a24d5-6033-4d2c-ace6-e47e591a8b6bManager "ptcp:6640:127.0.0.1"is_connected: trueBridge br-exController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort phy-br-exInterface phy-br-extype: patchoptions: {peer=int-br-ex}Port br-exInterface br-extype: internalPort bond1Interface bond1Bridge br-tunController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort br-tunInterface br-tuntype: internalPort patch-intInterface patch-inttype: patchoptions: {peer=patch-tun}Bridge br-intController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort patch-tunInterface patch-tuntype: patchoptions: {peer=patch-int}Port br-intInterface br-inttype: internalPort int-br-exInterface int-br-extype: patchoptions: {peer=phy-br-ex} (neutron-openvswitch-agent)[root@procontroller03 /]#3.3.6.2.2 計算節點
計算點 主要是修改neutron-openvswitch 交換機里面的配置，需要新增網橋并與外部物理網卡對應起來。
[root@procompute01 neutron-openvswitch-agent]# docker exec -it -u root neutron_openvswitch_agent /bin/bash (neutron-openvswitch-agent)[root@procompute01 /]# ovs-vsctl add-br br-ex (neutron-openvswitch-agent)[root@procompute01 /]# ovs-vsctl add-port br-ex bond1 (neutron-openvswitch-agent)[root@procompute01 /]# exit exit新增 bridge_mappings = physnet1:br-ex
[root@procompute01 neutron-openvswitch-agent]# cp openvswitch_agent.ini openvswitch_agent.ini.bak.orig [root@procompute01 neutron-openvswitch-agent]# vim openvswitch_agent.ini [root@procompute01 neutron-openvswitch-agent]#[agent] tunnel_types = vxlan l2_population = true arp_responder = true[securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver[ovs] bridge_mappings = physnet1:br-ex datapath_type = system ovsdb_connection = tcp:127.0.0.1:6640 local_ip = 10.3.140.21計算節點添加port、bridge 并將 bridge 指定網卡
[root@procompute01 neutron-openvswitch-agent]# docker restart neutron_openvswitch_agent neutron_openvswitch_agent [root@procompute01 neutron-openvswitch-agent]# docker exec -it -u root neutron_openvswitch_agent /bin/bash (neutron-openvswitch-agent)[root@procompute01 /]# ovs-vsctl show d90e91de-9445-47d3-b3c8-04937855be2aManager "ptcp:6640:127.0.0.1"is_connected: trueBridge br-exController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort phy-br-exInterface phy-br-extype: patchoptions: {peer=int-br-ex}Port br-exInterface br-extype: internalPort bond1Interface bond1Bridge br-tunController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort br-tunInterface br-tuntype: internalPort patch-intInterface patch-inttype: patchoptions: {peer=patch-tun}Bridge br-intController "tcp:127.0.0.1:6633"is_connected: truefail_mode: securedatapath_type: systemPort br-intInterface br-inttype: internalPort patch-tunInterface patch-tuntype: patchoptions: {peer=patch-int}Port int-br-exInterface int-br-extype: patchoptions: {peer=phy-br-ex} (neutron-openvswitch-agent)[root@procompute01 /]#重啟計算節點網絡服務器
[root@procompute01 ~]# docker restart neutron_openvswitch_agent neutron_openvswitch_agent [root@procompute01 ~]#3.3.6.3 運行initrunonce
修改原版initrunonce 鏡像地址，及網路參數即可
#IMAGE_PATH=/opt/cache/files/ IMAGE_PATH=/usr/local/share/kolla-ansible/ IMAGE_URL=https://github.com/cirros-dev/cirros/releases/download/0.5.1/ #IMAGE=cirros-0.5.1-${ARCH}-disk.img IMAGE=cirros-0.3.4-x86_64-disk.img IMAGE_NAME=cirros IMAGE_TYPE=linux# This EXT_NET_CIDR is your public network,that you want to connect to the internet via. ENABLE_EXT_NET=${ENABLE_EXT_NET:-1} EXT_NET_CIDR=${EXT_NET_CIDR:-'10.3.140.0/24'} EXT_NET_RANGE=${EXT_NET_RANGE:-'start=10.3.140.150,end=10.3.140.199'} EXT_NET_GATEWAY=${EXT_NET_GATEWAY:-'10.3.140.1'}$KOLLA_OPENSTACK_COMMAND network create demo-net $KOLLA_OPENSTACK_COMMAND subnet create --subnet-range 172.31.164.0/24 --network demo-net \--gateway 172.31.164.1 --dns-nameserver 8.8.8.8 demo-subnet執行命令
[root@procontroller01 kolla-ansible]# source /etc/kolla/admin-openrc.sh [root@procontroller01 kolla-ansible]# source /usr/local/share/kolla-ansible/init-runonce Checking for locally available cirros image. Using cached cirros image from the nodepool node. Creating glance image.3.3.6.4 手動創建網絡
有兩種方法通過命令行或者通過dashboard
3.3.6.4.1 通過命令行創建網絡
創建142 vlan
[root@procontroller01 kolla-ansible]# openstack network create provider_vlan_net142 --project admin --provider-network-type vlan --external --provider-physical-network physnet1 --provider-segment 142 --share[root@procontroller01 kolla-ansible]# openstack subnet create vlan_subnet142 --project admin --network provider_vlan_net142 --subnet-range 10.3.142.0/24 --allocation-pool start=10.3.142.10,end=10.3.142.2503.3.6.4.2 dashboard創建網絡
登錄dashboard,》管理員》網絡==》創建網絡
[root@procontroller01 kolla-ansible]# openstack network list +--------------------------------------+----------------------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+----------------------+--------------------------------------+ | 7536754e-f9b1-4939-8900-3bbed9d706ad | provider_vlan_net143 | a1e3da64-b633-42c0-b455-e44e0656571c | | a14f90bc-2094-41a1-bf23-1daa4618ad59 | provider_vlan_net142 | a524df0d-5ccb-4817-a5e0-3b45b11eef8b | +--------------------------------------+----------------------+--------------------------------------+ [root@procontroller01 kolla-ansible]#3.3.6.5 測試驗證
總結
以上是生活随笔為你收集整理的linux （centos 8.1）生产环境基于9台物理机 安装 opentstack ussuri集群以及集成ceph (已转gitee)的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： regnam r语言_求助：R语言进行增
- 下一篇： linux 联机游戏下载,星露谷物语多人