1.1 環(huán)境準(zhǔn)備

主機(jī)名IP地址

ceph-admin	192.168.16.220
ceph-node1,ceph-mon	192.168.16.221
ceph-node2,ceph-mon	192.168.16.222
ceph-node3,ceph-mon	192.168.16.223

1.1.1 配置ssh密鑰訪問

ssh-keygen -f ~/.ssh/id_rsa; ssh-copy-id -i root@192.168.16.221-223

1.1.2 添加hosts，配置主機(jī)名

cat /etc/hosts 192.168.16.220 ceph-admin 192.168.16.221 ceph-node1 ceph-mon 192.168.16.222 ceph-node2 ceph-mon 192.168.16.223 ceph-node3 ceph-mon for n in `seq 3`;do \ scp /etc/hosts 192.168.16.22$n:/etc/hosts; \ ssh 192.168.16.22$n "hostnamectl set-hostname ceph-node$n;" \ done

1.1.3 關(guān)閉firewalld、selinux

for n in `seq 3`;do \ ssh 192.168.16.22$n "systemctl stop firewalld; \ systemctl disable firewalld; \ sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config; \ setenforce 0 "; \ done

1.1.4 時(shí)間更新同步

for n in `seq 3`;do \ ssh 192.168.16.22$n "yum install ntpdate -y; \ ntpdate asia.pool.ntp.org" \ done

1.1.5 替換repo

export CEPH_DEPLOY_REPO_URL=http://172.18.210.253/repo/ceph-el7/jewel export CEPH_DEPLOY_GPG_URL=http://172.18.210.253/repo/ceph-el7/jewel/release.asc 注：如果本地源有問題或不在本地時(shí)，使用國(guó)內(nèi)的源即可 export CEPH_DEPLOY_REPO_URL=http://mirrors.163.com/ceph/rpm-jewel/el7 export CEPH_DEPLOY_GPG_URL=http://mirrors.163.com/ceph/keys/release.asc

1.2 安裝 ceph

1.2.1 安裝ceph管理工具ceph-deploy

yum install -y ceph-deploy

1.2.2 創(chuàng)建工作目錄

mkdir /ceph ; cd /ceph

1.2.3 安裝ceph客戶端

yum install -y ceph 或者 在管理節(jié)點(diǎn)上之行 ceph-deploy install ceph-admin ceph-node1 ceph-node2 ceph-node3

1.2.4 創(chuàng)建ceph集群

ceph-deploy new ceph-node1 ceph-node2 ceph-node3 #建議是奇數(shù)# cat ceph.conf fsid = 7e1daeea-417e-43e3-a2fe-56d9444f2fbf mon_initial_members = ceph-node1, ceph-node2, ceph-node3 mon_host = 192.168.16.221,192.168.16.222,192.168.16.223 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx rbd_default_features = 1 mon clock drift allowed = 2 mon clock drift warn backoff = 30

注意：
1、部分操作系統(tǒng)kernel只支持layering，所以最好直接在配置文件指明創(chuàng)建rbd時(shí)默認(rèn)支持的特性
rbd_default_features = 1
2、由于ceph分布式對(duì)時(shí)鐘的同步要求很高，可以將同步誤差范圍調(diào)大；
mon clock drift allowed = 2
mon clock drift warn backoff = 30

ceph-deploy mon create-initial

1.2.5 創(chuàng)建osd

• 創(chuàng)建osd有兩種方式
  1、使用系統(tǒng)裸盤，作為存儲(chǔ)空間；
  2、使用現(xiàn)有文件系統(tǒng)，以目錄或分區(qū)作為存儲(chǔ)空間，官方建議為 OSD 及其日志使用獨(dú)立硬盤或分區(qū)作為存儲(chǔ)空間

1.2.5.1 使用分區(qū)

ceph-deploy disk zap ceph-node1:/dev/sdb1 ceph-node2:/dev/sdb1 ceph-node3:/dev/sdb1 ceph-deploy osd prepare ceph-node1:/dev/sdb1 ceph-node2:/dev/sdb1 ceph-node3:/dev/sdb1 ceph-deploy osd activate ceph-node1:/dev/sdb1 ceph-node2:/dev/sdb1 ceph-node3:/dev/sdb1

1.2.5.2 使用目錄

ssh ceph-node1 “mkdir /datal/osd0;chown -R ceph:ceph /data/osd0" ssh ceph-node2 “mkdir /datal/osd1;chown -R ceph:ceph /data/osd1" ssh ceph-node3 “mkdir /datal/osd2;chown -R ceph:ceph /data/osd2" ceph-deploy osd prepare ceph-node1:/data/osd0 ceph-node2:/data/osd1 ceph-node3:/data/osd2 ceph-deploy osd acivate ceph-node1:/data/osd0 ceph-node2:/data/osd1 ceph-node3:/data/osd2

1.2.6 賦予管理員權(quán)限

ceph-deploy admin ceph-admin # ceph -scluster 7e1daeea-417e-43e3-a2fe-56d9444f2fbf health HEALTH_OK monmap e1: 3 mons at {ceph-node1=192.168.16.221:6789/0,ceph-node2=192.168.16.222:6789/0,ceph-node3=192.168.16.223:6789/0}election epoch 4, quorum 0,1,2 ceph-node1,ceph-node2,ceph-node3 osdmap e14: 3 osds: 3 up, 3 inflags sortbitwise,require_jewel_osds pgmap v24: 64 pgs, 1 pools, 0 bytes data, 0 objects15460 MB used, 2742 GB / 2757 GB avail64 active+clean # ceph health HEALTH_OK

1.2.7 創(chuàng)建pool

ceph osd pool create image 64 ceph osd pool delete rbd rbd --yes-i-really-really-mean-it

ceph建好后默認(rèn)有個(gè)rbd池，可以考慮刪除

rbd create test --size 1024 -p image

注：創(chuàng)建一個(gè)鏡像,-p參數(shù)指定池的名稱，-size單位為M

1.3 常用操作

1.3.1 ceph reset

ceph-deploy purge 節(jié)點(diǎn)1 節(jié)點(diǎn)2 .... ceph-deploy purgedata 節(jié)點(diǎn)1 節(jié)點(diǎn)2 .... ceph-deploy forgetkeys

1.3.2 常用命令

rados lspools 查看池子ceph -s 或 ceph status 查看集群狀態(tài)ceph -w 觀察集群健康狀態(tài)ceph quorum_status --format json-pretty 檢查ceph monitor仲裁狀態(tài)ceph df 檢查集群使用情況ceph mon stat 檢查monitor狀態(tài)ceph osd stat 檢查osd狀態(tài)ceph pg stat 檢查pg配置組狀態(tài)ceph pg dump 列出PGceph osd lspools 列出存儲(chǔ)池ceph osd tree 檢查osd的crush mapceph auth list 列出集群的認(rèn)證密鑰ceph 獲取每個(gè)osd上pg的數(shù)量

轉(zhuǎn)載于:https://www.cnblogs.com/zw12078/p/8793381.html

總結(jié)

以上是生活随笔為你收集整理的CentOS7.2 部署Ceph分布式存储的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。