zabbix數(shù)據(jù)庫(kù)優(yōu)化之?dāng)?shù)據(jù)庫(kù)優(yōu)化二

簡(jiǎn)介

數(shù)據(jù)庫(kù)history設(shè)置是保存7天。然后如果你沒有分區(qū)。雖然數(shù)據(jù)在減少但是表空間不會(huì)減少。浪費(fèi)硬盤空間的同事緩存內(nèi)的cache部分也沒有被釋放。分區(qū)后可以遷移分區(qū)合并分區(qū)刪除已經(jīng)沒有數(shù)據(jù)的分區(qū)優(yōu)化表空間優(yōu)化buffer內(nèi)存。

一，前期操作

清空表

Truncate?table?table_name;

導(dǎo)出庫(kù)

mysqldump?-uroot?-p?-all-databases?>zabbix.sql

整理表空間碎片

Alter?table?tables_name?engine=innodb;

二，遷移mysql

1調(diào)整升級(jí)mysql到5.7最新的存儲(chǔ)過(guò)程需要

2調(diào)整mysql參數(shù)

檢測(cè)你的參數(shù)是不是設(shè)置的合理

pt-variable-advisor?--source-of-variables?vars.vxt

三，分區(qū)操作

分區(qū)資料鏈接如下

核心部分

DELIMITER?$$

CREATE?PROCEDURE?`partition_maintenance_all`(SCHEMA_NAME?VARCHAR(32))

BEGIN

CALL?partition_maintenance(SCHEMA_NAME,?'history',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'history_log',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'history_str',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'history_text',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'history_uint',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'trends',?20,?24,?20);

CALL?partition_maintenance(SCHEMA_NAME,?'trends_uint',?20,?24,?20);

END$$

DELIMITER?;

創(chuàng)建分區(qū)的存儲(chǔ)過(guò)程

CALL?partition_maintenance(SCHEMA_NAME,?'history',?20,?24,?20);

保留天數(shù)20天

時(shí)間間隔24小時(shí)

創(chuàng)建分區(qū)數(shù)???例如??保留20天??創(chuàng)建分區(qū)數(shù)20

時(shí)間間隔1小時(shí)

創(chuàng)建分區(qū)數(shù)量?24*20

CALL?partition_maintenance(SCHEMA_NAME,?;hisuozy',?20,?1,?22*24):

調(diào)用存儲(chǔ)過(guò)程;創(chuàng)建分區(qū);

call?maintenance(‘zabbix’);

四，手動(dòng)操作分區(qū)

手動(dòng)清數(shù)據(jù)???清除7月30號(hào)前分區(qū)數(shù)據(jù)

call?partition_drop('zabbix','history','201707300000');

call?partition_drop('zabbix','history_log','201707300000');

call?partition_drop('zabbix','history_str','201707300000');

call?partition_drop('zabbix','history_text','201707300000');

call?partition_drop('zabbix','history_uint','201707300000');

call?partition_drop('zabbix','trends','201707300000');

call?partition_drop('zabbix','trends_uint','201707300000');

五，注意事項(xiàng)

注意??partition_maintenance存儲(chǔ)過(guò)程假如你設(shè)置的是保留20天??那么20天前的分區(qū)會(huì)被清除

清除實(shí)際調(diào)用的

call?partition_drop('zabbix','history',201707110000’);

分組報(bào)錯(cuò)

注意:

5.7??group?by需要。默認(rèn)是不能select多col9`n6

set?sql_mode=NO_ENGINE_SUBSTITUTION;

六，效果展示

七，附:

#########刪除201708110000之前的所有數(shù)據(jù)。同時(shí)可以刪除已經(jīng)沒有數(shù)據(jù)的分區(qū)

call partition_drop('zabbix','history','201708110000');

call partition_drop('zabbix','history_log','201708110000');

call partition_drop('zabbix','history_str','201708110000');

call partition_drop('zabbix','history_text','201708110000');

call partition_drop('zabbix','history_uint','201708110000');

call partition_drop('zabbix','trends','201708110000'); ? ?#建議保留

call partition_drop('zabbix','trends_uint','201708110000'); ?#建議保留

########## ?創(chuàng)建分區(qū)

CALL partition_maintenance('zabbix', 'history', 5, 24, 10);

CALL partition_maintenance('zabbix', 'history_log', 5, 24, 10);

CALL partition_maintenance('zabbix', 'history_str', 5, 24, 10);

CALL partition_maintenance('zabbix', 'history_text', 5, 24, 10);

CALL partition_maintenance('zabbix', 'history_uint', 5, 24, 10);

CALL partition_maintenance('zabbix', 'trends', 5, 24, 10);

CALL partition_maintenance('zabbix', 'trends_uint', 5, 24, 10);

保留天數(shù) ?5天 ? ? ? 注意5天前將被刪除

創(chuàng)漿蕬間間隔 ?24小時(shí)

創(chuàng)建分區(qū)數(shù) ?10 ? ? ?分區(qū)不會(huì)自動(dòng)創(chuàng)建所以可以創(chuàng)建多點(diǎn)一共保留15個(gè)分區(qū)

附件1

服務(wù)器配置24c??64g

[mysqld]

datadir=/database/

socket=/var/lib/mysql/mysql.sock

symbolic-links=0

##數(shù)據(jù)更新時(shí)timestamp列自動(dòng)更新

explicit_defaults_for_timestamp=1

innodb_file_per_table=on

skip_name_resolve=1

#事物提交刷新事物日志

innodb_flush_log_at_trx_commit=0

#Key_read_requests從緩存讀取索引的請(qǐng)求次數(shù)。

#Key_reads從磁盤讀取索引的請(qǐng)求次數(shù)。

#自適應(yīng)hash索引

innodb_adaptive_hash_index=on

#開啟臨接頁(yè)刷新 insert多開啟。update多不開啟

innodb_flush_neighbors=1

#異步io

innodb_use_native_aio=on

#更改時(shí)區(qū)設(shè)置避免system阻塞

#time_zone ='+8:00'

#InnoDB 用于寫入磁盤上日志文件 的緩沖區(qū)大小innodb_page_size=32k或64k innodb_log_buffer_size至少16M

innodb_page_size=32k

innodb_log_buffer_size=64M

key_buffer_size=128M

##grup by分組必須加大

tmp_table_size=512M

max_heap_table_size=512M

#其掃描 每個(gè)線程為其掃描的每個(gè) 表分配一個(gè)大小以字節(jié)為單位的緩沖區(qū)

read_buffer_size=4M

#多范圍讀取優(yōu)化緩存區(qū)大小。注意order by排序順序讀取表。每session

read_rnd_buffer_size=16M

#排序緩沖

sort_buffer_size=512M

max_connections=1024

#開啟查詢緩存

query_cache_type=1

query_cache_size=64M

innodb_write_io_threads = 24

innodb_read_io_threads = 24

innodb_page_cleaners=4

#設(shè)置change_buffer占buffer比例(insert update)

innodb_change_buffer_max_size=50

#預(yù)讀當(dāng)你連續(xù)讀取設(shè)定的數(shù)量的page后會(huì)觸發(fā)讀取這個(gè)extent的剩余page。

innkdb_rcntom_read_ahead=on

#并發(fā)

innodb_thread_concurrency = 24

#緩存線程

thread_cache_size=500

max_connect_errors=3000

max_connections=3000

#自動(dòng)遞增鎖模式

innodb_autoinc_lock_mode=2

#所有的change都緩沖

innodb_change_buffering=all

#緩沖池配置20個(gè)池每個(gè)1G

innodb_buffer_pool_chunk_size=1G

innodb_buffer_pool_instances=28

innodb_buffer_pool_size=28G

[mysqld_safe]

log-error=/database/error.log

pid-file=/database/mysqld.pid

#

# include all files from the config directory

#

!includedir /etc/my.cnf.d

補(bǔ)充自動(dòng)清除分區(qū)py:#!/usr/local/python-3.5.0/bin/python3.5

#-*-?coding:utf-8?-*-

import?pymysql,datetime

class?db_action(object):

def?__init__(self,host,port,user,password,database="mysql",charset="utf8"):

self.host=host

self.port=port

self.user=user

self.password=password

self.database=database

self.charset=charset

self.__cursor=self.__conn().cursor()

def?__conn(self):

return?pymysql.Connect(host=self.host,port=self.port,user=self.user,\

password=self.password,database=self.database,\

charset=self.charset)

def?callproc(self,pl,t):

ret=self.__cursor.callproc(pl,args=t)

return?ret

if?__name__?==?'__main__':

db=db_action(host='172.16.7.145',port=3306,user='root',password="zabbix",database="zabbix",charset='utf8')

rets=[]

ret=db.callproc('partition_drop',('zabbix','history',(datetime.datetime.now()+datetime.timedelta(days=-1)).strftime('%Y%m%d0000'),))

rets.append(ret)

ret=db.callproc('partition_drop',('zabbix','history_log',(datetime.datetime.now()+datetime.timedelta(days=-1)).strftime('%Y%m%d0000'),))

rets.append(ret)

ret=db.callproc('partition_drop',('zabbix','history_str',(datetime.datetime.now()+datetime.timedelta(days=-1)).strftime('%Y%m%d0000'),))

rets.append(ret)

ret=db.callproc('partition_drop',('zabbix','history_text',(datetime.datetime.now()+datetime.timedelta(days=-1)).strftime('%Y%m%d0000'),))

rets.append(ret)

ret=db.callproc('partition_drop',('zabbix','history_uint',(datetime.datetime.now()+datetime.timedelta(days=-1)).strftime('%Y%m%d0000'),))

rets.append(ret)

with?open('/root/pariton_clear.log',encoding='utf-8',mode='a+')?as?f:

f.write(str(rets)+'\n')

contable00?10?*?*?*?/usr/local/python-3.5.0/bin/python3.5?/etc/zabbix/shell/Partition_Clear.py

自動(dòng)創(chuàng)建分區(qū)

#!/usr/bin/env python

#-*- coding:utf-8 -*-

import pymysql,datetime,time

class DB_Action(object):

def __init__(self,host,port,user,password,database="mysql",charset="utf8"):

self.host=host

self.port=port

self.user=user

self.password=password

self.database=database

self.charset=charset

self.__conn=self.__connect()

self.__cursor=self.__conn.cursor()

def __connect(self):

return pymysql.Connect(host=self.host,port=self.port,user=self.user,\

password=self.password,database=self.database,\

charset=self.charset)

def callproc(self,partition_name,tuple_args):

'''存儲(chǔ)過(guò)程'''

self.__cursor.callproc(partition_name,tuple_args)

self.__conn.commit()

def execute(self, tables_name):

'''sql'''

ret = self.__cursor.execute('select count(*) from PARTITIONS WHERE TABLE_NAME=%s and TABLE_ROWS=0',\

[tables_name])

return self.__cursor.fetchone()[0]

def __del__(self):

self.__cursor.close()

self.__conn.close()

if __name__ == '__main__':

datatables_list=['history','history_log','history_str','history_text','history_uint','trends','trends_uint']

par_conn=DB_Action(host='172.16.7.145',port=3306,user='root',password="zabbix",database="information_schema",charset='utf8')

par_count=par_conn.execute('history')

with open('/root/pariton_create.log', encoding='utf-8', mode='a+') as f:

f.write('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@'+str(time.strftime('%Y-%m-%d',time.localtime()))+'\n')

for tables in datatables_list:

f.write('{0}分區(qū)統(tǒng)計(jì)'.format(tables) + str(par_count)+'\n')

call_count=DB_Action(host='172.16.7.145',port=3306,user='root',password="zabbix",database="zabbix",charset='utf8')

if par_count<=10:

for tables in datatables_list:

call_count.callproc('partition_maintenance',('zabbix', tables, 365, 24, 90))

par_count=par_conn.execute('history')

with open('/root/pariton_create.log', encoding='utf-8', mode='a+') as f:

f.write('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@'+str(time.strftime('%Y-%m-%d',time.localtime()))+'\n')

for tables in datatables_list:

f.write('{0}分區(qū)統(tǒng)計(jì)'.format(tables) + str(par_count)+'\n')

補(bǔ)充zabbix server配置文件LogFile=/var/log/zabbix/zabbix_server.log

LogFileSize=1024

DebugLevel=3?????????????????#日志級(jí)別

PidFile=/var/run/zabbix/zabbix_server.pid

DBHost=localhost

DBName=zabbix

DBUser=zabbix

DBPassword=zabbix

DBSocket=/var/lib/mysql/mysql.sock

StartPollers=100??????????????#poller進(jìn)程??100

StartPollersUnreachable=30????#無(wú)法訪問的主機(jī)輪詢進(jìn)程30

StartPingers=30???????????????#ping輪詢數(shù)量

StartDiscoverers=30

StartTimers=10

SenderFrequency=30?????#發(fā)送報(bào)警超時(shí)

SNMPTrapperFile=/var/log/snmptrap/snmptrap.log

CacheSize=4096M??????????????#存儲(chǔ)主機(jī)，項(xiàng)目和觸發(fā)器數(shù)據(jù)的共享內(nèi)存

CacheUpdateFrequency=120??????#執(zhí)行配置緩存的更新頻率

StartDBSyncers=24????????????#數(shù)據(jù)庫(kù)同步進(jìn)程

HistoryCacheSize=2048M

HistoryIndexCacheSize=2048M

TrendCacheSize=2048M?????????#趨勢(shì)數(shù)據(jù)最大2G

ValueCacheSize=2048M?????????#緩存項(xiàng)歷史數(shù)據(jù)請(qǐng)求，歷史值緩存

Timeout=30

UnreachablePeriod=120????????#幾秒鐘的不可達(dá)性將主機(jī)視為不可用。??不可用

UnavailableDelay=60??????????#主機(jī)在不可用期間內(nèi)檢查可用性的頻率(秒)。??不可用

UnreachableDelay=5???????????#不可達(dá)檢測(cè)頻率???解決wait?for?3?seconds

AlertScriptsPath=/usr/lib/zabbix/alertscripts

ExternalScripts=/usr/lib/zabbix/externalscripts

LogSlowQueries=2000??????????#記錄慢查詢

HousekeepingFrequency=1??????#從歷史記錄，警報(bào)和警報(bào)表中刪除不必要的信息??不超過(guò)4個(gè)小時(shí)??每隔1小時(shí)啟動(dòng)一次，刪除過(guò)期數(shù)據(jù)

MaxHousekeeperDelete=1000000?#清除過(guò)期數(shù)據(jù)，超過(guò)這個(gè)閥值的行都會(huì)被清理。

補(bǔ)充圖片

1，系統(tǒng)任務(wù)隊(duì)列查看

2，mysql status計(jì)量分析

3，io吞吐查看

4，my.cnf配置文件檢查

5，io buys計(jì)量

6，engine計(jì)量

總結(jié)

以上是生活随笔為你收集整理的zabbix mysql优化 my.cnf_zabbix数据库优化之数据库优化（二）的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。