openstack基于mysql的問題修復

?

同事一臺openstack單節點服務器由于未知原因，無法正常工作。

遠程引導同事檢查故障。?

[root@localhost ~]# nova-manage service list

?

出現錯誤:SQL connection failed 。貌似沒有連接數據庫,看來可能是服務或數據庫有問題。

引導同事檢測openstack服務及數據庫：

[root@sxun?init.d]#?/etc/init.d/openstack-keystone?status keystone?(pid??12593)?is?running... [root@sxun?init.d]#?/etc/init.d/openstack-nova-api?status openstack-nova-api?(pid??7446)?is?running... [root@sxun?init.d]#?/etc/init.d/openstack-nova-compute?status openstack-nova-compute?(pid??11728)?is?running... [root@sxun?init.d]#?/etc/init.d/openstack-nova-conductor?status openstack-nova-conductor?(pid??7704)?is?running... [root@sxun?init.d]#?/etc/init.d/mysqld?status?
mysql?is?stoppd? [root@sxun?init.d]#?/etc/init.d/mysqld?start starting?mysqld:?[FAILED]

restart也是如此，看來是mysqld服務沒有啟動的問題。為什么沒有啟動了，找到了mysqld的日志查看：

?tail?-100?/var/lib/mysql/sxun.err?

?錯誤日志報大致錯誤如下:

170420?15:00:20??InnoDB:?Database?was?not?shut?down?normally! InnoDB:?Starting?crash?recovery.

......

因圖不完整，引用以下網址文本http://blog.chinaunix.net/uid-27038861-id-3667209.html?bsh_bid=310758849?，也按照其方法處，沒動腦地操作了一下試試

##############以下引用#############

?130508?16:30:53?mysqld_safe?Starting?mysqlddaemon?with?databases?from?/data/mysql 130508?16:30:54?[Warning]?The?syntax'--log-slow-queries'?is?deprecated?and?will?be?removed?in?a?future?release.Please?use?'--slow-query-log'/'--slow-query-log-file'?instead. 130508?16:30:54?InnoDB:?The?InnoDB?memoryheap?is?disabled 130508?16:30:54?InnoDB:?Mutexes?andrw_locks?use?GCC?atomic?builtins 130508?16:30:54?InnoDB:?Compressed?tablesuse?zlib?1.2.3 130508?16:30:54?InnoDB:?Initializing?bufferpool,?size?=?500.0M 130508?16:30:54?InnoDB:?Completedinitialization?of?buffer?pool 130508?16:30:54?InnoDB:?highest?supportedfile?format?is?Barracuda. InnoDB:?Log?scan?progressed?past?thecheckpoint?lsn?27219795678 130508?16:30:54??InnoDB:?Database?was?not?shut?down?normally! InnoDB:?Starting?crash?recovery. InnoDB:?Reading?tablespace?information?fromthe?.ibd?files... InnoDB:?Restoring?possible?half-writtendata?pages?from?the?doublewrite InnoDB:?buffer... InnoDB:?Doing?recovery:?scanned?up?to?logsequence?number?27219928522

?##############引用，中間省略#############

解決辦法：?
刪除掉?
ibdata* ,
ib_logfile* ,
所有的日志。
重啟后可啟動數據庫，因數據庫是非正常關閉引起的。所有會造成數據丟失。要做好備份工作

?##############以上引用#############

[root@sxun?mysql]#?ll total?225356 drwx------?2?mysql?mysql??????4096?Feb?15?16:06?cinder drwx------?2?mysql?mysql??????4096?Oct??1??2016?glance -rw-rw----?1?mysql?mysql?220200960?Apr?20?18:08?ibdata1 -rw-rw----?1?mysql?mysql???5242880?Apr?20?18:08?ib_logfile0 -rw-rw----?1?mysql?mysql???5242880?Apr?20?18:08?ib_logfile1 drwx------?2?mysql?mysql??????4096?Oct??1??2016?keystone drwx------?2?mysql?mysql??????4096?Oct??1??2016?mysql srwxrwxrwx?1?mysql?mysql?????????0?Feb?16?14:52?mysql.sock drwx------?2?mysql?mysql??????4096?Oct??1??2016?nova drwx------?2?mysql?mysql??????4096?Oct??1??2016?ovs_neutron -rw-r-----?1?mysql?root??????38548?Feb?16?14:52?sxun.err drwx------?2?mysql?mysql??????4096?Oct??1??2016?test

生產環境不敢刪除，先移動其它地方備份記

?[root@sxun?init.d]#mv?ibdata*??/root/[root@sxun?init.d]#mv?ib_logfile*???/root/[root@sxun?init.d]#?/etc/init.d/mysqld?start starting?mysqld:???????????????????[OK]

服務器的數據庫已經是啟動起來了，也生成了新的ib*文件。測試一下keystone和nova服務，但是：

[root@sxun?instances]#?nova-manage?service?list CRITICAL?nova[req-97fd8f67-8a06-4ab7-b421-01d36bc6ad02?None?None]?(ProgrammingError)?(1146,"Table?'nova.services'?doesn't?exist")?'SELECT?services.created_at?ASservices_created_at,?services.updated_at?AS?services_updated_at,services.deleted_at?AS?services_deleted_at,?services.deleted?ASservices_deleted,?services.id?AS?services_id,?services.host?AS?services_host,services.`binary`?AS?services_binary,?services.topic?AS?services_topic,services.report_count?AS?services_report_count,?services.disabled?ASservices_disabled,?services.disabled_reason?AS?services_disabled_reason?\nFROMservices?\nWHERE?services.deleted?=?%s'?(0,)[root@sxun?instances]#?keystone?user-list Authorization?Failed:?An?unexpected?errorprevented?the?server?from?fulfilling?your?request.?(ProgrammingError)?(1146,"Table?'keystone.domain'?doesn't?exist")?'SELECT?domain.id?ASdomain_id,?domain.name?AS?domain_name,?domain.enabled?AS?domain_enabled,domain.extra?AS?domain_extra?\nFROM?domain?\nWHERE?domain.id?=?%s'?('default',)(HTTP?500)

? mysqld起來了,但由于ib*文件已經更換，數據不一樣了。nova等各項服務已經出錯了，?嘗試reboot重啟了一下服務器，仍然不行，ib*得還原，但還原又啟動不了mysqld了。?

直接抄襲主義不行，關鍵還得想辦法：

InnoDB:?Database?was?not?shut?down?normally! InnoDB:?Starting?crash?recovery.

問題在于這個錯誤，數據庫非正常關機，然后數據庫重啟后有一個恢復操作，恢復不了就掛了。

?

后續的解決方案：

1、還原ib*文件，

2、在配置文件加入?不強制恢復數據的?關掉?#innodb_force_recover = 6。

3、啟動數據庫，mysqldump將數據庫導出。
4、刪除數據庫文件，mysql啟動完成后重新導入mysql數據庫

?

同事操的刀，具體操作如下：

[root@sxun01?mysql(keystone_admin)]#mysqldump?-uroot??--all-database??>>?my_mysql.dump --?Warning:?Skipping?the?data?of?tablemysql.event.?Specify?the?--events?option?explicitly.[root@sxun01?mysql(keystone_admin)]#/etc/init.d/mysqld?stop Stopping?mysqld:??????????????????????????????????????????[??OK??][root@sxun01?mysql(keystone_admin)]#?rm?-rfibdata1?ib_logfile0?ib_logfile1 [root@sxun01?mysql(keystone_admin)]#?vi/etc/my.cnf [client] port???=?3306 socket???=?/var/lib/mysql/mysql.sock [mysqld_safe] socket???=?/var/lib/mysql/mysql.sock [client] port???=?3306 socket???=?/var/lib/mysql/mysql.sock [mysqld_safe] socket???=?/var/lib/mysql/mysql.sock nice???=?0 [mysqld] user???=?mysql pid-file?=?/var/run/mysqld/mysqld.pid socket???=?/var/lib/mysql/mysql.sock port?????=?3306 basedir??=?/usr datadir??=?/var/lib/mysql tmpdir???=?/tmp skip-external-locking#innodb_force_recover?=?6 bind-address????=?0.0.0.0key_buffer?????????=?16M max_allowed_packet?=?16M thread_stack???????=?192K thread_cache_size??=?8 myisam-recover?????=?BACKUP query_cache_limit??=?1M query_cache_size???=?16M log_error??????????=?/var/log/mysqld.log expire_logs_days???=?10 max_binlog_size????=?100Mdefault-storage-engine?=?InnoDB[mysqldump] quick quote-names "/etc/my.cnf"?43L,?811C?written[root@sxun01?mysql(keystone_admin)]#?mkdirdatabase_bak [root@sxun01?mysql(keystone_admin)]#?mvcinder/?glance/?keystone/?nova/?ovs_neutron/?database_bak/ [root@sxun01?mysql(keystone_admin)]#?lldatabase_bak/ total?20 drwx------?2?mysql?mysql?4096?Oct?25?15:08cinder drwx------?2?mysql?mysql?4096?Oct?25?15:08glance drwx------?2?mysql?mysql?4096?Oct?25?15:08keystone drwx------?2?mysql?mysql?4096?Oct?25?15:08nova drwx------?2?mysql?mysql?4096?Oct?25?15:08ovs_neutron [root@sxun01?mysql(keystone_admin)]#?ll total?29648 drwxr-xr-x?2?root??root?????4096?Apr?20?15:15?aaa drwxr-xr-x?2?root??root?????4096?Apr?20?15:33?bbb drwxr-xr-x?7?root??root?????4096?Apr?20?15:47?database_bak -rw-r--r--?1?root??root?29610746?Apr?20?15:45?my_mysql.dump drwx------?2?mysql?mysql?????4096?Oct?25?15:05?mysql -rw-r-----?1?mysql?root?????82317?Apr?20?15:46?sxun01.err -rw-r-----?1?mysql?root????637765?Apr?20?15:35?sxun01.err-- drwx------?2?mysql?mysql?????4096?Oct?25?15:05?test [root@sxun01?mysql(keystone_admin)]# [root@sxun01?mysql(keystone_admin)]# [root@sxun01?mysql(keystone_admin)]#/etc/init.d/mysqld?restart Stopping?mysqld:??????????????????????????????????????????[??OK??] Starting?mysqld:??????????????????????????????????????????[??OK??] [root@sxun01?mysql(keystone_admin)]#?mysql Welcome?to?the?MySQL?monitor.??Commands?end?with?;?or?\g. Your?MySQL?connection?id?is?22 Server?version:?5.1.73?Source?distributionCopyright?(c)?2000,?2013,?Oracle?and/or?itsaffiliates.?All?rights?reserved.Oracle?is?a?registered?trademark?of?OracleCorporation?and/or?its affiliates.?Other?names?may?be?trademarksof?their?respective owners.Type?'help;'?or?'\h'?for?help.?Type?'\c'?toclear?the?current?input?statement.mysql>?quit Bye [root@sxun01?mysql(keystone_admin)]#?mysql-uroot?<?my_mysql.dump [root@sxun01?mysql(keystone_admin)]# [root@sxun01?mysql(keystone_admin)]# [root@sxun01?mysql(keystone_admin)]#?mysql Welcome?to?the?MySQL?monitor.??Commands?end?with?;?or?\g. Your?MySQL?connection?id?is?93 Server?version:?5.1.73?Source?distributionCopyright?(c)?2000,?2013,?Oracle?and/or?itsaffiliates.?All?rights?reserved.Oracle?is?a?registered?trademark?of?OracleCorporation?and/or?its affiliates.?Other?names?may?be?trademarksof?their?respective owners.Type?'help;'?or?'\h'?for?help.?Type?'\c'?toclear?the?current?input?statement.mysql>?use?nova?; Reading?table?information?for?completion?oftable?and?column?names You?can?turn?off?this?feature?to?get?aquicker?startup?with?-ADatabase?changedmysql>?select?*?from??instances;mysql>?quit Bye[root@sxun01?mysql(keystone_admin)]#nova-manage?service?list Binary???????????Host?????????????????????????????????Zone?????????????Status?????State?Updated_At nova-consoleauth?sxun01???????????????????????????????internal?????????enabled????XXX??2017-04-20?05:31:48 nova-scheduler???sxun01???????????????????????????????internal?????????enabled????XXX??2017-04-20?05:31:54 nova-conductor???sxun01???????????????????????????????internal?????????enabled????:-)??2017-04-20?07:49:07 nova-cert????????sxun01???????????????????????????????internal?????????enabled????XXX??2017-04-20?05:31:55 nova-compute?????sxun01???????????????????????????????nova?????????????enabled????XXX??2017-04-20?05:31:53[root@sxun01?mysql(keystone_admin)]#?reboot [root@sxun01?mysql(keystone_admin)]# Broadcast?message?from?root@sxun01(/dev/pts/0)?at?15:49?...The?system?is?going?down?for?reboot?NOW! Last?login:?Thu?Apr?20?15:28:26?2017[root@sxun01?~]#?nova-manage?service?list Binary???????????Host?????????????????????????????????Zone?????????????Status?????State?Updated_At nova-consoleauth?sxun01???????????????????????????????internal?????????enabled????:-)??2017-04-20?07:53:28 nova-scheduler???sxun01???????????????????????????????internal?????????enabled????:-)??2017-04-20?07:53:28 nova-conductor???sxun01???????????????????????????????internal?????????enabled????:-)??2017-04-20?07:53:28 nova-cert????????sxun01???????????????????????????????internal?????????enabled????:-)??2017-04-20?07:53:28 nova-compute?????sxun01???????????????????????????????nova?????????????enabled????:-)??2017-04-20?07:53:24

最終結果，可喜可賀。

轉載于:https://blog.51cto.com/fzhddn/1917888

總結

以上是生活随笔為你收集整理的openstack基于mysql的问题修复的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。