linux系统分析工具续-SystemTap和火焰图（Flame Graph）
生活随笔
收集整理的這篇文章主要介紹了
linux系统分析工具续-SystemTap和火焰图（Flame Graph）
小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.
本文為網(wǎng)上各位大神文章的綜合簡單實踐篇，參考文章較多，有些總結(jié)性東西，自認暫無法詳細寫出，建議讀文中列出的參考文檔，相信會受益頗多。下面開始吧（本文出自 “cclo的博客” 博客，請務必保留此出處http://xuclv.blog.51cto.com/5503169/1184517） SystemTap簡介： SystemTap provides free software (GPL) infrastructure to simplify the gathering of information about the running Linux system. This assists diagnosis of a performance or functional problem. SystemTap eliminates the need for the developer to go through the tedious and disruptive instrument, recompile, install, and reboot sequence that may be otherwise required to collect data.SystemTap provides a simple command line interface and scripting language for writing instrumentation for a live running system. We are publishing samples, as well as enlarging the internal "tapset" script library to aid reuse and abstraction.
詳情參考： $ sudo vim /usr/share/nginx/www/index.php #隨便建個測試頁
訪問http://192.168.1.94/ #訪問確保正常 6> 用systemtap進行nginx的分析，并用flameGraph畫出相應的火焰圖。 1：編寫systemtap腳本: vim ngx.stp
2:執(zhí)行如下命令： $ sudo stap --ldd -d /usr/sbin/nginx --all-modules -D MAXMAPENTRIES=256 -D MAXACTION=20000 -D MAXTRACE=100 -D MAXSTRINGLEN=4096 -D MAXBACKTRACE=100 -x 2082 ngx.stp --vp 0001 > ngx.out可能需修改的地方：/usr/sbin/nginx #nginx的安裝位置，一般是此-D MAXMAPENTRIES=256 #本機是個虛擬機，只給了其512M內(nèi)存，為了保證內(nèi)存不溢出，設為256-x 2082 #指定其中一個nginx worker進程的pid在執(zhí)行以上命令的同時保證對nginx的壓力測試，另開終端執(zhí)行：$ ab -n 900000 -c 50 http://192.168.1.94/index.php $ perl stackcollapse-stap.pl ngx.out > ngx.out2 $ perl flamegraph.pl ngx.out2 > ngx.svg#以上2個perl腳本在https://github.com/brendangregg/FlameGraph 7> 訪問192.168.1.94/ngx.svg #火焰圖出爐了
每個框代表一個棧里的一個函數(shù)Y軸代表棧深度（棧楨數(shù)）。最頂端的框顯示正在運行的函數(shù)，這之下的框都是調(diào)用者。在下面的函數(shù)是上面函數(shù)的父函數(shù)X軸代表采樣總量。從左到右并不代表時間變化，從左到右也不具備順序性框的寬度代表占用CPU總時間。寬的框代表的函數(shù)可能比窄的運行慢，或者被調(diào)用了更多次數(shù)。框的顏色深淺也沒有任何意義如果是多線程同時采樣，采樣總數(shù)會超過總時間 8> 可能遇到的問題： 1：如果有類似如下行，執(zhí)行 $ sudo rm .systemtap -rfPass 4: using cached /home/ubuntu/.systemtap/cache/24/stap_2479e8647ccf262def735ebf0eeb57c5_5866.ko 2：如果有類似如下行，那應該有指望Pass 4: compiled C into "stap_2711449662ecf03b09d98a743c2122d0_5946.ko" in 24060usr/6960sys/103610rea 3：如果有類似如下行，可以考慮忽略下WARNING: Missing unwind data for module, rerun with 'stap -d stap_2711449662ecf03b09d98a743c2122d0_3437' 4：如果火焰圖只出現(xiàn)大量的init函數(shù)，說明完全不正確，暫不知道怎么解決，nginx如果是apt-get安裝，嘗試執(zhí)行如下行，可能有用$ sudo apt-get install nginx-full-dbg 5：WARNING: missing unwind/symbol data for module 'kernel' :user-space facilities not available without kernel CONFIG_UTRACE 這個錯誤是說你的 kernel 沒有提供 utrace/uprobes 用戶態(tài)支持。兩種解法：1. 自己給Ubuntu自帶的老kernel應用utrace補丁，并重新編譯它。chaoslawful老師寫過一篇博客分享過ubuntu上的步驟：http://chaoslawful.iteye.com/blog/14635642. 將kernel升級到官方最新的3.5或以上的版本。最新的kernel默認包含了uprobes機制，不再需要utrace補丁了（這是為什么用ubuntu12.10的原因） 9> 火焰圖生成參考的文檔：
http://sourceware.org/systemtap/wiki
http://www.ibm.com/developerworks/cn/linux/l-systemtap/
http://blog.yufeng.info/archives/855
火焰圖簡介： 通過性能分析來確定什么原因?qū)е翪PU繁忙是日常工作中長做的事情，這往往會涉及到棧性能分析。通過定期采樣的方式來確定哪些代碼是導致CPU繁忙的原因是一個比較粗糙的方法。一個更好的方式是創(chuàng)建一個定時中斷來收集程序運行時的計數(shù)，函數(shù)地址，甚至整個堆棧回溯，最后打印為我們可閱讀的報告.我們常用的性能分析工具有oprofile,gprof,dtracesystemtap 等Flame Graph：火焰圖，是一個把采樣所得到的堆棧跟蹤可視化展示的工具。它是基于上面提到的性能分析工具的結(jié)果，Flame graph本身并不具備性能檢測的能力。 參考：https://github.com/brendangregg/FlameGraph
http://dtrace.org/blogs/brendan/2011/12/16/flame-graphs/
http://dtrace.org/blogs/brendan/2012/03/17/linux-kernel-performance-flame-graphs/
systemtap安裝及生成nginx的火焰圖： OS：ubuntu12.10 #支持uprobes機制systemtap：2.1 #盡少的bug 1> 安裝systemtap，這里源碼編譯 1:安裝編譯環(huán)境：$ sudo apt-get install build-essential 2:安裝systemtap$ wget http://sourceware.org/systemtap/ftp/releases/systemtap-2.1.tar.gz$ tar zxvf systemtap-2.1.tar.gz$ sudo ./configure -prefix=/opt/systemtap -disable-docs -disable-publican -disable-refdocs如報以下錯：configure: error: missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel$ sudo apt-get install libdw-dev #其實缺的該是libdw1這個包，按提示安裝libdw-dev罷了，另三個包都木有$ sudo make$ sudo make install 3:創(chuàng)建個軟鏈接：$ sudo ln -s /opt/systemtap/bin/stap /usr/sbin/stap 2> 安裝debug packages (Kernel debug info packages on Ubuntu, which will aid in providing information for bugs) 1：$ uname -r3.5.0-17-generic 2:在http://ddebs.ubuntu.com/pool/main/l/linux/網(wǎng)址找到對應內(nèi)核的的debug packages，下載并安裝之$ sudo dpkg -i linux-image-3.5.0-17-generic-dbgsym_3.5.0-17.28_amd64.ddeb 如沒有找到自己系統(tǒng)對于的debug packages，建一個，方法如下：(How do I build a debuginfo kernel if one isn't available?)（官網(wǎng)原文，本文不涉及）$ cd $HOME$ sudo apt-get install dpkg-dev debhelper gawk$ mkdir tmp$ cd tmp$ sudo apt-get build-dep --no-install-recommends linux-image-$(uname -r)$ apt-get source linux-image-$(uname -r)$ cd linux-2.6.31 (this is currently the kernel version of 9.10)$ fakeroot debian/rules clean$ AUTOBUILD=1 fakeroot debian/rules binary-generic skipdbg=false$ sudo dpkg -i ../linux-image-debug-2.6.31-19-generic_2.6.31-19.56_amd64.ddeb 3> 測試systemtap，出現(xiàn)hello world，安裝完成 $ sudo stap -e 'probe kernel.function("sys_open") {log("hello world") exit()}' 4> systemtap安裝參考：?
http://sourceware.org/systemtap/wiki/SystemtapOnUbuntu
https://wiki.ubuntu.com/Kernel/Systemtap
http://www.dcshi.com/?p=124
5> 安裝LNMP（非必須，安裝nginx即可） $ sudo apt-get install nginx mysql-server mysql-client php5 php5-fpm php5-mysql $ sudo vim /etc/nginx/sites-enabled/default #啟用下面幾行location ~ \.PHP$ {
? ? ? ?fastcgi_split_path_info ^(.+\.php)(/.+)$;
# ? ? ? # NOTE: You should have "cgi.fix_pathinfo = 0;"in php.ini
#
# ? ? ? # With php5-cgi alone:
# ? ? ? fastcgi_pass 127.0.0.1:9000;
# ? ? ? # With php5-fpm:
? ? ? ?fastcgi_pass unix:/var/run/php5-fpm.sock;
? ? ? ?fastcgi_index index.php;
? ? ? ?include fastcgi_params;
}
<html><body><h1>It is my works!!</h1></body></html>
<?php
$link=mysql_connect('localhost','root','root');
if ($link)
? ?echo "Success";
else
? ?echo "Failure";
mysql_close();
phpinfo();
?>
global s;
global quit = 0;
probe timer.profile {
? ?if (pid() == target()) {
? ? ? ?if (quit) {
? ? ? ? ? ?foreach (i in s-) {
? ? ? ? ? ? ? ?print_ustack(i);
? ? ? ? ? ? ? ?printf("\t%d\n", @count(s[i]));
? ? ? ? ? ?}
? ? ? ? ? ?exit()
? ? ? ?} else {
? ? ? ? ? ?s[ubacktrace()] <<< 1;
? ? ? ?}
? ?}
}
probe timer.s(20) {
? ?quit = 1
}
http://www.dcshi.com/?p=287
https://groups.google.com/forum/#!msg/openresty/u-puKWWONMk/bxsyQdWMkJIJ
http://idning.github.io/systemtaprst.html
附： $ stap -h Systemtap translator/driver (version 2.1/0.153, non-git sources) Copyright (C) 2005-2013 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: TR1_UNORDERED_MAP NLSUsage: stap [options] FILE Run script in file.or: stap [options] - Run script on stdin.or: stap [options] -e SCRIPT Run given script.or: stap [options] -l PROBE List matching probes.or: stap [options] -L PROBE List matching probes and local variables.Options (in /home/ubuntu/.systemtap/rc and on command line):-- end of translator options, script options follow-h --help show help-V --version show version-p NUM stop after pass NUM 1-5, instead of 5 (parse, elaborate, translate, compile, run)-v add verbosity to all passes--vp {N}+ add per-pass verbosity [00000]-k keep temporary directory-u unoptimized translation -w suppress warnings -W turn warnings into errors -g guru mode -P prologue-searching for function probes -b bulk (percpu file) mode -s NUM buffer size in megabytes, instead of 0-I DIR look in DIR for additional .stp script files, in addition to/opt/systemtap/share/systemtap/tapset-D NM=VAL emit macro definition into generated C code-B NM=VAL pass option to kbuild make--modinfo NM=VALinclude a MODULE_INFO(NM,VAL) in the generated C code-G VAR=VAL set global variable to value-R DIR look in DIR for runtime, instead of/opt/systemtap/share/systemtap/runtime-r DIR cross-compile to kernel with given build tree; or else-r RELEASE cross-compile to kernel /lib/modules/RELEASE/build, instead of/lib/modules/3.5.0-17-generic/build-a ARCH cross-compile to given architecture, instead of x86_64-m MODULE set probe module name, instead of stap_4736-o FILE send script output to file, instead of stdout. This supportsstrftime(3) formats for FILE-c CMD start the probes, run CMD, and exit when it finishes-x PID sets target() to PID-F run as on-file flight recorder with -o.run as on-memory flight recorder without -o.-S size[,n] set maximum of the size and the number of files.-d OBJECT add unwind/symbol data for OBJECT file--ldd add unwind/symbol data for all referenced object files.--all-modulesadd unwind/symbol data for all loaded kernel objects.-t collect probe timing information--runtime=MODEset the pass-5 runtime mode, instead of kernel--privilege=PRIVILEGE_LEVELcheck the script for constructs not allowed at the given privilege level--unprivilegedequivalent to --privilege=stapusr--compatible=VERSIONsuppress incompatible language/tapset changes beyond VERSION,instead of 2.1--check-versiondisplays warnings where a syntax element may be version dependent--skip-badvarssubstitute zero for bad context $variables--suppress-handler-errorscatch all runtime errors, quietly skip probe handlers--use-server[=SERVER-SPEC]specify systemtap compile-servers--list-servers[=PROPERTIES]report on the status of the specified compile-servers:all,specified,online,trusted,signer,compatible--remote=HOSTNAMErun pass 5 on the specified ssh host.may be repeated for targeting multiple hosts.--remote-prefixprefix each line of remote output with a host index.--tmpdir=NAMEspecify name of temporary directory to be used.--download-debuginfo[=OPTION]automatically download debuginfo using ABRT.yes,no,ask,<timeout value>--dump-probe-typesshow a list of available probe types.--sysroot=DIRspecify sysroot directory where target files (executables,libraries, etc.) are located.--sysenv=VAR=VALUEprovide an alternate value for an environment variablewhere the value on a remote system differs. Pathvariables (e.g. PATH, LD_LIBRARY_PATH) are assumed to berelative to the sysroot.--suppress-time-limitsdisable -DSTP_NO_OVERLOAD -DMAXACTION and -DMAXTRYACTION limits本文出自 “cclo的博客” 博客，請務必保留此出處http://xuclv.blog.51cto.com/5503169/1184517?
轉(zhuǎn)載于:https://www.cnblogs.com/wicub/p/6733094.html
總結(jié)
以上是生活随笔為你收集整理的linux系统分析工具续-SystemTap和火焰图（Flame Graph）的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 梦到捡翡翠什么预兆
- 下一篇： linux使用flock解决cronta