内存检测工具：sanitizer
1. 背景
內存泄漏是一個比較常見的問題，之前使用的是valgrind來實現內存檢查的情況比較多，這里介紹一種更加便利的內存檢測工具， 那就是gcc自帶的sanitizer。
2. sanitizer 的用法
2.1 sanitizer的基本簡介
Sanitizers 是谷歌發起的開源工具集，包括AddressSanitizer，MemorySanitizer, ThreadSanitizer, LeakSanitizer, Sanitizers項目本身是llvm項目的一部分，
gcc自帶的工具， gcc從4.8版本開始支持Address和Thread Sanitizer，4.9版本開始支持Leak Sanitizer和UBSanitizer。
可以支持的內存檢測：
- Use after free
- Heap buffer overflow
- Stack buffer overflow
- Global buffer overflow
- Use after return
- Use after scope
- Initialization order bugs
- Memory leaks
具體錯誤類型解釋：
2.2 升級高版本的gcc和安裝相關的依賴庫(centos 7)
yum -y install centos-release-scl yum -y install devtoolset-7-gcc devtoolset-7-gcc-c++ devtoolset-7-binutils yum -y install devtoolset-7-libasan-devel.x86_64 devtoolset-7-liblsan-devel.x86_64 devtoolset-7-libtsan-devel.x86_64 devtoolset-7-libubsan-devel.x86_64 scl enable devtoolset-7 bash echo "source /opt/rh/devtoolset-7/enable" >>/etc/profile這里需要特別注意的是：Address Sanitizer 會替換malloc和free， 如果采用第三方的內存申請庫，則無法替換，會造成功能缺失。
可以檢查的內存問題包括：
1. Out-of-bounds accesses to heap, stack and globals 2. Use-after-free 3. Use-after-return (runtime flag) 4. ASAN_OPTIONS=detect_stack_use_after_return=1) 5. Use-after-scope (clang flag -fsanitize-address-use-after-scope) 6. Double-free, invalid free 7. Memory leaks (experimental)2.3 實踐測試
2.3.1 stack overflow
其中CMakeLists.txt如下：
cmake_minimum_required (VERSION 2.8) project (sanitizer) set(CMAKE_CXX_FLAGS "-g -fsanitize=leak -fsanitize=address -fno-omit-frame-pointer") add_executable(sanitizer_stack_overflow src/sanitizer_stack_overflow.cpp)-fsanitize=address 使能Address Sanitizer工具
-fsanitize=leak 只使能Leak Sanitizer，檢測內存泄漏問題
-fno-omit-frame-pointer 檢測到內存錯誤時打印函數調用棧
-O1 代碼優化選項，可以打印更清晰的函數調用棧
其中src/sanitizer_stack_overflow.cpp如下：
#include <stdio.h> #include <stdlib.h> #include <string.h>int func0(void) {char str[4] = {0};strcpy(str, "1234");return 0; }int main(int argc, char *argv[]) {func0();return 0; }執行結果如下：
==10098==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffdcae5ca24 at pc 0x7faae0b134ca bp 0x7ffdcae5c9f0 sp 0x7ffdcae5c198 WRITE of size 5 at 0x7ffdcae5ca24 thread T0#0 0x7faae0b134c9 (/lib64/libasan.so.4+0x794c9)#1 0x400a6a in func0() /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:7#2 0x400ad2 in main /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:12#3 0x7faadfecf554 in __libc_start_main (/lib64/libc.so.6+0x22554)#4 0x4008f8 (/root/code/cmake_project/app/sanitizer/build/sanitizer_stack_overflow+0x4008f8)Address 0x7ffdcae5ca24 is located in stack of thread T0 at offset 36 in frame#0 0x4009b6 in func0() /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:5This frame has 1 object(s):[32, 36) 'str' <== Memory access at offset 36 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext(longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow (/lib64/libasan.so.4+0x794c9) Shadow bytes around the buggy address:說明：
2.3.2 heap overflow
src/sanitizer_heap_overflow.cpp 代碼如下：
#include <stdio.h> #include <stdlib.h> #include <string.h>int func1(void) {char *p = (char*)malloc(sizeof(char)*4);char chs[] = {"12345"};memset(p, 0x0, 4);if (p != NULL) {memcpy(p, chs, 5);}return 0; }int main(int argc, char *argv[]) {func1();return 0; }執行結果如下：
================================================================= ==10373==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000014 at pc 0x7f8f772ba4ca bp 0x7fff5a93fc10 sp 0x7fff5a93f3b8 WRITE of size 5 at 0x602000000014 thread T0#0 0x7f8f772ba4c9 (/lib64/libasan.so.4+0x794c9)#1 0x400ac2 in func1() /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:10#2 0x400b2c in main /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:16#3 0x7f8f76676554 in __libc_start_main (/lib64/libc.so.6+0x22554)#4 0x4008d8 (/root/code/cmake_project/app/sanitizer/build/sanitizer_heap_overflow+0x4008d8)0x602000000014 is located 0 bytes to the right of 4-byte region [0x602000000010,0x602000000014) allocated by thread T0 here:#0 0x7f8f7731f8a0 in malloc (/lib64/libasan.so.4+0xde8a0)#1 0x400a0a in func1() /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:6#2 0x400b2c in main /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:16#3 0x7f8f76676554 in __libc_start_main (/lib64/libc.so.6+0x22554)SUMMARY: AddressSanitizer: heap-buffer-overflow (/lib64/libasan.so.4+0x794c9) Shadow bytes around the buggy address:說明：
2.3.2 use after free
#include <stdio.h> #include <stdlib.h> #include <string.h>void func2(void) {int * a = (int*)malloc(sizeof(int)*1);if ( a != NULL ) {*a = 1;printf("a is:%d.",*a);free(a);*a = 2;printf("error a is:%d.",*a);} }int main(int argc, char *argv[]) {func2();return 0; }執行結果如下：
================================================================= ==3838==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000000010 at pc 0x000000400a43 bp 0x7ffcdbefd570 sp 0x7ffcdbefd560 WRITE of size 4 at 0x602000000010 thread T0#0 0x400a42 in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:11#1 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17#2 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)#3 0x4008d8 (/root/Public/cmake_code/cmake_project/app/sanitizer/build/sanitizer_use_after_free+0x4008d8)0x602000000010 is located 0 bytes inside of 4-byte region [0x602000000010,0x602000000014) freed by thread T0 here:#0 0x7ff739e7f508 in __interceptor_free (/lib64/libasan.so.4+0xde508)#1 0x400a0b in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:10#2 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17#3 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)previously allocated by thread T0 here:#0 0x7ff739e7f8a0 in malloc (/lib64/libasan.so.4+0xde8a0)#1 0x400998 in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:6#2 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17#3 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)SUMMARY: AddressSanitizer: heap-use-after-free /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:11 in func2() Shadow bytes around the buggy address:說明：
2.3.3 global_buffer_overflow
src/sanitizer_global_buffer_overflow.cpp 代碼如下：
#include <stdio.h>int g_abc[11];int func3(void) {int i = 0;for (i = 0; i <= 100; i++) {printf("value:%d\t",g_abc[i]);if (i%10 == 0 && i != 0) {printf("\n");}}return g_abc[12]; }int main() {func3();return 0; }執行結果如下：
value:0 value:0 value:0 value:0 value:0 value:0 value:0 value:0 value:0 value:0 value:0 ================================================================= ==4137==ERROR: AddressSanitizer: global-buffer-overflow on address 0x00000060216c at pc 0x0000004009e9 bp 0x7ffc3d837020 sp 0x7ffc3d837010 READ of size 4 at 0x00000060216c thread T0#0 0x4009e8 in func3() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:8#1 0x400a86 in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:18#2 0x7fd1fe1a2554 in __libc_start_main (/lib64/libc.so.6+0x22554)#3 0x4008d8 (/root/Public/cmake_code/cmake_project/app/sanitizer/build/sanitizer_global_buffer_overflow+0x4008d8)0x00000060216c is located 0 bytes to the right of global variable 'g_abc' defined in '/root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:3:5' (0x602140) of size 44 SUMMARY: AddressSanitizer: global-buffer-overflow /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:8 in func3() Shadow bytes around the buggy address:說明：
2.3.4 memory leaks
src/sanitizer_memory_leaks.cpp 代碼如下：
#include <stdlib.h>char func4() {char *x = (char*)malloc(10 * sizeof(char*));return x[5]; }int main(int argc, char *argv[]) {func4();return 0; }ASAN_OPTIONS=detect_leaks=1 ./sanitizer_memory_leaks
================================================================= ==5501==ERROR: LeakSanitizer: detected memory leaksDirect leak of 80 byte(s) in 1 object(s) allocated from:#0 0x7f4d1a3848a0 in malloc (/lib64/libasan.so.4+0xde8a0)#1 0x400848 in func4() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_memory_leaks.cpp:4#2 0x4008a2 in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_memory_leaks.cpp:9#3 0x7f4d196db554 in __libc_start_main (/lib64/libc.so.6+0x22554)SUMMARY: AddressSanitizer: 80 byte(s) leaked in 1 allocation(s).3. sanitizer 原理介紹
AddressSanitizer主要包括兩部分：
- 插樁(Instrumentation)
- 動態運行庫(Run-time library)。
插樁主要是針對在llvm編譯器級別對訪問內存的操作(store，load，alloca等)，將它們進行處理。
動態運行庫主要提供一些運行時的復雜的功能(比如poison/unpoison shadow memory)以及將malloc,free等系統調用函數hook住。
該算法的思路是：如果想防住Buffer Overflow漏洞，只需要在每塊內存區域右端（或兩端，能防overflow和underflow）加一塊區域（RedZone），使RedZone的區域的影子內存（Shadow Memory)設置為不可寫即可。具體的示意圖如下圖所示。
- 內存映射
AddressSanitizer保護的主要原理是對程序中的虛擬內存提供粗粒度的影子內存(每8個字節的內存對應一個字節的影子內存)，為了減少overhead，采用了直接內存映射策略，所采用的具體策略如下：Shadow=(Mem >> 3) + offset。每8個字節的內存對應一個字節的影子內存，影子內存中每個字節存取一個數字k,如果k=0，則表示該影子內存對應的8個字節的內存都能訪問，如果0<k<7,表示前k個字節可以訪問，如果k為負數，不同的數字表示不同的錯誤（e.g. Stack buffer overflow, Heap buffer overflow)。
- 插樁
為了防止buffer overflow，需要將原來分配的內存兩邊分配額外的內存Redzone，并將這兩邊的內存加鎖，設為不能訪問狀態，這樣可以有效的防止buffer overflow(但不能杜絕buffer overflow)。以下是在棧中插樁的一個例子。
未插樁的代碼：
插樁后的代碼：
插樁后的代碼：
在動態運行庫中將malloc/free函數進行了替換。在malloc函數中額外的分配了Redzone區域的內存，將與Redzone區域對應的影子內存加鎖，主要的內存區域對應的影子內存不加鎖。
free函數將所有分配的內存區域加鎖，并放到了隔離區域的隊列中(保證在一定的時間內不會再被malloc函數分配)，可檢測Use after free類的問題。
詳細了解ASan算法原理可以訪問以下地址：
https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm
4. 對比 sanitizer 和 valgrind
4.1 sanitizer
-
包括address, memory, leak等多種sanitizer檢測工具
-
在使用gcc或者clang編譯時，加入額外編譯選項"-fsanitize=leak"
-
memsanitizer和leaksanitizer只能夠在clang中使用
-
能夠準確檢測出任何memory leak或者error
-
如果需要定位到源文件，需要指定以下環境: (否則只會定位到內存地址)
export ASAN_OPTIONS=‘abort_on_error=1’ ==> 過將環境變量 ASAN_OPTIONS 修改成如下形式來迫使軟件崩潰
-
優點： 定位準確， 檢查全面, 性能預計降低2倍左右
-
缺點： 需要重新編譯可執行文件
4.2 valgrind
linux平臺下的內存檢測工具包含多種tool包
- massif
- memcheck
輸出：內存泄漏、越界的代碼位置
作用：檢測內存泄漏或者內存越界。
如果是debug版本的程序，可以直接定位到行。
Note：still reacheable部分可以忽略。
常見問題：
5.1 malloc, calloc 與free不配對提前return或者goto使用時，造成possible leak
5.2 free 多次同一內存free未初始化的內存
5.3 如果使用了tcmalloc4 代替原始的malloc, 會使得valgrind失效
-
優點： 可以對任何可執行文件使用， 可視化圖像顯示內存使用
-
缺點： 常常會有誤報， 受編譯環境影響較大， 性能預計降低10倍左右
4. 參考資料
- https://www.jianshu.com/p/9e85345e500b
- https://www.bynav.com/cn/resource/bywork/healthy-work/70.html
- https://juejin.im/post/6844904067538370573
總結
以上是生活随笔為你收集整理的内存检测工具：sanitizer的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 内存错误检测工具——kfence工作原理
- 下一篇： 在线租车app开发