內存使用統計

# Server

redis_version:3.0.7

# Memory

used_memory:7270580648

?- used_memory: Total number of bytes allocated by Redis using its allocator (either standard libc, jemalloc, or an alternative allocator such as tcmalloc)

used_memory_rss:7979900928

?- used_memory_rss: Number of bytes that Redis allocated as seen by the operating system (a.k.a resident set size). This is the number reported by tools such as top(1) and ps(1)

?

used_memory_peak:8110394944

?- used_memory_peak: Peak memory consumed by Redis (in bytes)

used_memory_lua:36864

?- used_memory_lua: Number of bytes used by the Lua engine

?

mem_fragmentation_ratio:1.10

?- mem_fragmentation_ratio: Ratio between used_memory_rss and used_memory

?

mem_allocator:jemalloc-3.6.0

?- mem_allocator: Memory allocator, chosen at compile time

內存消耗劃分

Redis進程自身內存，對象內存，緩沖內存，和內存碎片。其中一個空的Redis進程消耗的內存可忽略不計，主要看后三者。

1.對象內存

其是Redis內存占用最大的一塊，存儲著應用的所有數據。Redis中的數據采用key-value形式，創建鍵值對時，至少創建兩個類型對象，key對象和value對象。對象內存消耗可簡單理解為sizeof(keys)+sizeof(values)。鍵對象都是字符串。value對象較復雜，主要包括5種基本數據類型：字符串，列表，哈希，集合，有序集合。其它的數據類型都是在這5種數據結構上實現的。

使用Redis時，應控制過長的key，和控制value的規模，避免響應延時和內存溢出。

2.緩沖內存

主要包括：客戶端緩沖，復制積壓緩沖區，和AOF緩沖區。

客戶端緩沖指的是所有接入Redis服務的TCP連接的輸入輸出緩沖。輸入緩沖無法控制，最大空間為1G，超過將斷開。

輸出緩沖通過client-output-buffer-limit控制，看下redis.conf中對這個參數的解釋：

# The client output buffer limits can be used to force disconnection of clients

# that are not reading data from the server fast enough for some reason (a

# common reason is that a Pub/Sub client can't consume messages as fast as the

# publisher can produce them).

# The limit can be set differently for the three different classes of clients:

# normal -> normal clients including MONITOR clients

# slave? -> slave clients

# pubsub -> clients subscribed to at least one pubsub channel or pattern

# The syntax of every client-output-buffer-limit directive is the following:

# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>

# A client is immediately disconnected once the hard limit is reached, or if

# the soft limit is reached and remains reached for the specified number of

# seconds (continuously).

# So for instance if the hard limit is 32 megabytes and the soft limit is

# 16 megabytes / 10 seconds, the client will get disconnected immediately

# if the size of the output buffers reach 32 megabytes, but will also get

# disconnected if the client reaches 16 megabytes and continuously overcomes

# the limit for 10 seconds.

# By default normal clients are not limited because they don't receive data

# without asking (in a push way), but just after a request, so only

# asynchronous clients may create a scenario where data is requested faster

# than it can read.

# Instead there is a default limit for pubsub and slave clients, since

# subscribers and slaves receive data in a push fashion.

# Both the hard or the soft limit can be disabled by setting them to zero.

client-output-buffer-limit normal 0 0 0

client-output-buffer-limit slave 256mb 64mb 60

client-output-buffer-limit pubsub 32mb 8mb 60

可見客戶端可分為3類：普通客戶端，從客戶端，和訂閱客戶端。Redis對普通客戶端默認沒有限制，一般普通客戶端的內存消耗可忽略不計。但配置中提到了MONITOR命令，即當使用有大量數據輸出的命令，且數據無法及時推送給客戶端時，這部分內存消耗就不能忽略了。使用CLIENT LIST，輸出中omem代表這部分內存。

id=39 addr=192.168.4.124:53000 fd=44 name= age=346151 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get

?- omem: The amount of memory used by the client for the output buffer.

Redis對從客戶端和訂閱客戶端默認進行了限制，主要是Redis推送數據給該類客戶端，防止連接緩慢或消費緩慢，造成緩沖區溢出。

對于復制積壓緩沖去，Redis配置提供了repl-backlog-size控制，對于該緩沖區主節點只申請一次，從節點共享。設置較大的此緩沖區，可以有效避免全量復制（默認才1mb）。

# Set the replication backlog size. The backlog is a buffer that accumulates

# slave data when slaves are disconnected for some time, so that when a slave

# wants to reconnect again, often a full resync is not needed, but a partial

# resync is enough, just passing the portion of data the slave missed while

# disconnected.

# The bigger the replication backlog, the longer the time the slave can be

# disconnected and later be able to perform a partial resynchronization.

# The backlog is only allocated once there is at least a slave connected.

# repl-backlog-size 1mb

AOF緩沖區，這部分空間用于Redis重寫期間保存最近的寫入命令，大小用戶無法控制，取決于AOF重寫時間和寫入命令量，通常很小。

3.內存碎片

Redis默認使用jemalloc內存分配器，可選的還有glibc，tcmalloc。內存分配器是為了更好的管理和重復利用內存，內存分配策略一般采用固定大小的內存塊進行。如保存5KB的對象時，jemalloc可能會采用8KB的塊存儲，而剩下的3KB空間就不好再分配給其它對象，變成了內存碎片。

jemalloc針對內存碎片問題專門做了優化，一般不會存在過度碎片化的問題，正常的碎片率在1.03左右，但如下場景容易出現內存碎片的問題：

?- 頻繁更新操作，對已存在的鍵執行APPEND，SETRANGE等更新操作。

?- 大量刪除過期鍵，鍵對象刪除后，釋放的空間無法得到充分利用，導致碎片上升。

內存碎片問題常見的解決方法：

?- 數據對齊：數據盡量采用固定長度的字符串，但這視具體業務而定。

?- 安全重啟：借助高可用架構Sentinel或Cluster，主從切換，重啟。

還有一部分內存使用，子進程內存消耗，主要指Redis執行AOF或RDB重寫時，fork子進程的內存消耗。看下RDB重寫的過程，Redis利用了Linux的copy-on-write特性，優化了內存的使用。

?- Redis forks. We now have a child and a parent process.

?- The child starts to write the dataset to a temporary RDB file.

?- When the child is done writing the new RDB file, it replaces the old one.

對于copy-on-write，可參考鏈接https://linux.die.net/man/2/fork，從此也可知道Redis fork子進程并不需要消耗1倍的父進程內存，實際消耗根據期間寫入命令量決定。

另外此處要設置兩個內核參數，從Redis的啟動日志中可以看到該提示：

# WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

設置vm.overcommit_memory = 1，允許內核可以分配所有的物理內存，防止Redis執行fork時，因系統剩余內存不足而失敗。

# WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

設置echo never > /sys/kernel/mm/transparent_hugepage/enabled，關閉THP，防止降低fork的速度，和copy-on-write期間內存過度消耗。

轉載于:https://blog.51cto.com/coveringindex/2385617

總結

以上是生活随笔為你收集整理的Redis的内存消耗的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。