文章目錄

• PreView
• 模擬寫入一個BigKey
• 如何發(fā)現(xiàn)BigKey
• • redis-cli --bigkeys
  • debug object
• 如何優(yōu)雅的刪除BigKey （lazy delete）
• 關(guān)于lazyfree的幾個配置項
• 如何優(yōu)化bigKey

---

PreView

Redis進(jìn)階-Redis鍵值設(shè)計及BigKey問題粗略的討論了BigKey，這里我們繼續(xù)再來研究下BigKey

Redis Version : 5.0.3

---

模擬寫入一個BigKey

---

如何發(fā)現(xiàn)BigKey

總結(jié)下無非幾個點(diǎn)

• 應(yīng)用異常日志 （沒法直接從日志關(guān)鍵字中看出來，因為這個bigkey阻塞以后，會引發(fā)一些奇怪的問題，比如從連接池獲取不到連接等等）
• redis-cli --bigkeys
• 監(jiān)控系統(tǒng)上報
• 其他…

redis-cli --bigkeys

先看下 redis-cli 都支持什么參數(shù)

[redis@artisan bin]$ ./redis-cli --help redis-cli 5.0.3Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]-h <hostname> Server hostname (default: 127.0.0.1).-p <port> Server port (default: 6379).-s <socket> Server socket (overrides hostname and port).-a <password> Password to use when connecting to the server.You can also use the REDISCLI_AUTH environmentvariable to pass this password more safely(if both are used, this argument takes predecence).-u <uri> Server URI.-r <repeat> Execute specified command N times.-i <interval> When -r is used, waits <interval> seconds per command.It is possible to specify sub-second times like -i 0.1.-n <db> Database number.-x Read last argument from STDIN.-d <delimiter> Multi-bulk delimiter in for raw formatting (default: \n).-c Enable cluster mode (follow -ASK and -MOVED redirections).--raw Use raw formatting for replies (default when STDOUT isnot a tty).--no-raw Force formatted output even when STDOUT is not a tty.--csv Output in CSV format.--stat Print rolling stats about server: mem, clients, ...--latency Enter a special mode continuously sampling latency.If you use this mode in an interactive session it runsforever displaying real-time stats. Otherwise if --raw or--csv is specified, or if you redirect the output to a nonTTY, it samples the latency for 1 second (you can use-i to change the interval), then produces a single outputand exits.--latency-history Like --latency but tracking latency changes over time.Default time interval is 15 sec. Change it using -i.--latency-dist Shows latency as a spectrum, requires xterm 256 colors.Default time interval is 1 sec. Change it using -i.--lru-test <keys> Simulate a cache workload with an 80-20 distribution.--replica Simulate a replica showing commands received from the master.--rdb <filename> Transfer an RDB dump from remote server to local file.--pipe Transfer raw Redis protocol from stdin to server.--pipe-timeout <n> In --pipe mode, abort with error if after sending all data.no reply is received within <n> seconds.Default timeout: 30. Use 0 to wait forever.--bigkeys Sample Redis keys looking for big keys.--hotkeys Sample Redis keys looking for hot keys.only works when maxmemory-policy is *lfu.--scan List all keys using the SCAN command.--pattern <pat> Useful with --scan to specify a SCAN pattern.--intrinsic-latency <sec> Run a test to measure intrinsic system latency.The test will run for the specified amount of seconds.--eval <file> Send an EVAL command using the Lua script at <file>.--ldb Used with --eval enable the Redis Lua debugger.--ldb-sync-mode Like --ldb but uses the synchronous Lua debugger, inthis mode the server is blocked and script changes arenot rolled back from the server memory.--cluster <command> [args...] [opts...]Cluster Manager command and arguments (see below).--verbose Verbose mode.--no-auth-warning Don't show warning message when using password on commandline interface.--help Output this help and exit.--version Output version and exit.Cluster Manager Commands:Use --cluster help to list all available cluster manager commands.Examples:cat /etc/passwd | redis-cli -x set mypasswdredis-cli get mypasswdredis-cli -r 100 lpush mylist xredis-cli -r 100 -i 1 info | grep used_memory_human:redis-cli --eval myscript.lua key1 key2 , arg1 arg2 arg3redis-cli --scan --pattern '*:12345*'(Note: when using --eval the comma separates KEYS[] from ARGV[] items)When no command is given, redis-cli starts in interactive mode. Type "help" in interactive mode for information on available commands and settings.[redis@artisan bin]$

官方提供的工具

執(zhí)行檢測

[redis@artisan bin]$ ./redis-cli --bigkeys# Scanning the entire keyspace to find biggest keys as well as # average sizes per key type. You can use -i 0.1 to sleep 0.1 sec # per 100 SCAN commands (not usually needed).[00.00%] Biggest string found so far 'artisan43206' with 12 bytes [37.40%] Biggest string found so far 'artisanBIG' with 26890 bytes-------- summary -------Sampled 100004 keys in the keyspace! Total key length in bytes is 1188921 (avg len 11.89)Biggest string found 'artisanBIG' has 26890 bytes100004 strings with 1215805 bytes (100.00% of keys, avg size 12.16) 0 lists with 0 items (00.00% of keys, avg size 0.00) 0 sets with 0 members (00.00% of keys, avg size 0.00) 0 hashs with 0 fields (00.00% of keys, avg size 0.00) 0 zsets with 0 members (00.00% of keys, avg size 0.00) 0 streams with 0 entries (00.00% of keys, avg size 0.00) [redis@artisan bin]$

redis 也是通過scan方式對key進(jìn)行統(tǒng)計，無需擔(dān)心對redis造成阻塞 ，從結(jié)果上看 發(fā)現(xiàn)了 artisanBIG 這個bigkey

---

debug object

這個命令本身會阻塞Redis，如果 O（N） ，需要關(guān)注N的大小。

# 查看artisanBIG序列化后的長度 [redis@artisan bin]$ ./redis-cli debug object artisanBIG Value at:0x7fa527edd0b0 refcount:1 encoding:raw serializedlength:5044 lru:7863297 lru_seconds_idle:112

輸出參數(shù)說明：

• Value at：key的內(nèi)存地址
• refcount：引用次數(shù)
• encoding：編碼類型
• serializedlength：序列化長度 5044 —> 5KB ,并不代表真實(shí)的大小，僅供參考
• lru_seconds_idle：空閑時間
• …

object命令： https://redis.io/commands/object

---

如何優(yōu)雅的刪除BigKey （lazy delete）

• 傳統(tǒng)的方式: DEL 唄

DEL 毫無疑問，阻塞命令，如果 O(N) N 很大，需要考慮。

• 另外一點(diǎn)需要考慮的： 有些key 設(shè)置了過期時間、rename操作等，這些key redis 自己維護(hù) ，那處理這些BigKey , 同樣的會阻塞。 并且這行操作不會記錄到慢查詢中，你都發(fā)現(xiàn)不了。 （慢查詢只記錄客戶端的行為，Redis server本身的行為并不會被記錄

---

• 推薦： Redis 4.0 提供了 lazy delete (unlink命令)

那4.0 一下的版本，怎么處理這些bigKey呢？ -----》 scan , 分批次刪除

我們重點(diǎn)來看 lazy delete -------------> UNLINK 命令

Lazy Redis is better Redis

UNLINK 官方文檔： https://redis.io/commands/unlink

127.0.0.1:6379> UNLINK artisanBIG (integer) 1 127.0.0.1:6379>

---

關(guān)于lazyfree的幾個配置項

• lazyfree-lazy-eviction：是否異步驅(qū)逐key，當(dāng)內(nèi)存達(dá)到上限，分配失敗后
• lazyfree-lazy-expire：是否異步進(jìn)行key過期事件的處理
• lazyfree-lazy-server-del：del命令是否異步執(zhí)行刪除操作，類似unlink
• replica-lazy-flush：replica client做全同步的時候，是否異步flush本地db

默認(rèn)都是no，按需開啟

---

如何優(yōu)化bigKey

• 優(yōu)化數(shù)據(jù)結(jié)構(gòu) : 字符串？ hash? 。。。。
• 合理拆分key ： 1個大key 拆分成N個小key
• 命令優(yōu)化 ： getall–> mget
• 物理隔離或者升級網(wǎng)卡（治標(biāo)不治本）

Redis進(jìn)階-Redis鍵值設(shè)計及BigKey問題

總結(jié)

以上是生活随笔為你收集整理的Redis进阶-如何发现和优雅的处理BigKey一二事的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。