本章主要講解redis的使用案例，相對于以往的技術，redis在這些領域將大大簡化或者提高程序的便利和穩定。比如日志記錄，相對于以往的文件記錄方式將更加靈活，便于數據操作，

日志記錄

以往的日志記錄采用本地文件存儲的方式，這種方式有一種弊端，由于是文本存儲，各個服務器之間很難協調，很難對多個數據文件進行聚合，導致之后的數據分析，將顯得很不方便，相對而言，由于redis數據庫可以在不同的服務器之間通信，加上自帶數據處理函數，所以將更加方便。
對不同的數據分析，不同的日志有不同的好處，為了方便管理或者處理，需要對相應的日志做分類處理。比如對于一個網站而言，日志是很多的，如果大量記錄是不現實的，但是，很多情況下，比如我們服務器突然出現問題，或者說現在我們需要立刻查看最近的操作的數據情況，這是，我們就可以使用最新的日志，這些日志只會記錄最近發生的響應，而不會記錄很久之前的數據情況，這樣就便于我們查錯，或者查看最近的操作情況。

• 最新日志：分別記錄最近的debug\info\warning\error\critical等日志情況，便于查看（這里使用的是列表存儲，方便添加和刪除日志）

• 常見日志：常見日志即我們認為重要的日志，比如用戶的操作，加入購物車，購買等信息。這些都是很重要的日志信息，可用于數據挖掘。但是文章中并沒有給出這個例子，而是，對不同的日志信息進行排序，對每個動作進行打分，從而只保存經常操作的那些動作，即表明這些操作是最重要的，有很高的商業價值。（同樣是列表）
  

• 計數器和數據統計
  

計數器是什么？這里的計數器并不是電子電路里面的時鐘計數器，而是一種數據記錄，比如用戶的登陸次數，百度上某個醫療廣告的點擊量，整個百度網頁每個渲染頁面的點擊量等等，這些都是很重要的價值信息。能夠從中發掘大量的商業價值。
數據統計。在統計學中，有幾個重要的數據衡量指標，分別是最大值、最小值、平均值、數量、總分數、標準差等等；比如對一個網站進行優化，我么你需要記錄網站各個頁面的響應時間，這些數據有利于我們有正對性的進行優化。

• 本行計數器案例主要模擬，分別在1秒、5秒、一分鐘、一小時等，網站的點擊量，由于需要進行排序，所以使用的是有序表和哈希集結合

• 數據統計主要是模擬網頁的加載時間

• IP地址等excel本地數據查詢

  我們經常會使用搜索，但是這里搜索并不是百度搜索，而是有現成的數據。比如我們一個存儲數據的excel表，亦或是文本等等。例如，有時候我們會查詢某個ip所對應的網址，但是這個數據很大，如果存儲在一般的關系型數據庫，查詢將會有點慢，因此，這時候就可以使用redis數據庫，大大加快查詢速度。

• 服務配置信息動態更改

  開發一個應用會存在很多配置文件，只寫配置文件并不是一成不變，以往的人工配置，會出現，一旦一個配置文件發生改變，就需要人工取重新配置，這樣大大降低了效率。因此，我們這里考慮使用程序自動化配置，即將配置信息保存在redis中，將其寫入到程序中作為守護線程，每當配置文件發生改變都會自動進行配置

• 文中的樣例為：redis配置redis的連接配置，會將redis的連接信息保存在redis中，自動檢查新的配置文件，然后更新配置到程序中

詳細代碼：

import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import org.apache.commons.csv.CSVFormat; import org.apache.commons.csv.CSVParser; import org.apache.commons.csv.CSVRecord; import org.javatuples.Pair; import redis.clients.jedis.Jedis; import redis.clients.jedis.Pipeline; import redis.clients.jedis.Transaction; import redis.clients.jedis.Tuple; import redis.clients.jedis.ZParams;import java.io.File; import java.io.FileNotFoundException; import java.io.FileReader; import java.io.IOException; import java.text.Collator; import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.Date; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Set; import java.util.UUID;/*** @author: ZouTai* @date: 2018/7/6* @description:* @create: 2018-07-06 15:23*/ public class Chapter05 {public static final String INFO = "info";public static final SimpleDateFormat TIMESTAMP =new SimpleDateFormat("EEE MMM dd HH:00:00 YYYY");public static final SimpleDateFormat ISO_FORMAT =new SimpleDateFormat("yyyy-MM-dd'T'HH:00:00");public static final Collator COLLATOR = Collator.getInstance();private static final int[] PRECISION = new int[]{1, 5, 60, 300, 3600, 18000, 86400};public static void main(String[] args) throws InterruptedException {new Chapter05().run();}private void run() throws InterruptedException {Jedis conn = new Jedis("localhost");conn.select(15);// 測試1：redis記錄日志 // testLogRecent(conn); // testLogCommon(conn);// 測試2：計數器和統計數據 // testCounters(conn); // testStats(conn); // testAccessTime(conn);// 測試3：查找ip所在的城市及其詳細信息 // testIpLookup(conn);// 測試4：服務發現與配置 // testIsUnderMaintenance(conn);testConfig(conn);}private void testConfig(Jedis conn) throws InterruptedException {System.out.println("首先，創建一個配置文件map：");Map<String, Object> config = new HashMap<String, Object>();config.put("db", 15);// 添加到redis中setConfig(conn, "redis", "test", config);Jedis conn2 = redisConnection("test");System.out.println("新的連接是否存在：" + (conn2.info() != null));}private static final Map<String, Jedis> REDIS_CONNECTIONS = new HashMap<String, Jedis>();private static final Map<String, Map<String, Object>> CONFIGS =new HashMap<String, Map<String, Object>>();private static final Map<String, Long> CHECKED = new HashMap<String, Long>();private Jedis redisConnection(String component) {Jedis configConn = REDIS_CONNECTIONS.get("config");if (configConn == null) {configConn = new Jedis("localhost");configConn.select(15);REDIS_CONNECTIONS.put("config", configConn);}String key = "config:redis:" + component;Map<String, Object> oldConfig = CONFIGS.get(key);Map<String, Object> newConfig = getConfig(configConn, "redis", component);// 判斷配置文件是否相等，不相等，更改當前配置文件if (!newConfig.equals(oldConfig)) {Jedis conn = new Jedis("localhost");conn.select(((Double) newConfig.get("db")).intValue());REDIS_CONNECTIONS.put(key, conn);}return REDIS_CONNECTIONS.get(key);}/*** 從redis獲取新的配置文件，與當前程序中的配置文件CONFIGS對比*/@SuppressWarnings("unchecked")private Map<String, Object> getConfig(Jedis conn, String type, String component) {String key = "config:" + type + ":" + component;long wait = 1000;if (CHECKED.get(key) == null || CHECKED.get(key) < System.currentTimeMillis() - wait) {CHECKED.put(key, System.currentTimeMillis());String value = conn.get(key);Map<String, Object> config = null;if (value != null) {Gson gson = new Gson();config = (Map<String, Object>) gson.fromJson(value, new TypeToken<Map<String, Object>>() {}.getType());} else {config = new HashMap<String, Object>();}CONFIGS.put(key, config);}return CONFIGS.get(key);}private void setConfig(Jedis conn, String type, String component, Map<String, Object> config) {Gson gson = new Gson();conn.set("config:" + type + ":" + component, gson.toJson(config));}private void testIsUnderMaintenance(Jedis conn) throws InterruptedException {boolean flag = false;flag = isUnderMaintenance(conn);System.out.println("是否在維護:" + flag);conn.set("is-under-maintenance", "yes");flag = isUnderMaintenance(conn);System.out.println("改變后，是否在維護:" + flag);Thread.sleep(1000);flag = isUnderMaintenance(conn);System.out.println("停留1秒鐘，是否在維護:" + flag);conn.del("is-under-maintenance");Thread.sleep(1000);flag = isUnderMaintenance(conn);System.out.println("清除后，是否在維護:" + flag);}private long lastChecked;private boolean underMaintenance;private boolean isUnderMaintenance(Jedis conn) {if (lastChecked < System.currentTimeMillis() - 1000) {String flag = conn.get("is-under-maintenance");underMaintenance = "yes".equals(flag);}return underMaintenance;}private void testIpLookup(Jedis conn) {String cwd = System.getProperty("user.dir");File blocks = new File(cwd + "/GeoLiteCity-Blocks.csv");File locations = new File(cwd + "/GeoLiteCity-Location.csv");if (!blocks.exists()) {System.out.println("文件不存在：" + blocks);}if (!locations.exists()) {System.out.println("文件不存在：" + locations);}System.out.println("將IP數據載入到redis："); // importIpsToRedis(conn, blocks);long ipSum = conn.zcard("ip2cityId:");System.out.println("IP數量為：" + ipSum);System.out.println("將城市數據載入redis："); // importCitiesToRedis(conn, locations);long citySum = conn.hlen("cityId2City:");System.out.println("城市數量為：" + citySum);System.out.println("隨機查找ip");for (int i = 0; i < 5; i++) {String ip = randomOctet(255) + "."+ randomOctet(256) + "."+ randomOctet(256) + "."+ randomOctet(256);String cityMessage = Arrays.toString(findCityByIp(conn, ip));System.out.println("所在城市信息為：");System.out.println(cityMessage);}}private void importCitiesToRedis(Jedis conn, File file) {FileReader reader = null;Gson gson = new Gson();try {reader = new FileReader(file);CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT);for (CSVRecord record : parser) {if (record.size() < 4 || !Character.isDigit(record.get(0).charAt(0))) {continue;}String cityId = record.get(0);String country = record.get(1);String region = record.get(2);String city = record.get(3);String postalCode = record.get(4);String latitude = record.get(5);String longitude = record.get(6);String metroCode = record.get(7);String areaCode = record.get(8);String json = gson.toJson(new String[]{country, region, city, postalCode, latitude, longitude, metroCode, areaCode});conn.hset("cityId2City:", cityId, json);}} catch (FileNotFoundException e) {e.printStackTrace();} catch (IOException e) {e.printStackTrace();} finally {try {reader.close();} catch (IOException e) {e.printStackTrace();}}}private String[] findCityByIp(Jedis conn, String ipAddress) {long score = ipToScore(ipAddress);Set<String> rangs = conn.zrevrangeByScore("ip2cityId:", score, 0, 0, 1);if (rangs.size() == 0) {return null;}String cityId = rangs.iterator().next();cityId = cityId.substring(0, cityId.indexOf("_"));return new Gson().fromJson(conn.hget("cityId2City:", cityId), String[].class);}private String randomOctet(int max) {return String.valueOf((int) (Math.random() * max));}private void importIpsToRedis(Jedis conn, File file) {FileReader fileReader = null;try {int number = 0;fileReader = new FileReader(file);CSVParser parser = new CSVParser(fileReader, CSVFormat.DEFAULT);for (CSVRecord csvRecord : parser) {String startIp = csvRecord.get(0);if (startIp.toLowerCase().indexOf('i') != -1) {continue;}long score = 0;if (startIp.indexOf('.') != -1) {score = ipToScore(startIp);} else {score = Long.parseLong(startIp, 10);}System.out.println(number);String cityIp = csvRecord.get(2) + "_" + number;number++;conn.zadd("ip2cityId:", score, cityIp);}} catch (FileNotFoundException e) {e.printStackTrace();} catch (IOException e) {e.printStackTrace();} finally {try {fileReader.close();} catch (IOException e) {e.printStackTrace();}}}private long ipToScore(String ipAddress) {long score = 0;// "."需要轉義for (String str : ipAddress.split("\\.")) {score = score * 256 + Integer.parseInt(str, 10);}return score;}/*** 記錄程序執行時長** @param conn*/private void testAccessTime(Jedis conn) throws InterruptedException {AccessTimer timer = new AccessTimer(conn);for (int i = 0; i < 5; i++) {timer.setStart(System.currentTimeMillis());Thread.sleep((long) (Math.random() * 4000 + 1));timer.stop("request-" + i);}System.out.println("輸出所有的分數，只緩存最多的100個：");Set<Tuple> stats = conn.zrangeWithScores("slowest:AccessTime", 0, -1);for (Tuple tuple : stats) {System.out.println(tuple.getElement() + " -- " + tuple.getScore());}}private void testStats(Jedis conn) {List<Object> rs = null;System.out.println("寫入數據：");for (int i = 0; i < 5; i++) {int value = (int) (Math.random() * 11 + 5);rs = updateState(conn, "context", "type", value);}HashMap<String, Double> stats = getStats(conn, "context", "type");System.out.println("打印數據報表：");System.out.println(stats);}private HashMap<String, Double> getStats(Jedis conn, String context, String type) {String keys = "stats:" + context + ":" + type;Set<Tuple> datas = conn.zrangeWithScores(keys, 0, -1);HashMap<String, Double> stats = new HashMap<String, Double>();for (Tuple tuple : datas) {stats.put(tuple.getElement(), tuple.getScore());}stats.put("average", stats.get("sum") / stats.get("count"));// 標準差的公式需要自行推導double numerator = stats.get("sumsq") - Math.pow(stats.get("sum"), 2) / stats.get("count");double count = stats.get("count");stats.put("stddev", Math.pow(numerator / (count > 1 ? count - 1 : 1), .5));return stats;}private List<Object> updateState(Jedis conn, String context, String type, long value) {String destination = "stats:" + context + ":" + type;String startKey = destination + ":start";int timeout = 5000;long end = System.currentTimeMillis() + timeout;while (System.currentTimeMillis() < end) {conn.watch(startKey);long now = System.currentTimeMillis();String hourstart = ISO_FORMAT.format(new Date());String existing = conn.get(startKey);Transaction trans = conn.multi();if (existing != null && COLLATOR.compare(existing, now) < 0) {trans.rename(destination, destination + ":last");trans.rename(startKey, destination + ":plast");trans.set(startKey, hourstart);}String tkey1 = UUID.randomUUID().toString();String tkey2 = UUID.randomUUID().toString();trans.zadd(tkey1, value, "min");trans.zadd(tkey1, value, "max");trans.zunionstore(destination, new ZParams().aggregate(ZParams.Aggregate.MIN), destination, tkey1);trans.zunionstore(destination, new ZParams().aggregate(ZParams.Aggregate.MAX), destination, tkey2);trans.del(tkey1, tkey2);trans.zincrby(destination, 1, "count");trans.zincrby(destination, value, "sum");trans.zincrby(destination, value * value, "sumsq");List<Object> results = trans.exec();if (results == null) {continue;}return results.subList(results.size() - 3, results.size());}return null;}private void testCounters(Jedis conn) throws InterruptedException {conn.del("common:test:info");long now = System.currentTimeMillis() / 1000;for (int i = 0; i < 10; i++) {int count = (int) (Math.random() * 5) + 1; // 隨機計數加量大小updateCounter(conn, "test", count, now + i);}System.out.println("測試1秒鐘的：");ArrayList<Pair<Integer, Integer>> counter = getCounter(conn, "test", 1);for (Pair<Integer, Integer> count : counter) {System.out.println(" " + count);}assert counter.size() >= 10;System.out.println("測試5秒鐘的：");counter = getCounter(conn, "test", 5);for (Pair<Integer, Integer> count : counter) {System.out.println(" " + count);}assert counter.size() >= 2;CleanCountersThread thread = new CleanCountersThread(0, 2 * 86400);thread.start();Thread.sleep(10000);thread.quit();thread.interrupt();counter = getCounter(conn, "test", 86400);System.out.println("Did we clean out all of the counters? " + (counter.size() == 0));assert counter.size() == 0;}private ArrayList<Pair<Integer, Integer>> getCounter(Jedis conn, String name, int precision) {String hash = String.valueOf(precision) + ":" + name;Map<String, String> datas = conn.hgetAll("count:" + hash);ArrayList<Pair<Integer, Integer>> results = new ArrayList<Pair<Integer, Integer>>();for (Map.Entry<String, String> entry : datas.entrySet()) {results.add(new Pair<Integer, Integer>(Integer.parseInt(entry.getKey()),Integer.parseInt(entry.getValue())));}Collections.sort(results); // 之所有組裝，是為了進行排序，將舊的數據放在前面（按時間）return results;}private void updateCounter(Jedis conn, String name, int count, long now) {Transaction trans = conn.multi();for (int prec : PRECISION) {// 計數器的開始時間（有多個）long pnow = (now / prec) * prec;String hash = String.valueOf(prec) + ":" + name;trans.zadd("known:", 0, hash); // 有序表-記錄分數trans.hincrBy("count:" + hash, String.valueOf(pnow), count); // 哈希表記錄每個日志信息：（起始時間-計數次數）}trans.exec();}/*** 測試1.2* 記錄重要日志，并篩選** @param conn*/private void testLogCommon(Jedis conn) {conn.del("common:test:info");System.out.println("添加測試日志");for (int count = 1; count < 6; count++) {for (int i = 0; i < count; i++) {logCommon(conn, "test", "message-" + count);}}Set<Tuple> commonsets = conn.zrevrangeWithScores("common:test:info", 0, -1);for (Tuple tuple : commonsets) {System.out.println(tuple.getElement() + " : " + tuple.getScore());}assert commonsets.size() >= 5 : "日志數量不夠，出錯";}private void logCommon(Jedis conn, String name, String message) {logCommon(conn, name, message, INFO, 5000);}private void logCommon(Jedis conn, String name, String message, String severity, int timeout) {String commonDest = "common:" + name + ":" + severity;String startKey = commonDest + ":start";long end = System.currentTimeMillis() + timeout;while (System.currentTimeMillis() < end) {conn.watch(startKey);String existing = conn.get(startKey);Transaction trans = conn.multi(); // 創建事務String hourStart = ISO_FORMAT.format(new Date());/*** if用于判斷，當過了一個小時時，將數據進行持久化存儲。即程序只會記錄最近一個小時之內的日志，很久以前的將刪除* 更確切的說：“start”記錄當前一個小時的，“last”記錄上一個小時的（完整的一個小時）* commonDest用于保存日志分數值* startKey用于保存日志最后存儲的時間（按所在小時記錄）*/if (existing != null && COLLATOR.compare(existing, hourStart) < 0) {trans.rename(commonDest, commonDest + ":last");trans.rename(startKey, commonDest + ":pstart");trans.set(startKey, hourStart);}trans.zincrby(commonDest, 1, message); // 添加日志，加分// 同步到最近的日志，減少客戶端服務器交互：因為如果這里不處理，返回給客戶端，客戶端又要請求服務器保存。// 多了一次往返String recentDest = "recent:" + name + ':' + severity;trans.lpush(recentDest, TIMESTAMP.format(new Date()) + ' ' + message);trans.ltrim(recentDest, 0, 99);List<Object> results = trans.exec();if (results == null) {continue;}return;}}/*** 測試1.1* 記錄登錄日志** @param conn*/private void testLogRecent(Jedis conn) {conn.del("recent:test:info");for (int i = 0; i < 5; i++) {logRecent(conn, "test", "this message is : " + i);System.out.println("\n");}}private void logRecent(Jedis conn, String name, String message) {logRecent(conn, name, message, INFO);List<String> recentLogs = conn.lrange("recent:test:info", 0, -1);System.out.println("當前所有的登陸日志有：");for (String log : recentLogs) {System.out.println(log);}assert recentLogs.size() >= 5 : "日志數量不夠，出錯";}private void logRecent(Jedis conn, String name, String message, String severity) {String destination = "recent:" + name + ":" + severity;Pipeline pipe = conn.pipelined();pipe.lpush(destination, TIMESTAMP.format(new Date()) + " " + message);pipe.ltrim(destination, 0, 99); // 截取信息，只保留前100條pipe.sync(); // 同步管道數據，獲取所有的響應值}private class CleanCountersThread extends Thread {private Jedis conn;private int sampleCount = 100;private boolean quit;private long timeOffset; // used to mimic a time in the future.public CleanCountersThread(int sampleCount, int timeOffset) {this.conn = new Jedis("localhost");this.conn.select(15);this.sampleCount = sampleCount;this.timeOffset = timeOffset;}public void quit() {quit = true;}@Overridepublic void run() {int passes = 0; // 清理的次數while (!quit) {long start = System.currentTimeMillis();int index = 0; // 記錄表下標[1,5,60...86400]while (index < conn.zcard("known:")) {Set<String> oneSet = conn.zrange("known:", index, index);index++;// 處理了一個// 如果表中無數據，則返回if (oneSet.size() == 0) {break;}String onehash = oneSet.iterator().next();int prec = Integer.parseInt(onehash.substring(0, onehash.indexOf(':')));int numPrec = (int) Math.floor(prec / 60);if (numPrec == 0) {numPrec = 1;}// 對更新在60秒之內的，為防止清理多次，即只清理一次，需要計數判斷// 此時numPrec==1，下列計算將不等于0，不再進行刪除if ((passes % numPrec) != 0) {continue;}String hkey = "count:" + onehash;ArrayList<String> keyLists = new ArrayList<String>(conn.hkeys(hkey));Collections.sort(keyLists);String cutoff = String.valueOf((System.currentTimeMillis() + timeOffset) / 1000 - sampleCount * prec);int delNum = bisectRight(keyLists, cutoff);// 刪除數據System.out.println(hkey.toString() + "刪除數量為：" + delNum);if (delNum != 0) {conn.hdel(hkey, keyLists.subList(0, delNum).toArray(new String[0]));// 如果刪除的數據剛好等于數據的總數，則表明表已經清空，需要同時刪除有序表if (delNum == keyLists.size()) {conn.watch(hkey);if (conn.hlen(hkey) == 0) {Transaction trans = conn.multi();trans.zrem("known:", onehash);trans.exec();index--;// 直接刪除了，需要處理index}} else {conn.unwatch(); // 不為空時，解除監控}}}// 清理一次+1passes++;// 使程序60秒執行一次// 程序持續時間long duration = Math.min(System.currentTimeMillis() + timeOffset - start + 1000, 60000);// 多余的時間停留一下try {sleep(Math.max(60000 - duration, 1000));} catch (InterruptedException e) {Thread.currentThread().interrupt();}}}private int bisectRight(ArrayList<String> keyLists, String cutoff) {int index = Collections.binarySearch(keyLists, cutoff);return index > 0 ? index + 1 : Math.abs(index) - 1;}}private class AccessTimer {private Jedis conn;private long start;public AccessTimer(Jedis conn) {this.conn = conn;}public void setStart(long start) {this.start = start;}/*** 記錄停留時間-即類似執行時間*/public void stop(String context) {long delta = System.currentTimeMillis() - start;List<Object> stats = updateState(conn, context, "AccessTime", delta / 1000);double average = (Double) stats.get(1) / (Double) stats.get(0);Transaction trans = conn.multi();trans.zadd("slowest:AccessTime", average, context);trans.zremrangeByRank("slowest:AccessTime", 0, -101);trans.exec();}} }

總結

以上是生活随笔為你收集整理的Redis实战：第五章-使用Redis构建支持程序的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。