當前位置：首頁 >

使用ELK(Elasticsearch + Logstash + Kibana) 搭建日志集中分析平台实践--转载

發布時間：2025/4/5 42 豆豆

生活随笔收集整理的這篇文章主要介紹了使用ELK(Elasticsearch + Logstash + Kibana) 搭建日志集中分析平台实践--转载小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

原文地址：https://wsgzao.github.io/post/elk/

另外可以參考：https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04

前言

Elasticsearch + Logstash + Kibana（ELK）是一套開源的日志管理方案，分析網站的訪問情況時我們一般會借助Google/百度/CNZZ等方式嵌入JS做數據統計，但是當網站訪問異?；蛘弑还魰r我們需要在后臺分析如Nginx的具體日志，而Nginx日志分割/GoAccess/Awstats都是相對簡單的單節點解決方案，針對分布式集群或者數據量級較大時會顯得心有余而力不足，而ELK的出現可以使我們從容面對新的挑戰。

Logstash：負責日志的收集，處理和儲存
Elasticsearch：負責日志檢索和分析
Kibana：負責日志的可視化

ELK(Elasticsearch + Logstash + Kibana)

更新記錄

2015年08月31日 - 初稿

閱讀原文 -?http://wsgzao.github.io/post/elk/

擴展閱讀

CentOS 7.x安裝ELK(Elasticsearch+Logstash+Kibana) -?http://www.chenshake.com/centos-install-7-x-elk-elasticsearchlogstashkibana/
Centos 6.5 安裝nginx日志分析系統 elasticsearch + logstash + redis + kibana -?http://blog.chinaunix.net/xmlrpc.php?r=blog/article&uid=17291169&id=4898582
logstash-forwarder and grok examples -?https://www.ulyaoth.net/threads/logstash-forwarder-and-grok-examples.32413/
三斗室 -?http://chenlinux.com/
elastic -?https://www.elastic.co/guide
LTMP索引 -?http://wsgzao.github.io/index/#LTMP

組件預覽

JDK -?http://www.oracle.com/technetwork/java/javase/downloads/index.html
Elasticsearch -?https://www.elastic.co/downloads/elasticsearch
Logstash -?https://www.elastic.co/downloads/logstash
Kibana -?https://www.elastic.co/downloads/kibana
redis -?http://redis.io/download

設置FQDN

創建SSL證書的時候需要配置FQDN

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#修改hostname
cat /etc/hostname
elk

#修改hosts
cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

127.0.0.1 10-10-87-19
10.10.87.19 elk.ooxx.com elk

#刷新環境
hostname -F /etc/hostname

#復查結果
hostname -f
elk.ooxx.com

hostname
elk

服務端

Java

1
2
3
4
5
6
7
8
9

cat /etc/redhat-release
CentOS release 6.5 (Final)

yum install java-1.7.0-openjdk
java -version

java version "1.7.0_85"
OpenJDK Runtime Environment (rhel-2.6.1.3.el6_6-x86_64 u85-b01)
OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

Elasticsearch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#下載安裝
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.noarch.rpm
yum localinstall elasticsearch-1.7.1.noarch.rpm

#啟動相關服務
service elasticsearch start
service elasticsearch status

#查看Elasticsearch的配置文件
rpm -qc elasticsearch

/etc/elasticsearch/elasticsearch.yml
/etc/elasticsearch/logging.yml
/etc/init.d/elasticsearch
/etc/sysconfig/elasticsearch
/usr/lib/sysctl.d/elasticsearch.conf
/usr/lib/systemd/system/elasticsearch.service
/usr/lib/tmpfiles.d/elasticsearch.conf

#查看端口使用情況
netstat -nltp

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:9200 0.0.0.0:* LISTEN 1765/java
tcp 0 0 0.0.0.0:9300 0.0.0.0:* LISTEN 1765/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1509/sshd
tcp 0 0 :::22 :::* LISTEN 1509/sshd

#測試訪問
curl -X GET http://localhost:9200/

Kibana

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

#下載tar包
wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz
#解壓
tar zxf kibana-4.1.1-linux-x64.tar.gz -C /usr/local/
cd /usr/local/
mv kibana-4.1.1-linux-x64 kibana

#創建kibana服務
vi /etc/rc.d/init.d/kibana

#!/bin/bash
### BEGIN INIT INFO
# Provides: kibana
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Runs kibana daemon
# Description: Runs the kibana daemon as a non-root user
### END INIT INFO

# Process name
NAME=kibana
DESC="Kibana4"
PROG="/etc/init.d/kibana"

# Configure location of Kibana bin
KIBANA_BIN=/usr/local/kibana/bin

# PID Info
PID_FOLDER=/var/run/kibana/
PID_FILE=/var/run/kibana/$NAME.pid
LOCK_FILE=/var/lock/subsys/$NAME
PATH=/bin:/usr/bin:/sbin:/usr/sbin:$KIBANA_BIN
DAEMON=$KIBANA_BIN/$NAME

# Configure User to run daemon process
DAEMON_USER=root
# Configure logging location
KIBANA_LOG=/var/log/kibana.log

# Begin Script
RETVAL=0

if [ `id -u` -ne 0 ]; then
echo "You need root privileges to run this script"
exit 1
fi

# Function library
. /etc/init.d/functions

start() {
echo -n "Starting $DESC : "

pid=`pidofproc -p $PID_FILE kibana`
if [ -n "$pid" ] ; then
echo "Already running."
exit 0
else
# Start Daemon
if [ ! -d "$PID_FOLDER" ] ; then
mkdir $PID_FOLDER
fi
daemon --user=$DAEMON_USER --pidfile=$PID_FILE $DAEMON 1>"$KIBANA_LOG" 2>&1 &
sleep 2
pidofproc node > $PID_FILE
RETVAL=$?
[[ $? -eq 0 ]] && success || failure
echo
[ $RETVAL = 0 ] && touch $LOCK_FILE
return $RETVAL
fi
}

reload()
{
echo "Reload command is not implemented for this service."
return $RETVAL
}

stop() {
echo -n "Stopping $DESC : "
killproc -p $PID_FILE $DAEMON
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $PID_FILE $LOCK_FILE
}

case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $PID_FILE $DAEMON
RETVAL=$?
;;
restart)
stop
start
;;
reload)
reload
;;
*)
# Invalid Arguments, print the following message.
echo "Usage: $0 {start|stop|status|restart}" >&2
exit 2
;;
esac

#修改啟動權限
chmod +x /etc/rc.d/init.d/kibana

#啟動kibana服務
service kibana start
service kibana status

#查看端口

轉載于:https://www.cnblogs.com/davidwang456/p/5238973.html