1、yum源配置

[root@localhost ~]# cat > /etc/yum.repos.d/logstash.repo <<EOF [logstash-1.5] name=logstash repository for 1.5.x packages baseurl=http://packages.elasticsearch.org/logstash/1.5/centos gpgcheck=1 gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch enabled=1 EOF [root@localhost ~]# yum clean all[root@localhost ~]# yum makecache

2、安裝logstash

[root@localhost ~]# yum install logstash

3、yum安裝的目錄如下

[root@logstash]$ whereis logstash logstash: /etc/logstash /opt/logstash/bin/logstash.bat /opt/logstash/bin/logstash/opt/logstash/bin/logstash #執行文件 /etc/logstash/conf.d/ #配置文件目錄

4、java環境變量配置

因為logstash默認回去/usr/sbin/和/usr/bin/里找java，如果安裝java時沒有配置相關環境變量，則可在/usr/sbin/和/usr/bin/下做個軟連接即可。 [root@localhost nginx]# which java /usr/java/jdk1.8.0_60/bin/java [root@localhost bin]# ln -s /usr/java/jdk1.8.0_60/bin/java java

注：每個 logstash 過濾插件，都會有四個方法叫 add_tag, remove_tag, add_field 和remove_field。它們在插件過濾匹配成功時生效。

5、配置文件（具體配置方法和說明可查看本人博客《logstash配置語法》

[root@logstash]$ more /etc/logstash/conf.d/logstash-nginx.conf input {file {path => "/data/logs/nginx/*.log"start_position => beginning} } filter {if [path] =~ "access" {mutate { replace => { type => "apache_access" } }grok {match => { "message" => "%{IPORHOST:clientip} - %{USER:ident} \[%{HTTPDATE:timestamp}\] %{NUMBER:reqLen} \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{INT:status} %{NUMBER:respLen} %{NUMBER:duration} %{QS:referrer} %{QS:userAgent} %{QS:xforward} %{INT:conn}:%{INT:reqs}" }}date {match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]}} else if [path] =~ "error" {mutate { replace => { type => "apache_error" } }grok {match => { "message" => "(?<datetime>\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d) \[(?<errtype>\w+)\] \S+: \*\d+ (?<errmsg>[^,]+), (?<errinfo>.*)$" }}mutate {rename => [ "host", "fromhost" ]gsub => [ "errmsg", "too large body: \d+ bytes", "too large body" ]}if [errinfo]{ruby {code => "event.append(Hash[event['errinfo'].split(', ').map{|l| l.split(': ')}])"}}grok {match => { "request" => '"%{WORD:verb} %{URIPATH:urlpath}?(?: HTTP/%{NUMBER:httpversion})"' }patterns_dir => "/etc/logstash/patterns"remove_field => [ "message", "errinfo", "request" ]}} else {mutate { replace => { type => "random_logs" } }}mutate {convert => ["duration", "float", #將字段由str類型轉換成數值類型"status", "integer","respLen", "integer"]}geoip {source => "clientip"#add_tag => [ "geoip" ]database => "/etc/logstash/GeoLiteCity.dat" #本地ip庫}geoip {source => "client"#add_tag => [ "geoip" ]database => "/etc/logstash/GeoLiteCity.dat" #本地ip庫} } output {elasticsearch {host => ["10.173.17.71","10.172.198.108","10.170.237.100"] #數據以輪詢的方式傳輸到數組中的主機列表，充分發揮elasticsearch集群的作用protocol => "http"ndex => "uustore-nginx-log-%{+yyyy.MM}" #索引名稱，以日期進行命名會每天生產一個索引文件，當需要刪除時可按時間進行清理而不影響其他索引使用，但需要注意的是在kibana設置索引時就需要使用通配符“*”。}stdout {codec => rubydebug} }

6、時間戳（記錄每次日志已抽取的時間點和行數）

[root@logstash]$ locate sincedb /root/.sincedb_53b7f195d5f913db850de77bc552cec0

如果要重新抽取日志的話需要刪除時間戳（如果不是以service logstash start方式啟動服務的，則時間戳放在/root目錄）

[root@localhost ~]# rm -f /var/lib/logstash/.sincedb_e90b8ae60d1c692cb46b94ebbf869e32

注：不同方式安裝、不同方式啟動，.sincedb文件有可以在/root/目錄下也可以在/var/lib/logstash/目錄下，看個人情況，上面就是本人兩種不同啟動方式生成的.sincedb存放路徑

7、logstash有個插件geoip，提供公共的ip庫進行查詢

[root@localhost nginx]# cd /etc/logstash/[root@localhost nginx]# wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz[root@localhost nginx]# gunzip GeoLiteCity.dat.gz[root@localhost nginx]# cd conf.d/[root@localhost nginx]# /opt/logstash/bin/logstash -f logstash-nginx.conf

轉載于:https://www.cnblogs.com/dengtr/p/5288449.html

總結

以上是生活随笔為你收集整理的ELK日志管理之——logstash部署的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。