注意：修改配置后建議重新創(chuàng)建index

1、nginx 日志文件格式

1 2 3

log_format?elk?"$http_clientip?|?$http_x_forwarded_for?|?$time_local?|?$request?|?$status?|?$body_bytes_sent?|?" ?"$request_body?|?$content_length?|?$http_referer?|?$http_user_agent?|?" ?"$http_cookie?|?$remote_addr?|?$hostname?|?$upstream_addr?|?$upstream_response_time?|?$request_time";

2、logstash nginx 服務器上的配置文件 agent.conf

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

input?{?? ????????file?{?? ????????????????type?=>?"elk_frontend_access"?? ????????????????path?=>?["/data/logs/flight1-access_log"]?? ????????} }?? filter?{ ruby?{ init?=>?"@kname?=?['http_clientip','http_x_forwarded_for','time_local','request','status','body_bytes_sent','request_body','content_length','http_referer','http_user_agent','http_cookie','remote_addr','hostname','upstream_addr','upstream_response_time','request_time']" code?=>?"new_event?=?LogStash::Event.new(Hash[@kname.zip(event.get('message').split('|'))]) new_event.remove('@timestamp') event.append(new_event)" } if?[request]?{ ruby?{ init?=>?"@kname?=?['method','uri','verb']" code?=>?"new_event?=?LogStash::Event.new(Hash[@kname.zip(event.get('request').split('?'))]) new_event.remove('@timestamp') event.append(new_event) " } if?[uri]?{ ruby?{ init?=>?"@kname?=?['url_path','url_args']" code?=>?"new_event?=?LogStash::Event.new(Hash[@kname.zip(event.get('uri').split('?'))]) new_event.remove('@timestamp') event.append(new_event) " } kv?{ prefix?=>?"url_" source?=>?"url_args" field_split?=>?"&?" remove_field?=>?[?"url_args","uri","request"?] } } } mutate?{ convert?=>?["body_bytes_sent"?,?"integer",?"content_length",?"integer",?"upstream_response_time",?"float","request_time",?"float"] } date?{ match?=>?[?"time_local",?"dd/MMM/yyyy:hh:mm:ss?Z"?] locale?=>?"en" } ????????grok?{ ????????match?=>?{?"message"?=>?"%{IP:clientip}"?} ??} ????????geoip? { ????????source?=>?"clientip" ????????} } output?{ ????????redis?{?? ????????????????host?=>?"10.10.45.200"?? ????????????????data_type?=>?"list"?? ????????????????key?=>?"elk_frontend_access:redis"?? ????????????????port=>"5379"?? ????????}?? }

3、logstash elk服務器上的配置文件server.conf

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

input?{?? ????????redis?{?? ????????????????host?=>?"10.10.45.200"?? ????????????????data_type?=>?"list"?? ????????????????key?=>?"elk_frontend_access:redis"?? ????????????????port?=>"5379"?? ????????}?? }?? output?{?? ????????elasticsearch?{?? ????????????????hosts?=>?"10.10.45.200:8200"?? ????????????????index?=>?"logstash-zjzc-frontend-%{+YYYY.MM.dd}"?? ????????}?? ????????stdout?{?? ????????????codec?=>?rubydebug?? ????????}?? }

注意：如果修改后沒有生效，在kibana上重建索引。

? ? ?本文轉自1321385590 51CTO博客，原文鏈接：http://blog.51cto.com/linux10000/1922391，如需轉載請自行聯(lián)系原作者

總結

以上是生活随笔為你收集整理的ELK 分析 nginx access 日志的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。