免费推广的预期效果/优化大师官方
EFK收集nginx日志
- 在上一篇的基础上 【153端】操作
- 1.安装epel和ab压测
- 2.安装nginx并启动
- 3.验证端口
- 4.编辑filebeat配置文件
- 5. 启动filebeat
- 6.添加nginx配置文件【154端】
- 7.修改管道【化一为二】
- 8.编写正则匹配
- 9.重启/验证
- 10.【153端】
- 11.登录测试
在上一篇的基础上 【153端】操作
1.安装epel和ab压测
yum -y install epel-release httpd-tools
2.安装nginx并启动
yum -y install nginx
nginx
3.验证端口
ss -nlpt |grep 80
4.编辑filebeat配置文件
vim /etc/filebeat/filebeat.yml
内容如下:
filebeat.inputs:
- type: logenabled: truepaths:- /var/log/messagesfields:log_topics: msg007
###nginx##
- type: logenabled: truepaths:- /var/log/nginx/access.logfields:log_topics: nginx007output.kafka:enabled: truehosts: ["192.168.112.153:9092","192.168.112.154:9092","192.168.112.155:9092"]topic: '%{[fields][log_topics]}'
5. 启动filebeat
systemctl restart filebeat
6.添加nginx配置文件【154端】
vim /etc/logstash/conf.d/nginx.conf
内容如下:
input{kafka{bootstrap_servers => ["192.168.10.130:9092,192.168.10.131:9092,192.168.10.132:9092"]group_id => "logstash"topics => "nginx007"consumer_threads => 5}}
filter {json{source => "message"}mutate {remove_field => ["host","prospector","fields","input","log"]}grok {match => { "message" => "%{NGX}" }}}output{elasticsearch {hosts => "192.168.10.130:9200"index => "nginx-%{+YYYY.MM.dd}"}
}
7.修改管道【化一为二】
vim /etc/logstash/pipelines.yml
8.编写正则匹配
路径
vim /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/nginx
内容
NGX %{IPORHOST:client_ip} (%{USER:ident}|- ) (%{USER:auth}|-) \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} (%{NOTSPACE:request}|-)(?: HTTP/%{NUMBER:http_version})?|-)" %{NUMBER:status} (?:%{NUMBER:bytes}|-) "(?:%{URI:referrer}|-)" "%{GREEDYDATA:agent}"
9.重启/验证
systemctl restart logstash
ss -nlpt|grep 9600
10.【153端】
触发生成数据
chmod -R 777 /var/log/* 【加权限】ab -n 100 -c 100 http://1923168.112.153/index.html 【压测数据】
11.登录测试
返回上一层
