如何通过ELK系统监控防范Web恶意攻击

背景

上一篇文章利用ELK系统分析Nginx日志并对数据进行展示简单的写了下ELK展示分析Nginx日志，但如何对日志进行监控？如果有人攻击我们，如何在第一时间知晓，本文将介绍如何通过ELK系统监控Web攻击，直接上教程：

安装ElastAlert

一、下载安装ElastAlert

安装 epel 库
yum -y install epel-release wget
安装 git
yum -y install gcc git
安装 python3.6
yum -y install python36 python36-devel
配置 py3 虚拟环境
python3.6 -m venv /opt/py3
source /opt/py3/bin/activate
cd /opt/
下载ElastAlert
git clone https://github.com/Yelp/elastalert.git
cd elastalert
python setup.py install
pip install -r requirements.txt
cp config.yaml.example config.yaml

二、配置config.yml

rules_folder: rules
run_every:  ##用来设置定时向elasticsearch发送请求
  minutes: 1
buffer_time: ##用来设置请求里时间字段的范围，默认是15分钟
  minutes: 15
es_host: 192.168.21.158 ##elasticsearch的host地址
es_port: 9200 ##elasticsearch 对应的端口号
writeback_index: elastalert_status  ##elastalert产生的日志在elasticsearch中的创建的索引
writeback_alias: elastalert_alerts
alert_time_limit:  ##失败重试的时间限制
  days: 2

配置完成后运行

elastalert-create-index

三、配置告警规则

新建rules目录，并在web目录中新建yaml配置文件webattack.yml

name: web attack
realert:
  minutes: 5
type: frequency
num_events: 19
index: logstash-nginx*  #对应logstash的配置文件中output elasticsearch index前缀
timeframe:
  minutes: 1
​
​filter:
- query_string:   ##告警规则，通过查询ES进行匹配
# sql insert  xss detect
        query: "request: select.+(from|limit) OR request: union(.*?)select OR request: into.+(dump|out)file OR
        request: (base64_decode|sleep|benchmark|and.+1=1|and.+1=2|or%20|exec|information_schema|where%20|union%20|%2ctable_name%20|cmdshell|table_schema) OR
        request: (iframe|script|body|img|layer|div|meta|style|base|object|input|onmouseover|onerror|onload) OR
        request: .+etc.+passwd OR http_user_agent：(HTTrack|harvest|audit|dirbuster|pangolin|nmap|sqln|-scan|hydra|Parser|libwww|BBBike|sqlmap|w3af|owasp|Nikto|fimap|havij|PycURL|zmeu|BabyKrokodil|netsparker|httperf|bench) OR
        status: (400|404|500|501)
        NOT (request:_health.html OR remote_addr:222.222.222.222  )
        "
#配置告警
smtp_host: smtp.qiye.163.com
smtp_port: 25
user: xxx@163.com
password: password
#回复给那个邮箱
email_reply_to: xxx@163.com
#从哪个邮箱发送
from_addr: xxx@163.com
alert:
- "email"
email:
- "xx@163.com"
alert_text: |
  你好，服务器({})可能正在受到web攻击，请采取手段阻止！！！！
  ### 截止发邮件前匹配到的请求数：{}
  > 发生时间: {}
  > timestamp:{}
  > attacker's ip: {}
  > request: {}
  > status:{}
  > UA头:{}
  >>> 参考来源：{}
alert_text_args:
  - host
  - num_hits
  - time
  - "@timestamp"
  - client_ip
  - url
  - status
  - http_user_agent
  - source

启动elastalert

nohup python -m elastalert.elastalert --verbose --rule rules/webattack.yaml >/dev/null 2>&1 &

运行效果

当匹配到自定义攻击规则的时候，ElastAlert将会以邮件方式发送告警信息：

web attack may be by 104.38.xx.xx at @[xx/xx/2020:16:06:58 +0800]
xxx 发给 xx   
你好，服务器(xx.xx.xx.xx)可能正在受到web攻击，请采取手段阻止！！！！
### 截止发邮件前匹配到的请求数：20
> 发生时间: [xx/xx/2020:16:06:58 +0800]
> timestamp:2018-01-13T08:07:04.930Z
> attacker's ip: 184.233.9.121
> request: GET /dbadmin/scripts/setup.php?id=1'and 1=1 HTTP/1.0
> status:200
> UA头:ZmEu
>>> 参考来源：/var/log/nginx/access.log