es集群可能存在的问题
- 集群索引每天日渐增长,索引数量越来越多,机器负载越来越高
- 很久以前的索引没有关闭,导致打开的索引太多,拖慢集群速度
解决方案
- 删除不需要的索引,提升es集群速度
- 关闭不需要经常查看的索引,一般关闭七天之前的索引,打开的索引会极大的占用es资源
curator部署(ubuntu)
官方文档
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/apt-repository.html
配置源:
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/curator.list
deb [arch=amd64] https://packages.elastic.co/curator/5/debian stable main
安装
apt-get update && apt-get install elasticsearch-curator
创建目录
mkdir /etc/curator
创建curator系统配置
~]# cat /etc/curator/curator.yml
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
client:
hosts:
- 10.10.110.28
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']
创建curator的action配置
actions:
1: #清理索引操作,清理30以外的 applog-%Y.%m.%d 格式的索引
action: delete_indices
description: "Delete selected indices"
options:
disable_action: False
timeout_override: 300
continue_if_exception: True
ignore_empty_list: True # 如果目标列表为空,不加这句会报错No actionable items in list
filters:
- filtertype: pattern
kind: regex
value: '^.*-'
- filtertype: pattern
kind: prefix
value: applog-
exclude: True
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 30
2: #清理索引操作, 清理所有索引,排除以applog-开头的索引
action: delete_indices
description: "Delete selected indices"
options:
disable_action: False
timeout_override: 300
continue_if_exception: True
ignore_empty_list: True # 如果目标列表为空,不加这句会报错No actionable items in list
filters:
- filtertype: pattern
kind: prefix
value: 'applog-'
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 180
3: # 关闭索引操作,关闭超过七天的所有索引
action: close
description: >-
Close indices older than 7 days (based on index name), for .*-
prefixed indices.
options:
ignore_empty_list: True
delete_aliases: False
# disable_action: True #代表这个action是否激活,True为不激活,默认为False
filters:
- filtertype: pattern
kind: regex
value: '^.*-'
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 7
...