es集群可能存在的问题

  1. 集群索引每天日渐增长,索引数量越来越多,机器负载越来越高
  2. 很久以前的索引没有关闭,导致打开的索引太多,拖慢集群速度

解决方案

  1. 删除不需要的索引,提升es集群速度
  2. 关闭不需要经常查看的索引,一般关闭七天之前的索引,打开的索引会极大的占用es资源

curator部署(ubuntu)

官方文档

https://www.elastic.co/guide/en/elasticsearch/client/curator/current/apt-repository.html

配置源：

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/curator.list
deb [arch=amd64] https://packages.elastic.co/curator/5/debian stable main

安装

apt-get update && apt-get install elasticsearch-curator

创建目录

mkdir /etc/curator

创建curator系统配置

~]# cat /etc/curator/curator.yml
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
	- 10.10.110.28
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 30
  master_only: False

logging:
  loglevel: INFO
  logfile: /var/log/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

创建curator的action配置

actions:
  1: #清理索引操作,清理30以外的 applog-%Y.%m.%d 格式的索引
    action: delete_indices
    description: "Delete selected indices"
    options:
      disable_action: False
      timeout_override: 300
      continue_if_exception: True
      ignore_empty_list: True  # 如果目标列表为空,不加这句会报错No actionable items in list
    filters:
    - filtertype: pattern
      kind: regex
      value: '^.*-'
    - filtertype: pattern
      kind: prefix
      value: applog-
      exclude: True
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 30
  2: #清理索引操作, 清理所有索引,排除以applog-开头的索引
    action: delete_indices
    description: "Delete selected indices"
    options:
      disable_action: False
      timeout_override: 300
      continue_if_exception: True
      ignore_empty_list: True  # 如果目标列表为空,不加这句会报错No actionable items in list
    filters:
    - filtertype: pattern
      kind: prefix
      value: 'applog-'
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 180
  3:  # 关闭索引操作,关闭超过七天的所有索引
    action: close
    description: >-
      Close indices older than 7 days (based on index name), for .*-
      prefixed indices.
    options:
      ignore_empty_list: True
      delete_aliases: False
 #     disable_action: True  #代表这个action是否激活，True为不激活,默认为False
    filters:
    - filtertype: pattern
      kind: regex
      value: '^.*-'
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 7