It should be easy to install an ElasticSearch plugin with Chef, right? It is…​ but I couldn’t find a simple template anywhere. The "logstash" cookbook itself, which we weren’t using, was the inspiration for this quick recipe. The purpose of installing Curator is so we could purge old indices from our structured logging setup (ELK.) This let’s us use smaller machines to save money and keeps the Kibana UI responding quickly.

metadata.rb

depends 'python'
depends 'cron'

recipes/default.rb

include_recipe 'mycookbook::curator'

recipes/curator.rb

include_recipe 'python'

python_pip 'elasticsearch-curator'

cmd = [
  '/usr/local/bin/curator',
  '--host',
  node['mycookbook']['curator']['host'],
  'delete indices',
  '--time-unit',
  node['mycookbook']['curator']['time_unit'],
  '--older-than',
  node['mycookbook']['curator']['older_than'],
  '--timestring',
  node['mycookbook']['curator']['timestring'],
  '&>>',
  node['mycookbook']['curator']['logfile']
]

cron "curator-delete" do

    # e.g. /usr/local/bin/curator --host 10.0.0.1 delete indices --time-unit days --older-than 60 --timestring %Y.%m.%d &>> /tmp/curator-delete.log
    command cmd.join(' ')
    minute  node['mycookbook']['curator']['cron_minute']
    hour    node['mycookbook']['curator']['cron_hour']

end

attributes/default.rb

default['mycookbook']['curator']['time_unit']    = 'days'
default['mycookbook']['curator']['older_than']   = 60
default['mycookbook']['curator']['timestring']   = '%Y.%m.%d'
default['mycookbook']['curator']['logfile']      = '/tmp/curator-delete.log'
default['mycookbook']['curator']['host']         = node['ipaddress']
# stagger the hours and minutes by server
default['mycookbook']['curator']['cron_minute']  = node['ipaddress'].gsub('.','').to_i % 60
default['mycookbook']['curator']['cron_hour']    = node['ipaddress'].gsub('.','').to_i % 24

I went a bit wild using attributes for everything but it’s frustrating when you’re reusing someone’s cookbook and they haven’t left room for extension. Also, be sure to use the host IP where ElasticSearch is listening. There may be much better ways to achieve the "trick" for staggering the cron timing :-)

That’s all!