daeltar's

Capistrano task to generate Logrotate config

Motivation: your logs are growing request by request and your free disk space is smaller and smaller. Solution is log rotation with logrotate.

Requirements:

• nginx + passenger (installed in /opt/nginx)
• logrotate (aptitude install logrotate)
• capistrano (deploying into /home/web/YOUR_APPLICATION_NAME)
• logs to rotate (production.log, access.log, error.log, newrelic_agent.log, …)

Capistrano part

Logrotate configuration will be generated from ERB template:

# logrotate.erb.conf
# Logrotate config for <%= application %>
# Generated at <%= Time.now.strftime("%d.%m.%Y, %H:%M") %>
<%= shared_path %>/log/*.log {
  daily
  missingok
  rotate 30
  compress
  delaycompress
  sharedscripts
  olddir <%= shared_path %>/log/old
  postrotate
    test ! -f /opt/nginx/logs/nginx.pid || kill -USR1 `cat /opt/nginx/logs/nginx.pid`
  endscript
}

Notes:

• old logs moved under old directory, compressed and kept for 30 days
• no slow copytruncate, after rotation of all logs in directory (sharedscripts) send USR1 signal to nginx to reopen logs

To generate logrotate configuration use following capistrano task:

# logrotate.rb Capistrano task
namespace :logrotate do
  desc "Create logrotate configuration file"
  task :create_conf, :roles => :web do
    template = File.read(File.join(File.dirname(__FILE__), "logrotate.conf.erb"))
    buffer   = ERB.new(template).result(binding)
    put buffer, "#{shared_path}/config/logrotate.conf"
  end
end
after 'setup', 'logrotate:create_conf'

Server part

To include generated configuration file add following to server’s /etc/logrotate.conf

include /home/web/YOUR_APPLICATION_NAME/shared/config/logrotate.conf

To ensure periodic rotation logrotate relies on daily execution by cron. In Debian it is placed in /etc/cron.daily/logrotate.

Resume

I have seen examples promoting virtual host generation from ERB template by capistrano tasks but why not apply it to generating logrotate configuration? You got it bundled (and versioned) with your application sources and that is big win.

Cron tasks with style: Whenever

Whenever is simple and very useful tool for defining cron jobs in ruby.

First things first, add whenever to your Gemfile (you use bundler, right?):

gem 'whenever', :require => false

Configuration is kept in config/schedule.rb. It uses plain ruby DSL. Periodically executed code is wrapper in every blocks, in which you specify your command or rake task to be run.

It is good idea to set output redirection and disable default cron behaviour of emailing standard output (otherwise you end up with thousands local emails).

set :output, '/home/web/fakturoid/shared/log/cron.log'
env :MAILTO, "''"

Basic example can be running rake task for cleaning stale database sessions:

every 2.hours do
  rake "db:sessions:clean_nonactive"
end

I prefer application related tasks to be grouped under app namespace:

every 1.day, :at => '0:30 am' do
  rake "app:mark_as_overdue"
  rake "app:invoices_from_generators"
end

Backups should be also scheduled to run nightly:

every 1.day, :at => '2:30 am' do
  REMOTE = "someaccount@someserver.com:backups"
  rake "backup BACKUP_REMOTE=#{REMOTE}"
end

As you can see, I use rake tasks to perform scheduled jobs. But if you have background job queue system in place, it would be wise to use cron only for placing task in queue.

To generate cron configuration from ruby DSL run whenever command (via bundle exec whenever). But real fun starts with capistrano integration to regenerate cron settings after deploy. All your cron tasks will run under user specified in capistrano application variable.

# whenever.rb capistrano task
namespace :deploy do
  desc "Update the crontab file"
  task :update_crontab, :roles => :db do
    run "cd #{current_path} && /opt/ruby-enterprise/bin/bundle exec whenever --update-crontab #{application}"
  end
end
after "deploy:restart", "deploy:update_crontab"