daeltar’s

Capistrano task to generate Logrotate config

Motivation: your logs are growing request by request and your free disk space is smaller and smaller. Solution is log rotation with logrotate.

Requirements:

• nginx + passenger (installed in /opt/nginx)
• logrotate (aptitude install logrotate)
• capistrano (deploying into /home/web/YOUR_APPLICATION_NAME)
• logs to rotate (production.log, access.log, error.log, newrelic_agent.log, …)

Capistrano part

Logrotate configuration will be generated from ERB template:

# logrotate.erb.conf
# Logrotate config for <%= application %>
# Generated at <%= Time.now.strftime("%d.%m.%Y, %H:%M") %>
<%= shared_path %>/log/*.log {
  daily
  missingok
  rotate 30
  compress
  delaycompress
  sharedscripts
  olddir <%= shared_path %>/log/old
  postrotate
    test ! -f /opt/nginx/logs/nginx.pid || kill -USR1 `cat /opt/nginx/logs/nginx.pid`
  endscript
}

Notes:

• old logs moved under old directory, compressed and kept for 30 days
• no slow copytruncate, after rotation of all logs in directory (sharedscripts) send USR1 signal to nginx to reopen logs

To generate logrotate configuration use following capistrano task:

# logrotate.rb Capistrano task
namespace :logrotate do
  desc "Create logrotate configuration file"
  task :create_conf, :roles => :web do
    template = File.read(File.join(File.dirname(__FILE__), "logrotate.conf.erb"))
    buffer   = ERB.new(template).result(binding)
    put buffer, "#{shared_path}/config/logrotate.conf"
  end
end
after 'setup', 'logrotate:create_conf'

Server part

To include generated configuration file add following to server’s /etc/logrotate.conf

include /home/web/YOUR_APPLICATION_NAME/shared/config/logrotate.conf

To ensure periodic rotation logrotate relies on daily execution by cron. In Debian it is placed in /etc/cron.daily/logrotate.

Resume

I have seen examples promoting virtual host generation from ERB template by capistrano tasks but why not apply it to generating logrotate configuration? You got it bundled (and versioned) with your application sources and that is big win.

Cron tasks with style: Whenever

Whenever is simple and very useful tool for defining cron jobs in ruby.

First things first, add whenever to your Gemfile (you use bundler, right?):

gem 'whenever', :require => false

Configuration is kept in config/schedule.rb. It uses plain ruby DSL. Periodically executed code is wrapper in every blocks, in which you specify your command or rake task to be run.

It is good idea to set output redirection and disable default cron behaviour of emailing standard output (otherwise you end up with thousands local emails).

set :output, '/home/web/fakturoid/shared/log/cron.log'
env :MAILTO, "''"

Basic example can be running rake task for cleaning stale database sessions:

every 2.hours do
  rake "db:sessions:clean_nonactive"
end

I prefer application related tasks to be grouped under app namespace:

every 1.day, :at => '0:30 am' do
  rake "app:mark_as_overdue"
  rake "app:invoices_from_generators"
end

Backups should be also scheduled to run nightly:

every 1.day, :at => '2:30 am' do
  REMOTE = "someaccount@someserver.com:backups"
  rake "backup BACKUP_REMOTE=#{REMOTE}"
end

As you can see, I use rake tasks to perform scheduled jobs. But if you have background job queue system in place, it would be wise to use cron only for placing task in queue.

To generate cron configuration from ruby DSL run whenever command (via bundle exec whenever). But real fun starts with capistrano integration to regenerate cron settings after deploy. All your cron tasks will run under user specified in capistrano application variable.

# whenever.rb capistrano task
namespace :deploy do
  desc "Update the crontab file"
  task :update_crontab, :roles => :db do
    run "cd #{current_path} && /opt/ruby-enterprise/bin/bundle exec whenever --update-crontab #{application}"
  end
end
after "deploy:restart", "deploy:update_crontab"

How to start with Rails

Lots of my friends are starting with Ruby on Rails development and all of them have similar questions. They often have previous PHP experience. This post is about directions and sources which helped me. You may also read my previous article about setting Mac for RoR development.

1) Ruby

You can learn Ruby (the language) as you go by learning primary Rails, it worked for me.

• Version: Use Ruby 1.8.7 for smoothest ride (for both RoR 2.3 and 3 as of mid 2010)
• Book: Programming Ruby 1.9
• Legendary Why’s (Poignant) Guide to Ruby – online book
• Ruby for Rails – book recommended by Vojto
• Standard Library Documentation

2) Rails

• Version: Go with Rails 3.
• Book: Agile Web Development with Rails, 4th Edition
• Guides: Rails guides – great, great, great learning source
• API doc: Rails API documentation
• Test driven development: If you are not used to writing tests for your code it is best to change that quickly.
• Railscasts – free screencasts (about 10 min each)
• PeepCode – paid screencasts (about 1h each)
• Ruby Toolbox – use existing plugins/gems before developing something on your own (like paperclip for image uploads)

3) Git and GitHub

Install git and get familiar with it.

• Book: Pro Git – free html book
• Screeencasts: GitCasts
• GitHub: Grab account, you will use GitHub a lot. Almost all Rails related code is hosted there (including plugins, gems and RoR itself).

4) Asking questions

• Stackoverflow – browse rails and ruby tags
• RailsForum

5) Czech sources

• RoR fórum – czech discussion board
• CSRUG – facebook group for czech RoR users
• #rubyonrails.cz – IRC channel

Bonus sources

• RubyFlow – community links
• Planet – rails related blog aggregator
• RubyShow – podcast with Dan Benjamin and Jason Seifer

Now go and develop some great Rails application. Remember, it is not about tools (Rails in this case), but about what you can do with them.

How to setup your Mac for RoR development

1) Developer tools – install Xcode

2) Package manager homebrew

We will install homebrew + some common packages (git, ImageMagick, PostgreSQL, …) into /usr/local and take ownership over directory. This way we will not need to use sudo and all binaries will be in our $PATH (because /usr/local/bin is by default).

sudo chown -R $USER /usr/local
ruby -e "$(curl -fsS http://gist.github.com/raw/323731/install_homebrew.rb)"
brew install wget git postgresql sqlite imagemagick

To initialize a database cluster for PostgreSQL use initdb /usr/local/var/postgres. (it will be owned by your user, again, it should not be problem)

3) Ruby version manager – rvm

RVM allows you to have multiple rubies (1.8.7, 1.9.2, jruby, ..) in parallel. It installs everything into your home directory, so again, you can forget about sudo and messing with $PATH variable. Run as normal user:

bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
echo 'if [[ -s "$HOME/.rvm/scripts/rvm" ]]  ; then source "$HOME/.rvm/scripts/rvm" ; fi' >> .bashrc

Open new tab in terminal and rvm should work (be sure your .bash_profile contains source $HOME/.bashrc).

Now you can install last ruby 1.8.7 and set it as default:

rvm install 1.8.7
rvm use 1.8.7 --default
gem install bundler rails capistrano unicorn

Additional ruby interprets can be installed such as:

rvm install 1.9.2
rvm use 1.9.2

4) Git GUI and text editor

• GitX
• TextMate

5) You are all set

I would suggest using bundler for managing your project gems (even on RoR 2.3.x). Than just cd PROJECT_PATH; bundle install and ejoy simple management of rubies and packages via rvm and homebrew.

Evolution of tools, spring edition

I got more into GTD. I started using Things.app for managing personal and most of work related task. In our Fakturoid startup we moved from Backpack to Basecamp for project management. It is great, but it is slow. I bought iPhone which is great gaming device, mobile web browser, organizer, calendar, map and you can call and text too :). It brings some crazy and useful concepts.

New HW

• That iPhone 3GS.
• Magic Mouse from Apple. 
• New external HDD box with old disk. Time Machine enabled.
• SSD in MBP - that is game-changer. Way faster, instant application opening, definitely worth money.

iPhone apps

• first screen: Ego, Things, Tweetie, Reeder, Articles
• second: Skype, Dropbox, Meebo, 1Password, TouchTerm, Foursquare
• games: GoldRush, Plants vs. Zombies, Gelex, Spider, Dungeon Hunter

What tools I use

Hardware

Mac Book Pro 15" as my main station. When home, paired with Apple Keyboard, Logitech bluetooth mouse and 24" HP ISP LCD panel.  It is connected via Wi-Fi or via ethernet to router and therefore to home backup server (whitebox with big drives). I have photo of my desk on deskography.

Software

Text editor, web browser and terminal application can be run on any system, on Mac it just make sense for me. (Only less known tools are linked.)

• Snow Leopard (on notebook) or Linux/Solaris (servers)
• Terminal (for SSH, tail -f, scrip/console and others)
• Finder (previews of images and PDFs and drag&drops to Terminal)
• TextMate (with great bundles for Ruby, Rails and Git)
• GitX (mainly for viewing)
• Spark (keyboard shortcuts for new Terminal or Finder window)
• Safari (for dev and normal use + reading RSS feeds)
• Chrome, Opera and Firefox for testing
• Communication
  • Mail.app (Gmail over IMAP)
  • Adium (jabber and icq)
  • Tweetie (great Cocoa twitter app)
  • Skype (mainly voice, there is no need for video)
• iCal.app (for local calendar and remote subscription to Backpack)
• iTunes (music, podcasts - Ruby5, RailsEnvy, TED talks)
• Dockdrop (drag file on dock icon and get it uploaded with URL in your clipboard ready to paste in IM or mail)
• Pixelmator 
• MindNote (for structural ideas)
• VirtualBox (with Windows 7 and various Linux distributions for testing)
• Cyberduck (for FTP needs only, because mostly I am using SCP from shell or git over SSH for file transfers)

Web based SW

I like simplicity of 37signals products (it is inspiration for my own apps).

• Backpack (general purpose tool to keep me organized)
• Campfire (trying Wave from Google)
• Fakturoid (I am user and also developer of this invoicing tool)
• GitHub (open source project hosting with style)
• Writeboard (for preparing blogposts) 
• small self-hosted apps (for private use)

Ideas for improvement

Bigger screen, faster drive (or SSD), more battery life, even more memory, even more connected SW with better import/export abilities.