delayed_job (and other daemons) in a production environment

Posted by Michael Mon, 16 Aug 2010 02:10:00 GMT

The delayed_job plugin for Rails does a good job for pushing tasks that take some time to process into the background, so that your users (and your Rails processes) can do other things than to wait. It uses daemons to process the backgrounded tasks, so for your system to work correctly it is essential that those daemons are running. Thus you want to make sure that those daemons are getting started when the server boots and are restarted in case they die. Those points are not addressed by delayed_job.

The usual way to get processes started at boot time under Linux is to use an init.d script. But init.d scripts only address the boot process - if the daemon dies, it won't get restarted.

D.J. Bernstein's daemontools make it very simple to create system services which achieve both of the above points: starting your daemons at system boot time and restarting them in case they die. And the best feature: to create a new service, you don't even need to include any of the typical daemon features (such as backgrounding the process) into your program. So while delayed_job uses the daemons library to provide those features, we won't be needing those.

The following steps show how to set up a new service. This assumes that you already installed daemontools (available for Ubuntu/Debian for instance: apt-get install daemontools-run).

  1. Create a new directory. This is going to be the service directory.
  2. Create a shell script "run" in the service directory which runs your program. This can be as simple as exec /path/to/my/program.
  3. Create a symlink from the system service directory (for Ubuntu that'd be /etc/service), pointing to your new service directory.

That's it. To control your service, use the svc tool. See the manpage for more information.

In case of delayed_job, we are using the following run script:

#!/bin/sh
export RAILS_ENV=production
exec 2>/dev/null
exec setuidgid railsuser /srv/railsuser/project/current/script/delayed_job run

The script changes the user to "railsuser" (you don't want to run your delayed job processing under root; change it to match your setup), and then starts the usual delayed_job script, telling it to not put itself into the background.

One specialty to note is the handling of stderr. We redirect it to /dev/null to avoid potential "Broken pipe" exceptions in case something writes to stderr, which isn't available. Redirecting sdterr to stdout did not work.

Now, when updating, you will want to restart the delayed_job service. With capistrano we use the following task definitions:

namespace :delayed_job do
  desc "Start delayed job (if not running)"
  task :start, :roles => :app do
    sudo "svc -u /etc/service/#{application}_#{rails_env}_delayed_job"
  end
  desc "Stop delayed job"
  task :stop, :roles => :app do
    sudo "svc -d /etc/service/#{application}_#{rails_env}_delayed_job"
  end
  desc "Restart delayed job"
  task :restart, :roles => :app do
    sudo "svc -t /etc/service/#{application}_#{rails_env}_delayed_job"
  end
end

after "deploy:start", "delayed_job:start" 
after "deploy:stop", "delayed_job:stop" 
after "deploy:restart", "delayed_job:restart"

This requires that your deployment user will be able to run svc using sudo, so make sure to add this to your sudoers.

Also note that for the service names in the system service directory we use the pattern #{application}_#{rails_env}_delayed_job. Those are links to the service's directory, which are located under /srv/railsuser/project/services for our setup.

With this setup we have a pretty reliable delayed_job, and can use the same framework to run most (if not all) other services we might need with very little effort.

Enabling url parameter based sessions in Ruby on Rails

Posted by Paul McMahon Fri, 09 Apr 2010 04:33:00 GMT

Out of the box, Ruby on Rails uses cookies to store a user's session ID. This is fine for most applications, but doesn't work if your application needs to support browsers that don't support cookies, such as some mobile browsers. Instead of putting the session ID in a cookie, it must be put in the URL. This increases the possibility of session fixation attacks, where one user can take over another's session, however if security isn't paramount, this is an acceptable trade off. See the Ruby on Rails Security Guide for more details on these kinds of attacks.

To enable parameter based sessions in Rails, there are a number of changes you need to make. First, in config/initializers/session_store.rb, change the default file from containing something like

ActionController::Base.session = {
  :key         => '_application_session',
  :secret      => 'secret'
}

to

ActionController::Base.session = {
  :key         => '_application_session',
  :secret      => 'secret',
  :cookie_only => false # allow session to be loaded from params
}

# Overwrite default cookie based store
ActionController::Base.session_store = :active_record_store

Now Rails can read the session ID from the URL parameters, and doesn't store the session in a cookie (the default behaviour). Besides the ActiveRecord session store, other server based stores, such as the memcache store, work as well.

In addition to enabling Rails to read the session ID from the URL parameters, the session ID must be added as a parameter. The most basic way of doing this is to define default_url_options in the ApplicationController:

def default_url_options(options = nil)
  { request.session_options[:key] => request.session_options[:id] }
end

This will ensure that the session ID is always set in the URL.

The session ID can also be added conditionally, by doing something like the following:

def default_url_options(options = nil)
  if cookies_supported?
    super
  else
    { request.session_options[:key] => request.session_options[:id] }
  end
end

If the session ID is not included in the parameters, it will fall back to the cookie.