Last night I presented at Cookpad’s Tokyo Rails event. This installment had the largest turn out I’ve seen so far, with at least forty participants. I think word’s gotten out that Tokyo Rails has the best food of any Ruby group in the city (and great people too)! This time we had some great home cooked Korean food.

My presentation was about converting Rails plugins to gems. Fortuitously, there was an announcement last week that vendor/plugins will be deprecated, and gem plugins would become the standard way of building a Rails plugin. So the timing of my presentation worked out great.

Unfortunately, I don’t think my slides showed up so well on the projector. I’ve attached them below, so if anyone is interested, please have a look at them.

For a class I'm organizing on metaprograming in Ruby, we’re using Metaprograming Ruby as a textbook. In the chapter Tuesday: Methods, the book introduces two techniques for removing duplication: dynamically defining methods and using method_missing. It gives examples of applying both techniques to some sample code of which a simplified version is reproduced below:

# initial code
class Computer
  def initialize(data_source)
    @data_source = data_source
  end

  def mouse
    puts "Price: #{data_source.mouse_price}"
  end

  def keyboard
    puts "Price: #{data_source.mouse_price}"
  end

  def monitor
    puts "Price: #{data_source.mouse_price}"
  end
  
  #...
end

# dynamic methods
class Computer
  def initialize(data_source)
    @data_source.methods.grep(/^(.*)_price$/) { Computer.define_component $1 }
  end
  
  def self.define_component(name)
    define_method(name) do
      puts "Price: #{data_source.send("#{name}_price")
    end
  end
end

# method_missing
class Computer
  instance_methods.each do |m|
    undef_method m unless m.to_s =~ /^__|method_missing|respond_to?/
  end

  def method_missing(name, *args)
    super if !respond_to?(name)
    puts "Price: #{data_source.send("#{name}_price")
  end

  def respond_to?(method)
    @data_source.respond_to?("#{name}_price")
  end
end

However, the book doesn't discuss when you should use one technique versus another. Here’s my rule: only use method_missing when it is infeasible to use dynamic methods.

One good example of using method_missing is ActiveRecord's find_by method. With it, you can call a method like find_by_name_and_birthdate. While ActiveRecord could theoretically use the dynamic method technique, generating every possible method would be overkill, thus using method_missing makes sense.

One counter example can be found in an example in the Metaprograming Ruby book itself:

class Roulette
  def method_missing(name, *args)
    person = name.to_s.capitalize 
    super unless %w[Bob Frank Bill].include? person
    number = 0
    3.times do
      number = rand(10) + 1
      puts "#{number}..."
    end
    "#{person} got a #{number}"
  end
end

In this case, we have three methods we want Roulette to respond to: bob, frank, and bill. As we know these methods in advance, you should use the dynamic_method strategy:

class Roulette
  %w[bob frank bill].each do |name|
    define_method name do
      number = 0
      3.times do
        number = rand(10) + 1
        puts "#{number}..."
      end
      "#{name.to_s.capitalize} got a #{number}"
    end
  end
end

Now back to our initial example with Computer, which technique should we apply? Given that we can apply the dynamic methods technique, we should use it. Oh, and assuming we are always passed an instance of DataStore, I'd refactor it like so:

# dynamic methods
class Computer
  DataSource.methods.grep(/^(.*)_price$/).each do |name|
    define_method(name) do
      puts "Price: #{data_source.send("#{name}_price")
    end
  end
end

For our Japanese invoicing solution, 請求書.jp, we record the initial HTTP referrer for each user who signs up to our service. Search engines have standardize on the q parameter to represent a search query, such as 請求書テンプレート, so we use this parameter to guess what query a user signs up by. Unfortunately, search engines have not standardized on an encoding to use, and the query parameter can be encoded in any one of UTF-8, EUC-JP, or Shift_JIS. To work around this, we use the following Ruby 1.9 code:

This code allows us to try each of the three expected encodings, and then encode the result into UTF-8 for display within our admin interface.

For 請求書.jp, we wanted to provide our users with a way to generate PDF versions of invoices. We used the PDFKit gem (https://github.com/jdpace/PDFKit), which by itself is a thin wrapper around wkhtmltopdf (http://code.google.com/p/wkhtmltopdf/), to easily generate PDFs. With this tool chain it is possible to generate PDFs for any page, using the HTML and CSS you already have in place.

Installation

Installation is quite simple:

1. add PDFKit to your Gemfile, install the wkhtmltopdf-binary gem
2. add the following to your config/application.rb (Rails 3) to hook PDFKit into Rack:
   

   config.middleware.use PDFKit::Middleware
   

3. add an initializer (config/initializers/pdfkit.rb) like this:
   

   PDFKit.configure do |config|
     config.default_options = { page_size: 'A4', print_media_type: true }
   end
   

Basically that’s it, but please see https://github.com/jdpace/PDFKit for more details. Anyway, pretty easy, no?

Getting it to work on Heroku

Well, the tricky part starts when you want to deploy this to Heroku.

First, you’ll need to include a statically linked 64bit Linux version of wkhtmltopdf in your application. You can download it from http://code.google.com/p/wkhtmltopdf/downloads/list (wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2 for instance). To use the same binaries in development and production, we removed the wkhtmltopdf-binary gem again, downloaded the binaries for Linux and Mac so all our development environments are covered, and extended config/initializers/pdfkit.rb like this:

PDFKit.configure do |config|
  config.default_options = { page_size: 'A4', print_media_type: true }
  if RUBY_PLATFORM =~ /linux/
    wkhtmltopdf_executable = 'wkhtmltopdf-amd64'
  elsif RUBY_PLATFORM =~ /darwin/
    wkhtmltopdf_executable = 'wkhtmltopdf-osx'
  else
    raise "Unsupported. Must be running linux or intel-based Mac OS."
  end
  config.wkhtmltopdf = Rails.root.join('vendor', 'bin', wkhtmltopdf_executable).to_s
end

After deploying to Heroku, the second issue you are likely to notice is the following: Requesting a PDF page just sits there for a while and then returns an application error. That is because you are most likely using external stylesheets and wkhtmltopdf will request them in order to generate the PDF. But your Dyno is currently busy with generating the PDF, so it can’t respond to the incoming request regarding the stylesheets.

Unfortunately starting a second Dyno or using unicorn on the cedar stack doesn’t really help here, because Heroku’s router doesn’t know which Dynos are busy but rather distributes the workload according to different parameters. So basically doing a request from within your application to your application won’t work reliably. Putting a cache / CDN (like AWS’ cloudfront) in front of your assets mitigates the problem, but users might still see an application error because the cache / CDN needs to request those files from time to time. Serving all your assets altogether from S3 works of course, but makes deployment to Heroku harder.

The solution we employed in the end was to avoid the additional requests entirely and instead embed the printing styles into the HTML itself, using the following code in the layout (HAML):

!!!
%html{:lang => "ja"}
  %head
    %title= 請求書.jp
    - if request_from_pdfkit?
      %style{type: "text/css"}
        = File.read(Rails.root.join("public","stylesheets","print.css"))
    - else
      = javascript_include_tag 'application'
      = stylesheet_link_tag ‘application’

And in the application_helper.rb:

  def request_from_pdfkit?
    # when generating a PDF, PDFKit::Middleware will set this flag
    request.env["Rack-Middleware-PDFKit"] == "true"
  end

The downside of this solution is though that we need to have a static print.css in the public/stylesheets directory and can’t use Rails 3.1’s asset pipeline as before.

The third challenge we had to address was caused by using non-Latin script, Japanese in our case. To have Japanese script in the PDF, you need Japanese fonts installed on the server doing the PDF generation, so it can embed the font. There are no Japanese fonts installed on Heroku though. Fortunately we could come up with a way to install the required font within the Heroku environment. For that to work, we added the font to vendor/fonts in the Rails project and added the following initializer:

if Rails.env.production?
  font_dir = File.join(Dir.home, ".fonts")
  Dir.mkdir(font_dir) unless Dir.exists?(font_dir)

  Dir.glob(Rails.root.join("vendor", "fonts", "*")).each do |font|
    target = File.join(font_dir, File.basename(font))
    File.symlink(font, target) unless File.exists?(target)
  end
end

With this setup we can now reliably generate nice looking PDFs for our users.

請求書.jp allows Japanese freelancers and small businesses to easily create invoices. We’ve built it using Rails 3.0 in conjunction with Coffee Script and Sass, and host the application on Heroku. Although CoffeeScript and Sass have made developing the service easier, getting them setup on Heroku is a bit of a hassle. However, Rails 3.1 introduces the asset pipeline, which not only makes it easier to use CoffeeScript and Sass, but also handles the packaging of these resources into a single file for increased performance.

Although Rails 3.1 has not been officially released yet, it is out of beta and into the fourth release candidate. Given how attractive the asset pipeline was, I decided to give upgrading 請求書.jp a shot.

The biggest challenge was to find information about how the asset pipeline works. I was able to come across a couple articles and a presentation DHH gave, but overall the information was sparse on how it actually worked. The best success I had in understanding how everything worked was to generate a new rails project with 3.1 and then use the scaffold command to generate a simple resource. By studying the generated code, I was able to figure out how to convert our application. The following is a summary of the asset pipeline specific changes I made.

# Gemfile
gem 'rails', '3.1.0.rc4'
gem 'sprockets', '= 2.0.0.beta.10' # rails 3.1.0.rc4 compatible
gem 'sass-rails', "~> 3.1.0.rc"
gem 'coffee-script'
gem 'uglifier'

# app/assets/javascripts/application.js
//= require jquery
//= require jquery_ujs
//= require ../../../vendor/assets/javascripts/externals
//= require_tree .

# vendor/assets/javascripts/externals.js:
//= require ./jshashtable-2.1.js
//= require ./jquery.numberformatter-1.2.2.min.js

# app/assets/javascripts/invoices.coffee
// Invoicing specific CoffeeScript that is dependent on jquery.numberformatter

# app/assets/stylesheets/application.css
/*
 *= require_self
 *= require_tree ./web
 */

# app/assets/stylesheets/_compatibility.css.sass
/* Macros for browser compatibility */

# app/assets/stylesheets/web/*.css.sass
@import compatibility
/* Styling of various elements */

# app/assets/stylesheets/print.css.sass
/* Print specific CSS */

# config/application.rb
config.assets.enabled = true

# config/environments/production.rb
config.assets.compress = true
config.assets.js_compressor = :uglifier

After getting everything migrated to the asset pipeline, the rest of the upgrade involved identifying outstanding issues in Rails 3.1. These issues appeared in places where I was doing something that was a bit unusual, such as accessing a relation in an after_initialize block or storing an unsaved ActiveRecord object to the session. Rather than delving into the internals of Rails, I’ve worked around these issues.

After resolving these issues, we deployed it to Heroku. There we discovered one issue - that the New Relic plugin isn’t compatible with Rails 3.1. Once removing the plugin, we had no other issues.

Upgrading 請求書.jp to Rails 3.1 took me a total of eleven hours. In its current state, Rails 3.1 is usable, but upgrading requires a fair amount of independent research and debugging. Once it is actually released, these hiccups should go away, and upgrading your application should go smoothly.

We have just finished updating one of our bigger projects from Rails 2.3.8 to 3.0.3 and wanted to share our experience.

The update process:

For the update we performed the following tasks:

• basic Rails 3 framework update, including new routes and switch to bundler
• update the gems and plugins, replace some gems / plugins with others or with custom code (see below)
• resolve Rails and shoulda deprecations
• adapt to Rails 3 new escaping behavior
• port all mailers to the new ActionMailer API

For the update process itself, we followed the various advise out there.

Finding replacements for the gems and plugins we used wasn't always straight forward.  The replacements we had to do:

• AppConfig -> RailsConfig
• super exception notifier -> exception_notification and custom code using rescue_from to handle error display
• globalize2 -> globalize3
• subdomain_fu -> custom code; the new router already takes care of some of subdomain_fu's functionality

Some numbers:

To give you an impression of the size of the application, here are the sloccount numbers:

• test: 4047 sloc
• app: 2497 sloc
• vendor: 606 sloc
• config: 332 sloc
• lib: 188 sloc

Change statistics (taken from git, comparing before starting the upgrade with upgrade finished):

• all directories excluding vendor:
  268 of 698 files changed with 2191 insertions, 1886 deletions
• app, config, lib, test only:
  260 of 263 files changes with 1957 insertions, 1801 deletions

As you can see, changes were needed to nearly every file in app, config. The main reason we also had to change so many test cases were the deprecations in shoulda.

In total we spent 51 hours on the upgrade, plus 10 hours debugging a very strange issue.

The basic update and getting our test cases running again took us around 2 days. Another 5 days we then spent mainly on:

• Resolving deprecations and minor issues which were caused by changed behavior in new Rails or gems.
• Updating our own plugins and gems. This was rather challenging because for some parts of Rails the documentation is still lacking or outdated.
• The new escaping behavior wasn't caught by our test cases. So manual testing to find any escaped HTML chunks in was required.
• Locale in URL handling: The new router still isn't flexible enough and we couldn't DRY this up completely. For simpler sites routing-filter works nicely though.

All in all, the update took a bit longer than we expected (which was about a week). We attribute this to the amount of detail that needed to be addressed.

Rails default date input is functional, but not very user friendly. For instance, the following is what date input previously looked like in Doorkeeper.

However, by using the Javascript library jQuery Tools' Dateinput, I changed improved the user interface to the following.

The issue with Dateinput, and other date selection widgets, is that they don't use the Rails style drop downs for inputs, but rather a text input (or HTML 5's date input in the case of Dateinput). To work around this, there are two general strategies: adapt your Rails application to support text inputs for dates, or adapt the widget to use Rails date drop downs. I prefer the second strategy as it is overall much less intrusive.

To accomplish this, I adapted a snippet I found for adapting jQuery UI's datepicker to Rails to the following which is available as a gist.

// Based on http://snipt.net/boriscy/datetime-jquery-formtastic/

$.tools.dateinput.localize("ja", {
  months: '1月,2月,3月,4月,5月,6月,7月,8月,9月,10月,11月,12月',
  shortMonths: '1月,2月,3月,4月,5月,6月,7月,8月,9月,10月,11月,12月',
  days: '日曜日,月曜日,火曜日,水曜日,木曜日,金曜日,土曜日',
  shortDays: '日,月,火,水,木,金,土'
});
$.tools.dateinput.conf.format = 'yyyy-mm-dd';

$(document).ready(function() {
  $.tools.dateinput.conf.lang = $('html').attr('lang');
  $('div.date, div.datetime').each(function(i, el) {
    var sels = $(el).find("select:lt(3)");
    var d = new Date(sels[0].value, parseInt(sels[1].value) - 1, sels[2].value);
    var dateinput = $(">input type="date" /<").dateinput({ value: d} );

    // Without this, the field is initially blank
    dateinput.val(dateinput.data().dateinput.getValue($.tools.dateinput.conf.format));

    dateinput.change(function(event, date) {
      $(sels[0]).val(date.getFullYear());
      $(sels[1]).val(date.getMonth() + 1);
      $(sels[2]).val(date.getDate());
    });
    $(sels[0]).before(dateinput);
    $(sels).hide();
  });
});

By just adding this javascript file, along with the base dateinput widget, you can convert all your date inputs to use the widget. No modification of any internal application or view logic required. The Japanese localization is also included, along with automatic locale switching based on the html's lang attribute.

Kara-mail (空メール) is a user interface design pattern commonly seen in Japanese mobile web sites. As all Japanese mobile phones have an email address, registration for mobile sites usually requires an email address. However, entering an email address is relatively time consuming for a user to do. So instead of asking the user to input the email address, sites have a user send them an empty email, hence the name kara-mail (kara is Japanese for empty). After the site receives the email, the site will reply with an email containing a sign up link. Upon clicking the sign up link, the user can then complete the sign up process.

On Doorkeeper, we use kara-mail to allow users to easily sign up to attend an event via their mobile.

Kara-mail Registration with Doorkeeper

Because mobile spam is particularly annoying, mobile carriers have a setting that will allow users to block all email that do not come from other mobile devices or a specifically whitelisted domain. To help users whitelist our domain, we link directly to the carrier settings page. Additionally, as most Japanese mobiles do not support copying of web page text, but do allow copying of form input text, we put our domain in a form input so a user can easily copy it.

Depending on the mobile carrier, we show an appropriate link to the settings. The links are

For more tips on designing for Japanese mobiles, see our Keitai Web Technology Guide.

For many of our development projects, we use Ruby as our programming language. The language is not only interesting from a software development perspective, but also from a cultural one. Ruby was originally created by Yukihiro Matsumoto, and many of the core developers are also Japanese.  Although this programming language began in Japan, it is now popular throughout the world. 

Because many of the Japanese ruby developers are not so comfortable with English, there is a bit of a rift between the two communities.  This divide between the international and Japanese Ruby communities came up several times during RubyKaigi.  At the same time, both sides want to work to close this gap.

As a Canadian in Japan I feel I am in an excellent position to help bring these two communities together.  I am both tapped into the English speaking Ruby community around the world through the internet, and the Japanese Ruby community by virtue of my location.

With this in mind, I created Tokyo Rubyist Meetup. This group will give Japanese Rubyists a chance to meet with non-Japanese Rubyists like my self.  Within an hour of the announcement of the group over twitter, we already had 25 people sign up, so it seems there is a demand for such a group.

We are going to have our first meeting towards the end of this month (the exact date is to be decided).  If you are interested in Ruby and living in Tokyo please join us. 

The delayed_job plugin for Rails does a good job for pushing tasks that take some time to process into the background, so that your users (and your Rails processes) can do other things than to wait. It uses daemons to process the backgrounded tasks, so for your system to work correctly it is essential that those daemons are running. Thus you want to make sure that those daemons are getting started when the server boots and are restarted in case they die. Those points are not addressed by delayed_job.

The usual way to get processes started at boot time under Linux is to use an init.d script. But init.d scripts only address the boot process - if the daemon dies, it won't get restarted.

D.J. Bernstein's daemontools make it very simple to create system services which achieve both of the above points: starting your daemons at system boot time and restarting them in case they die. And the best feature: to create a new service, you don't even need to include any of the typical daemon features (such as backgrounding the process) into your program. So while delayed_job uses the daemons library to provide those features, we won't be needing those.

The following steps show how to set up a new service. This assumes that you already installed daemontools (available for Ubuntu/Debian for instance: apt-get install daemontools-run).

1. Create a new directory. This is going to be the service directory.
2. Create a shell script "run" in the service directory which runs your program. This can be as simple as exec /path/to/my/program.
3. Create a symlink from the system service directory (for Ubuntu that'd be /etc/service), pointing to your new service directory.

That's it. To control your service, use the svc tool. See the manpage for more information.

In case of delayed_job, we are using the following run script:

#!/bin/sh
export RAILS_ENV=production
exec 2>/dev/null
exec setuidgid railsuser /srv/railsuser/project/current/script/delayed_job run

The script changes the user to "railsuser" (you don't want to run your delayed job processing under root; change it to match your setup), and then starts the usual delayed_job script, telling it to not put itself into the background.

One specialty to note is the handling of stderr. We redirect it to /dev/null to avoid potential "Broken pipe" exceptions in case something writes to stderr, which isn't available. Redirecting sdterr to stdout did not work.

Now, when updating, you will want to restart the delayed_job service. With capistrano we use the following task definitions:

namespace :delayed_job do
  desc "Start delayed job (if not running)"
  task :start, :roles => :app do
    sudo "svc -u /etc/service/#{application}_#{rails_env}_delayed_job"
  end
  desc "Stop delayed job"
  task :stop, :roles => :app do
    sudo "svc -d /etc/service/#{application}_#{rails_env}_delayed_job"
  end
  desc "Restart delayed job"
  task :restart, :roles => :app do
    sudo "svc -t /etc/service/#{application}_#{rails_env}_delayed_job"
  end
end

after "deploy:start", "delayed_job:start" 
after "deploy:stop", "delayed_job:stop" 
after "deploy:restart", "delayed_job:restart"

This requires that your deployment user will be able to run svc using sudo, so make sure to add this to your sudoers.

Also note that for the service names in the system service directory we use the pattern #{application}_#{rails_env}_delayed_job. Those are links to the service's directory, which are located under /srv/railsuser/project/services for our setup.

With this setup we have a pretty reliable delayed_job, and can use the same framework to run most (if not all) other services we might need with very little effort.