RubyGems mirror update

Due to the increasing time/resources needed to generate the indices for my rubygems mirror, I decided to stop generating the ones for old RubyGems versions (<= 1.2.0) and switch to the —update option.

This greatly help with the load generated on the server by the mirroring/indexing process. From now on I will only generate the legacy indices (for RubyGems <= 1.2.0) twice a day : 12AM UTC+2/12PM UTC+2.

[…]

RSpec 1.1

David Chelimsky anounced this morning the release of RSpec 1.1 (as of now the website is not up to date) RSpec 1.1 and just now the brand new website hosting.

What’s in this release ?

• StoryRunner : merge from Dan North ’s RBehave library. It aimed at providing a clean way to express and automate Acceptance Tests.

• Nested Example Groups : allows you to nest your describe blocks resulting in group sharing common specifications.

• Support for Rails 2.0.1

• Test::Unit interoperability : switch smoothly from Test::Unit to RSpec by allowing you to run your Test::Unit tests with RSpec. The goal here is to provide a way to progressively transition your tests to RSpec syntax.

More infos on David blog post about the RSpec 1.1 release

Congratulation and many thanks to everyone involved in this release ![…]

RubyGems mirror update

My RubyGems mirror just got an update :

• switch to rubygems 1.9.5 0.9.5
• index generation

If all goes well, it should be compatible with older rubygems versions and give a significantly boost to those using rubygems >= 1.9.5. 0.9.5[…]

Date, Time and old days

Not so long ago a bug emerged on my current application, a vicious one.
Let’s say you have a user and store its bithday date in database (say MySQL) as a DateTime (yep I know, it’s a little to precise for this kind of data).
Rails conveniently retrieve this field as a Time, but as you probably do not be aware of is the range limitation of Time’s dates. In fact the Time class is often used because it’s faster than Date or DateTime, but the tradeoff is it handle date based on epoch time (01-01-1970), where Date/DateTime use Complex class and as such can handle a broader range of date.

Well, my user’s birthday date was somewhere in 60s and so can’t be handle by a the Time class. But it’s not as simple, Rails fallback to DateTime if parsing MySQL datetime field does not work with Time.parse. This results in an object perfectly valid when using it, the problem arise when you try to update the corresponding record in database.

Remember my initial datetime field was fetched as a DateTime ruby object and as Rails include all fields from the record on update it tries to translate the birthday to something MySQL recognize. ActiveSupport include some convenients methods to do that : Time#to_s and Date#to_s takes a parameter allowing us to do something like Time.today.to_s(:db) as well as some convenient Date to Time conversion. Unfortunately, no out of range detection is done in Date#to_time which leads me to the first headache. Then ActiveSupport::CoreExtensions::Date::Conversions::DATE_FORMATS hash does not contains a :db key so calling Date#to_s(:db) results in an erroneous string for the db. (this is corrected in rails edge but not in 1.2-stable branch).

All in all :

1. be aware of Time class limitation
2. ruby open classes are wonderful : just drop this code in your lib directory and that should make it until rails is patched.

module ActiveSupport
  module CoreExtensions
    module Date
      module Conversions
        # Add db and rfc822 format
        DATE_FORMATS.merge!({
          :db       => "%Y-%m-%d %H:%M:%S",
          :rfc822   => "%a, %d %b %Y %H:%M:%S %z"
        })
      end
    end
  end
end 

[…]

Ruby 1.8.6 and Date

If you have a localized application and use Gettext, you probably use some method to handle localized Date.
Mine was a widely used approach consisting of redefining various Date.rb constants (Date::ABBR_DAYNAMES, Date::MONTHNAMES, etc.) to use the Gettext Object#_() method.

Recently I was forced to use Windows™® as my primary work OS, so I installed latest Ruby One-Click Installer which comes with Ruby 1.8.6. (I’m running 1.8.5 on all other machines).

This is where I was bitten by the Ruby 1.8.6 Date change, all my lovely constants are now frozen and I can’t localized them anymore (at least so simply).

So stick with 1.8.5 as long as you don’t really need to upgrade, there are also some threads issues with this release… so you also have weird issues if you use backgroundrb.[…]

Gettext and Rails

I encountered some problems with gettext rails integration recently, so here they are with explanation and resolution (if I found it).

Localization of ActiveRecordHelper error messages :

I choose to change the error message as explained by the gettext tutorial :

in application.rb

ActionView::Helpers::ActiveRecordHelper::L10n.set_error_message_title(Nn_("An error is occured", "%{num} errors are occured"))  
ActionView::Helpers::ActiveRecordHelper::L10n.set_error_message_explanation(Nn_("The error is:", "The errors are:"))

this works great, the error message displayed by the helper is the good one… except it only appears in the base language (English in my case). And the text never appears in my .pot file. After some searches in the gettext files, I found the problem : the textdomain for the helper is ‘rails’, which is great because you got validations and errors localized by gettext team for free. But it means you can’t set your own message without modifying the rails.pot in gettext package, which is not a solution because you need to maintain your own gettext gem or apply modifications to all you servers installs, and no you can’t install gettext as a rails plugin as gettext gem compile a gettext.so|.dylib on install which depends of your system.

The solution ? I didn’t find any satisfying one, if you force the textdomain to your app’s one then you need to duplicate and update all the rails translations strings from gettext package.

Gettext#updatepo and models with observers :

One of gettext nifty feature is its ability to extract your model columns names for translation. But the implementation cause some trouble when you have declared some observers in your environment.rb.

To accomplish its tasks, gettext first load rails environment. Then each files are parsed by Gettext::ActiveRecordParser#parse where you have :

old_constants = Object.constants
begin
  eval(open(file).read, TOPLEVEL_BINDING)
rescue
  $stderr.puts _("Ignored '%{file}'. Solve dependencies first.") % {:file => file}
  $stderr.puts $! 
end
loaded_constants = Object.constants - old_constants

This code try to catch the constants the current file add to Object.constants and then parse them either as ruby or ActiveRecord subclass. But if you declared an observer in your environment.rb, the model and its observer is already in the Object.constants and are just ignored by the gettext active record parser.

Solution ? Comment the config.observers line in your environment.rb each time you need to run gettext:updatepo task, not really sexy but it works.[…]

How to use a custom RDoc template from a rake task

I just need to do that, use a custom rdoc template to generate a documentation for a Rails project.

My main concern, regarding the different solutions I found on the web, was I didn’t want to have a template in the ruby installation directory but one that resides in a subdirectory of my rails application.

In fact it’s not a problem … when you find the good command. Here’s my rake task to generate the documentation (I put the Jamis Buck customized template in RAILS_ROOT/doc/rdoc_template/jamis.rb)

namespace :doc do 
  desc "Generate documentation for the application"
  Rake::RDocTask.new("app_jamis") { |rdoc|
    rdoc.rdoc_dir = 'doc/app_jamis'
    rdoc.title    = "My Website Documentation"
    rdoc.template = "doc/rdoc_template/jamis.rb"
    rdoc.options << '--line-numbers' << '--inline-source'
    rdoc.rdoc_files.include('doc/README_FOR_APP')
    rdoc.rdoc_files.include('app/**/*.rb')
    rdoc.rdoc_files.include('lib/**/*.rb')
  }
end

The important part is :

rdoc.template = "doc/rdoc_template/jamis.rb"

I tried to use the —template= and -T options in rdoc.options but it didn’t work… then I found that the RDocTask appends its template variable to the options passed to RDoc bypassing mine.[…]

PostgreSQL, MySQL and Ruby Drivers with MacPorts

If you’re on MacOS X and you’re developing database aware applications you certainly want to have installed locally one or both of those populars RDBMS : PostgreSQL or MySQL.

My prefered way of doing it is through MacPorts (previously known as DarwinPorts), a package manager designed initially for Darwin and now focusing on MacOS X.

I assume you already have installed MacPorts as explain on their website.

So let’s open a console with your favorite application (I tend to prefer iTerm over Terminal.app) and enter those commands :

sudo port install mysql5 +server

this will install MySQL 5.x database and the launchd script needed to start it (on-demand or at startup).
When the installation is done you’ll be asked to install the launchd script, just ignore this part and jump to the initialization part.

To initialize MySQL and secure access to it, just enter those commands

sudo -u mysql mysql_install_db5
sudo -u mysql /opt/local/lib/mysql5/bin/mysqld_safe &
/opt/local/lib/mysql5/bin/mysqladmin -u root password 'your_password'

You should now have a shinny new MySQL 5 installation with a password for your root account.

Then PostgreSQL, let’s type in :

sudo port install postgresql81-server

this will install PostgreSQL 8.1.x database and its launchd script.
When the installation is done, as with MySQL skip the launchd part and go to the initialization by issuing those commands :

sudo mkdir -p /opt/local/var/db/postgresql81/defaultdb
sudo chown postgres81:postgres /opt/local/var/db/postgresql81/defaultdb
sudo su postgres81 -c '/opt/local/lib/postgresql81/bin/initdb -D /opt/local/var/db/postgresql81/defaultdb'

PostgreSQL access authorization is something beyond this article, so I’ll don’t cover it.

If all goes well you should now have 2 working installations, but no simple way to start/stop them. A simple and powerful way of handling this (and launchd scripts, create/edit/load/unload) is Lingon. Just download the last version and go to the “Users Daemons” tab and you will see your two launchd scripts are there waiting for you to load/unload/enable/disable them.

To finish this installation, if you’re a Ruby kind of guy you probably want to access those databases from it. There are two ways to install the needed libraries : MacPorts or RubyGems.

For MySQL, there is no problem with MacPorts just type :

sudo port install rb-mysql

If you prefer the RubyGems solution the command is little more tricky :

sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

[Update]
For MacOS X.5 (Leopard) stock Ruby 1.8.6 :

sudo env ARCHFLAGS="-arch i386" gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

For PostgreSQL, you can’t do it through MacPorts unless you choose to install postgresql8 (and not postgresql81), because of the dependencies. But you can install the library through RubyGems with :

 sudo gem install postgres -- --with-pgsql-include-dir=/opt/local/include/postgresql81/ --with-pgsql-lib-dir=/opt/local/lib/postgresql81/ 

[Update]
For MacOS X.5 (Leopard) stock Ruby 1.8.6 :

 sudo env ARCHFLAGS="-arch i386" gem install postgres -- --with-pgsql-include=/opt/local/include/postgresql81/ --with-pgsql-lib=/opt/local/lib/postgresql81/ 

This is a simple and yet functional setup to work.[…]

Capistrano tips

If you use (and you definitely should) Capistrano to handle your Rails application deployement, you should have faced some troubles to handle passwords for svn + database, at least if you’re like me and don’t want to put those production passwords in your subversion repository.

My first concern was to have a different user/password to checkout code from subversion, ssh to the server and connect to the database.

So here’s some tricks which allows you to avoid puting passwords in subversion and use different passwords for each part of the process (ssh, svn, db), put those in your config/deploy.rb.

Use whatever SVN User/Password you want :

[UPDATE] A little update to this part to avoid remote svn user/password to be cached locally.

Old :

 set :svn_user, ENV['svn_user'] || "jonathan"
 set :svn_password, Proc.new { Capistrano::CLI.password_prompt('SVN Password: ') }
 set :repository,
  Proc.new { "--username #{svn_user} " +
             "--password #{svn_password} " +
             "http://your.domain.tld/path/to/svn/#{application}/trunk/" }

New :

 set :svn_user, ENV['svn_user'] || "jonathan"
 set :svn_password, Proc.new { Capistrano::CLI.password_prompt('SVN Password: ') }
 set :repository,
  Proc.new { "--username #{svn_user} " +
             "--password #{svn_password} " +
             "--no-auth-cache " +
             "http://your.domain.tld/path/to/svn/#{application}/trunk/" }

So you can now do things like :

  svn_user=your_name cap deploy

and the svn checkout will use ‘your_name’ as svn username, prompt you for the subversion password (remember, you’re using ssh, so no clear password transit) and then do the checkout normaly.

Generate config/database.yml on-the-fly

Just like passwords, it’s a good idea to avoid having your database configuration in subversion if you’re more than one developer to commit code. So why not generate it locally and put it through our lovely ssh connection, at deployement time ?

  desc "After updating code we need to populate a new database.yml"
task :after_update_code, :roles => :app do
  require "yaml"
  set :production_database_password, proc { Capistrano::CLI.password_prompt("Production database remote Password : ") }

  buffer = YAML::load_file('config/database.yml.template')
  # get ride of uneeded configurations
  buffer.delete('test')
  buffer.delete('development')
  
  # Populate production element
  buffer['production']['adapter'] = "postgresql"
  buffer['production']['database'] = "vms_production"
  buffer['production']['username'] = "you_db_username
  buffer['production']['password'] = production_database_password
  buffer['production']['host'] = "localhost"
  
  put YAML::dump(buffer), "#{release_path}/config/database.yml", :mode => 0664
end

This code uses the config/database.yml.template file (it’s up to you to have one, I don’t think this one is hard :D), to generate a simple database.yml containing only the production database configuration. Additionnaly it prompts you for the database connection password.

Isn’t this great ?[…]