PostgreSQL, MySQL and Ruby Drivers with MacPorts

If you’re on MacOS X and you’re developing database aware applications you certainly want to have installed locally one or both of those populars RDBMS : PostgreSQL or MySQL.

My prefered way of doing it is through MacPorts (previously known as DarwinPorts), a package manager designed initially for Darwin and now focusing on MacOS X.

I assume you already have installed MacPorts as explain on their website.

So let’s open a console with your favorite application (I tend to prefer iTerm over Terminal.app) and enter those commands :

sudo port install mysql5 +server

this will install MySQL 5.x database and the launchd script needed to start it (on-demand or at startup).
When the installation is done you’ll be asked to install the launchd script, just ignore this part and jump to the initialization part.

To initialize MySQL and secure access to it, just enter those commands

sudo -u mysql mysql_install_db5
sudo -u mysql /opt/local/lib/mysql5/bin/mysqld_safe &
/opt/local/lib/mysql5/bin/mysqladmin -u root password 'your_password'

You should now have a shinny new MySQL 5 installation with a password for your root account.

Then PostgreSQL, let’s type in :

sudo port install postgresql81-server

this will install PostgreSQL 8.1.x database and its launchd script.
When the installation is done, as with MySQL skip the launchd part and go to the initialization by issuing those commands :

sudo mkdir -p /opt/local/var/db/postgresql81/defaultdb
sudo chown postgres81:postgres /opt/local/var/db/postgresql81/defaultdb
sudo su postgres81 -c '/opt/local/lib/postgresql81/bin/initdb -D /opt/local/var/db/postgresql81/defaultdb'

PostgreSQL access authorization is something beyond this article, so I’ll don’t cover it.

If all goes well you should now have 2 working installations, but no simple way to start/stop them. A simple and powerful way of handling this (and launchd scripts, create/edit/load/unload) is Lingon. Just download the last version and go to the “Users Daemons” tab and you will see your two launchd scripts are there waiting for you to load/unload/enable/disable them.

To finish this installation, if you’re a Ruby kind of guy you probably want to access those databases from it. There are two ways to install the needed libraries : MacPorts or RubyGems.

For MySQL, there is no problem with MacPorts just type :

sudo port install rb-mysql

If you prefer the RubyGems solution the command is little more tricky :

sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

[Update]
For MacOS X.5 (Leopard) stock Ruby 1.8.6 :

sudo env ARCHFLAGS="-arch i386" gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

For PostgreSQL, you can’t do it through MacPorts unless you choose to install postgresql8 (and not postgresql81), because of the dependencies. But you can install the library through RubyGems with :

 sudo gem install postgres -- --with-pgsql-include-dir=/opt/local/include/postgresql81/ --with-pgsql-lib-dir=/opt/local/lib/postgresql81/ 

[Update]
For MacOS X.5 (Leopard) stock Ruby 1.8.6 :

 sudo env ARCHFLAGS="-arch i386" gem install postgres -- --with-pgsql-include=/opt/local/include/postgresql81/ --with-pgsql-lib=/opt/local/lib/postgresql81/ 

This is a simple and yet functional setup to work.

Published on Sun, 19 Nov 2006 13:06

Subversion Automation : automate branching/tagging

I recently discovered a gem (no joke here) called Subversion Automation, it’s aimed to provide an unified layout to handle your projects branching/tagging in subversion and more importantly automate the process through a simple command : sc .

It currently handle some key aspects of a project management :

  • release :
Allows you to create a release branch (major.minor), or tag a release (macro). It creates a branch for the release and a tag for it.
  • bug
Allows you to work with release branch bug fixes, in fact it handle the creation of a unique branch based on a bug name/id and then the merge to the corresponding release branche and to the trunk. Really useful to track bug fix changes.
  • experimental
Allows you to work on some experimental code in a branch wich can be merge with the trunk in both direction.

Subversion Automation is really worth a try, even if it could be hard to use on existing project due to the subversion layout changes implied, but it’s up to you.

Be sure to watch the marvelous screencast on author website : Subversion Automation Screencast

Published on Wed, 15 Nov 2006 11:37

Multiple identities in one account with Apple Mail.app

Over the time I tend to accumulate mail adresses and accounts. Not so long ago, I decided to switch from a multiple mail accounts to a single imap account where I aggregated all others.

I used to be an Apple Mail.app user, but this switch caused me some troubles. How to specify different (legitimate) sender addresses with only one account configured ? I thought it was impossible and so decided to switch to Mozilla Thunderbird which allows you to set different sender addresses by account.

But recently, I found it was possible in Mail.app and here’s how :

In your account configuration, enter a comma-separated list of adresses in the “Email Address” field.

Multiple identities for one account

Then, you’ll be able to choose which sender address should be used when writing a new mail.

Choose which adresse to use when writing a new mail

For those of you interested in my configuration, I operate a simple linux box with Postfix, Courier-Imapd and Maildrop. Then I use Fetchmail to grab mails from my different accounts and put them in my imap setup.

[Update]
Drew leave a comment about a solution to have different names for each addresses. It require you edit the preferences plist file : Mac OS X Mail app config – EmailDiscussions.com

Published on Sun, 05 Nov 2006 09:30

Prototype's Enumerable

My previous posts about Prototype were focused on shortcuts and functions to work on forms. This one is all about the Enumerable object.

Well, Enumerable is one of those terrific objects of Prototype. Basically it allows you to add enumeration to your objects or add prototype’s way to do it to standard javascript objects. But as javascript doesn’t allow you to redefine the way your object (or standard object) implements enumeration, it broke the standard javascript enumeration. As you should know, javascript allows you to enumerate through an object properties ( with the “for in” syntax) and takes care to step over unwanted properties, for example :

var myArray = [1,2,3,4]; // Let's build an array object

for ( i in myArray ) {
  console.info(i);
}

// Output to firebug console:
// 1 
// 2 
// 3
// 4

As you’ve seen, even if Array is an object, the enumeration step over methods to only gives you values. If you do the same thing with Prototype library loaded, you’ll get the following :

  var myArray = [1,2,3,4]; // Let's build an array object
  
  for ( i in myArray ) {
    console.info(i);
  }
  
  // Output to firebug console:
  // 0
  // 1
  // 2
  // 3
  // each
  // all
  // any
  // collect
  // detect
  // findAll
  // ...

So I let you imagine how this can break existing code. That’s why so many other javascript framework developers don’t like Prototype’s way of doing and in some way they are right, this is a constraint of Prototype you should be aware of : Prototype don’t play nicely with others.

Then, after this little precautions let’s examine what Prototype’s Enumerable objects can give us.
All you need, to add enumerability to your objects, is to define a function _each which takes a function as parameter and call this function with each iterable elements of your objects.

For example if your objects stores its elements in an Array, you can make your objects enumerable like this :

var MyObject = Class.create();
MyObject.prototype = {
  initialize: function() {
    this.elements = $A(arguments);
  }
  
  // The magic function :)
  _each: function(iterator) {
    for ( var i = 0; i < this.elements.length; i++)
      iterator(this.elements[i]);
  }
};
// Extends MyObject with Enumerable functions
Object.extend(MyObject.prototype, Enumerable);

Now our new object have the following methods :

  • each()
  • all()
  • any()
  • collect()
  • detect()
  • findAll()
  • grep()
  • include()
  • inject()
  • invoke()
  • max()
  • min()
  • partition()
  • pluck()
  • reject()
  • sortBy()
  • toArray()
  • zip()
  • inspect()

Nice for a single method added, isn’t it ?
Let’s see what we can do with this :

// Let's create a new MyObject object :)
var anObject = new MyObject(3,5,4);

// Iterate through values
anObject.each( function(element) {
  doSomething(element);
});

// Verify if all our values are lesser than 6
// return true
anObject.all( function(value) {
  return value < 6;
});

// Verify if any value is greater than 4
// return true
anObject.any( function(value) {
  return value > 4;
});

// Get an array containing all values as string prefixed by 'id_'
// return ['id_3','id_5','id_4']
valuesAsId = anObject.collect( function(value) {
  return 'id_' + value;
});

// Get the first element greater than 3
// return 5
anObject.detect( function(value) {
  return value > 3;
});

// Find all elements greater than 3
// return [5,4]
anObject.findAll( function(value) {
  return value > 3;
});

// Find all elements matching a given pattern
// here we assume our initialization was anObject = new MyObject('jonathan', 'tron');

// return ['jonathan']
anObject.grep(/h/)

// return ['JONATHAN']
anObject.grep(/h/, function(value){
  return value.toUpperCase();
});

// Find if 3 is an element of our object
// return true
anObject.include(3);

// Get the sum of our elements + 10
// return 22
anObject.inject(10, function(sum, value) {
  return sum + value;
});

// Want to call a method on each elements ?
// return ["3","5","4"]
anObject.invoke('toString');

// Want to call a method on each elements but this time with params ?
// return ["11","101","100"]
anObject.invoke('toString', 2);

// Get the greatest element
// return 5
anObject.max();

// Get the smallest element
// return 3
anObject.min();

// Separate the elements in two groups :
// those matching a condition and those no matching it
// return [ [3], [5,4] ]
anObject.partition( function(value) {
  return value <= 3;
});

// Get an array of a particular properties of each element
// here we assume our initialization was :
// anObject = new MyObject( {firstname: 'jonathan', lastname: 'tron'}, 
//                          {firstname: 'sam', lastname: 'stephenson'});
// return ['jonathan','sam']                            
anObject.pluck('firstname');

// Return all element for whose the given function return true
// return [3]
anObject.reject(function(value) {
  return value != 3;
});

// Sort using the given compare function
// here we assume our initialization was :
// anObject = new MyObject( {firstname: 'jonathan', lastname: 'tron'}, 
//                          {firstname: 'sam', lastname: 'stephenson'});
// return [{firstname: 'sam', lastname: 'stephenson'},{firstname: 'jonathan', lastname: 'tron'}]
anObject.sortBy(function(value) {
  return value.lastname.toLowerCase();
});

// Get a new array of elements
// return [3,5,4]
anObject.toArray();

// And the last one, I did not have any need for it by now, but I can imagine it could be helpful when dealing with some sort of columns and rows
// here we assume our initialization was :
// anObject = new MyObject( [1, 2, 3] );
// return [ [1,4,7], [2,5,8], [3,6,9] ]
anObject.zip([4,5,6], [7,8,9]);

But there’s more, what if you want to enumerate but stop the enumeration at some point or skip an iteration ?

Prototype defines two objects for this : $break and $continue.

// Iterate over our values but break if value is 2
[1,2,3].each( function(value) {
  if ( value == 2 ) throw $break;
});

// Iterate over our values but skip to next element if value is 2
[1,2,3].each( function(value) {
  if ( value == 2 ) throw $continue;
});
Published on Thu, 12 Oct 2006 14:45

Apache 2.0.x + Mongrel + mod_proxy + mod_rewrite configuration

When I decided to try the switch from fcgid to mod_proxy + mongrel for this blog (and now the majority of my projects at work), I discovered there was no example of such configuration or at least not all in one place.
This post will try to fill this lack of a complete example. The distro i’m using here is Debian, but the configuration should be easily ported to yours.

Thereafter I assume you already have a working configuration with :

- Apache 2.0.x - mod_proxy_http (thanks AMicky) - mod_proxy - mod_rewrite

mod_proxy configuration :

There’s not so much to change here, just allow proxying to anyone for local domains.

/etc/apache2/mods-available/proxy.conf

  <Proxy *>
     Order deny,allow
     # next line was Deny from all
     Allow from all
  </Proxy>

A sample vhost file

This configuration is intended to be used with a RubyOnRails application served by Mongrel and deployed with Capistrano.

So you should notice some nice things, like auto-disabling website with a personalized message when there’s a maintenance.html page found, serving static contents directly by apache (no uneeded hit on mongrel) and conserving access to cgi-bin directory (see my previous post about awstats).

/etc/apache2/sites-available/zone.mydomain.tld.conf

<VirtualHost *>
    # Set the domain
    ServerName mydomain.tld
    # Set domain aliases
    ServerAlias www.mydomain.tld 
    ServerAlias web.mydomain.tld

    # Set the DocumentRoot, this is where apache will look for static files
    # This is the public directory of your rails application 
    # As previously stated, my app is deployed with Capistrano, so I set DocumentRoot to the current/public directory
    DocumentRoot /var/www/mydomain.tld/current/public/

    # Set where you want apache put its logs
    # note I use deflate as log format, this is because I use mod_deflate to compress all outputs
    CustomLog /var/www/mydomain.tld/logs/access.log deflate
    ErrorLog /var/www/mydomain.tld/logs/error.log
    
    # if you want to have server-wide statistics,
    # you can also tell apache to log in a common file shared by all your domains 
    CustomLog /var/log/apache2/access.log deflate
    ErrorLog /var/log/apache2/error.log

    # I use utf-8 for all my projects, so I force apache to send the good charset by default.
    # This is needed if you use page caching and want apache serves these with the good charset.
    AddDefaultCharset utf-8

    # Allow limited access to your public directory
    # dont allow user to list directories
    # allow apache to FollowSymlinks
    <Directory /var/www/mydomain.tld/current/public/>
      Options -Indexes FollowSymLinks

      AllowOverride All
      Order allow,deny
      Allow from all
    </Directory>

    # Be sure mod_rewrite is activated for you vhost
    RewriteEngine On

    # If you use applications which resides in cgi-bin, like awstats
    # the following mod_rewrite RewriteRule will allow apache to know it should not proxy these request and the PT switch will allow the ScriptAlias directive to work
    RewriteRule "^/cgi-bin/.*" "$0" [QSA,PT,L]
    RewriteRule "^/awstatsicon/.*" "$0" [PT,L]

    # Set a ScriptAlias for /cgi-bin/ url
    # All url begining with /cgi-bin/ will be served from the specified cgi-bin deirectory
    ScriptAlias /cgi-bin/ /var/www/mydomain.tld/cgi-bin/  
    <Directory /var/www/mydomain.tld/cgi-bin/>
      AllowOverride All
      Options ExecCGI FollowSymLinks
      Order allow,deny
      Allow from all
    </Directory>

   # Do not allow open proxying, allow only requests starting with a /
   <LocationMatch "^[^/]">
      Deny from all
   </LocationMatch>
   
   # Avoid open you server to proxying
   ProxyRequests Off

   # Let apache correctly rewrite redirect
   ProxyPassReverse / http://localhost:8001/

   # Let apache pass the original host not the ProxyPass one
   ProxyPreserveHost On

   # As I mentioned earlier, Capistrano disable_web task allows you to disable your application
   # In fact it only create a simple page named maintenance.html in the directory public/system/ of your application
   # The following mod_rewrite rules will tell apache to directly serve the maintenance.html pages if it find it out.
   RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
   RewriteCond %{SCRIPT_FILENAME} !maintenance.html
   RewriteRule ^.*$ /system/maintenance.html [L]
   
   # For static files it's good to avoid hitting your mongrel process
   # so let apache knows it should serve it directly
   # for a rails application it means, files in images / stylesheets / javascripts
   RewriteRule "^/(images|stylesheets|javascripts)/?(.*)" "$0" [L]
   # Try to match a cached pages
   RewriteRule ^([^.]+)$ $1.html [QSA]
   # if the cached page does not exists
   RewriteCond %{REQUEST_FILENAME} !-f
   # proxy requests to your mongrel instance
   RewriteRule "^/(.*)" "http://localhost:8001/$1" [P,QSA,L]
</VirtualHost>
Published on Wed, 26 Jul 2006 23:28

RewriteRule for ScriptAlias with mod_proxy enabled

Since my switch to Apache2+mod_proxy+mongrel, I had some troubles figuring how to make urls defined by a ScriptAlias work.

As an example, take Awstats, tipically it’s configuration involve a ScriptAlias directive in you vhost configuration.

Something like :

ScriptAlias /awstats/ /var/www/your_site_dir/cgi-bin/awstats/
<Directory /var/www/your_site_dir/cgi-bin/awstats/>
   AllowOverride All
   Options ExecCGI FollowSymLinks
   Order allow,deny
   Allow from all
</Directory>

But when mod_proxy is enabled, all requests are proxied, ignoring your ScriptAlias directive.

A solution if you also use mod_rewrite is to write something like this :

RewriteRule "^/awstats/.*" "$0" [PT,L]

The magic switch is PT which enable rewrite rules to be shared between differents mods.

Published on Wed, 26 Jul 2006 17:22

Working on Forms with Prototype

Prototype besides its nices shortcuts provides some function to work with forms elements. So let’s see what prototype can do for us.

Published on Fri, 21 Jul 2006 22:11

Capistrano tips

If you use (and you definitely should) Capistrano to handle your Rails application deployement, you should have faced some troubles to handle passwords for svn + database, at least if you’re like me and don’t want to put those production passwords in your subversion repository.

My first concern was to have a different user/password to checkout code from subversion, ssh to the server and connect to the database.

So here’s some tricks which allows you to avoid puting passwords in subversion and use different passwords for each part of the process (ssh, svn, db), put those in your config/deploy.rb.

Use whatever SVN User/Password you want :

[UPDATE] A little update to this part to avoid remote svn user/password to be cached locally.

Old :

 set :svn_user, ENV['svn_user'] || "jonathan"
 set :svn_password, Proc.new { Capistrano::CLI.password_prompt('SVN Password: ') }
 set :repository,
  Proc.new { "--username #{svn_user} " +
             "--password #{svn_password} " +
             "http://your.domain.tld/path/to/svn/#{application}/trunk/" }

New :

 set :svn_user, ENV['svn_user'] || "jonathan"
 set :svn_password, Proc.new { Capistrano::CLI.password_prompt('SVN Password: ') }
 set :repository,
  Proc.new { "--username #{svn_user} " +
             "--password #{svn_password} " +
             "--no-auth-cache " +
             "http://your.domain.tld/path/to/svn/#{application}/trunk/" }

So you can now do things like :

  svn_user=your_name cap deploy

and the svn checkout will use ‘your_name’ as svn username, prompt you for the subversion password (remember, you’re using ssh, so no clear password transit) and then do the checkout normaly.

Generate config/database.yml on-the-fly

Just like passwords, it’s a good idea to avoid having your database configuration in subversion if you’re more than one developer to commit code. So why not generate it locally and put it through our lovely ssh connection, at deployement time ?

  desc "After updating code we need to populate a new database.yml"
task :after_update_code, :roles => :app do
  require "yaml"
  set :production_database_password, proc { Capistrano::CLI.password_prompt("Production database remote Password : ") }

  buffer = YAML::load_file('config/database.yml.template')
  # get ride of uneeded configurations
  buffer.delete('test')
  buffer.delete('development')
  
  # Populate production element
  buffer['production']['adapter'] = "postgresql"
  buffer['production']['database'] = "vms_production"
  buffer['production']['username'] = "you_db_username
  buffer['production']['password'] = production_database_password
  buffer['production']['host'] = "localhost"
  
  put YAML::dump(buffer), "#{release_path}/config/database.yml", :mode => 0664
end

This code uses the config/database.yml.template file (it’s up to you to have one, I don’t think this one is hard :D), to generate a simple database.yml containing only the production database configuration. Additionnaly it prompts you for the database connection password.

Isn’t this great ?

Published on Sat, 15 Jul 2006 16:33

Switch to Apache + mod_proxy + Mongrel

In my quest to find the best rails configuration, I decided to try a new alternative to Apache+FastCGI/Apache+FCGID/Lighttpd+FastCGI.

Everyone seems to be focused on Mongrel combined with Apache 2.2 + mod_proxy_balancer. It looks good but Apache 2.2 isn’t actually in Debian (not even in unstable), and as I choose to switch this blog from a Fedora Core 2 (ooch !) server to another server of mine powered by Debian, it really doesn’t fits my needs.

So I choosed to try a different approach and use Apache 2.0.x + mod_proxy + Mongrel. This solution, gives me the possibility to use mongrel_cluster for process management, even if actually I don’t have multiple mongrel instances, and add a load balancer (Pen/Balance/Pound) if needed.

This setup seems to be sufficient for this blog, which runs the latest Typo (3.99.3).

Let me know if you see any differences with the previous configuration.

Published on Sat, 15 Jul 2006 14:29

Enhanced Localization Plugin for Rails

For a project I had at work, I need to have a website in different languages. I assumed this was a common task nowadays, but I quickly found the limits of current solutions in Rails or at least I didn’t found one which matched my criteras for this project : lightweight, allows texts localizations and more importantly permits URLs localizations.

So I took one of the simpler plugins for localization tasks : Localization from Thomas Fuchs. Then I tweaked it a little bit to make it fits my needs and here comes the result : Localization Enhanced. It allows you to easily translate your texts using simple hashes loaded at startup, have as many languages as you want and have controllers/actions translations as well, all using the standards Rails methods (link_to, url_for, etc.). Additionally, I added a little refinement to ActionView::Helpers::AssetTagHelper#image_tag which allows you to use different images based on the current languages where you need it (images-based menu bar, …), by passing :loc => true as an option to image_tag. This one results in image tag generating an url prefixed by the current language (EN_en, FR_fr, etc.) unless the current language is the default one.

If you want to give it a try here’s the subversion repository url to get it :

http://projects.tron.name/svn/railsplugins/localization_enhanced/

or directly install it with :

./script plugin install http://projects.tron.name/svn/railsplugins/localization_enhanced/

License is the same as the original one : MIT Licence (at least I think it is this one as I didn’t find any mention of one in the source).

Published on Sun, 09 Jul 2006 16:09

RSS