The Big Leap October 15th, 2010

Today was my last day at Openminds!

As some of you know I’ve been working with Openminds, a Belgian hosting and development company, for about 4 years now. I was their first employee, when the business was still vested in a small condo, and I’ve grown with it while the company moved 2 times to bigger locations, and only got bigger and better. I already knew my way around computers, but during my time at Openminds I learned a whole lot more about servers and hosting. I also met Ruby and Rails there, I got to learn it, and I can say I’ve become a professional Ruby programmer now. I’ve also enjoyed working with the colleagues there, who are all the best at what they do.

But now it’s time to take the big leap, and no longer take the safe bet of being an employee. From now on I’m an Entrepreneur! A few months ago my colleague Jens asked me and Tijs to start a new business with him, and after some days I was sure now was the moment!

We’ll be doing what we’re good at:

  • Consultancy in web-applications, from starting a new app to rescuing a dying one and optimizing an existing one.
  • Convert good idea’s into full fledged web applications
  • Give regular web applications an awesome mobile interface.

I will be responsible for the Ruby-department of this story, with Ruby consultancy and Ruby development. I’ll also share my expertise in server-setups with our customers.

The first weeks of this plan we were known as Unnamed BVBA, but as of today we’ll be known as SumoCoders, where we’ll wrestle applications and code like real rikishi (wrestlers, “literally men of power”)


More info (in dutch) on our website

tags: l

Ruby and Rails beginner talk October 8th, 2009

Last night I gave a talk for Zeus about Ruby and Rails, talking about Ruby fundamentals and a short intro to the Rails philosophy and MVC.

I don’t know if you’ll be much with the slides without hearing the talk, but you can find them on http://defv.be/ruby.and.rails.pdf.

If you want to see the entire presentation you should come to ArrrrCamp, where I’ll be giving it again.

tags: l

Ruby Hash coolness September 28th, 2009

Don’t shoot me if you already use this every day, but this is new to me.

In the past I’ve often had code like this

@hosts = {}

@accounts.each do |account|
  @hosts[account[:host]] ||= []
  @hosts[account[:host]] << account
end

But why do this hen you can just add some initializer code to your hash.

@hosts = Hash.new { |h, k| h[k] = [] }

@accounts.each do |account|
  @hosts[account[:host]] << account
end

Where Hash automatically invokes the block when an unknown hash key is called. Great!

tags: l

MySQL password hashing in Ruby May 12th, 2009

An old database used MySQL’s PASSWORD() hashing functionality for storing user credentials. We’re creating a Ruby app now to interface with that database, and wanted to hash the password in ruby. After looking at the code it looked like they just SHA1 hash the password twice and prepend a *. Implementation in Ruby is easy:

require 'digest/sha1'

def hash_mysql_password pass
  "*" + Digest::SHA1.hexdigest(Digest::SHA1.digest(pass)).upcase
end

Which gives us in MySQL

mysql> SELECT PASSWORD('foo');
+-------------------------------------------+
| PASSWORD('foo')                           |
+-------------------------------------------+
| *F3A2A51A9B0F2BE2468926B4132313728C250DBF | 
+-------------------------------------------+

And in Ruby

>> hash_mysql_password 'foo'
=> "*F3A2A51A9B0F2BE2468926B4132313728C250DBF"
tags: l

Custom ActiveRecord timestamps April 27th, 2009

Some legacy databases have already defined their own created_at or updated_at fields. They can be easily filled in with ActiveRecord before_save filters, but after implementing this behaviour a few times the urge arose to write something reusable, so I created the custom_timestamps plugin.

With custom_timestamps you can easily define the columns you want to be updated on creation/change

class LegacyModel < ActiveRecord::Base
  set_create_column :creation_time
  set_update_column :change_time
end

The plugin can be found on github.

tags: l

ArrrrCamp Belgium April 2nd, 2009

We have been talking between ourselves about hosting a Belgian Ruby conference since the last Railsconf Europe, and we have finally decided to proceed with the idea. Colleague Joren put things in motion and today we’re proud to announce Arrrrcamp, which stands for “About Ruby, Rails, Radiant and Rum Camp”.

The event will take place on friday, May 8th, in Ghent, Belgium. The idea is to have a barcamp-like organisation of talks, where everybody is encouraged to participate by either doing talks, taking pictures, do some hacking, … You can read all about the event on the website.

tags: l

Radiant 0.7.x and Ruby 1.8.4 March 13th, 2009

A client of ours complained that Radiant 0.7.1 was not working out of the box on one of our shared servers. When running the radiant command he got an error:

/usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:199:in `set_load_path': undefined method `load_paths' for nil:NilClass (NoMethodError)
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:97:in `run'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:101:in `run'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:72:in `load_initializer'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:90:in `load_initializer'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:61:in `run'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:19:in `boot!'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:161
  from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
  from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/radiant:3
  from /usr/bin/radiant:19

It worked fine on our other servers, as well as on our local machines, the only thing that was different was the ruby version, which was 1.8.4 on that specific server. Apparently that version handles super a bit different. The solution was to change 1 line in /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:

# In /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb line 101:
-     super
+     super(command, configuration)
tags: l

Application-wide timeranges February 2nd, 2009

In some applications you need to work with time-ranges a lot. Displaying todays messages, a count of all messages that were posted this month, or last month, … ActiveRecord has pretty good support for times and ranges, so you could do it the default way.

Message.find(:all, :conditions => {:created_at => Time.now.beginning_of_month..Time.end_of_month})

And although this works perfectly it’s not very easy on the eyes. It’s also not very DRY since I need this on a lot of models. It’d be a lot better if I could have it in a scope, like this

Message.in(:this_month)

So how do we achieve this?

In our config/initializer folder we have a file called timeranges.rb.

# timeranges.rb
TimeRanges = {
  :today      => lambda { Time.now.beginning_of_day..Time.now.end_of_day },
  :yesterday  => lambda { 1.day.ago.beginning_of_day..1.day.ago.end_of_day },
  :this_week  => lambda { Time.now.beginning_of_week..Time.now.end_of_week },
  :last_week  => lambda { 1.week.ago.beginning_of_week..1.week.ago.end_of_week},
  :this_month => lambda { Time.now.beginning_of_month..Time.now.end_of_month },
  :last_month => lambda { 1.month.ago.beginning_of_month..1.month.ago.end_of_month }
}

Which gives us a constant TimeRanges we can use all over the application. So in our Message we define the named_scope in

class Message < ActiveRecord::Base
  named_scope :in, lambda { |period| 
      :conditions => {:created_at => TimeRanges[period].call}
  }
end

Now we could even make this more DRY and make a module that defines the named_scope when it’s included, but in my application I had a lot of different use-cases for the TimeRanges, and didn’t really need the scope a lot. Leave me a note if you want it in a module but don’t know how to, else it’s an exercise for the interested and easily challenged reader ;-)

tags: l

Using Datamapper on legacy databases December 18th, 2008

Yesterday I had to map a legacy database schema to some Ruby classes. Just before that I had been looking into the advantages of DataMapper, and it looked like the perfect match for my current “challenge”.

from the DataMapper site:

DataMapper only issues updates or creates for the properties it knows about. So it plays well with others. You can use it in an Integration Database without worrying that your application will be a bad actor causing trouble for all of your other processes.

The system

The system is running with 2 databases. One database holds all the information for the legacy app, the other database holds all information for system management.

mysql app
mysql> desc cpClients;
+--------------+--------------+------+-----+---------+----------------+
| Field        | Type         | Null | Key | Default | Extra          |
+--------------+--------------+------+-----+---------+----------------+
| clientID     | int(255)     | NO   | PRI | NULL    | auto_increment | 
| login        | varchar(255) | NO   | UNI |         |                | 
| pass         | varchar(255) | NO   |     |         |                | 
| type         | int(2)       | NO   |     | 0       |                | 
| traffic      | bigint(255)  | NO   |     | 0       |                | 
<snip>
| VAT          | varchar(255) | YES  |     | NULL    |                | 
+--------------+--------------+------+-----+---------+----------------+
mysql> desc cpURL;
+----------+--------------+------+-----+---------+-------+
| Field    | Type         | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| url      | varchar(255) | NO   | PRI |         |       | 
| clientID | int(255)     | NO   |     | 0       |       | 
+----------+--------------+------+-----+---------+-------+
> exit
mysql management
mysql> desc ftp_users;
+--------------+------------------+------+-----+-------------------+-------+
| Field        | Type             | Null | Key | Default           | Extra |
+--------------+------------------+------+-----+-------------------+-------+
| username     | varchar(60)      | NO   | PRI | NULL              |       | 
| password     | varchar(30)      | YES  |     | NULL              |       | 
<snip>
| homedir      | text             | YES  |     | NULL              |       | 
| shell        | varchar(15)      | NO   |     | /bin/false        |       | 
+--------------+------------------+------+-----+-------------------+-------+

One Client has many URL’s, One client also has many ftp_users, but the link in that can’t be captured in a simple has many relationship.

Solving it with DataMapper

Setting up the connections

Setting up 2 connections to the databases was quite easy and well documented. I chose the app database to be the default connection, and management as another.

DataMapper.setup(:default, 'mysql://localhost/app')
DataMapper.setup(:management, 'mysql://localhost/management')
Having different database names

The first problem I had was that I wanted the cpClient table to be mapped to the User class, and cpURL to be mapped to a Domain class. This can be solved with storage_names.

class User
    include DataMapper::Resource
    storage_names[:default] = 'cpClients'
end

class Domain
    include DataMapper::Resource
    storage_names[:default] = 'cpURL'
end
Defining primary keys

Defining the primary key of my cpURL table was quite easy since DataMapper does natural keys out of the box.

class Domain
  include DataMapper::Resource
  storage_names[:default] = 'cpURL'

  property :url, String, :key => true
end

Defining the primary key of the cpClients table was a tad more difficult since it was poorly documented. I wanted the property to be called ID, but it should still use clientID in the database. The magic option to this is :field, which allows you to set a custom field for a propery.

class User
    include DataMapper::Resource
    storage_names[:default] = 'cpClients'

    property :id,                    Serial, :field => 'clientID'
end
Defining associations

The hardest part was defining associations. When I’d just define them with has n without extra options, DataMapper would look for user_id, so I’d have to tell which key was used. Since it wasn’t very well documented I had some failed attempts

has n, :domains, :child_key => 'clientID'
# => +options[:child_key]+ should be Array, but was String (ArgumentError)

has n, :domains, :child_key => ['clientID']
# => ArgumentError: +name+ should be Symbol, but was String 

has n, :domains, :child_key => [:clientID]
# => MysqlError: Unknown column 'client_i_d' in 'field list' (mysql_error_code=0001)

So I need to use a symbol, but apparently DataMapper does some automatic magic to translate keys. When you have a key like me you can disable this magic by changing field_naming_convention’s proc to your own. In the proc we return the stringified value’s name (to_s is needed because the key is given as a symbol)

repository(:default).adapter.field_naming_convention = lambda { |value| value.name.to_s }

And then the association worked. Do note the :child_key option needs to be set at both sides.

class User
  has n, :domains, :child_key => [:clientID]
end

class Domain
  belongs_to :user, :child_key => [:clientID]
end
Setting models to automatically connect to another database

The FtpUser model automatically needs to connect to the :management database, and there doesn’t seem to be a function to set this. If you temporarily want to connect to another database you can do it with the repository command.

repository(:management) { FtpUser.first }

However, if you want a model to always connect to a database other then the default, defining self.default_repository_name with the correct repository works very well. (Databases are referred to as repositories)

class FtpUser
  include DataMapper::Resource

  def self.default_repository_name
      :services
  end
end

All of this was needed to get my legacy database mapped, I hope some of you find this information useful. If you see things I can do better / different, do let me know in the comments. I’ve only used DataMapper for about 2 hours so some of my assumptions may be way of.

tags: l

Mollom gem news November 17th, 2008

Since last time I talked about the Mollom gem there have been some small changes to it. Not really worth a blog post, but I figured some people might be interested.

key_ok?

I had implemented the key_ok? function merely as a test-function for myself. Turns out people are using it to verify if the user-entered key is correct, so I made some changes so it actually returns true and false, instead of true and raising an error.

server_list

I’m querying 3 Mollom servers for a server list instead of just 1. This was an update to the API docs and makes sure that if a server fails we can still get a response from another server. I also added some documentation on setting a cached server_list (Yes, you should cache your server list!)

Ruby 1.9

It didn’t take a lot of effort, but the Mollom Gem is now Ruby 1.9 compatible.. Party Hats!

tags: l

libxslt-ruby and Ubuntu Dapper October 15th, 2008

If you try to install the gem libxslt-ruby on Ubuntu Dapper you will find yourself in a pickle, since it doesn’t co-operate. Given a default Dapper installation with apt-get’ed Ruby and rubygems 1.3, I got this:

root@dapper:~# gem install libxml-ruby
Building native extensions.  This could take a while...
Successfully installed libxml-ruby-0.8.3
1 gem installed
root@dapper:~# gem install libxslt-ruby
Building native extensions.  This could take a while...
ERROR:  Error installing libxslt-ruby:
  ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb install libxslt-ruby
Gem::SourceIndex#search support for String patterns is deprecated
extconf.rb:18 is outdated
checking for atan() in -lm... no
checking for atan() in -lm... yes
checking for inflate() in -lz... yes
checking for xmlXPtrNewRange() in -lxml2... yes
checking for libxml/xmlversion.h... no
checking for libxml/xmlversion.h... yes
checking for xsltApplyStylesheet() in -lxslt... yes
checking for xslt.h... no
checking for xslt.h... yes
checking for libxml/ruby_libxml.h... yes
checking for libxml/ruby_xml_document.h... yes
creating extconf.h
creating Makefile

make
gcc -fPIC -Wall -g -O2  -fPIC  -I. -I/usr/lib/ruby/1.8/i486-linux -I/usr/lib/ruby/1.8/i486-linux -I. -DHAVE_ZLIB_H -DHAVE_LIBXML_RUBY_LIBXML_H -DHAVE_LIBXML_RUBY_XML_DOCUMENT_H  -c libxslt.c
In file included from libxslt.c:5:
libxslt.h:10:27: error: libxml/parser.h: No such file or directory
libxslt.h:11:29: error: libxml/debugXML.h: No such file or directory
In file included from libxslt.h:12,
                 from libxslt.c:5:
/usr/include/libxslt/extra.h:14:26: error: libxml/xpath.h: No such file or directory
In file included from /usr/include/libxslt/extra.h:16,
                 from libxslt.h:12,
                 from libxslt.c:5:
/usr/include/libxslt/xsltInternals.h:16:25: error: libxml/tree.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:17:25: error: libxml/hash.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:19:29: error: libxml/xmlerror.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:20:25: error: libxml/dict.h: No such file or directory
In file included from /usr/include/libxslt/xsltInternals.h:23,
                 from /usr/include/libxslt/extra.h:16,
                 from libxslt.h:12,
                 from libxslt.c:5:
...
<snip>
...
In file included from libxslt.h:19,
                 from libxslt.c:5:
ruby_xslt_stylesheet.h:9:32: error: libxml/ruby_libxml.h: No such file or directory
ruby_xslt_stylesheet.h:10:38: error: libxml/ruby_xml_document.h: No such file or directory
In file included from libxslt.c:5:
libxslt.h:25:2: error: #error "Incompatible LibXML-Ruby headers - please install same major/micro version"
libxslt.c:6:31: error: libxml/xmlversion.h: No such file or directory
libxslt.c: In function 'Init_libxslt_ruby':
libxslt.c:40: error: 'LIBXML_TEST_VERSION' undeclared (first use in this function)
libxslt.c:40: error: (Each undeclared identifier is reported only once
libxslt.c:40: error: for each function it appears in.)
libxslt.c:50: error: 'xmlChar' undeclared (first use in this function)
libxslt.c:50: error: syntax error before ')' token
libxslt.c:54: error: syntax error before ')' token
libxslt.c:55: error: syntax error before ')' token
libxslt.c:56: error: syntax error before ')' token
libxslt.c:57: error: syntax error before ')' token
libxslt.c:58: error: syntax error before ')' token
make: *** [libxslt.o] Error 1


Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/libxslt-ruby-0.8.2 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/libxslt-ruby-0.8.2/ext/libxslt/gem_make.out

After a lot of searching the problem seemed to be with the ruby-version Dapper was running on. When I redid everything with a custom-built Ruby 1.8.6, it installed without a problem.

tags: l

RailsXLS Revived October 2nd, 2008

RailsXLS is a plugin to generate Excel files from a .rxls view. It uses a Java Bridge to talk to the Jakarta POI library. I’ve used this plugin in a few projects and it works great for generating .xls files.

On a recent project I once again needed an “Export to Excel” feature. The project is on Rails Edge, and apparently there were some changes to how TemplateHandlers are handled since the plugin was written. I dove into the code and recreated the plugin for Rails Edge / 2.2.

You can find the new plugin on github

It’s usage is quite easy:

# in your environment.rb
Mime::Type.register "application/excel", :xls

# in controller
def index
  @clients = Client.all
  respond_to do |speaks|
    speaks.html
    speaks.xls { render :layout => false }
  end
end

# in index.xls.rxls
sheet = workbook.createSheet("Client List")

@clients.each_with_index do |client, index|
  row = sheet.createRow(index)
  row.createCell(0).setCellValue(client.id)
  row.createCell(1).setCellValue(client.name)
end

As you can see there are still some rough patches in the view. I’m working on a wrapper around this, so all obvious tasks can be easily performed.

Any comments/suggestions for the plugin are very welcome! A big thanks to Venkata Subramaniyan for writing the initial plugin, which can be found here

tags: l

Do Not Forget: NamedScope and dates September 29th, 2008

One of the scopes I had in a project was from_this_month

class Transaction
  named_scope :from_this_month, :conditions => {:created_at => Time.now.beginning_of_month..Time.now}
  ...
end

Which worked great to know all transactions from a user in a certain month. I could do current_user.transactions.from_this_month. When I tested this in developer mode all went well.. Production, however, didn’t work at all.

As we all know, Developer mode reloads classes every time you make a request. Production mode obviously doesn’t, so in Production, the first time you load the page you get a correct time-scope, if you do it a day later, however, the scope is still the same of yesterday. This can be solved with a proc or a lambda, which will get called on each request!

So, just to make sure I never forget, Use a lambda!

class Transaction
  named_scope :from_this_month, lambda { {:conditions => {:created_at => Time.now.beginning_of_month..Time.now}} }
  ...
end
tags: l

Adding custom states to god September 18th, 2008

God is a monitoring-framework written in Ruby, with Ruby config files.

For one of our projects I’m using it to monitor some of the application processes (mongrels, a stirling, a juggernaut server, an engine). Most of these have a normal lifecycle: they get initialized, they start, they crash/go down, and god restarts them.

the engine process, however, has a different life-cycle. It should start when god does, but when it crashes, it has to stay offline, and I should receive an email saying it crashed. Then I could go in to inspect what made it crash. Apparently this isn’t included in god by default. Luckily, ruby’s open class infrastructure allows you to add items to constant-arrays, so you can just add your own states.

ENGINE_ROOT=PROJECT_ROOT + '/engine/'

# Add the stopped state
God::Watch::VALID_STATES << :stopped

God.watch do |w|
  w.name = "engine"

  w.interval = 5.seconds # default

  w.start = "#{ENGINE_ROOT}/bin/engine start"
  w.stop  = "#{ENGINE_ROOT}/bin/engine start"
  w.pid_file = "#{ENGINE_ROOT}/log/engine.pid"

..snip..
  # Just mail us if the process dies
  w.transition(:up, :stopped) do |on|
    on.condition(:process_running) do |c|
      c.interval = 3
      c.notify = {:contacts => ['openminds'], :priority => 'URGENT'}
      c.running = false
    end
  end
..snip..
end

And that’s how easy that is ;-)

tags: l

Time#upto September 17th, 2008

In of of our applications I had to create an XML file to generate a time-based graph. For this I needed the time in certain intervals, for example from 1 year ago to now with an interval of 1 day. I looked around but there were no obvious solutions, so I came up with my own!

# Rails convenience functions are used in the examples
>> 1.week.ago.upto(:now) { |day| puts day }
Wed Sep 10 14:58:42 +0200 2008
Thu Sep 11 14:58:42 +0200 2008
Fri Sep 12 14:58:42 +0200 2008
Sat Sep 13 14:58:42 +0200 2008
Sun Sep 14 14:58:42 +0200 2008
Mon Sep 15 14:58:42 +0200 2008
Tue Sep 16 14:58:42 +0200 2008
Wed Sep 17 14:58:42 +0200 2008

>> 5.hours.ago.upto(:now, 1.hour) { |day| puts day }
Wed Sep 17 09:59:24 +0200 2008
Wed Sep 17 10:59:24 +0200 2008
Wed Sep 17 11:59:24 +0200 2008
Wed Sep 17 12:59:24 +0200 2008
Wed Sep 17 13:59:24 +0200 2008
Wed Sep 17 14:59:24 +0200 2008

>> 5.hours.ago.upto(1.hour.ago, 1.hour) { |day| puts day }
Wed Sep 17 09:59:58 +0200 2008
Wed Sep 17 10:59:58 +0200 2008
Wed Sep 17 11:59:58 +0200 2008
Wed Sep 17 12:59:58 +0200 2008
Wed Sep 17 13:59:58 +0200 2008

The upto function does all the stupid time logic for you! If you want to use this function yourself, this is the code:

class Time
  def upto(date, step = 86400) # 1.day
    time = self.dup
    until_time = case date
    when :now
      Time.now
    else
      date
    end
    until time > until_time
      yield time
      time += step
    end
  end
end
tags: l