MySQL password hashing in Ruby May 12th, 2009
An old database used MySQL’s PASSWORD() hashing functionality for storing user credentials. We’re creating a Ruby app now to interface with that database, and wanted to hash the password in ruby. After looking at the code it looked like they just SHA1 hash the password twice and prepend a *. Implementation in Ruby is easy:
require 'digest/sha1' def hash_mysql_password pass "*" + Digest::SHA1.hexdigest(Digest::SHA1.digest(pass)).upcase end
Which gives us in MySQL
mysql> SELECT PASSWORD('foo');
+-------------------------------------------+
| PASSWORD('foo') |
+-------------------------------------------+
| *F3A2A51A9B0F2BE2468926B4132313728C250DBF |
+-------------------------------------------+
And in Ruby
>> hash_mysql_password 'foo' => "*F3A2A51A9B0F2BE2468926B4132313728C250DBF"
Custom ActiveRecord timestamps April 27th, 2009
Some legacy databases have already defined their own created_at or updated_at fields. They can be easily filled in with ActiveRecord before_save filters, but after implementing this behaviour a few times the urge arose to write something reusable, so I created the custom_timestamps plugin.
With custom_timestamps you can easily define the columns you want to be updated on creation/change
class LegacyModel < ActiveRecord::Base set_create_column :creation_time set_update_column :change_time end
The plugin can be found on github.
l 2 comments »ArrrrCamp Belgium April 2nd, 2009
We have been talking between ourselves about hosting a Belgian Ruby conference since the last Railsconf Europe, and we have finally decided to proceed with the idea. Colleague Joren put things in motion and today we’re proud to announce Arrrrcamp, which stands for “About Ruby, Rails, Radiant and Rum Camp”.
The event will take place on friday, May 8th, in Ghent, Belgium. The idea is to have a barcamp-like organisation of talks, where everybody is encouraged to participate by either doing talks, taking pictures, do some hacking, … You can read all about the event on the website.
l 0 comments »Radiant 0.7.x and Ruby 1.8.4 March 13th, 2009
A client of ours complained that Radiant 0.7.1 was not working out of the box on one of our shared servers. When running the radiant command he got an error:
/usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:199:in `set_load_path': undefined method `load_paths' for nil:NilClass (NoMethodError) from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:97:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:101:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:72:in `load_initializer' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:90:in `load_initializer' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:61:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:19:in `boot!' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:161 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/radiant:3 from /usr/bin/radiant:19
It worked fine on our other servers, as well as on our local machines, the only thing that was different was the ruby version, which was 1.8.4 on that specific server. Apparently that version handles super a bit different. The solution was to change 1 line in /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:
# In /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb line 101: - super + super(command, configuration)
Application-wide timeranges February 2nd, 2009
In some applications you need to work with time-ranges a lot. Displaying todays messages, a count of all messages that were posted this month, or last month, … ActiveRecord has pretty good support for times and ranges, so you could do it the default way.
Message.find(:all, :conditions => {:created_at => Time.now.beginning_of_month..Time.end_of_month})
And although this works perfectly it’s not very easy on the eyes. It’s also not very DRY since I need this on a lot of models. It’d be a lot better if I could have it in a scope, like this
Message.in(:this_month)
So how do we achieve this?
In our config/initializer folder we have a file called timeranges.rb.
# timeranges.rb TimeRanges = { :today => lambda { Time.now.beginning_of_day..Time.now.end_of_day }, :yesterday => lambda { 1.day.ago.beginning_of_day..1.day.ago.end_of_day }, :this_week => lambda { Time.now.beginning_of_week..Time.now.end_of_week }, :last_week => lambda { 1.week.ago.beginning_of_week..1.week.ago.end_of_week}, :this_month => lambda { Time.now.beginning_of_month..Time.now.end_of_month }, :last_month => lambda { 1.month.ago.beginning_of_month..1.month.ago.end_of_month } }
Which gives us a constant TimeRanges we can use all over the application. So in our Message we define the named_scope in
class Message < ActiveRecord::Base named_scope :in, lambda { |period| :conditions => {:created_at => TimeRanges[period].call} } end
Now we could even make this more DRY and make a module that defines the named_scope when it’s included, but in my application I had a lot of different use-cases for the TimeRanges, and didn’t really need the scope a lot. Leave me a note if you want it in a module but don’t know how to, else it’s an exercise for the interested and easily challenged reader ;-)
l 0 comments »Using Datamapper on legacy databases December 18th, 2008
Yesterday I had to map a legacy database schema to some Ruby classes. Just before that I had been looking into the advantages of DataMapper, and it looked like the perfect match for my current “challenge”.
from the DataMapper site:
DataMapper only issues updates or creates for the properties it knows about. So it plays well with others. You can use it in an Integration Database without worrying that your application will be a bad actor causing trouble for all of your other processes.
The system
The system is running with 2 databases. One database holds all the information for the legacy app, the other database holds all information for system management.
mysql app mysql> desc cpClients; +--------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+----------------+ | clientID | int(255) | NO | PRI | NULL | auto_increment | | login | varchar(255) | NO | UNI | | | | pass | varchar(255) | NO | | | | | type | int(2) | NO | | 0 | | | traffic | bigint(255) | NO | | 0 | | <snip> | VAT | varchar(255) | YES | | NULL | | +--------------+--------------+------+-----+---------+----------------+ mysql> desc cpURL; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | url | varchar(255) | NO | PRI | | | | clientID | int(255) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ > exit mysql management mysql> desc ftp_users; +--------------+------------------+------+-----+-------------------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+------------------+------+-----+-------------------+-------+ | username | varchar(60) | NO | PRI | NULL | | | password | varchar(30) | YES | | NULL | | <snip> | homedir | text | YES | | NULL | | | shell | varchar(15) | NO | | /bin/false | | +--------------+------------------+------+-----+-------------------+-------+
One Client has many URL’s, One client also has many ftp_users, but the link in that can’t be captured in a simple has many relationship.
Solving it with DataMapper
Setting up the connections
Setting up 2 connections to the databases was quite easy and well documented. I chose the app database to be the default connection, and management as another.
DataMapper.setup(:default, 'mysql://localhost/app') DataMapper.setup(:management, 'mysql://localhost/management')
Having different database names
The first problem I had was that I wanted the cpClient table to be mapped to the User class, and cpURL to be mapped to a Domain class. This can be solved with storage_names.
class User include DataMapper::Resource storage_names[:default] = 'cpClients' end class Domain include DataMapper::Resource storage_names[:default] = 'cpURL' end
Defining primary keys
Defining the primary key of my cpURL table was quite easy since DataMapper does natural keys out of the box.
class Domain include DataMapper::Resource storage_names[:default] = 'cpURL' property :url, String, :key => true end
Defining the primary key of the cpClients table was a tad more difficult since it was poorly documented. I wanted the property to be called ID, but it should still use clientID in the database. The magic option to this is :field, which allows you to set a custom field for a propery.
class User include DataMapper::Resource storage_names[:default] = 'cpClients' property :id, Serial, :field => 'clientID' end
Defining associations
The hardest part was defining associations. When I’d just define them with has n without extra options, DataMapper would look for user_id, so I’d have to tell which key was used. Since it wasn’t very well documented I had some failed attempts
has n, :domains, :child_key => 'clientID' # => +options[:child_key]+ should be Array, but was String (ArgumentError) has n, :domains, :child_key => ['clientID'] # => ArgumentError: +name+ should be Symbol, but was String has n, :domains, :child_key => [:clientID] # => MysqlError: Unknown column 'client_i_d' in 'field list' (mysql_error_code=0001)
So I need to use a symbol, but apparently DataMapper does some automatic magic to translate keys. When you have a key like me you can disable this magic by changing field_naming_convention’s proc to your own. In the proc we return the stringified value’s name (to_s is needed because the key is given as a symbol)
repository(:default).adapter.field_naming_convention = lambda { |value| value.name.to_s }
And then the association worked. Do note the :child_key option needs to be set at both sides.
class User has n, :domains, :child_key => [:clientID] end class Domain belongs_to :user, :child_key => [:clientID] end
Setting models to automatically connect to another database
The FtpUser model automatically needs to connect to the :management database, and there doesn’t seem to be a function to set this. If you temporarily want to connect to another database you can do it with the repository command.
repository(:management) { FtpUser.first }
However, if you want a model to always connect to a database other then the default, defining self.default_repository_name with the correct repository works very well. (Databases are referred to as repositories)
class FtpUser include DataMapper::Resource def self.default_repository_name :services end end
All of this was needed to get my legacy database mapped, I hope some of you find this information useful. If you see things I can do better / different, do let me know in the comments. I’ve only used DataMapper for about 2 hours so some of my assumptions may be way of.
l 1 comments »small less-tip December 3rd, 2008
When you’re using less to view your Rails log, and you want your colours back, you can use less with the -R option, or you can automate this by setting the $LESS variable in your environment. So for a bash environment:
# ~/.profile export LESS="-R"
Update!
Since setting $LESS in your environment affects git commands like status (git status automatically uses less) I updated by profile a bit:
# ~/.profile # Disabled: Fucks up git status # export LESS="-R" alias less="less -R"
Mollom gem news November 17th, 2008
Since last time I talked about the Mollom gem there have been some small changes to it. Not really worth a blog post, but I figured some people might be interested.
key_ok?
I had implemented the key_ok? function merely as a test-function for myself. Turns out people are using it to verify if the user-entered key is correct, so I made some changes so it actually returns true and false, instead of true and raising an error.
server_list
I’m querying 3 Mollom servers for a server list instead of just 1. This was an update to the API docs and makes sure that if a server fails we can still get a response from another server. I also added some documentation on setting a cached server_list (Yes, you should cache your server list!)
Ruby 1.9
It didn’t take a lot of effort, but the Mollom Gem is now Ruby 1.9 compatible.. Party Hats!
l 0 comments »Testing Forgery Protection November 3rd, 2008
Try to follow here:
I needed to test 1 controller to see if he was not protected from forgery protection, because requests to that controller come from an external source.
So basically I need to test if my code has
skip_before_filter :verify_authenticity_token
In the test environment, the authenticity_token check is disabled. You can re-enable it in your test like this:
class AccountsControllerTest < ActionController::TestCase def setup super AccountsController.allow_forgery_protection = true # Make sure we have forgery protection before filter turned off end end
When you do a post and you haven’t skipped the filter, the test will fail.
l 1 comments »session_lifetime plugin October 30th, 2008
One of the things I take for granted on the internet is session expiry. I’m used to the fact that if I’m not active for a certain amount of time, I automatically get logged out. This happens with my Online Banking, with ordering tickets online, …. Most information-critical applications have this.
Because of that, I was surprised that Rails doesn’t have this functionality on board. There is no way to set an expiry date on your session. I googled a bit and stumbled upon the dynamic session exp plugin. This gives you the possibility to expire your session through the cookie’s expires-flag.
An example
# in environment.rb CGI::Session.expire_after 1.day
This plugin worked fine until I wanted to send a message to the webuser, informing him why this happened. Because we’re working with the expire-setting in a cookie, this isn’t possible. Because of that, I wrote my own plugin that gave me that possibility.
With session_lifetime you can set after how much time of inactivity your session should expire, you can execute an action when the session expires, and you can set where to redirect_to after session expiry.
An example
class ApplicationController << ActionController::Base expires_session :time => 2.hours, :redirect_to => '/login' protected def on_expiry flash[:notice] = "Your session has been expired, and you have been logged out." end end
More information can be found on github.
Missing something?
I wrote this plugin to solve the problems I had with default Rails session handling. If you have an additional need which you think would be great for this plugin, give me a shout at sessionlifetime@defv.be, or through GitHub, and I’ll be more then happy to implement your proposal.
l 4 comments »The Rumble October 27th, 2008
The rumble’s been over for a whole week now, and reflecting on it, we had a very good time and it was a lot of fun. Some things could’ve gone better, but it was good overall. The application we made is Locatr.
More information about voting and about locatr itself can be found on our Rumble Team Page
And some rumblin music:
l 0 comments »RailsRumble Preparations October 16th, 2008
This week I will be competing in the RailsRumble, which is a competition to write a Ruby on Rails application in 48hours or less. We’ve been doing some preparations, like drawing out the resources we’ll have, thinking out every “difficult” part, inspecting APIs we don’t know yet, and, most importantly, arranging everything about catering and such (Which will be handled by Openminds boss Frank, thank you Frank!).
I can’t tell you a lot about the application we’ll be developing yet, but I think it’ll be quite cool (I hope!). This evening we have a last meetup with the whole team before the Rumble starts, and there are still a lot of things to be decided.
If you want to follow us this weekend we’ve also set up a twitter account on which we’ll be posting all the cool things we do, problems we encounter, some sneak previews of the application, and some pictures of the time we spend at the office.
l 0 comments »libxslt-ruby and Ubuntu Dapper October 15th, 2008
If you try to install the gem libxslt-ruby on Ubuntu Dapper you will find yourself in a pickle, since it doesn’t co-operate. Given a default Dapper installation with apt-get’ed Ruby and rubygems 1.3, I got this:
root@dapper:~# gem install libxml-ruby
Building native extensions. This could take a while...
Successfully installed libxml-ruby-0.8.3
1 gem installed
root@dapper:~# gem install libxslt-ruby
Building native extensions. This could take a while...
ERROR: Error installing libxslt-ruby:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.8 extconf.rb install libxslt-ruby
Gem::SourceIndex#search support for String patterns is deprecated
extconf.rb:18 is outdated
checking for atan() in -lm... no
checking for atan() in -lm... yes
checking for inflate() in -lz... yes
checking for xmlXPtrNewRange() in -lxml2... yes
checking for libxml/xmlversion.h... no
checking for libxml/xmlversion.h... yes
checking for xsltApplyStylesheet() in -lxslt... yes
checking for xslt.h... no
checking for xslt.h... yes
checking for libxml/ruby_libxml.h... yes
checking for libxml/ruby_xml_document.h... yes
creating extconf.h
creating Makefile
make
gcc -fPIC -Wall -g -O2 -fPIC -I. -I/usr/lib/ruby/1.8/i486-linux -I/usr/lib/ruby/1.8/i486-linux -I. -DHAVE_ZLIB_H -DHAVE_LIBXML_RUBY_LIBXML_H -DHAVE_LIBXML_RUBY_XML_DOCUMENT_H -c libxslt.c
In file included from libxslt.c:5:
libxslt.h:10:27: error: libxml/parser.h: No such file or directory
libxslt.h:11:29: error: libxml/debugXML.h: No such file or directory
In file included from libxslt.h:12,
from libxslt.c:5:
/usr/include/libxslt/extra.h:14:26: error: libxml/xpath.h: No such file or directory
In file included from /usr/include/libxslt/extra.h:16,
from libxslt.h:12,
from libxslt.c:5:
/usr/include/libxslt/xsltInternals.h:16:25: error: libxml/tree.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:17:25: error: libxml/hash.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:19:29: error: libxml/xmlerror.h: No such file or directory
/usr/include/libxslt/xsltInternals.h:20:25: error: libxml/dict.h: No such file or directory
In file included from /usr/include/libxslt/xsltInternals.h:23,
from /usr/include/libxslt/extra.h:16,
from libxslt.h:12,
from libxslt.c:5:
...
<snip>
...
In file included from libxslt.h:19,
from libxslt.c:5:
ruby_xslt_stylesheet.h:9:32: error: libxml/ruby_libxml.h: No such file or directory
ruby_xslt_stylesheet.h:10:38: error: libxml/ruby_xml_document.h: No such file or directory
In file included from libxslt.c:5:
libxslt.h:25:2: error: #error "Incompatible LibXML-Ruby headers - please install same major/micro version"
libxslt.c:6:31: error: libxml/xmlversion.h: No such file or directory
libxslt.c: In function 'Init_libxslt_ruby':
libxslt.c:40: error: 'LIBXML_TEST_VERSION' undeclared (first use in this function)
libxslt.c:40: error: (Each undeclared identifier is reported only once
libxslt.c:40: error: for each function it appears in.)
libxslt.c:50: error: 'xmlChar' undeclared (first use in this function)
libxslt.c:50: error: syntax error before ')' token
libxslt.c:54: error: syntax error before ')' token
libxslt.c:55: error: syntax error before ')' token
libxslt.c:56: error: syntax error before ')' token
libxslt.c:57: error: syntax error before ')' token
libxslt.c:58: error: syntax error before ')' token
make: *** [libxslt.o] Error 1
Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/libxslt-ruby-0.8.2 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/libxslt-ruby-0.8.2/ext/libxslt/gem_make.out
After a lot of searching the problem seemed to be with the ruby-version Dapper was running on. When I redid everything with a custom-built Ruby 1.8.6, it installed without a problem.
l 0 comments »RailsXLS Revived October 2nd, 2008
RailsXLS is a plugin to generate Excel files from a .rxls view. It uses a Java Bridge to talk to the Jakarta POI library. I’ve used this plugin in a few projects and it works great for generating .xls files.
On a recent project I once again needed an “Export to Excel” feature. The project is on Rails Edge, and apparently there were some changes to how TemplateHandlers are handled since the plugin was written. I dove into the code and recreated the plugin for Rails Edge / 2.2.
You can find the new plugin on github
It’s usage is quite easy:
# in your environment.rb Mime::Type.register "application/excel", :xls # in controller def index @clients = Client.all respond_to do |speaks| speaks.html speaks.xls { render :layout => false } end end # in index.xls.rxls sheet = workbook.createSheet("Client List") @clients.each_with_index do |client, index| row = sheet.createRow(index) row.createCell(0).setCellValue(client.id) row.createCell(1).setCellValue(client.name) end
As you can see there are still some rough patches in the view. I’m working on a wrapper around this, so all obvious tasks can be easily performed.
Any comments/suggestions for the plugin are very welcome! A big thanks to Venkata Subramaniyan for writing the initial plugin, which can be found here
l 12 comments »Removing a remote branch in git September 30th, 2008
Removing a remote branch in git is fairly simple, although not very well documented:
jan:project jan$ git push origin :old_branch To git@openminds.be:project.git - [deleted] old_branch