Compiling mongodb 'stable' as a debian lenny package January 14th, 2010
Today I had the pleasure of having to install MongoDb on one of our production servers. Pleasure because MongoDb is an awesome document database, and I also like new things.
Most of our production servers run Debian, and we try to keep all installed packages up to date with apt. Since Debian Lenny didn’t have a mongodb package (it does have a lot of other packages Mongolian language packs) I had to make my own. Luckily the mongodb git repository comes with a ‘debian’ folder, so making a .deb package should be easy as pie.
There are quite a few gotchas to get it to build, though. This is what you’ll need to do:
Get last mongodb ‘stable’ source
git clone git://github.com/mongodb/mongo.git cd mongo git checkout r1.2.1
The debian/control file lists all deps including one we can’t fulfill on Lenny, xulrunner-1.9-dev. We only have xulrunner-dev. So change the debian/control file:
# the Build-Depends line
-Build-Depends: debhelper (>= 7), libboost-dev, libpcre3, libpcre3-dev, scons, xulrunner-1.9-dev, libboost-thread-dev, libboost-filesystem-dev, libboost-program-options-dev, libboost-date-time-dev
+Build-Depends: debhelper (>= 7), libboost-dev, libpcre3, libpcre3-dev, scons, xulrunner-dev, libboost-thread-dev, libboost-filesystem-dev, libboost-program-options-dev, libboost-date-time-dev
# the Depends line
-Depends: ${shlibs:Depends}, ${misc:Depends}, xulrunner-1.9-dev
+Depends: ${shlibs:Depends}, ${misc:Depends}, xulrunner-dev
Also, be sure to install all these build-deps
apt-get install debhelper libboost-dev libpcre3 libpcre3-dev scons xulrunner-dev libboost-thread-dev libboost-filesystem-dev libboost-program-options-dev libboost-date-time-dev
Next up, we’ll need to change a script (buildscripts/hacks_ubuntu.py) which sets old-xulrunner-compatibility. Apparently the bug I encountered was already fixed but didn’t work on my system. I just deleted the condition since I was sure I needed the old-compatability mode.
diff --git a/buildscripts/hacks_ubuntu.py b/buildscripts/hacks_ubuntu.py
index 67c5d78..baebfe6 100644
--- a/buildscripts/hacks_ubuntu.py
+++ b/buildscripts/hacks_ubuntu.py
@@ -42,6 +42,5 @@ def foundxulrunner( env , options ):
incroot + "unstable/" ] )
env.Append( CPPDEFINES=[ "XULRUNNER" , "OLDJS" ] )
- if best.find( "1.9.0" ) >= 0:
- env.Append( CPPDEFINES=[ "XULRUNNER190" ] )
+ env.Append( CPPDEFINES=[ "XULRUNNER190" ] )
return True
That’s all changes needed to get it to compile. Next step is running dpkg-buildpackage and installing the package!
dpkg-buildpackage -rfakeroot -b # <it builds..> dpkg -i ../mongodb_1.2.1*.deb
Ruby and Rails beginner talk October 8th, 2009
Last night I gave a talk for Zeus about Ruby and Rails, talking about Ruby fundamentals and a short intro to the Rails philosophy and MVC.
I don’t know if you’ll be much with the slides without hearing the talk, but you can find them on http://defv.be/ruby.and.rails.pdf.
If you want to see the entire presentation you should come to ArrrrCamp, where I’ll be giving it again.
l 0 comments »Ruby Hash coolness September 28th, 2009
Don’t shoot me if you already use this every day, but this is new to me.
In the past I’ve often had code like this
@hosts = {} @accounts.each do |account| @hosts[account[:host]] ||= [] @hosts[account[:host]] << account end
But why do this hen you can just add some initializer code to your hash.
@hosts = Hash.new { |h, k| h[k] = [] } @accounts.each do |account| @hosts[account[:host]] << account end
Where Hash automatically invokes the block when an unknown hash key is called. Great!
l 1 comments »Autoload JQuery Plugins August 5th, 2009
Rails has the convenient helper javascript_include_tag where you can give up :default, which automatically loads prototype / effects, or with the jrails plugin adds jquery, jquery-ui and jrails.
But when using jQuery you want to use more than those files, you want to use jQuery plugins, or write your own, all in small .
Loading this can be a pain in the ass. Thats why I wrote this little initializer which adds everything in public/javascripts/jquery/*.js to the defaults.
# config/initializers/add_jquery_to_defaults.rb Dir[File.join(ActionView::Helpers::AssetTagHelper::JAVASCRIPTS_DIR,'jquery','*.js')].each do |js| ActionView::Helpers::AssetTagHelper.register_javascript_include_default "jquery/#{File.basename(js)}" end
Rack::Cache and Lighttpd+FCGI July 15th, 2009
On some of the Openminds shared hosting servers we run a Lighttpd+FCGI stack for Ruby/Rails applications. This week we got a complaint from a user saying they had a problem with the latest Radiant 0.8. Every page he loaded was the same as the first page he opened since the last server restart.
When digging into the problem it was clear that it was a problem with Rack::Cache, which only created 1 meta-store entry and served that from there on. On investigation Rack::Cache::Key.call(@request) always returned http://example.com:80/dispatch.fcgi?. Apparently the last part of the key gets created by scriptname and pathinfo:
..snip..
parts << @request.script_name
parts << @request.path_info
..snip..
When looking at the request it seems Lighttpd+FCGI doesn’t fill in the env[‘PATH_INFO’], which results in always having the same cache key, thus the same page.
To solve this problem I’ve created a small Rack middleware that sets PATHINFO from REQUESTURI (Like Passenger does with Nginx)
module ::Rack class LighttpdFix def initialize(app) @app = app end def call(env) env['PATH_INFO'] = env['REQUEST_URI'].split('?', 2).first @app.call(env) end end end # in production.rb config.middleware.insert_before ::Rack::Cache, ::Rack::LighttpdFix # or for Radiant config.middleware.insert_before ::Radiant::Cache, ::Rack::LighttpdFix
I’ll make a gem out of this tomorrow.I made a gem out of this, you can find it on GitHub and install it using RubyGems.
sudo gem install DefV-rack_lighttpd_fix
MySQL password hashing in Ruby May 12th, 2009
An old database used MySQL’s PASSWORD() hashing functionality for storing user credentials. We’re creating a Ruby app now to interface with that database, and wanted to hash the password in ruby. After looking at the code it looked like they just SHA1 hash the password twice and prepend a *. Implementation in Ruby is easy:
require 'digest/sha1' def hash_mysql_password pass "*" + Digest::SHA1.hexdigest(Digest::SHA1.digest(pass)).upcase end
Which gives us in MySQL
mysql> SELECT PASSWORD('foo');
+-------------------------------------------+
| PASSWORD('foo') |
+-------------------------------------------+
| *F3A2A51A9B0F2BE2468926B4132313728C250DBF |
+-------------------------------------------+
And in Ruby
>> hash_mysql_password 'foo' => "*F3A2A51A9B0F2BE2468926B4132313728C250DBF"
Custom ActiveRecord timestamps April 27th, 2009
Some legacy databases have already defined their own created_at or updated_at fields. They can be easily filled in with ActiveRecord before_save filters, but after implementing this behaviour a few times the urge arose to write something reusable, so I created the custom_timestamps plugin.
With custom_timestamps you can easily define the columns you want to be updated on creation/change
class LegacyModel < ActiveRecord::Base set_create_column :creation_time set_update_column :change_time end
The plugin can be found on github.
l 2 comments »ArrrrCamp Belgium April 2nd, 2009
We have been talking between ourselves about hosting a Belgian Ruby conference since the last Railsconf Europe, and we have finally decided to proceed with the idea. Colleague Joren put things in motion and today we’re proud to announce Arrrrcamp, which stands for “About Ruby, Rails, Radiant and Rum Camp”.
The event will take place on friday, May 8th, in Ghent, Belgium. The idea is to have a barcamp-like organisation of talks, where everybody is encouraged to participate by either doing talks, taking pictures, do some hacking, … You can read all about the event on the website.
l 0 comments »Radiant 0.7.x and Ruby 1.8.4 March 13th, 2009
A client of ours complained that Radiant 0.7.1 was not working out of the box on one of our shared servers. When running the radiant command he got an error:
/usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:199:in `set_load_path': undefined method `load_paths' for nil:NilClass (NoMethodError) from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/vendor/rails/railties/lib/initializer.rb:97:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:101:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:72:in `load_initializer' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:90:in `load_initializer' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:61:in `run' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:19:in `boot!' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/../config/boot.rb:161 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/bin/radiant:3 from /usr/bin/radiant:19
It worked fine on our other servers, as well as on our local machines, the only thing that was different was the ruby version, which was 1.8.4 on that specific server. Apparently that version handles super a bit different. The solution was to change 1 line in /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb:
# In /usr/lib/ruby/gems/1.8/gems/radiant-0.7.1/lib/radiant/initializer.rb line 101: - super + super(command, configuration)
Application-wide timeranges February 2nd, 2009
In some applications you need to work with time-ranges a lot. Displaying todays messages, a count of all messages that were posted this month, or last month, … ActiveRecord has pretty good support for times and ranges, so you could do it the default way.
Message.find(:all, :conditions => {:created_at => Time.now.beginning_of_month..Time.end_of_month})
And although this works perfectly it’s not very easy on the eyes. It’s also not very DRY since I need this on a lot of models. It’d be a lot better if I could have it in a scope, like this
Message.in(:this_month)
So how do we achieve this?
In our config/initializer folder we have a file called timeranges.rb.
# timeranges.rb TimeRanges = { :today => lambda { Time.now.beginning_of_day..Time.now.end_of_day }, :yesterday => lambda { 1.day.ago.beginning_of_day..1.day.ago.end_of_day }, :this_week => lambda { Time.now.beginning_of_week..Time.now.end_of_week }, :last_week => lambda { 1.week.ago.beginning_of_week..1.week.ago.end_of_week}, :this_month => lambda { Time.now.beginning_of_month..Time.now.end_of_month }, :last_month => lambda { 1.month.ago.beginning_of_month..1.month.ago.end_of_month } }
Which gives us a constant TimeRanges we can use all over the application. So in our Message we define the named_scope in
class Message < ActiveRecord::Base named_scope :in, lambda { |period| :conditions => {:created_at => TimeRanges[period].call} } end
Now we could even make this more DRY and make a module that defines the named_scope when it’s included, but in my application I had a lot of different use-cases for the TimeRanges, and didn’t really need the scope a lot. Leave me a note if you want it in a module but don’t know how to, else it’s an exercise for the interested and easily challenged reader ;-)
l 0 comments »Using Datamapper on legacy databases December 18th, 2008
Yesterday I had to map a legacy database schema to some Ruby classes. Just before that I had been looking into the advantages of DataMapper, and it looked like the perfect match for my current “challenge”.
from the DataMapper site:
DataMapper only issues updates or creates for the properties it knows about. So it plays well with others. You can use it in an Integration Database without worrying that your application will be a bad actor causing trouble for all of your other processes.
The system
The system is running with 2 databases. One database holds all the information for the legacy app, the other database holds all information for system management.
mysql app mysql> desc cpClients; +--------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+----------------+ | clientID | int(255) | NO | PRI | NULL | auto_increment | | login | varchar(255) | NO | UNI | | | | pass | varchar(255) | NO | | | | | type | int(2) | NO | | 0 | | | traffic | bigint(255) | NO | | 0 | | <snip> | VAT | varchar(255) | YES | | NULL | | +--------------+--------------+------+-----+---------+----------------+ mysql> desc cpURL; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | url | varchar(255) | NO | PRI | | | | clientID | int(255) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ > exit mysql management mysql> desc ftp_users; +--------------+------------------+------+-----+-------------------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+------------------+------+-----+-------------------+-------+ | username | varchar(60) | NO | PRI | NULL | | | password | varchar(30) | YES | | NULL | | <snip> | homedir | text | YES | | NULL | | | shell | varchar(15) | NO | | /bin/false | | +--------------+------------------+------+-----+-------------------+-------+
One Client has many URL’s, One client also has many ftp_users, but the link in that can’t be captured in a simple has many relationship.
Solving it with DataMapper
Setting up the connections
Setting up 2 connections to the databases was quite easy and well documented. I chose the app database to be the default connection, and management as another.
DataMapper.setup(:default, 'mysql://localhost/app') DataMapper.setup(:management, 'mysql://localhost/management')
Having different database names
The first problem I had was that I wanted the cpClient table to be mapped to the User class, and cpURL to be mapped to a Domain class. This can be solved with storage_names.
class User include DataMapper::Resource storage_names[:default] = 'cpClients' end class Domain include DataMapper::Resource storage_names[:default] = 'cpURL' end
Defining primary keys
Defining the primary key of my cpURL table was quite easy since DataMapper does natural keys out of the box.
class Domain include DataMapper::Resource storage_names[:default] = 'cpURL' property :url, String, :key => true end
Defining the primary key of the cpClients table was a tad more difficult since it was poorly documented. I wanted the property to be called ID, but it should still use clientID in the database. The magic option to this is :field, which allows you to set a custom field for a propery.
class User include DataMapper::Resource storage_names[:default] = 'cpClients' property :id, Serial, :field => 'clientID' end
Defining associations
The hardest part was defining associations. When I’d just define them with has n without extra options, DataMapper would look for user_id, so I’d have to tell which key was used. Since it wasn’t very well documented I had some failed attempts
has n, :domains, :child_key => 'clientID' # => +options[:child_key]+ should be Array, but was String (ArgumentError) has n, :domains, :child_key => ['clientID'] # => ArgumentError: +name+ should be Symbol, but was String has n, :domains, :child_key => [:clientID] # => MysqlError: Unknown column 'client_i_d' in 'field list' (mysql_error_code=0001)
So I need to use a symbol, but apparently DataMapper does some automatic magic to translate keys. When you have a key like me you can disable this magic by changing field_naming_convention’s proc to your own. In the proc we return the stringified value’s name (to_s is needed because the key is given as a symbol)
repository(:default).adapter.field_naming_convention = lambda { |value| value.name.to_s }
And then the association worked. Do note the :child_key option needs to be set at both sides.
class User has n, :domains, :child_key => [:clientID] end class Domain belongs_to :user, :child_key => [:clientID] end
Setting models to automatically connect to another database
The FtpUser model automatically needs to connect to the :management database, and there doesn’t seem to be a function to set this. If you temporarily want to connect to another database you can do it with the repository command.
repository(:management) { FtpUser.first }
However, if you want a model to always connect to a database other then the default, defining self.default_repository_name with the correct repository works very well. (Databases are referred to as repositories)
class FtpUser include DataMapper::Resource def self.default_repository_name :services end end
All of this was needed to get my legacy database mapped, I hope some of you find this information useful. If you see things I can do better / different, do let me know in the comments. I’ve only used DataMapper for about 2 hours so some of my assumptions may be way of.
l 1 comments »small less-tip December 3rd, 2008
When you’re using less to view your Rails log, and you want your colours back, you can use less with the -R option, or you can automate this by setting the $LESS variable in your environment. So for a bash environment:
# ~/.profile export LESS="-R"
Update!
Since setting $LESS in your environment affects git commands like status (git status automatically uses less) I updated by profile a bit:
# ~/.profile # Disabled: Fucks up git status # export LESS="-R" alias less="less -R"
Mollom gem news November 17th, 2008
Since last time I talked about the Mollom gem there have been some small changes to it. Not really worth a blog post, but I figured some people might be interested.
key_ok?
I had implemented the key_ok? function merely as a test-function for myself. Turns out people are using it to verify if the user-entered key is correct, so I made some changes so it actually returns true and false, instead of true and raising an error.
server_list
I’m querying 3 Mollom servers for a server list instead of just 1. This was an update to the API docs and makes sure that if a server fails we can still get a response from another server. I also added some documentation on setting a cached server_list (Yes, you should cache your server list!)
Ruby 1.9
It didn’t take a lot of effort, but the Mollom Gem is now Ruby 1.9 compatible.. Party Hats!
l 0 comments »Testing Forgery Protection November 3rd, 2008
Try to follow here:
I needed to test 1 controller to see if he was not protected from forgery protection, because requests to that controller come from an external source.
So basically I need to test if my code has
skip_before_filter :verify_authenticity_token
In the test environment, the authenticity_token check is disabled. You can re-enable it in your test like this:
class AccountsControllerTest < ActionController::TestCase def setup super AccountsController.allow_forgery_protection = true # Make sure we have forgery protection before filter turned off end end
When you do a post and you haven’t skipped the filter, the test will fail.
l 1 comments »session_lifetime plugin October 30th, 2008
One of the things I take for granted on the internet is session expiry. I’m used to the fact that if I’m not active for a certain amount of time, I automatically get logged out. This happens with my Online Banking, with ordering tickets online, …. Most information-critical applications have this.
Because of that, I was surprised that Rails doesn’t have this functionality on board. There is no way to set an expiry date on your session. I googled a bit and stumbled upon the dynamic session exp plugin. This gives you the possibility to expire your session through the cookie’s expires-flag.
An example
# in environment.rb CGI::Session.expire_after 1.day
This plugin worked fine until I wanted to send a message to the webuser, informing him why this happened. Because we’re working with the expire-setting in a cookie, this isn’t possible. Because of that, I wrote my own plugin that gave me that possibility.
With session_lifetime you can set after how much time of inactivity your session should expire, you can execute an action when the session expires, and you can set where to redirect_to after session expiry.
An example
class ApplicationController << ActionController::Base expires_session :time => 2.hours, :redirect_to => '/login' protected def on_expiry flash[:notice] = "Your session has been expired, and you have been logged out." end end
More information can be found on github.
Missing something?
I wrote this plugin to solve the problems I had with default Rails session handling. If you have an additional need which you think would be great for this plugin, give me a shout at sessionlifetime@defv.be, or through GitHub, and I’ll be more then happy to implement your proposal.
l 4 comments »