Article RSS Feed

MiniTest, Rails functional tests and Devise

Wed, 04 Jul 2012 04:34:36 GMT

MiniTest, Rails functional tests and Devise

Here's how I got MiniTest specs running on Rails and Devise, it pieces together various helpers and other bits of info I found with the google.

Padrino CMS Template

Tue, 28 Feb 2012 00:00:00 GMT

Padrino CMS Template

The idea behind this cms template is to give the padrino user a quick and simple way to populate content and build pages, menus, sidebars etc. without losing any of the versatility of Padrino. This is achieved by setting a low priority route which maps to '/*path', the path parameter is then used to retrieve content.

I intend to package this as a gem when I have a few more items ticked off the TODO list as there are some limitations at the moment (the orm is hardwired as ActiveRecord and rendering as HAML) but as a first draft it's usable. I've generated the base of Mathew Staplehurst's CBT Website using it.

The project is available on Github here. Any issues/suggestions are most welcome.

blog comments powered by Disqus

Hotrod Hayride

Mon, 01 Aug 2011 00:00:00 GMT

Wall of Death

No suspension, no padding, fuel tank frame.

Two Lane Blacktop

Most Rated

Thu, 28 Jul 2011 00:00:00 GMT

Check out MostRated.com

ActionView date_select calendar logic in javascript

Fri, 24 Jun 2011 00:00:00 GMT

ActionView date_select calendar logic in javascript

Here's the Gist

Javascript bitwise encoding

Fri, 17 Jun 2011 00:00:00 GMT

Here's a quick sample of how to do bitwise encoding in javascript:

Independent Stove Installations

Sat, 19 Feb 2011 00:00:00 GMT

Independent Stove Installations

A good friend of mine from Bristol has trained as a HETAS engineer which means he is able to install wood burning and multi fuel stoves to the required legal standard for UK building regulations.

With heating costs climbing and bitter winter weather it's a shrewd move training in this field, plus there's nothing like a good fire.

Naturally every business needs a website and that's where I come in, I put together Independent Stove Installations another Heroku & Radiant website.

I've used the Radiant gallery extension to show the various installations ISI have completed.

RubyProf, Oink and ActiveRecord bloat

Fri, 03 Dec 2010 03:26:54 GMT

RubyProf, Oink and ActiveRecord bloat

Posted by Steve on December 03, 2010|Comments

There are some applications that are always gonna hammer the OS, I'm not talking about your common-or-garden web app, think huge amounts of I/O in the shape of huge file uploads, image processing, iso creation, hungry hungry hungry.

So what to do when you see ruby processes (spawned from passenger) gobbling up 1gb of RAM? The app was already in production so we had to be careful in our approach to profiling and code changes. Time to grab the production db and associated file data and see if the problem could be reproduced locally.

I'd say this is the hard part of profiling, where to start looking in a fairly large in-production app? A bit f common sense helped here, we knew that there were some expensive processes mainly dealing with large amounts of image processing and large amounts of copying files. My very experienced colleague pointed me at this article detailing common mistakes made when developing with active record.

So a quick search through the codebase for all.each and find(:all as mentioned in the article lit up some hotspots in the app code where potentially we could be experiencing bloat.

Next step was to wrap these code blocks in a ruby profiler block as detailed here then make the app execute the profiled code block - no changes to the code yet - this is just to get a benchmark.

Now we made a small but apparently very important change removing a programmatic filter on a large collection (one of those all.each... buggers) and swapped in a more precise active record find with conditions and some outer joins to get us the desired collection. The app was restarted and the same test run again leaving the profiling block in place and bingo processing time was one sixth of the previous run, big improvement in changing 4 lines of code.

But all this was done in a local production copy, we needed verification of what was causing bloat in production, adding profiling to production was a risk we couldn't take but what we did do was add and configure the oink gem which gave us a memory stat in the production logs and after a few hours of normal app use the ability to do $ oink -t 50 log/production.log which gives the insightful output:

---- MEMORY THRESHOLD ----
THRESHOLD: 50 MB

-- SUMMARY --
Worst Requests:
1. Dec 02 11:11:08, 143136 KB, BarController#foo

so verification of the bloat as this method was executing the all.each mentioned above. All the evidence was converging on this point in code. Marvellous!

I had never profiled a Rails app before admittedly this was not your average app but the tools used here definitely took us from hunches and presuppositions to good solid evidence and the beginnings of some simple but effective performance refactorings.

Hope it helps you when your 400lb Gorilla app starts playing up.

blog comments powered by Disqus

Fun fun fun fun fun

Wed, 24 Nov 2010 07:07:25 GMT

Fun fun fun fun fun

Posted by Steve on November 24, 2010|Comments

Say what you like about the tories, they know how to get the kids rioting...

blog comments powered by Disqus

Ruby 1.9 String encoding shitfest

Thu, 16 Sep 2010 08:56:32 GMT

There are countless posts out there, screams in the darkness by developers claiming that this gem doesn't work with that web framework using Ruby 1.9

In my case the problematic layers in the technology stack were Padrino 0.9.14 with Sinatra-1.0 and Typhoeus 0.1.31 plus some other String to byte array conversions in our codebase.

Testing had showed that when the app received input containing umlauts and other accented letters (i.e. multi byte characters) it failed. There are some lengthy and detailed explanations and discussions about how Ruby 1.8 handles character encoding differently from Ruby 1.9. The sort of thing I'd hoped I'd never have to read but some times you just have to get informed. Hopefully this article will help shortcut or reiterate some of this info.

So lets begin with the rendered view of your web app, a couple of things will help ease the pain here.

First make sure all your pages contain the correct content-type meta tag:

Secondly make sure that all your forms use the accept-charset attribute in the html form tag or whatever you use to generate the tag:

<form accept-charset="utf-8"...

now we know that the pages can display and post multi byte UTF-8 characters, the web application might be a different story...

One of the next quick fixes to apply which worked well for me was to apply a filter on all requests which forced the encoding of all incoming String params.

A branch of RKH's fork of Sinatra helped with this, there's a utility method in lib/sinatra/base.rb which does the trick...

if defined? Encoding

if Encoding.default_external.to_s =~ /^ASCII/

Encoding.default_external = "UTF-8"

end

Encoding.default_internal ||= Encoding.default_external

def force_encoding(data)

return if data == self

if data.respond_to? :force_encoding

data.force_encoding(Encoding.default_external)

elsif data.respond_to? :each_value

data.each_value { |v| force_encoding(v) }

elsif data.respond_to? :each

data.each { |v| force_encoding(v) }

end

else

def force_encoding(*) end

end

to work with this we need to set some default encodings for our webapp.

In Padrino this can be done in the app.rb file:

if RUBY_VERSION < '1.9'

$KCODE = 'u'

else

Encoding.default_external = Encoding::UTF_8

Encoding.default_internal = Encoding::UTF_8

end

if you are a padrino user who doesn't wanna touch the Sinatra gem you can create a general before filter in app.rb like this

before do

force_encoding(params)

end

there are likely other parts of your application stack that will not be UTF-8 friendly, noteably your tests might complain if you attempt to add multi byte characters to your test data.

Adding the code hint

# encoding: utf-8

to the first line of the test class files will alleviate this, it's a grubby fix but we live in a world of compromises, what can you do.