Heuristics for Improving Performance of Ruby Code

Finding a ruby program running like a three-legged dog on tranquilisers, I started to try to improve its performance. Searching the internet I could find no rules of thumb for improving this already documented in one place. So on Ruby-Talk:154733 I asked:

Subject: Ruby-specific performance heuristics?
From: Hugh Sasse <hgs dmu.ac.uk>
Date: Fri, 2 Sep 2005 21:34:17 +0900

I've been doing some stuff with CSV recently, having data in one
flat file that I'm trying to normalize into tables.  Thus, I'm
trying to ignore things I've seen before so I don't create 2 entries
for them, and that sort of thing.

Jon Bentley points out that "Data Structures Programs" -- i.e. how the
data is "shaped" determines how the program will be designed and
perform.  Often the best solution is not immediately obvious, as he
discusses.  So, for example, one can test for "seen before" by

   array = []

   unless array.include? item
     array << item
   end

which is much slower for clumped data than

   unless array.include? item
     array.unshift item
   end

or one could use a hash, or maybe a set.

So, my question is [a memeber of the set of "How long is a piece of
string?" type questions]:

   Are there any heuristics for performance of *Ruby* data structures
   available, which guide one to designing code to *often* have good
   performance?

The answer will be very much "it depends, you neet to test it, it's
a function of data size, ...", but I suspect thee
implementation, i.e. Ruby puts useful bounds on the problem.

My searching has not turned up anything, though there are good
performance hints in the PickAxe. I'm wondering if the info
exists or if it would be worth trying to create it, or
if it is just too broad a problem to be worth it.  However, since we
Rubyists often speak of the ability to put code together quickly,
having hints around on how to do so and yield efficient code seems
of some worth.

         Hugh
    

Contributions are as follows, but see the thread for full information.

• Robert Klemme said
• A hash is faster for lookups than Array (but not just for Ruby)
• freeze a string that you are going to use as a hash key because a String key will be duped on insertion if it is not frozen to prevent errors caused by aliasing
• Another general rule of thumb that is true with Java and also Ruby is that object creation is comparatively expensive. I.e. "foo" + "bar" is more expensive than "foo" << "bar" for example. There are only some exceptions to this rule (Fixnums for example).
• This is mentioned in the PickAxe
• Ara.T.Howard said
• use class variables of some sort : never store in an object what should be stored in a class
• write lazy attribute getters : if your parser loads 15,000 fields from a header but normally only 4 or 5 are use in code write the parser to parse the field on demand and cache the result
• don't write explicit IO : use mmap and let the kernel worry about it
• use processes not threads : ipc is so easy with ruby (using local drb objects) that it's ridiculous not to use multiple processes so multi-processor machines can actually take best advantage of your code.
• don't check things: assume an arg is an int - the stack trace is plenty good to tell you it wasn't and the check slows things down
• use [rbtree, sqlite, ruby inline], each is amazing
• use iterator methods for filesystem operations: prefer
  find('.'){|f| ... }
  over
  Dir['*/**'].each{|f| ... }
  the seconds kills you when a dir has 100,000 files in it.
• compose things out of array, hashes, and strings whenever possible. these objects are pretty optimized in ruby and serialize great as yaml meaning you never need to write hand parsers.
• eliminate variable creation where possible: prefer
  n = nil
obj.each{|elem| n = elem ** 2 and p n}
  over
  obj.each{|elem| n = elem ** 2 and p n}
  
• This is mentioned in the PickAxe
• When creating regular expressions, use an anchor unless you know exactly why you shouldn't.
• For regular expressions which contain interpolation, compile them only once (/#{var}/o) where you can.
• cache anything possible, for instance methods dynamically generated from method_missing: permanmently add the method to the object instead of generating it everytime
• See The Fully Upturned Bin and the related commentry on RedHanded for more on how garbage collection interacts with your program.
• I seem to be finding that:
• Explicitly declaring a class with attr_accessor methods and an initializer seems to be faster than using Struct to do this.
• Using String#split(Regexp) is faster than String#split(String), and certainly better than trying to use String#trim to simplify the regexp therein.
• Vincent Foley finds (here, ruby-talk:184106
• Using Set instead of Array when only uniqueness, not ordring is needed, will be faster

Created by Hugh Sasse on 02-SEP-2005

Last Modified by Hugh Sasse on 13-JUN-2006
$Id: index.html,v 1.5 2006-06-13 16:41:59+01 hgs Exp hgs $