Meta-programming

Ruby, Meta-programming, and Watir

Daniel Bernier Meta-programming Tags: 14 Comments

WATIR – Web App Testing In Ruby

I’ve been playing with Watir recently (thanks for the tip, Ken!), since the folks at my new job have (ahem) no real QA team or process. Watir lets you create and drive an Internet Explorer browser with Ruby (via the Win32 API library), so you can script (and save, and frequently run) a whole suite of regression tests. It’s a great library—it’s easy, clean, immediately rewarding, lots of fun, and implemented fairly simply. Let’s use it to search Google for ‘Watir’:

require 'watir'
ie = Watir::IE.new  # launch a new IE browser
ie.goto("http://www.google.com")   # send it to Google

# Get the query input (the text field named "q"),
# and set it to "Watir"
ie.text_field(:name, "q").set("Watir")

# Get the search button (whose value, or text, matches the
# Regexp /search/), and click it
ie.button(:value, /search/i).click

# Find the link that includes "web application testing",
# and click it
ie.link(:text, /web application testing/i).click

I did that all from memory, and made only two mistakes (I forgot the Regexps should be case-insensitive, and I looked for the “Search” button by :label instead of :value). Go ahead—install ruby, run gem install watir (what’s gem?), and try it out.Watir makes it easy to find any HTML element on the page, if it has a name, id, title, or value…but it doesn’t let you search by CSS class, which I consider a significant omission (plenty of generated HTML has consistent style names, but generated IDs). Looking at the source, I think the authors assumed you would be only looking for one element (ie.text_field(:name, "q")), or all of a given kind of element (ie.divs).Also, suppose you grabbed that paragraph about avocados, and now want the second link inside that paragraph—ie.p(:name, 'avocados').links(:index, 2) doesn’t work. If the HTML elements you want have no id or name, the ability to traverse the DOM hierarchy is pretty important. This also is a problem for me.So I set about trying to improve Watir, to see if I could add this functionality. Along the way, I decided to write my own wrapper around IE’s DOM interface (exposed through Ruby’s WIN32OLE object), and I’ll try to re-write the DOM parts of Watir using it. If it goes well, I’ll submit it to the Watir team, but even if it doesn’t, I’m learning a lot about Ruby, and really starting to like it. Call me a fanboy.Here, I’m writing about the prizes I find along the way—things I’m learning, the neat approaches, or things that confused me at first. It’s aimed at someone who (hey, just like me!) has learned enough Ruby to do the basics, but wants more information on the parts of Ruby that aren’t found in C-based, OO languages. If you stop reading here, you’ve already picked up a great tip—go download Watir. The curious rest of you, let’s proceed…

Delegation Made Trivial

class Node
    # Auto-wrap the @ole_node
    def method_missing(method_id)
        @ole_node.send(method_id.id2name)
    end
end

Watir hands out the OLE Document object, and the Node class (and its subclasses) wraps it, adding some functionality and OO pleasantness…but there are times we’d like to access the WIN32OLE’s methods. We could expose the OLE through an accessor method, but what we really want is to talk to the Node like it’s the OLE. Here’s the part I like: instead of writing a hundred lines of delegator methods, we implement method_missing. If a Node is sent a message it doesn’t understand (a call to a method it doesn’t implement), it just forwards the message to its WIN32OLE.To do this, we override Object’s method_missing method—it’s a hook for just this situation. We’re passed the name of the missing method, and we can use the object.send("method_name") syntax for calling methods. My example is really simple—every method of WIN32OLE is exposed through Node, with the same name. To restrict it, I’ll probably add an Array of methods I want to expose, and only delegate if wrapped_methods.include? method_name. If I want to get fancy, I’ll use a Hash to map the method names, so node.tag delegates to @ole_node.tagName.This trick, I think, is possible because Ruby is dynamically typed. Ruby’s willingness to send any message to any object is what makes this work.

Meta-programming, and Then Some

searchable_tags = %w[a p div span input table]

searchable_tags.each { |tag|
    code = <<METHOD_TEMPLATE
        def #{tag}s (*args, &proc)
            proc ||= make_filter(args)
            self.find_all { |n|
                n.tagName.upcase == "#{tag.upcase}" and proc.call(n)
            }
        end
METHOD_TEMPLATE
    module_eval(code)
}&#91;/sourcecode&#93;
This is maybe a bit a dense...I’ll explain first.
<h3>Let’s Just Find the Divs</h3>
I like Watir’s style of searching for elements: <code>ie.div(:name, "header")</code> returns the div named ‘header’, and <code>ie.divs</code> returns all divs.  I decided to copy this, but make it a bit more general:  both versions (div and divs) should accept multiple filter parameters, so you can search by zero, one, or more attributes.  The singular version (div) should return the first match, the plural (divs) should return all matches.  For example:
doc.div  # the first div
doc.div(:class, "blue_text") # the first div whose class is "blue_text"
doc.divs  # all divs
doc.divs(:class, "blue_text") # all divs whose class is "blue_text"

# all divs whose class is "blue_text", and whose title contains 'porpoise'
doc.divs(:class, "blue_text", :title, /porpoise/)

And just for fun, let’s let people pass in their own filter blocks:

doc.divs { |div|
    div.hasChildNodes
}

The code for the divs method will look like this:

def divs (*args, &proc)
    proc ||= make_filter(args)

    # Find all sub-nodes for which this block evaluates to 'true'
    self.find_all { |n|
        n.tagName.upcase == "div".upcase and proc.call(n)
    }
end

For those new to Ruby, *args wraps all the method’s parameters (arguments) in an array, so you can accept variable lists of them…it’s how the method supports calls like doc.divs(:class, "blue_text", :title, /porpoise/). Also, if the method was called with a block, instead of calling it via yield, we can treat it as a variable by putting &proc in the method definition, and run it later via proc.call.Speaking of that proc…if a block was passed, we’ll use that, but if we got a list of filter parameters, we want to make a filter proc out of them (via make_filter(args)). Enter Ruby’s ||= shortcut: x ||= y is the same as x = x || y. If x is non-null, then y doesn’t have to be checked, so x || y evaluates to the value of x. If x is null, then x || y evaluates to the value of y. It’s a nice idiom for setting optional parameters to default values: “if a value was passed, use it, but otherwise, use this default.” Here, proc ||= make_filter(args) sets proc either to the block that was passed, or to the proc that make_filter(args) returns. From there, it’s simple: find all elements where the tag is “DIV”, and the filter procedure returns true.

Generating Other Finder Methods

Now that’s fine for divs, but we want to search for lots of elements this way! I don’t want copy-paste versions of that method for span, a, p, img, table, and the rest of them…what a mess. Instead, I’ll write Ruby code to generate them for me, from a template. Let’s work from the inside out.

module_eval(code) takes a string of Ruby code, and evaluates it in the context of the current module. In other words, if you pass in code that defines a method, you can then execute that method for the module. Here, we’ll use it to add methods to the Element class.

But who wants to cram a bunch of code onto one line, in a string variable? Let’s use Ruby’s multi-line string:

code = <<METHOD_TEMPLATE
    def #{tag}s (*args, &proc)
        proc ||= make_filter(args)
        self.find_all { |n|
            n.tagName.upcase == "#{tag.upcase}" and proc.call(n)
        }
    end
METHOD_TEMPLATE&#91;/sourcecode&#93;
Everything between <code>&lt;&lt;METHOD_TEMPLATE</code> and <code>METHOD_TEMPLATE</code> is interpreted as a string, and stored in the code variable (the names “code” and “METHOD_TEMPLATE” can be whatever you want—they’re not specific to generating methods).Sharp readers will notice that that string won’t evaluate without a variable named ‘tag’ in scope, so let’s add that:
searchable_tags = %w[a p div span input table]

searchable_tags.each { |tag|
    code = <<METHOD_TEMPLATE
        def #{tag}s (*args, &proc)
            proc ||= make_filter(args)
            self.find_all { |n|
                n.tagName.upcase == "#{tag.upcase}" and proc.call(n)
            }
        end
METHOD_TEMPLATE
    module_eval(code)
}&#91;/sourcecode&#93;
This shows off Ruby’s nice <code>%w[ ... ]</code> short-cut for declaring an array of strings.

# Normal syntax
searchable_tags = ["a", "p", "div", "span", "input", "table"]

# %w cleans things up!
searchable_tags = %w[a p div span input table]

Once you’re used to it, the code is much clearer. Thanks to Greg Brown and his Nuby Gems column for shedding the light.

Meta-programming Wrap Up

Specifically from this example, I’ve learned:

  • Meta-programming is like a mini run-time code generator built right into Ruby, and it’ll save you lots of typing—it’s great for keeping your code light. Remember: module_eval affects classes, instance_eval affects objects.
  • For meta-programming, those multi-line strings make a nice template mechanism. Just remember: the end label must come right after a newline—no leading whitespace for indenting. That stumped me for a bit.
  • Ruby’s ||= is handy—just remember that a ||= b means a = a || b. If a is null, then a evaluates to false, so a is assigned the value of b.

If you want to read more about meta-programming, I’d suggest A Little Ruby, a Lot of Objects, even though it’s only partly about meta-programming. Dwemthy’s Array is fun (and deranged), but I found it hard to see the meta-programming through the other stuff. I’m currently chewing on these bits. I still need to crack the metaclass = (class < self; self; end) nut that I keep reading about…any advice?

One Last Thing—On-line Documentation

As much as I’m coming to really like the pickaxe, these Ruby API docs, made with RAnnotate, are really handy. As we start to really invest in Watir, I’ll probably set this up for us locally.