Archive for February, 2007

Functional Programming in JavaScript and Ruby

[UPDATE: I've been lucky enough to have some commenters who know much more about functional programming than I do. There's some good reading in the comments, and you especially should read them before using this stuff in production.]

I’ve been playing with functional JavaScript tonight. It’s not the greatest of OO-functional hybrid languages, but it’s good to supplant my fledgling functional skills with something besides Ruby. Plus, I’m not the only one doing it, so I can compare notes. And who doesn’t love JavaScript, right? …right?

Before I get much farther, I should thank Adam Turoff for his post What’s Wrong with the For Loop. It gets at the root of the why we’d use a functional programming language instead of an OO or procedural one, and (bonus!) it helped me grok Ruby’s inject method (it’s the same as my new friend fold). Go read his post, it’s good for you. And if you like it, we recommend the 1981 SICP lectures. Robusto!

Here we go!

Introductions to functional programming usually discuss three methods: map, find, and fold. The names aren’t always the same, but those are the three amigos of functional programming. They all do something different to arrays (or lists, or collections, or whatever you want to call them):

  • find does what it says. You give it some criteria, it returns all the matching elements. The criteria is expressed as a function, a little bit of code that says “yes, I want this one,” or “no, skip it.” If we’re picking out the red gumballs, you could say (in Ruby) gumballs.find_all { |ball| ball.isRed? } find is also known as filter. [Ruby only calls it find_all because it uses find for returning just the first match.] Po-tay-to, po-tah-to.
  • fold takes each element, and “folds” it into a running total. Think of summation—you’re folding each new number into a running tally. In Haskell (and, it seems, many functional languages), it comes in two varietes, fold_left and fold_right, which just mean “start from the left” and “start from the right”. Ruby calls it inject, which confuses some people (like me).
  • map is the mathiest of them (don’t be scared, it’s easy!). In English, we’d say “change each item like this, and give me the new results.” Let’s down-case a list of words. “Change each word to lowercase, and give me the downcased words.” words.map { |word| word.downcase }. The name “map” comes from functions in math (surprise), where you map each input to one output. The whole y = f(x) thing…f is for “function”. Ruby also calls this collect.

Now, none of these comes predefined in JavaScript. How can this be? Well, all the JavaScript engines out there are in browsers, and we know that “when browser elephants battle, it is the JavaScript grass that suffers.” The browser developers are busy with other things. But this oversight means we have some easy examples to write. It’s boring to re-implement existing stuff, just for the exercise. So here we go—I’ll compare examples in JavaScript and Ruby.

A quick word about JavaScript’s open classes

JavaScript has open classes, just like Ruby. In other words, you can re-open a class definition, give it a method, and then start using that method on instances of that class. JavaScript’s OO is prototype-based, not class-based, so it looks a little weird:

Array.prototype.first = function() {
    return this[0];
}
[1, 2, 3].first(); >> 1

Array.prototype.rest = function() {
    return this.slice(1);
}
[1, 2, 3].rest(); >> [2, 3]

Array.prototype.isEmpty = function() {
    return this.length == 0;
}
[].isEmpty();    >> true
[1].isEmpty();    >> false

“For the prototypical Array, the Array from which all other Arrays spring forth, create a property called ‘first’, which shall be this function, which returns the first element of the Array…” Just a Biblical way of defining a method for a class.

Open classes is good news, because it’ll make our job of adding map, find, and fold possible, since they’ll be going into the Array class.

Find

find is the easiest, so we’ll start there. Let’s take a look:

Array.prototype.find = function(isAMatch) {
    var matches = [];
    for (i = 0; i < this.length; i++) {
        if (isAMatch(this[i])) {
            matches.push(this[i]);
        }
    }
    return matches;
}

find is a function, and it takes the function isAMatch as a parameter. One of the hallmarks of the functional programming style is that functions are first-class citizens in the language: they can be treated like any other variable, so they can be passed around as parameters, and returned from other functions. [Closures and anonymous functions are also major features.] isAMatch is a function that tells you whether we should include any given element of the Array. Use find like this:

var evens = [1, 2, 3, 4, 5].find(
    function(i) {
        return i % 2 == 0;
    }
);

function isOdd(i) {
    return i % 2 != 0;
}
var odds = [1, 2, 3, 4, 5].find(isOdd);

The first example uses an in-line, anonymous function; the second uses a named function. Both work fine, but here is where we start to see JavaScript’s awkward syntax get in the way. Consider the Ruby version:

# find_all comes pre-defined in Array, via the Enumerable module,
# but this is what it would look like...
class Array
    def find_all
        matches = []
        self.each do |item|        # Ruby says 'self', not 'this'.
            if yield(item)         # yield says "Call the block we were given"
                matches << item    # Stuff item into matches
            end
        end
        return matches
    end
end

evens = [1, 2, 3, 4, 5].find_all { |i|
    i % 2 == 0
}
def isOdd(i)
    i % 2 != 0
end
odds = [1, 2, 3, 4, 5].find_all { |i|
    isOdd(i)
}

In Ruby, every expression returns a value, so return disappears. And Ruby’s blocks mean we don’t need to wrap our match condition inside function(i) { ... }. But Ruby’s find_all won’t take a reference to a method as a parameter:

def isOdd(i)
    i % 2 != 0
end
odds = [1, 2, 3, 4, 5].find_all(isOdd) >> `isOdd': wrong number of arguments (0 for 1) (ArgumentError)

Once you’ve defined a function in JavaScript, you can pass it around by name, like any other variable, but you need that function(i) { ... } syntax around your test. In Ruby, find_all takes a block instead of parameters, so you can’t pass a reference. Given how nice blocks are in Ruby, I guess this can be forgiven, but it’s a little weird.

Fold

Now we’ll get into the recursive guys. find is a pain to implement recursively in JavaScript, so I did it iteratively, but recursion works well for fold and map. Since recursion seems to be more idiomatic in functional languages, we’ll use it here.

I took two shots at fold in JavaScript (I’m learning). Here they are:

Array.prototype.fold_recursive = function(combineFunc, base) {
    if (this.isEmpty()) {      // I added isEmpty, first, and rest, as above
        return base;
    }
    else {
        return combineFunc(
            this.first(),
            this.rest().fold_recursive(combineFunc, base));
    }
}
Array.prototype.fold_iterative = function(combineFunc, tally) {
    if (this.isEmpty()) {
        return tally;
    }
    else {
        return this.rest().fold_iterative(
            combineFunc,
            combineFunc(this.first(),
                        tally));
    }
}

The first is straightforward recursion. If the list is empty, we’re done, so return the base value (zero, if we’re adding). Otherwise, combine the first value with whatever the rest of the items fold up to. [If you have trouble writing recursively, this is the canonical pattern you should probably follow: if done, do base case; else, do this element, then the rest of them.]

The second is a little different. You’ve got a starting base, the tally so far, and a combineFunc that folds each value into the tally. If the list is empty, we’re done, so return the tally. Otherwise, fold the first item into the tally for the rest of the list.

It the first, you hang around, waiting for the answer to the rest of the problem, before you add in your part of the sum. In the second, you push your part of the sum down into the next round of processing, so you don’t have to remember anything. When the answer comes back from the recursive call, it’ll already have your part of the sum in it.

[The first one is a “linear recursive process”, the second is “linear iterative process”, even though both are recursive procedures. If your interest is piqued, check out this SICP page, but it’s not needed for this article. For the rest of this post, I’ll be using the linear recursive version, because it’s conceptually clearer. Thanks to mfp for keeping me honest.]

Here’s how fold is used:

function add(a, b) { return a + b; }  // A handy adding function

Array.prototype.sum_recursive = function() {
    return this.fold_recursive(add, 0);
}
[1, 2, 3].sum_recursive();  >> 6

To sum the elements of an array, we want to fold the numbers together by add-ing them, starting at 0. Easy, isn’t it?

Here are two Ruby versions, based on fold_recursive…one that takes a function (called a “procedure object” in Ruby) as a parameter, one that takes a block:

class Array
    def rest        # We'll define rest, to keep the examples similar
        self[1..-1]
    end

    def fold_w_proc(combineFunc, base)
        if self.empty?
            base
        else
            combineFunc.call(    # "func.call(args)" instead of JavaScript's "func(args)"
                self.first,
                self.rest.fold_w_proc(
                    combineFunc,
                    base)
            )
        end
    end
    def fold_w_block(base, &combineFunc)
        if self.empty?
            base
        else
            combineFunc.call(
                self.first,
                self.rest.fold_w_block(
                    base,
                    &combineFunc)
            )
        end
    end

    def sum_w_proc
        fold_w_proc(lambda { |a, b| a + b }, 0)
    end
    def sum_w_block
        fold_w_block(0) { |a, b| a + b }
    end
end

[1, 2, 3].sum_w_proc    >> 6
[1, 2, 3].sum_w_block    >> 6

Notice how similar fold_w_block and fold_w_proc are to the JavaScript fold_recursive. The thing that differentiates fold_w_block and fold_w_proc is how they’re used. The definitions themselves are nearly identical, except for the order of the parameters. Look at sum_w_proc and sum_w_blocksum_w_block is a bit clearer, isn’t it? But if you use blocks, you lose the ability to pass a function reference as a parameter.

Map

map looks a lot like fold.

Array.prototype.map = function(mapFunc) {
    if (this.isEmpty()) {
        return this;
    }
    else {
        return [mapFunc(this.first())].concat(
                this.rest().map(mapFunc));
    }
}

function cube(i) { return i * i * i; }

[1, 2, 3].map(function(i) { return i * i; });  >> [1, 4, 9]
[1, 2, 3].map(cube);  >> [1, 8, 18]

Again, it’s basic recursion…if we’re out of items, return the empty list (ie, this). Otherwise, make an array of the first mapped item, and concatenate it with the rest of the mapped items. Again, with JavaScript, you can pass your function in-line, or by reference.

Here’s a recursive definition of map in Ruby:

class Array
    def map(&mapFunc)
        if self.empty?
            self
        else
            [mapFunc.call(self.first)] + self.rest.map(&mapFunc)
        end
    end
end

[1, 2, 3].map { |i| i * i }   >> [1, 4, 9]

As before, calling map with a block certainly looks nicer than by passing in functions, but you lose the ability to define a function somewhere, and pass it in by name, like we did with cube in JavaScript.

Wrap-up

If you want to explore functional programming, both JavaScript and Ruby are good candidates. They both have their goods and bads. Up to now, I’ve only used Ruby, but JavaScript certainly has its benefits, and exposure to both balances out my understanding.

I hope that, if you were curious about functional programming before, this was a helpful read. If you’re a functional programmer, and I got something wrong, please let me know…I’m still learning. For instance, I now better understand how recursive processes differ from linear recursive processes.

How to Design A Domain Specific Language, with David Pollak

I just finished watching David Pollack’s presentation at Google, How to Design A Domain Specific Language. It’s only an hour, and it’s got some interesting ideas. A nice jog, good for your mental health.

His main theme is bringing computing closer to the business users. Computers exist to help us solve problems, but most people can’t program, so we need all these programmers to translate for the business people, but the translation is often imperfect, so we go back and forth like couriers between the business and the machine. Don’t we have better things to be doing? This idea of letting business users program often riles programmers who fear being automated out of a job, but I think there’s enough hard stuff out there for anyone who’s up for cracking some hard nuts. I say, let’s make the drudge work easier, so we can get on with the real work.

The essence of a DSL is building the semantics of the business into a mini-language, so the business people can read the code. [Sure, they could probably write it themselves, but most don't want to.] And if they can read the code, they can sign off on the code. Why translate specs from business-speak into Java? Why not make the code so clear, it is the spec?

Pollack points out that if you try this, you have to convince the business that the translation works. I’ve seen this before…when your code is readable by the business, they’ll forget that it’s code. “How do we know the software does what the requirements say?” Because the requirements basically are the software. Once they grok it, though, you can almost remove yourself from the equation. Pollack says of his DSL SiteMap:

We were able to go through with…the business users themselves, to figure out [the navigation rules] for each page…We never had a failure where the business user didn’t get something they were expecting to get. We had to demonstrate early…that the implementation of the code matched the semantics of the DSL. Once we proved that, the code reviews went really really quick.

That’s a great place to be.

BTW, thanks to Peter Cooper for sharing that video. He also shared Code Generation With Ruby with Jack Herrington…Herrington’s book Code Generation in Action has been on my wishlist ever since I read the sample chapters Manning let me download, so I’m really looking forward to the video.

Programmable Text Editing on Windows with EmEditor

Windows lacks a decent programmable text editor.

I know there’s Emacs and Vim, but I want something that looks good, and is easy to learn. If it’s free or cheap, that’s good, too. Based on screencasts I’ve seen, I’d say I want something like TextMate, but I’ve never actually used it, so that’s just a guess.

It has to be programmable. I spend lots of time working with text, and I want an editor that lets me write code to process my text. I want to take a list of values, and generate SQL from it. I want to auto-format HTML. I want to insert custom code templates, based on keywords I type.

Really, I want something like Ruby’s text-mashing power, built into my editor. If I could send a buffer’s contents through some code, and replace the buffer with the script’s output, that’d be something. This would let me automate probably 99% of the repeatable things I do. It doesn’t have to be Ruby, but that’d be nice.

EmEditor’s Programmable Macros

I found EmEditor, which gets me pretty close. It has a 30-day trial, and it’s $40 to buy. It has programmable macros, with a decent set of variables that give you access to the buffers, editors, and all menu items. The macros are run through the Windows Script Host (WSH), which is a little scary, but it means that you can write your macros in JavaScript. Actually, since WSH is language-agnostic, you can write your macros in any language with a WSH interface. Including Ruby. Or Haskell, or Lua, which makes it easier for me to experiment with them now. And of course there’s Python and Perl. I think this is pretty nifty.

The Downside

Since it’s built on WSH, you don’t have any means of including other files. Nothing like Ruby’s require, Java’s import, C#’s using. This means no library re-use, no utility functions. No RedCloth for blog posting, no REXML for auto-formatting XML, no Hpricot for digging through generated HTML. How about a nice, clean interface to WSH’s hairy objects? I’d love to write one! (Well, I’d love to have one, so I’d be willing to write it.) But you couldn’t use it, if I did. I think this is a pretty significant problem. You have to start from scratch with each script. Someone please tell me I’m missing something here.

None of this is stopping me from rolling up my sleeves yet. Here are some tasks I’ve automated so far (you can get some of the .js files from my Google Pages…I’ll put up more when I have a minute):

  • Replacing “smart quotes” and other funny characters with their plain-text equivalents. Great for copying code and data out of MS Office products.
  • Delete all copies of whatever text is selected. When I’m cleaning up generated HTML, I remove lots of code (<td>, <tr>, class="foo"), so I can see the parts I care about. Just highlight, run, poof. All gone.
  • Make the word I just typed into an opened-and-closed HTML tag, with the cursor in between. I have this hooked up to Ctrl-space, and Shift-Ctrl-space adds in a newline and tab indent. “b” ctrl-space produces <b>|</b>, and the cursor is where the | is. “table” shift-ctrl-space “tr” shift-ctrl-space “td” ctrl-space. Much faster.
  • Run some JavaScript on each line, and replace the line with the code’s return value. You can use this for chewing on data, cleaning up formatting, calculating values, or whatever. If you have to change many lines of similar C# or Java, you can chew each line up, and rewrite the code with this. Powerful, easy to make a mess with, and all that. This is probably my favorite.

Things I haven’t gotten around to yet:

  • Tidying XML or HTML
  • Customizeable templates. These would work like my HTML-tag-izer, where you type a word, then Ctrl-space, and your word is replaced with a template of code. Eclipse has this, and TextMate comes with a whole mess of really nicely defined ones for Ruby.

Maybe someday, I’ll have learned emacs, and I’ll look back on this post and laugh at myself. “Windows Script Host?? What was wrong with me?” In the meantime, I have something that helps me along, lets me have fun automating things, and lets me explore other languages.

Gotham Ruby Conference

This April 21, in Google’s Manhattan offices, for $100, you can check out the Gotham Ruby Conference.

These regional Ruby conferences are a great idea, I think. If your job won’t pay for you to attend a huge (more expensive) Ruby conference, plus travel expenses, you can take advantage of the smaller, cheaper, local conferences springing up all around. Even if you’re a relative Ruby newbie, the price makes it a low-risk investment. For $100, and a full day of Ruby, it’s a bargain. Just as a cost comparison, I’m about to start another day of .Net training, and that costs about $500 a day.

See you there…

Ruby, Yoga for your Brain

Just a fun little bit of writing, for a contest

LISP has jokingly been described as “the most intelligent way to misuse a computer”. I think that description a great compliment because it transmits the full flavour of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts.

The analysis of the influence that programming languages have on thinking habits of its users, and the recognition that, by now, brainpower is by far our scarcest resource, they together give us a new collection of yardsticks for comparing the relative merits of various programming languages. The competent programmer is fully aware of the strictly limited size of his own skull…

- Edsger W. Dijkstra, The Humble Programmer

“Brainpower is by far our scarcest resource,” indeed. Why then, would I want mine blown? I’d rather gently stretch it. Extend it. Liberate it. Yoga for my brain, please.

In yoga, you honor your body’s suggested threshold of pain. If you can’t do the full posture, don’t hurt yourself, but try this modification, so you still benefit. When you’re ready, you can try the full posture.

Ruby is like yoga for your mind. Ruby encourages you to stretch to your limits, and come back when you’re ready for more. You can write getters and setters, and old-school for loops, and Ruby won’t complain…but it’ll gently encourage you to try attr_reader and attr_writer, and block iteration. You can write your own database access code, until you’re ready to try active record. You can write straight OO, and never leave the Kingdom of Nouns, until you’re ready to try something more functional.

Ruby lets you learn new things, in a familiar setting. With Ruby, you can honor your mind’s suggested threshold of pain. Ruby hasn’t blown my mind, and I trust it never will. A few nasties have blown my mind, and it’s never fun. It takes time to recover from abuse like that. I’d rather follow the yogis out there, and enjoy the benefits of life-long mental flexibility.


Say Hello

danbernier [at] gmail [dot] com
Twitter @danbernier
Hartford.rb
LinkedIn

Tweeting:

  • @rbazinet Dude, a Dell Latitude D820? That's my work laptop! I'm pretty "meh" about it. 33 minutes ago
  • @jpmccu "Clutter [is] a result of...excessive attachment." I think this relates to projects too. So many things out there to do! 2 days ago
  • @wigu If we say we LIKE the teeth, does that mean they'll go away sooner? 2 days ago
  • @dchec Invite them behind the counter, and move the event to a small town. You never know. 6 days ago
  • @kdaigle Where's the new digs again? Congrats on the move! 6 days ago

I’m a BackPack fan

Backpack

Categories