MapReduce runtime language choices

I'm not interested in language wars or general arguments over relative merits of a particular technology choice, but a recent blog post caught my attention as it flashed up on my Google Alert email. Entitled Yahoo's infrastructural disadvantage to Google: Java performance does not scale, the author (Kevin Lawton) presents some experimental results in benchmarking scalability of Java and C++.

Yahoo(YHOO) uses a Java-based MapReduce infrastructure called Hadoop. This article demonstrates why Java performance does not scale well for large scale compute settings, relative to C++, which is what Google(GOOG) uses for their MapReduce infrastructure.

What I particularly liked about this post is that Kevin does not just compare languages against each other, but compares scalability of each language against itself and then against the other.

We all have choices to make, and there will never be a one-size-fits-all technology. The great thing about software - and open-source in particular - is that a lot of very smart people work to provide alternative technology solutions so that we can make these choices freely to best suit our needs.