Apr 6 2009

The great Twitter Ruby vs Scala war debate


The executive summary:

Twitter prefer Scala rather than Ruby  for some back-end processes.
Fun ensues

Some of the Twitter developers were recently intervied by Bill Veners on Scala: Twitter on Scala

Which seemed to raise the ire of many of the Ruby crowd in the infosphere. The fact that Twitterer(?) Alex Payne has a new Scala book smells of vested interest to many.

As Payne posted in Mending The Bitter Absence of Reasoned Technical Discussion:

the story … had gone from “Scala is a nifty language and you should think about it for your business to “Twitter engineer spits on the grave of Ruby, exalts Scala as shining new deity”

Tony Arcieri (REIA, Erlang) has a really great analysis of message queues and Ruby – the best part is some of the implicated Twitter developers address his points in the comments. The team evaulated various message queues and went with their own implementation (a Scala-based app called Kestrel).

The Twitter guys know their code, know their environment. They have tried a bunch of technologies, and have developed something that works for them.

However, it’s comments like this from the interview that really pique my interest:

I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.

Jeremy McAnally summed up my own thoughts on Twitter:

If you have to use kind_of? all over your code to mimic a “type system,” you’re doing it wrong.

Twitter: you’re doing it wrong!

I realise that I have no real right to be calling Twitter out here as my credentials with regards to developing the next big thing with a growth curve that is a straight line all the way up to world domination are currently nil.

But the secret truth of most large software projects is that the code often sucks.

See the Big Ball of Mud for more details.

So given Twitter’s code probably sucks, and your code probably sucks, and my code definitely sucks, what can we do?

I guess we assume they picked the right tool to make some of their code suck a little less.

Well done Twitter.

My takeaways:

The Rubyists are a pretty defensive group. I love Ruby and Rails. but still, I am hopefully not a member of the “cult”.

The JVM is just about the best platform there is for high-scalability. Not Java itself as such, but the JVM. I am fairly certain the future is going to be languages running on the JVM. I think Clojure is interesting for this reason – although Scala may be good middle ground as a future-proof language option. Lisps have a long history of being the best lanaguage ever that never made it to the mainstream.


One other thing I do find very curious is that given the code-compile-deploy cycle still required by Scala is how it can really be as fast to develop in as Ruby or Python? During my time as a Java Developer, it was this cycle that was the real time killer. On any non-trival project the compile cycle will start eating minutes and deployment is necessarily complicated when pushing to servers with JAR/WAR deployment systems. I can understand how this trade-off may work for infrastructure (like a message queue) where changes are slow and largely internal, but for a web-level application where change is constant, it’s going to slow you down.