The great Twitter Ruby vs Scala war debate

tobyhede

The executive summary:

Twitter prefer Scala rather than Ruby  for some back-end processes.
Fun ensues

Some of the Twitter developers were recently intervied by Bill Veners on Scala: Twitter on Scala

Which seemed to raise the ire of many of the Ruby crowd in the infosphere. The fact that Twitterer(?) Alex Payne has a new Scala book smells of vested interest to many.

As Payne posted in Mending The Bitter Absence of Reasoned Technical Discussion:

the story … had gone from “Scala is a nifty language and you should think about it for your business to “Twitter engineer spits on the grave of Ruby, exalts Scala as shining new deity”

Tony Arcieri (REIA, Erlang) has a really great analysis of message queues and Ruby – the best part is some of the implicated Twitter developers address his points in the comments. The team evaulated various message queues and went with their own implementation (a Scala-based app called Kestrel).

The Twitter guys know their code, know their environment. They have tried a bunch of technologies, and have developed something that works for them.

However, it’s comments like this from the interview that really pique my interest:

I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.

Jeremy McAnally summed up my own thoughts on Twitter:

If you have to use kind_of? all over your code to mimic a “type system,” you’re doing it wrong.

Twitter: you’re doing it wrong!

I realise that I have no real right to be calling Twitter out here as my credentials with regards to developing the next big thing with a growth curve that is a straight line all the way up to world domination are currently nil.

But the secret truth of most large software projects is that the code often sucks.

See the Big Ball of Mud for more details.

So given Twitter’s code probably sucks, and your code probably sucks, and my code definitely sucks, what can we do?

I guess we assume they picked the right tool to make some of their code suck a little less.

Well done Twitter.

My takeaways:

The Rubyists are a pretty defensive group. I love Ruby and Rails. but still, I am hopefully not a member of the “cult”.

The JVM is just about the best platform there is for high-scalability. Not Java itself as such, but the JVM. I am fairly certain the future is going to be languages running on the JVM. I think Clojure is interesting for this reason – although Scala may be good middle ground as a future-proof language option. Lisps have a long history of being the best lanaguage ever that never made it to the mainstream.

Afterthought:

One other thing I do find very curious is that given the code-compile-deploy cycle still required by Scala is how it can really be as fast to develop in as Ruby or Python? During my time as a Java Developer, it was this cycle that was the real time killer. On any non-trival project the compile cycle will start eating minutes and deployment is necessarily complicated when pushing to servers with JAR/WAR deployment systems. I can understand how this trade-off may work for infrastructure (like a message queue) where changes are slow and largely internal, but for a web-level application where change is constant, it’s going to slow you down.


16 Responses to “The great Twitter Ruby vs Scala war debate”

  • Stefan Says:

    About J2EE deployment: I think it’s a critical point to get it right. For small to mid-size projects the best you can do is to use exploded EAR/WAR-directory and have just an ant task that copies the files from the ide to the appserver (or use something like myeclipse or jboss ide that does it automatically) . With that a redeployment cycle takes about 5 seconds.

  • Toby Hede Says:

    That’s a really good point. Things have definitely improved a lot since I first started Java web development.

  • nicholas f Says:

    We used to set up the source as an expoloded .war within the IDE (IDEA), and run the app. through a Jetty .jar there. The compiler would only ever recompile *changed* code, so the only lag in the entire sequence was how long it took to start up javac. The IDE allowed a hot deploy; most recompilations changes took seconds – i.e. change your code, hit save (IDEA would automatically compile), then refresh the browser.

    It didn’t interfere with development at all.

  • Joao Pedrosa Says:

    I think one of the pluses of dynamic languages like Ruby that are hard to copy in static typed languages is that Ruby code doesn’t have to load all files of the “project” in one go because the correctness of the code is checked at runtime and the code first needs to be run at runtime for it to be necessarily loaded. That is, it’s easy to make Ruby code modular and plugins work without having to rely on much else than basic Ruby.

    NetBeans for example has a very advanced plugin architecture with XML and stuff and Java has dynamic loading of classes I think. Even though I don’t know much about it, there’s the OSGi in Java-land whatever that works for. In Ruby, one a.rb file is a module for a b.rb file and add to that some more configuration and you have a dynamic architecture capable of expanding to tons of files that don’t all need to be loaded at the same time. For instance, Rails uses a lot of “autoload” nowadays, which is a native Ruby feature.

    Also, CRuby runs instantly, loading lots of files in likely under a second. JRuby might take a while longer due to the environment requiring more work and still not being as much optimized as possible. My point is that in dynamic languages, testing changes can take seconds, and the fewer seconds the more pleasant the experience. When coupled with loading just the needed files for testing the change it can be a killer feature. For instance, instead of loading the entire GUI application you could load just the GUI window you want to test it could be a killer feature. In NetBeans, you can test the window developed with the GUI designer but it doesn’t quite work beyond showing you the GUI widgets and their placements.

    Summing up, I don’t believe coding in Scala can be as pleasurable as coding in Ruby in most cases.

  • Darren Says:

    “The JVM is just about the best platform there is for high-scalability.”

    I (generally) agree with that. The JVM is a very impressive platform. With support for clojure, scala and ruby, it is a very flexible platform as well.

  • Roshan Shrestha Says:

    “The JVM is just about the best platform there is for high-scalability.”

    True, and this is so not because of just the virtual machine itself, but because Java has the most extensive libraries among all languages, stuff like JMS (which Twitter should have used), Lucene, Hadoop, iText, and many others.

    You can code using your favorite language and still use the Java libraries. Of course, you better code in Java if you want to write libraries for the JVM that others may use!

  • Ennuyer.net » Blog Archive » 2009-04-07- Today’s Ruby/Rails Reading Says:

    [...] The great Twitter Ruby vs Scala war debate — TopSecretProject [...]

  • Ivan Sanchez » Language wars Says:

    [...] it just me who’s loving this whole Ruby vs. Scala debate going on lately? So far it shows [...]

  • Rob Heittman Says:

    re: the Afterthought. I suspect the super-painful code/compile/deploy cycle isn’t Java’s fault, it’s JEE’s. In our OSGi-based environment we can write Java or Scala modules, export the classes with a click, and drop them into a running system to either augment or replace a specific component.

    I know this realtime mashup ability flies in the face of the very structured application release lifecycle preferred inside the enterprise … but it works very well for managing large and complex, always-up, web services, where you change one aspect at a time. This is no new idea for script-based Web developers … (PHP, Ruby) but the notion that you can do it with Java is somewhat heretical.

  • Daily Links 10/4/2009 Says:

    [...] Ruby vs Scala – Si, ancora [...]

  • Jonathan Says:

    A mindless Atrocity, oh how can you praise and give honor, money, and Pulitzer’s to them. Those who build tyranny , the suffering of men .That leave land stricken and with childless mom’s. Unveil their creation, the nuclear and atomic bomb. Men that create wealth out of another’s confusion. Paint a picture, but one of delusion. I will bring you water, I will dig your pits. But, I will be buried upon where you sit. Do you stop to hear me cry or am I silenced by the voices of my oppressor’s lies. Can you reach out to me, just to grab my hand? Can we talk for awhile? you might understand? They don’t know the true God as I do? Did he steal, kill, and destroy you mindless fool? Did he? Did he? Can you go on breathing, believing, and thinking you are safe? Bury your holes the end is near the hour grows late. Step back raise up, live, laugh, and love. Show your mercy and kindness as heaven above. So bring your chaos and your disorder, my nightmare, your New world Order

    You will be judged, raped, poor and rid. Not because you are guilty, but because you say “it’s not my kids”! People too skinny, frail, and broke. We drink our wine; eat our steak as we choke. When will the chaos end? When will we embrace each other not as acquaintances but friends? We say kill them all, let their house fall, which of you cares it is sin they bear. They don’t know the true God as I do? Did he steal, kill, and destroy you mindless fool? Did he? Did he? Can you go on breathing, believing, and thinking you are safe? Bury your holes the end is near the hour grows late. Step back raise up, live, laugh, and love. Show your mercy and kindness as heaven above. . So bring your chaos and your disorder, my nightmare, your New world Order

    Act like you don’t know me. Rolls your eyes as never to unfold me. Look at me are we not brothers? Abraham as our Father and Eve as our mother? Can we take a few steps in the sand or do we only glance to pass money by hand. Do I repulse you? The flies that consume me? Did you turn off your television in your air conditioned living room as they brought genocide and abused me? They don’t know the true God as I do? Did he steal, kill, and destroy you mindless fool? Did he? Did he? Can you go on breathing, believing, and thinking you are safe? Bury your holes the end is near the hour grows late. Step back raise up, live, laugh, and love. Show your mercy and kindness as heaven above. . So bring your chaos and your disorder, my nightmare, your New world Order

    Share your secrets about heaven above, as you bring chaos, missiles, and no message of love. Prey on my land steal my resources. Bring your law that lends it’s divorces. Transcend me, bring me into the brightest of light. Where jets drop bombs and bodies ignite. Is this your vision? The mystery to fathom? Is this the reason and my days as you have them? They don’t know the true God as I do? Did he steal, kill, and destroy you mindless fool? Did he? Did he? Can you go on breathing, believing, and thinking you are safe? Bury your holes the end is near the hour grows late. Step back raise up, live, laugh, and love. Show your mercy and kindness as heaven above. . So bring your chaos and your disorder, my nightmare, your New world Order
    Corporations without border, to feed the vile of greed. You plant on this earth the devil’s seed.

  • Toby Hede Says:

    I have no idea what this means, but it’s very well written.

  • Bill Says:

    I would like to know how,why, and what the solution is to the “you’re doing it wrong” comment on a ruby type-system. Personally, I don’t see how duck typing is that great unless you’re sure something is a duck and can quack in the first place …

  • Dobba Says:

    The only reason Twitter is so popular is because it is filled with wannabe web savants tweeting about how great twitter is. It is potentially just another vacuous example of the self-consuming technology hype cycle. Anyone remember Second Life?

  • ueckerman.net » Blog Archive » Book review: Programming Scala Says:

    [...] on Rails. 4 years later and Scala is getting some serious water-cooler time – especially since the Ruby vs Scala flame wars prompted by Twitters move from Ruby to Scala for back-end grunt. So I choose Programming Scala – [...]

  • Francis Fish Says:

    I was very amused by the exploded WAR file stuff at the beginning of the post … you missed out the 20 minute build of the thing … of course machines have got a bit faster over the last three years.

    I started in the late 80′s and it took half an hour to do edit/compile/run on an old Cobol system. My first Java job in 2007 it took *longer* while watching Ant do its stuff. This is progress? That’s why I jumped to Rails when I had a chance. Rails has a great productivity boost, when you encounter specific problems then debug the system and fix what need fixing – if this means using a more appropriate tech, then so what?

Leave a Reply